Tomorrow, I'm excited to present "Finite-Time Convergence Rates in Stochastic Stackelberg Games with Smooth Algorithmic Agents", which addresses how a principal can influence the behavior of competitive learning agents! #ICML2025 📍West Exhibition Hall, W-817, 11:00 - 1:30 🧵👇

Today we're announcing Treeline and $25M from a16z. Software and AI have crossed a threshold and we're rebuilding IT services around it - great software paired with experienced technicians, designed to be a foundation for growth. We're hiring, reach out!

Introducing ⚓ 𝗔𝗻𝗰𝗵𝗼𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴: a copyright mitigation strategy for any language model! With @uwnlp LMs today reproduce copyrighted text—raising concerns for creator consent and potential legal (and 💸 💸) liabilities for AI developers. 🫠 𝗔𝗻𝗰𝗵𝗼𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴 relies on two off-the-shelf LMs: 🧼A 𝘀𝗮𝗳𝗲 𝗟𝗠 trained only on permissively licensed text, ⚠️A higher-utility 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 trained on any data. The 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drives generation, but the 𝘀𝗮𝗳𝗲 𝗟𝗠 acts as an anchor. If the 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drifts into memorization, the 𝘀𝗮𝗳𝗲 𝗟𝗠 pulls it back ↩️. 🤝We provide a formal guarantee: outputs stays within a user-set budget of the 𝘀𝗮𝗳𝗲 𝗟𝗠. Details below! 👇 [1/⚓]

Decision-making is often done not only under uncertainty, but also in environments subject to the actions of learning agents in competition with one another (e.g. crowd-sourcing, multi-agent systems). A natural abstraction for shaping these agents' behavior is a Stackelberg game.

A bit of a belated announcement 😅 but I’ll be at ICML today presenting S4S, which enables few-NFE diffusion model sampling in <1 hour on 1 A100! 📍East Exhibition Hall, E-3210, 11:00 - 1:30. Looking forward to chatting more about all things diffusion! #ICML2025

Web data, the “fossil fuel of AI”, is being exhausted. What’s next?🤔 We propose Recycling the Web to break the data wall of pretraining via grounded synthetic data. It is more effective than standard data filtering methods, even with multi-epoch repeats! arxiv.org/abs/2506.04689

Thrilled to announce that I will be joining @UTAustin @UTCompSci as an assistant professor in fall 2026! I will continue working on language models, data challenges, learning paradigms, & AI for innovation. Looking forward to teaming up with new students & colleagues! 🤠🤘

🤯 We cracked RLVR with... Random Rewards?! Training Qwen2.5-Math-7B with our Spurious Rewards improved MATH-500 by: - Random rewards: 21% - Incorrect rewards: 25% - (FYI) Ground-truth rewards: 28.8% How could this even work⁉️ Here's why: 🧵 Blogpost: tinyurl.com/spurious-rewards

Excited to share that our paper "Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder" is accepted to #ACL2025! Preprint: arxiv.org/pdf/2411.05195 Thank @SimonShaoleiDu and @PangWeiKoh so much for your support and guidance throughout the journey!

I'm excited to announce that I will join @WisconsinCS as an assistant professor this fall! Time to get to it.

LLMs naturally memorize some verbatim of pre-training data. We study whether post-training can be an effective way to mitigate unintentional reproduction of pre-training data. 🛠️ No changes to pre-training or decoding 🔥 Training models to latently distinguish between memorized sequences and their paraphrases 🔍 Effectively reduced verbatim reproduction while still recalling famous quotes when requested

Think PII scrubbing ensures privacy? 🤔Think again‼️ In our paper, for the first time on unstructured text, we show that you can re-identify over 70% of private information *after* scrubbing! It’s time to move beyond surface-level anonymization. #Privacy #NLProc 🔗🧵

Long Range Navigator (LRN) 🧭— an approach to extend planning horizons for off-road navigation given no prior maps. Using vision LRN makes longer-range decisions by spotting navigation frontiers far beyond the range of metric maps. personalrobotics.github.io/l…

Our new paper (first one of my PhD!) on cooperative AI reveals a surprising insight: Environment Diversity > Partner Diversity. Agents trained in self-play across many environments learn cooperative norms that transfer to humans on novel tasks. shorturl.at/fqsNN🧵

🔭 Science relies on shared artifacts collected for the common good. 🛰 So we asked: what's missing in open language modeling? 🪐 DataDecide 🌌 charts the cosmos of pretraining—across scales and corpora—at a resolution beyond any public suite of models that has come before.