We've lost an absolute giant today. RIP Dimitri Bertsekas. His probability and optimization books got me through my masters. Massive loss for the MIT community and the field.

Self-correction for LLMs is usually done by critiquing an answer and regenerating the entire reasoning trace. We show targeted backtrack-and-resample works better: self-localize the first erroneous step, then sample a counterfactual. Repeat. [1\n]

Oh, to me it is the opposite. LLM RL is when you say supervised fine-tuning (SFT) instead of behavior cloning, RLVF instead of batch policy optimization, base policy instead of behavior policy, trace instead of trajectory, verifiable reward instead of reward, .. LOL

When I teach, I prepare my lecture notes by writing on Goodnotes, then export them as PDFs and share to the class. Recently I asked claude code to read these PDF files and convert them into markdown. This is a test I constantly gave to new LLMs and now claude is good enough to pass it. I converted my RL theory graduate-level course into markdowns and post the lecture notes here: zhuoranyang.github.io/sds685… When preparing the course, I learned a lot from the wonderful courses offered by @CsabaSzepesvari @chijinML @WenSun1 @nanjiang_cs and borrowed some good stuffs from their notes. Also I discussed what to teach in a RL course heavily with @zhaoran_wang and stole many of his insights.

✨New paper alert Does AI help or harm critical thinking? It’s one of the most pressing questions around AI. But maybe the better question is: under what conditions? In our #CHI2026 paper, we find that time can change everything. 🌐 criticalthinking-ai.app/ #AI #criticalthinking #humancognition #decisionmaking #civicparticipation (1/10)

Introducing Simile. Simulating human behavior is one of the most consequential and technically difficult problems of our time. We raised $100M from Index, Hanabi, A* BCV, @karpathy @drfeifei @adamdangelo @rauchg @scottbelsky among others.

Great work by @Ankur_Samanta_ on self-improvement in LLMs via RL post-training on “debate” outcomes!

What if we boost LLM output diversity by fusing base and aligned models? Excited to share this work led by @YichenZW, whose passion for understanding LLM diversity really shows here. More insights and fun ideas on LLM creativity coming soon as well!💡

TD Learning can suffer on long tasks: ↑ deep bellman recursions → ↓ poor scalability (despite big data) We introduce a new method (TRL) with a "divide-and-conquer" value update, which scales well with long horizons!

This paper shows that you can predict actual purchase intent (90% accuracy) by asking an LLM to impersonate a customer with a demographic profile, giving it a product & having it give its impressions, which another AI rates. No fine-tuning or training & beats classic ML methods.

Introducing *dual representations*! tl;dr: We represent a state by the "set of similarities" to all other states. This dual perspective has lots of nice properties and practical benefits in RL. Blog post: seohong.me/blog/dual-represe… Paper: arxiv.org/abs/2510.06714 ↓

Honored that our @RL_Conference paper won the Outstanding Paper Award on Empirical Reinforcement Learning Research! 📜Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-Functions 📎openreview.net/forum?id=H3jc… Grateful to my advisors @JosephLim_AI and @ebiyik_!

Use the same @ExpectedParrot survey you ran w/ LLMs with actual humans via our @Prolific integration (or bring your own panel) - it works!

Wrapped up Stanford CS336 (Language Models from Scratch), taught with an amazing team @tatsu_hashimoto @marcelroed @neilbband @rckpudi. Researchers are becoming detached from the technical details of how LMs work. In CS336, we try to fix that by having students build everything:

What does it mean to write and think with AI? What new possibilities and challenges does that bring? I spoke with THE AI (in Korean) about our group's research and the future of writing with AI. 👩🤖✍️ newstheai.com/news/articleVi…

CATransformers is a carbon-driven neural architecture and system hardware co-design framework. Using CATransformers, we discover greener CLIP models that achieve an average of 9.1% reduction potential in total lifecycle carbon emissions while maintaining accuracy (or increasing accuracy) and latency. This research is the first to look into carbon-driven neural architecture and system hardware co-design. It is enabled by a first-of-its-kind architectural carbon modeling tool – ACT, which we developed at FAIR. Check out: Our paper ➡️ go.fb.me/nedmh0; code repository ➡️ go.fb.me/7jcp6q; and the additional carbon design tools and research artifacts in Sustainable AI ➡️go.fb.me/csee32