New positions at FOCAL! cs.cmu.edu/~focal/index.html

One of my open math problems apparently got resolved by ChatGPT 5.5 Pro (Ryan O'Donnell prompted it better than I did!), though the proof was so hard for me to read that it seemed easier to just prove it myself. More thoughts on implications for math here: aifails.substack.com/p/even-…

@FOCAL_lab member @EmanuelTewolde is presenting his CoopEval work at EPFL on Monday (14:30) and it will be on zoom! (paper link in comment) memento.epfl.ch/event/ai-cen…

(1/4) Can remembering more of the past make AI agents less cooperative? In our new paper, we study LLM agents in repeated social dilemmas. The key variable is not how many rounds they play, but how much prior interaction history they can access when making each decision.

(2/4) Surprisingly, longer recall often degrades cooperation. Across 7 LLMs and 4 repeated social dilemma games, agents with longer histories often shift away from forward-looking cooperation and toward retrospective grievance-tracking.

(3/4) The mechanism is not just “too much context.” It is what the agents remember: replacing histories with synthetic cooperative records restores cooperation, and ablating explicit CoT reasoning often reduces the collapse. We call this the memory curse.

(4/4) Paper link: arxiv.org/abs/2605.08060 Big thanks to my collaborators! @Jack_Litq, @zzzoooeee321, Xin Luo, Haoxuan Zeng, @EmanuelTewolde, Tai Sing Lee, @tonghanwang, @ckingsford, @conitzer

In this paper led by Jerick Shi @Jerick1380, we propose a taxonomy of LLM deception. arxiv.org/abs/2604.04788

(3/3) Also, I don't understand how some people think AGI is just around the corner but the risks are easily manageable! Of course their positions may not be captured accurately here.

(2/3) I'm sure we all have thoughts on our descriptions -- I certainly worry about many other AI risks current and future in addition to scaled misinformation, and I actually think the world is too focused on LLMs-as-chatbots -- but still impressive.

(1/3) Wow, I (and many friends) made it onto this "Mapping AI" map! Does anyone know the methodology? Who is behind it? mapping-ai.org/map

(3/3) You can play with interleaving tokens yourself here! (... works better with better models...) cs.cmu.edu/~focal/alternatin…

(2/3) We've also been interested in interleaving tokens for philosophical reasons. This chapter based on a talk I gave at a Duke conference about tests of consciousness discusses how coherent LLM text doesn't necessarily come from any clear unit entity. cs.cmu.edu/~conitzer/LLMcons…

(1/3) In this new paper led by Jiayuan Liu, we show that interleaving tokens from multiple LLMs can be (somewhat) robust, even when a *majority* of the LLMs are corrupted (unlike if they all vote over tokens)! arxiv.org/abs/2604.17139

After about a year of work, I defended my MSCS thesis at CMU: Title: The Structure of Deception: How LLM Agents Lie, Break Promises, and Exploit Trust in Multi-Agent Settings Core claim: LLM deception in multi-agent settings isn't one phenomenon. It's a family of structurally distinct failure modes, each shaped by different features of the interaction. Some look like premeditated false commitments. Others look like strategic silence that message-level classifiers can't see at all. Aggregate lying rates hide this, and current monitoring approaches each fail against different parts of it. I would like to deeply thank to my advisors @conitzer and @ZhijingJin, @AdtRaghunathan for being part of the committee, and everyone in the @JinesisLab for all their time and effort shaping this work. Recording: youtu.be/Z3Q9AkriPxg @MPI_IS @ELLISforEurope @UofTCompSci @VectorInst @TorontoSRI @CIFAR_News @JinesisLab @EuroSafeAI @ELLISInst_Tue @CarnegieMellon @SCSatCMU #AIAgents #AISafety #MultiAgentAI

(4/4) Emin Berker is presenting (on Saturday at this time) "Designing Rules to Pick a Rule: Aggregation by Consistency" (an unusual approach to social choice where we let the choice of rule depend on the input (!)) arxiv.org/abs/2508.17177

(3/4) Ioannis Anagnostides (maybe with help from Emanuel Tewolde) is presenting (right now!) "Convergence of Regret Matching in Potential Games and Constrained Optimization" (studying the properties of regret matching beyond zero-sum games) arxiv.org/abs/2510.17067

(2/4) Cyrus Cousins is presenting (right now!) "Towards Cognitively-Faithful Decision-Making Models to Improve AI Alignment" (arguing for the importance of ensuring that models learned from people's decisions align with their cognitive processes) arxiv.org/abs/2509.04445

(1/4) Three papers with authors from our Foundations of Cooperative AI Lab at ICLR!

🏛️ Affiliations: @CarnegieMellon @SCSatCMU @FOCAL_lab @JinesisLab @EuroSafeAI @UofTCompSci @VectorInst @TorontoSRI @CIFAR_News @MPI_IS @ELLISforEurope @ELLISInst_Tue @ETH_en @ETH_AI_Center 🏷️ #GameTheory #AISafety #MultiAgent #AIGovernance #AIPolicy