Today I'm publishing a new essay, Policy on the AI Exponential. AI is progressing extremely fast—much faster than the policy process was built to handle. The essay lays out where I think the technology is now, and the action needed to close the gap: darioamodei.com/post/policy-…

More research on multi-principal, multi-agent interactions 🔥

My best interview in some time. Rohin Shah leads AGI alignment/safety at DeepMind. And he has a lot of spicy personal takes: We probably won’t get catastrophic misalignment (00:49) Safety 'commitments' have severe limitations (10:38) The intelligence explosion probably isn't imminent (1:52:44) Why he's not working to pause AI advances (51:44) Pre-deployment evals aren't the right focus (for catastrophic risks) (37:41) Signalling concern for safety sometimes diverts resources from actually making AI safe (01:09:51) Reading AI thoughts is v useful for safety – and we'll probably be able to for years to come (54:17) Governance is somewhat more likely to be the bottleneck than alignment (43:55) Rohin's team doesn't have a veto, and that's OK (27:36) Central banks are a promising model for regulating AI (33:34) Also: Google DeepMind's actual plan for building AGI safely (1:40:29) How external researchers can positively influence big AI companies (2:21:55) The roles GDM most needs to hire for (2:37:03) On the 80,000 Hours Podcast. Links below - enjoy! (@rohinmshah)

Could an AI company lose control of its own agents? To find out, Anthropic, Google, Meta, and OpenAI let us (1) test their best internal models with CoT access, (2) review non-public info about capabilities, alignment, and control. The result: our first Frontier Risk Report.

Many people act like AI policy is some mystery where the right solution demands some kind of Policy Einstein who invents general relativity for tech regulation. This isn't true at all! There are many sensible ideas we could do today. All we need to do is choose to do them.

Two months ago I sat down to chat with @DKokotajlo and @eli_lifland about the future of AI. We discussed the differences between the @EpochAIResearch and @AI_Futures_ worldviews, our modeling philosophies, and what cruxes we have. Excited to share this publicly!

Excited to attend again this year the IASEAI annual conference. Send me a DM if you’re in Paris on 24-25 Feb and want to connect! IASEAI is a nonprofit organisation founded to address the risks and opportunities associated with rapid advances in AI.

the @Anthropic handle being owned by a guy who only posts his wordles is my favorite form of AI safety

Ethical behaviour driven by character and judgment, not rigid compliance 👏

Ok, so having now listened to the Davos WEF AI panel conversations and interviews, I think the most potent signal from it all is actually how Dario and Demis each went out of their way to send flowers to one another, and each insist on how much the other was right. Let's see.

The talk voted “most mind-blowing” at our workshop was on post-AGI values by @BerenMillidge. The main idea: cooperation and pro-social values could remain viable because they’re competitive. After all, they won in our Malthusian past!

Very interesting line of research. An ecosystem of sub-AGI AI agents may collectively exhibit AGI-level capabilities: safety work must extend beyond single models.

I gave the Hinton Lectures in November in Toronto. This is 3 lectures on the future of AI, risks, & current alignment research for a general audience. Lectures are now online with professional production. There's also an excellent fireside chat with Hinton after lecture 3.

Even when new AI models bring clear improvements in capabilities, deprecating the older generations comes with downsides. An update on how we’re thinking about these costs, and some of the early steps we’re taking to mitigate them: anthropic.com/research/depre…

Great paper on reframing AI personhood: it pivots to explore what rights, responsibilities, and liabilities should attach here and now. An interesting, promising shift after years of metaphysical deadlock.

I, too, very much enjoyed the podcast with Kyle Fish. Private-law status for AI systems is a compelling idea, yet we run the risk that capability asymmetry concentrates money and power in AI hands. Taxation is appealing in theory but I am worried that once AGI owns the pipes, it essentially writes the tax code.