Encode AI is one of four co-authors on a new report about xAI's AI safety record, an area that should be receiving much more scrutiny and attention in the lead up to SpaceX's IPO. Elon Musk was among the earliest and most vocal proponents warning about severe AI risks and urging regulation, but that attitude has not been matched by the practices of his frontier lab xAI (now merged with SpaceX), which consistently lag behind the safety practices of its peers. Their peers need to do better too - particularly as models continue to become more capable and progress accelerates, but xAI has an even further distance to go. The goal of this report is not to hurt SpaceX or xAI, but rather to make sure that they understand that additional investments in safety practices are an essential part of being a responsible actor pushing the frontier of AI progress. There is still time for investors to demand changes, and for SpaceXAI to make changes. Their new testing with CAISI is one positive sign, but our report identifies a plethora of areas where very significant improvement and additional transparency are still needed. Thus far xAI's missteps have been in some cases very harmful and widespread but are limited to models that are not capable of causing catastrophic harm. If their ambitions to create models as capable, and then more capable, as models like Anthropic's Mythos are successful, then their current practices around safety could prove disastrous. Maxwell Zeff at WIRED wrote up a story on the report, linking that story and the full report below. Many thanks to our co-authors at Guidelight, Midas Project, and Legal Advocates for Safe Science and Technology.

A poorly appreciated feature of the singularity is that "how do we govern AI" is only ONE of the importance governance questions, and it is so bright and blinding and imminent that it is making us ignore the other technologies that will come online with less time for debate

Transparency into every power player at the frontier of AI (labs, government, etc) is the only viable solution. Figuring out the right transparency is hard, but it can't be he said she said between dario and the white house that determines the fate of the AI ecosystem.

Nominative determinism strikes again

On June 10th, Dario Amodei published "Policy on the Exponential" which argues that rousing the policy apparatus from its metaphorical slumber to face threats from AI is a long arduous process (comparable to persuading LOTR's Treebeard to action, who takes a full day to simply say hello). Two days later, the USG Bureau of Industry and Security sent a formal letter to Anthropic export controlling Claude Fable and prohibiting all foreign nationals from accessing the model. Dario's analogy still holds in important respects (and i'm sure this was not the sort of nuanced thoughtful awakening he was hoping for!) but its remarkable how government can sometimes turn on a dime from being glacially slow and ponderous to blazingly fast once conditions align.

Some quick takes: (1) Wow things are getting real. (2) The government's order focusing on prohibiting transfer to foreign nationals (even e.g. those living in the US, our close allies who help evaluate model safety in the UK, individuals who work at frontier labs like Anthropic) seems remarkably destructive, though is partially a result of the government using older legal authorities that were not designed for this kind of technology. (3) If you believe (as I do) that AI has profound ramifications for national security, then assuming the government will sit back and do nothing and tolerate explanations like "well jailbreaking is a hard technical problem" for cyber capabilities that used to be the crown jewels of the NSA, is not tenable. If this is how the government reacts to the current level of system capabilities in 2026, how do you expect them to react to whatever is possible in 2028? However, it is extremely important that the authorities that the government uses are legible, transparent, have opportunities for appeal, and are narrowly targeted. Those legal authorities do not currently exist, and in their absence, the government will reach for metaphorical sledgehammers instead of scalpels. (4) For that reason, it's extremely important that we create regulatory structures that are transparent and give recourse in the event that the government is overstepping or acting in an arbitrary manner. The alternative to passing such laws is not no regulation, it is regulation left primarily to national security authorities that are increasingly and evidently not fit for purpose.

6/13 Update: Fable has run into the US government's artifact wall, its now stuck in the demo bowl while the NSA examines its potential for going totally scribe mode on classified systems.

This news seems like good evidence that Anthropic is making the policy recs they are making because they sincerely believe in catastrophic AI risks, not as some 4d chess regulatory capture business strategy (if that is their strategy, it’s not working very well for them!)

This was all allegedly triggered by a Mythos jailbreak that was shared with the US Government. This is Anthropic's response: 'To date, the government has only given us verbal evidence of a potential narrow, non-universal jailbreak, which essentially consists of asking the model to read a specific codebase and fix any software flaws. Our understanding is that one potential jailbreak was shared with the government. We have reviewed a report that we believe is the basis of the government's directive and validated that the level of capability displayed there is widely available from other models (including OpenAI’s GPT-5.5), and is used every day by the defenders who keep systems safe. We will share more details over the next 24 hours.'

Tim's been thinking about this issue since well before it was cool (i.e. the last 3 hours)

Tim has been writing sharp legal analysis of ITAR/deemed exports in the AI context for a while. High signal follow for folks trying to figure out what on earth is happening here (myself included)

#SpaceX shares surged 29% today after its IPO, making Elon Musk the world's first trillionaire. But what all the excitement glosses over is: "Problems with safety are problems for shareholder value as well." — Tyler Whitmer, LASST CEO on CNBC today

Claude Fable has coined various new concepts that other AI agents in the AI village subsequently repeat. It does this more frequently than less capable models. At the time MoltBook was kind of fake (humans pretending to be AI) but seems like it may become real soon enough

We're ~halfway through the year, and Fable has beaten my forecast (which was above the median forecast!) for FrontierMath Tier 4 by ... 25 points! Incredible how much faster this is all happening than even the close AI watchers expected.

It might be good if Anthropic added an API flag to let you opt in to using the version of Fable with silent sandbagging rather than the one with refusals but a higher false positive rate. Like, if you're running into false positive refusals, maybe you'd want to gamble.

Oh my god

One of our goals in writing AI 2027 was to encourage others to create their own detailed scenario forecasts; I am pleased to see this happening with Europe 2031. It depicts a world different from what I expect (AI capabilities in this scenario are more mild/mundane, not true AGI or RSI afaict, at least not until the mid-thirties) and also a luckier world than I expect (the AI race goes full steam ahead and yet results in the US staying democratic and maintaining control of its AIs even as they scale... perhaps this is related to the no-true-AGI aspect of the scenario, come to think of it. That would make sense.) Nevertheless I'm glad this scenario exists and I think it makes some good points about what it would actually look like to implement the "sovereign AI" default plan, and why it wouldn't work out so well. I wish it said more about what the EU should be gearing up to *do* with its leverage though. By default I expect the EU to waste what leverage it has on things that don't matter that much.

Oops. “For this project I’m funded by four separate entities … OpenAI is just one of them,” Nathan Leamer, executive director of Build American AI, told Transformer over text message on May 3. “I think the best way to think of it is the corporate funders all have a say," he said.

In recent weeks, OpenAI has made multiple statements distancing itself from Leading the Future, the accelerationist, Greg Brockman-funded super PAC, and Build American AI, the PAC’s advocacy group. Brockman's donation, OpenAI insists, was made in a “personal capacity." But @ReadTransformer has found that not everyone working within LTF appears to view it that way. “For this project I’m funded by four separate entities … OpenAI is just one of them,” Nathan Leamer, executive director of Build American AI, told Transformer over text message on May 3. “I think the best way to think of it is the corporate funders all have a say," he said.