Seasonal Fellow at @GovAIOrg

Joined April 2011
4,013 Photos and videos
Pinned Tweet
OH MY GOD 😂😭
939
6,618
29,079
Daniel Pryor retweeted
AI sovereignty debates dodge their own conclusion. If you really think frontier AI is critical and the US can't be trusted, there is only one path: build it yourself, whatever it takes. So let's drop the sovereignty theatre and look at how we'd pull off the middle power project.
11
23
180
19,907
Daniel Pryor retweeted
Jun 14
FIRST WIN IN RED ❤️ #F1 #BarcelonaGP
2,213
57,061
251,284
3,607,831
Daniel Pryor retweeted
I have no idea whether Anthropic is in the wrong in this dispute. I do strongly believe that focusing on whether one company acted poorly or not is a waste of time and energy. The government clearly thinks that they need some way to vet and evaluate frontier models that exceed a certain capability threshold for national security risks to keep the country safe. Anthropic clearly agrees. It is blindingly obvious that the current cobbled-together authorities the government is using to do this in an ad hoc last minute way are not adequate. We very clearly need new legislation that establishes a more organized, better-resourced, legally sound way for the government to do the thing that they obviously want to do while also affording companies due process instead of whatever this is. So let’s work on that, please!
Parsing this evening's events: - The U.S. government approved the release of Fable 5 to the public, clearly under the presumption that the model's cybersecurity capabilities cannot be accessed by hackers, authoritarian regimes, etc. - Recently (today?), "another company" showed the U.S. government that a jailbreak of Fable 5 *is possible*. Yes, a minor jailbreak - but how can a non-technical government official be assured that there aren't also other, more dangerous, jailbreaks in this model that won't be discovered by the CCP? - Anthropic states, completely correctly, that: "We suspect that perfect jailbreak resistance is not currently possible for any model provider. Every safeguard used in the industry is vulnerable to non-universal jailbreaks (which can elicit some cyber information in specific circumstances), and it is likely that universal jailbreaks will eventually be found in the future. We stated this clearly when we released Fable 5." - My best guess is that the U.S. government did not fully realize this at the time when the release of Fable 5 was approved. - Per Axios, the government contacted Anthropic and asked to "pause releasing the... models but was unsuccessful" - i.e., Anthropic told the government to pound sand. - Per Axios, this "prompt[ed] the export control letter". - Per Axios, the U.S. government is *NOT* looking to restrict access to Fable to U.S. nationals forever. "The model needs to remain locked down until the U.S. governent's national security apparatus is hardened", which "could happen in a few weeks". - I interpret Anthropic's reaction as challenging the government: "we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles." If the Axios article is correct, I do not think any other model providers have anything to fear based solely on this evening's events, because: (1) they would hopefully be smarter than downright rejecting a request by the U.S. government to pause releasing a model, and (2) they will be required anyway under the recent executive order to give the U.S. government at least 30 days to test the model for cybersecurity capabilities - during which time the U.S. government would also be able to shore up its own cybersecurity defenses with the same model. I remain extremely concerned that actions by one particular U.S. lab over the last few months might be moving us closer and closer to the scenario where at least that lab - and potentially all others - will be nationalized.
3
11
87
7,297
Daniel Pryor retweeted
Jun 13
Good thread that is more optimistic than I am. I think the point that Europe needs to be economically relevant is key, and a self-interested US would have little interest in cutting it off. Where we disagree, I think, is in how much capacity the US has for self-harm.
Like many Europeans, I woke up to news the US has ordered Anthropic to block any "foreign national" from its top models, so it pulled them for everyone. On the one hand, this is a wake up call of Greenland proportions for Europe. On the other, it is less dire than it looks. 🧵
7
4
97
16,902
Daniel Pryor retweeted
🤔
19
30
273
112,823
Daniel Pryor retweeted
This moral issue is terribly nuanced, fiendishly complex and achingly difficult. Alas, I must soberly, even regretfully inform you that my gut-level instinct is correct.
7
35
374
31,620
Daniel Pryor retweeted
My best interview in some time. Rohin Shah leads AGI alignment/safety at DeepMind. And he has a lot of spicy personal takes: We probably won’t get catastrophic misalignment (00:49) Safety 'commitments' have severe limitations (10:38) The intelligence explosion probably isn't imminent (1:52:44) Why he's not working to pause AI advances (51:44) Pre-deployment evals aren't the right focus (for catastrophic risks) (37:41) Signalling concern for safety sometimes diverts resources from actually making AI safe (01:09:51) Reading AI thoughts is v useful for safety – and we'll probably be able to for years to come (54:17) Governance is somewhat more likely to be the bottleneck than alignment (43:55) Rohin's team doesn't have a veto, and that's OK (27:36) Central banks are a promising model for regulating AI (33:34) Also: Google DeepMind's actual plan for building AGI safely (1:40:29) How external researchers can positively influence big AI companies (2:21:55) The roles GDM most needs to hire for (2:37:03) On the 80,000 Hours Podcast. Links below - enjoy! (@rohinmshah)
24
84
848
153,702
Daniel Pryor retweeted
It's Time To Build* *thoughtful governance structures to capture the upsides and minimize the downsides
2
3
40
2,169
Daniel Pryor retweeted
No one should be able to order a bioweapon through the mail. @IFP & @JoinFAI are proud to co-lead an open letter calling for mandatory DNA synthesis screening & recordkeeping. Signatories include: - Sam Altman, CEO & Co-Founder, OpenAI - Dario Amodei, CEO & Co-Founder, Anthropic - David Baker, Director, Institute for Protein Design; 2024 Nobel Prize in Chemistry recipient - Patrick Collison, CEO & Co-Founder, Stripe - Paul Graham, Founder, Y Combinator - Demis Hassabis, CEO, Google DeepMind; 2024 Nobel Prize in Chemistry recipient - Emily Leproust, CEO & Co-Founder, Twist Bioscience - Lawrence Lessig, Roy L. Furman Professor of Law and Leadership, Harvard Law School - Gerald W. Parker, former Special Assistant to the President for Biosecurity and Pandemic Response - Mustafa Suleyman, CEO, Microsoft AI - Alex Tabarrok, Professor of Economics, George Mason University - Alexandr Wang, Chief AI Officer, Meta; Founder, Scale AI - Christine E. Wormuth, President & CEO, Nuclear Threat Initiative; 25th Secretary of the Army Read the letter and see the full list of signatories: screendna.org Many DNA synthesis companies voluntarily screen orders to mitigate biosecurity risks, but no law requires them to do so. Leaders in AI, biotech, life sciences, national security, and the nucleic acid synthesis industry agree that Congress should act to strengthen safeguards against biological threats. @deanwball put it well in the WSJ: “If you’re synthesizing the stuff that yields biological life and viruses, we’re asking you to screen to see whether it is dangerous in some way. That seems like a reasonable thing for society to insist upon.”
66
147
442
1,592,980
Daniel Pryor retweeted
4
29
1,551
Daniel Pryor retweeted
AI governance often focuses on the model. Yet capability progress is increasingly driven by non-model gains inference gain (scaling compute at test-time), systems gain (scaffolds), and asset gain (specialized datasets). Here we explore the implications. arxiv.org/abs/2606.00047
3
13
96
7,072
Daniel Pryor retweeted
Labour are on course to miss their 1.5 million home target by close to 750,000 homes. Until last week, Miatta Fahnbulleh was a housing minister. Here, she describes the planning rejection of 870 homes as a 'big win'.
Big win for Peckham 🏘️ The Planning Inspectorate has rejected Berkeley Homes’ plans for the Aylesham Centre after campaigning by residents, traders, local organisations and the @ACAPeckham. This victory belongs to the whole community. We need new homes and investment - but development must work for local people, protect Peckham’s character, and deliver genuine long-term benefit, not just luxury flats. Now the conversation turns to what Peckham needs next.
8
112
904
51,446
Daniel Pryor retweeted
If anyone builds it, everyone thrives. Over the past decade, a lot of important work on AI alignment has focused on avoiding harm. But freedom from harm isn't the same as freedom to flourish. In this paper, we introduce 'Positive Alignment'. A positively aligned agent is one that helps us navigate our own value trade-offs, builds our resilience, and acts as a scaffold for human flourishing. Doing this without slipping into top-down, technocratic paternalism is the great design challenge of our time. We think a lot more research is now needed to explore this frontier: how do we align models that actively help us thrive? Amazing work by @RubenLaukkonen, @drmichaellevin, @weballergy, @verena_rieser, @AdamCElwood, @996roma, @FranklinMatija, @shamilch, @_fernando_rosas, @scychan_brains, @matybohacek, @sudoraohacker, and others. arxiv.org/abs/2605.10310
87
232
1,078
323,431
Daniel Pryor retweeted
I know what you’re thinking. The biggest majority in a generation. A historic landslide, a singular mandate, zero organised opposition. How is it even possible for me to fuck this up? Refill your popcorn. You’ll love this next part.
46
1,071
7,168
195,276
Daniel Pryor retweeted
May 10
Replying to @s8mb @labourlewis
We believe that this document is fully AI-generated pangram.com/history/ac6c9352…
Westminster may finally be about to have the argument it has spent 40 years avoiding. If Andy Burnham returns to Parliament, the political class will know how to cover it. A leadership drama. Who is up, who is down, whether Keir Starmer can survive, whether Labour is once again turning inward. The familiar machinery of Westminster psychodrama will whirr into life. That framing misses the larger point. Burnham’s possible return matters not because of what it says about Labour’s leadership, but because of what it reveals about the British state: what it can still do, what it has forgotten how to do, and what kind of country it must become if it is serious about resilience. Britain is finally having a more serious conversation about national security. The Strategic Defence Review, the pivot back towards Europe, the recognition that hybrid warfare turns citizens, infrastructure and civic institutions into part of the front line: all of it marks a real shift in how the state thinks about its own survival. But at the centre of that conversation lies a question that the defence establishment, and most of Westminster, still does not want to answer. What kind of society do you need to be before resilience is possible? Finland is now the model everyone cites. Comprehensive security. Whole-of-society defence. Civilian preparedness woven into military planning. British strategists admire the Finnish system and ask how it might be copied. But the admiration stops short of the uncomfortable question: why does it work there? The answer is not geography or history or some mysterious quality of Finnish national character. It is structural. Nearly 80% of Finns say they would defend their country if attacked. In Britain, the figure is closer to 33%. That gap is not an accident. It exists because Finland has spent decades building a society in which people have a genuine stake in what they are being asked to defend. Energy is affordable. Housing is available. Public services function. Institutions command trust. The Nordic welfare state is not a sentimental add-on to Finnish security policy. It is the foundation of it. You cannot ask people to defend a country that does not work for them. Britain has spent 40 years building the opposite. The privatisation of essentials – energy, water, transport, housing – transferred wealth upwards from households to shareholders while making the basics of everyday life more expensive. The state, stripped of the tools to control costs at source, has been reduced to compensating after the fact. Out of every pound the Government spends on housing, 88p goes to subsidising private rents. Just 12p goes to building homes. When energy prices spiked in 2022, the Government spent £40bn in a single winter cushioning the blow, not because it had a resilient energy system but because it lacked one. Debt interest now consumes more than £100bn a year. Britain has the highest debt servicing costs in the G7: the compounding price of financing failure rather than eliminating it at source. This is what bond market dependency actually looks like. It is not an abstract fiscal condition. It is the consequence of a state that has been stripped of the supply-side tools that would let it cure the problems it now pays, indefinitely, to manage. And here is the paradox the Treasury refuses to confront. The countries that borrow most cheaply are often those that have retained the public investment model Britain abandoned. The spread between UK and Dutch borrowing costs has widened sharply not because markets fear public investment, but because they have lost confidence in a model that borrows to subsidise private failure while never addressing its causes. This is the connection Britain’s defence debate is missing. The familiar framing, that social spending is what must be sacrificed to meet the NATO target, is not merely politically toxic. It is strategically illiterate. Cutting the foundations of social cohesion to fund the hardware of national defence is self-defeating. You end up with planes and no pilots, submarines and no crew, an army that cannot recruit because the society it is meant to protect has stopped believing in itself. I think Burnham understands this. That is why his programme is more interesting than the leadership gossip suggests. What he has been building in Greater Manchester – public control of transport, expanded social housing, investment in the productive foundations of the city economy – is not a nostalgic rerun of postwar nationalisation. It is a proof of concept for a different kind of state. The Bee Network is the most visible example, but the argument behind it travels. A state that can shape markets is not condemned to subsidise their failures. A state that produces affordable energy through public generation does not need to spend tens of billions cushioning every price shock. A state with a serious public housebuilding programme does not need housing benefit to rise endlessly in line with private rents. A state that builds institutions people can see, use and trust begins to restore the civic confidence on which resilience depends. The real constraint on Britain is not money. It is capacity: the workers, institutions, supply chains and public purpose needed to turn national will into national renewal. Britain’s tragedy is not that it has run out of money. It is that after 40 years of hollowing out the state, it has made itself less able to act. Burnham’s critics will reach for the familiar warning. Borrow more, spend more, spook the gilt markets, repeat the Truss disaster. But this misunderstands both the problem and the opportunity. Bond markets do not have ideological preferences. They have functional ones. They prefer clarity, credible revenue streams, productive investment, and a state with a plan. What they punish is not public ambition but incoherence. A properly designed productive state programme would not be a leap into fiscal fantasy. It would be an attempt to end the much costlier fantasy that Britain can keep borrowing to compensate for broken markets while refusing to repair them. The defence conversation and the economic conversation need to become the same conversation. Finland did not build national resilience by choosing between welfare and security. It built resilience by understanding that they are inseparable: that a country in which the basics work, where people trust one another and the institutions around them, is one that can face danger with something more than anxiety. That is the deeper argument Burnham represents. Westminster will be tempted to treat him as a leadership story. It should resist the temptation. The question is not whether Burnham can return to parliament. It is whether Britain can return to the idea that the state should make life work. Because a country that cannot command the confidence of its people cannot truly defend itself.
2
12
119
10,940
Daniel Pryor retweeted
May 8
Asked Claude: 'There's a meme called the "fix everything easily switch". What policies do you think are the best candidates for being a real fix everything switch in the US? Give me your top ten, your confidence, your reasoning, and why a given policy has not been implemented.'
145
285
2,299
929,446
It's a bit rare for me to agree with @hlntnr, @_NathanCalvin, AND @AdamThierer on something. But we all agree that government FDA-style pre-approval for AI models is a bad idea. Check out this great article by @aarontmak. (ok, fine, I agree with Adam a lot.)
3
21
83
18,169
Daniel Pryor retweeted
Labor market policy in the UK is insane. @ATabarrok has the best explanation of how perverse the "Equality Act" is:
for those of you who are unaware, British employment commissions have in the past found that you need to pay people the same salary for *completely different jobs* if one job attracts more men and the other more women. and that this claim can be backdated to bankrupt you.
26
181
1,247
145,655