81 Photos and videos
My biggest project yet is now LIVE!
@hankgreen talks superintelligence, AI risk, and the many issues with AI today that present huge concerns as AI gets more powerful on SciShow.
Happy Halloween!
youtube.com/watch?v=90C3XVjU…
11
19
173
81,113
Samuel is an absolute powerhouse and it's been wildly impressive to see how quickly he's ASI-pilling the Canadian government!
19h
With just one member of staff in Canada, we've briefed over 100 politicians in just 9 months.
Since then, AI risk and superintelligence have been on the agenda in Parliament, and we've been testifying.
Here are some of the highlights:
5
13
944
One of the fundamental problems in AI is that Every. Single. Model. Is. Jailbreak-able.
All capabilities will be unleashed and publicly accessible.
It is grossly negligent to build something capable of cyberattacks and bioweapons design and release it to the public.
Jun 13
This allegedly was triggered by a successful jailbreak of Mythos by an unnamed group who reported it to the government. We will hopefully get details soon.
2
5
32
1,661
Banning superintelligence really doesn't seem that far fetched now, does it?
Still crazy to me that AI companies thought they could just build a doomsday machine in broad daylight, be open that it could be catastrophic, and nobody in power would catch on or do anything.
Jun 13
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.
The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.
Access to all other Claude models is not affected.
We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible.
Read our full statement: anthropic.com/news/fable-myt…
2
5
30
478
Max Winga retweeted
Jun 13
"The government will never do anything to hinder AI," they said.
Inaction yesterday does not imply inaction today. An export control directive came out of nowhere, and a ban on superintelligence could too. Inevitabilism is wrong.
Jun 13
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.
The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.
Access to all other Claude models is not affected.
We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible.
Read our full statement: anthropic.com/news/fable-myt…
13
40
441
21,869
As always, Anthropic's policies never disrupt their singular goal - to build superintelligence and take over the world before anyone else can.
"Regulation is slow and we're 6-18 months from RSI but guys I swear it's way too soon to regulate loss of control or superintelligence."
Jun 10
AI is advancing at a pace our policymaking institutions were never built for—and the gap between the two is becoming the central challenge of the technology. In his latest essay, our CEO Dario Amodei lays out how to close it.
We're launching three new initiatives to support the efforts he outlines.
2
7
30
8,036
As always, Anthropic's policies never disrupt their singular goal - to build superintelligence and take over the world before anyone else can.
"Regulation is slow and we're 6-18 months from RSI but guys I swear it's way too soon to regulate loss of control or superintelligence."
Jun 10
AI is advancing at a pace our policymaking institutions were never built for—and the gap between the two is becoming the central challenge of the technology. In his latest essay, our CEO Dario Amodei lays out how to close it.
We're launching three new initiatives to support the efforts he outlines.
2
7
30
8,036
See why Anthropic's plan is fundamentally doomed:
x.com/i/status/2064987734679…
Jun 11
This does not address catastrophic risks, and fails all three checks for a plan to address catastrophic risks.
Development, not deployment, of powerful AI needs to be restricted at a global level if we are to survive ASI.
x.com/DarioAmodei/status/206…
2
213
Max Winga retweeted
Jun 10
Anthropic and OpenAI did not call for a pause. Read the wording: "good for the world to have the option", "possible" to slow down "when needed".
This is how they signal safety to one audience, acceleration to another. What is not vague is their unequivocal commitment to RSI.
2
20
49
3,694
Very interesting to see the former Chief Scientist of UK AISI quit and say the better plan for AI alignment is to not built ASI yet and that we should work hard to make that happen!
Jun 10
But I just published “Automated alignment is harder than you think” (arxiv.org/abs/2605.06390)! Automated alignment is not the best plan! A better plan is to not build ASI yet, and the world should try hard to realise that plan. Alas, the speed of progress calls for backups.
ALT Automated alignment involves a mixture of tasks which are easy and hard to supervise correctly, and we could easily get fooled by the later.
7
24
1,130
Max Winga retweeted
Jun 10
Great post from @testdrivenzen !
Most plans about developing superintelligent AI are extremely naive about geopolitics and how governments will react to something as threatening as upcoming ASI.
All roads lead to catastrophe unless ASI development is prevented globally.
Jun 10
Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways:
- It sparks war between nuclear powers
- It causes a misaligned ASI to kill everyone
- It establishes a permanent dystopian dictatorship
5
18
694
If you work at an AI company and think you're doing good for ASI x-risk, you should challenge your beliefs with this post.
Jun 10
Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways:
- It sparks war between nuclear powers
- It causes a misaligned ASI to kill everyone
- It establishes a permanent dystopian dictatorship
14
713
Great post @testdrivenzen!
Far too much writing about ASI geopolitics assumes uninformed, irrational, & neutered versions of state actors, and it's good to see analysis that doesn't.
If you work at an AI company, it is worth challenging your theory of change with this post.
1
1
11
405
Even independent evals orgs are completely beholden to AI companies – strongly incentivized to avoid anything that goes against company interests – to maintain their privileged access.
We need governments to create binding regulations, not voluntary commitments.
Jun 10
There is a lot of justified anger at Anthropic for sandbagging Fable 5 for AI development tasks. But an unanticipated side effect is that third-party evaluators can no longer credibly use the model for evaluations.
Case in point: we are in the middle of running *really hard* AI R&D evaluations. Fable 5 would be a perfect test candidate. But because of Anthropic's guardrails, we can't know if the model failed or if their classifiers blocked the capability.
By the way, this is not just true for AI R&D. Since Anthropic doesn't make it clear when they are sandbagging, this could seep into any number of technical tasks, and the evaluators wouldn't have any way to know. So they can't credibly claim to evaluate state-of-the-art accuracy using the model.
1
3
16
377
Max Winga retweeted
Jun 8
ControlAI's CEO @andreamiotti: Recursive self-improvement could leave us with AIs far smarter than humans that we don't control.
Governments must prevent this, and we're already seeing movement from policymakers in the US, UK and Canada.
With @KamaliMelbourne on @SkyNews:
3
9
35
770
Max Winga retweeted
Jun 7
this is true, except that loudly and publicly pausing to see if others follow suit seems like a reasonable thing to try in dire straits such as these?
obviously I get this is a costly thing to do, and burning your lead is bad if you genuinely believe you’re the safest actor, but that just makes the signal even stronger. I think unilaterally pausing would be much more commensurate with Anthropic’s stated beliefs than writing a blog post
( even if they *do* believe they’re the safest, they clearly don’t think they are actually on track to solve the problem, so slamming the brakes seems like a better bet than trying to ‘win’)
“If Anthropic wants to pause, they can just pause” is obviously true. “If Anthropic wants *everyone* to pause, they can just pause”, meanwhile, is false.
5
8
62
6,314