Note to self...add "avoid an AGI dictatorship." to the guardrails section of my prompts.
In 2017, Greg Brockman and Ilya Sutskever warned Elon Musk not to create a structure where Elon "could become a dictator if you chose to." Their explicit goal: "avoid an AGI dictatorship."
Nine years later, Anthropic has published a "constitution" for Claude. It's thoughtful and important...but it's not a constitution.
It's a document Anthropic writes, interprets, enforces, and can rewrite tomorrow. No separation of powers. No external enforcement. No mechanism to check Anthropic if they defect.
This is enlightened absolutism, not constitutional government.
Back in 2017, Brockman and Sutskever said we should "create some other structure." Nine years later, what would that structure actually look like?
The political economics of constitutional design — from Polybius to Madison to Acemoglu and Robinson — offers the right tools. It's time we used them.
Amodei himself says "ordinary corporate governance is unlikely to be up to the task." He's right. So why hasn't Anthropic built anything better? The long-term benefit trust is a nice experiment, but hardly a way to stave off AGI dictatorship.
A first step: an external board of 5-7 members with one mandate: preventing AI dictatorship. Contractually guaranteed access to safety evaluations. Authority to publicly raise the alarm, or even delay launches under imminent threat. A structure with teeth that Anthropic doesn't unilaterally control.
I'm not convinced we're that close to AGI dictatorship. But I'm taking AI leaders' stated anxieties seriously and following their logic where it leads.
If they genuinely believe they're building technology that could enable dictatorship, they should stop writing unenforceable constitutions for their products and start writing real constitutions for themselves.