Competition is the only protection anyone has ever had against concentrated power.
Anthropic's valuation only works if open weights get banned by law. Their safety case and their business case are the same case.
We have four options:
-Anthropic ruling the world as a monopolist (or a duopolist) is catastrophic. We are having the first inklings of that with Fable release.
- The US government controlling the world by controlling the model releases (on the table right now): even worse. They will weaponize it and blackmail the hell out of everyone.
- The world government solution is cute, but not going to happen.
- So competition, with the open weights only a few months behind, is the only thing protecting us in the rest of the world. Models will need some regulation, like cars or planes do.
As a European, the first two options scare me more than the technology does. OpenAI has not been making these ridiculous Effective Altruist noises, they have quietly released models which are on the whole superior to Anthropic's in my view and there have been no risk to the world. I think Anthropic protests too much.