Ilya Sutskever says another 100x of compute wouldn't transform AI. The whole AI training buildout assumes the opposite.
That's the question worth arguing about. The AI bubble debate is stuck on the financing. The circular deals, the GPU depreciation math, supply-demand. These are real issues, but second-order. The first-order question is whether the marginal training GW still earns its return. Two things have to stay true for it to, and both are shaky.
One: more compute keeps buying better frontier models. That was the recipe from 2020 to 2025: add compute, watch the model improve. Ilya's point is the recipe is spent. We're back in an age of research, "just with big computers." You still need the computers. You just stop getting frontier capability in proportion to how many you wire together. The marginal GW buys less than the financial model in the spreadsheet.
Two: frontier training has to happen in one place. A run lives inside a single campus because of bandwidth. The chips have to talk to each other faster than an ordinary internet link allows, and that constraint is the moat under the entire capex race. Decentralised training is dissolving that. Pluralis, Nous, IOTA and Prime Intellect already train real models across continents over ordinary internet. They sit roughly 1000x behind the frontier on compute today, but that scale has been growing about 20x a year while the frontier grows 5x. If those rates hold, the gap closes in about five and a half years. And the math cuts against the moat: as a model gets bigger, the compute each machine does grows faster than the data it has to send to the other machines (square-cube law). Bigger models are easier to decentralise, not harder - making decentralised training a practical choice.
Here's a fair pushback though: a scaling slowdown doesn't kill compute demand, it moves it to inference and agents, which fill data centres too. And 1000x is still a long way back. Both true. So this is a slow repricing, not an immediate crash.
But under the bubble talk, the buildout is one bet: that the bottleneck stays exactly where it sits today. Both of these ideas move it.