GPUs aren’t getting better, they’re just getting bigger. In the past four years, compute density (TFLOPS/mm^2) has only improved by ~15%.
Next-gen GPUs (NVIDIA B200, AMD MI300X, Intel Gaudi 3, AWS Trainium2, etc.) are now counting two chips as one card to “double” their performance.
With Moore’s law slowing, the only way to improve performance is specialization.
The economics of scale are changing
Today, AI models cost $1B to train and will be used for $10B in inference. At this scale, a 1% improvement would justify a $50-100M custom chip project.
ASICs are 10-100x faster than GPUs. When bitcoin miners hit the market in 2014, it became cheaper to throw out GPUs than to use them to mine bitcoin.
With billions of dollars on the line, the same is happening for AI.