Software dev has already changed a lot since the beginning of the year
And seems like both Anthropic and OpenAI will have much better models by the end of the year.
Software has always been very inefficient to make. And now it will be not perfectly efficient but orders of magnitude more so.
Is there a new kind of Efficient Market Hypothesis for the software industry?
Ie if you only have public information that everyone else also has, you probably shouldn’t use it to trade stocks or build a startup on. There’s little alpha there, and roaming apex predators with more GPUs than you.
I find enterprise interesting because it’s a non-public slog. Long procurement processes, no public docs, bespoke fractals of internal processes and jargon and messy human context that isn’t public or legible to labs or others yet.
Previously, context (eg a well written internal Google doc) was cheap relative to the cost of building software. Now it’s flipped, weirdly
If building software becomes more efficient, where are the “private context slogs” worth making?
Ie curating non-public context that (when combined with public agents) unlocks new value to businesses
Has anyone been thinking about the Efficient Software Hypothesis, and its implications?
On Spud:
'The way that our development process works is you have pre-training. So you produce a new base model, that then is the foundation that we build further improvements on top of. And that is always a huge effort across many people in the company. And that's where I've actually been spending most of my efforts over the past eighteen months has been really focused on our GPU infrastructure, on supporting the teams that do all of the training frameworks to scale up at these big runs.
....
So I think of Spud as a new base, as a new pre-train, and ... I'd say it's like we have maybe two years worth of research that is coming to fruition in this model. It's going to be very exciting, and I think that the way that the world will experience it is just improved capabilities.'