Leaked Internal Google Document Claims Open Source (Generative) AI Will Outcompete Google and OpenAI.
1) "We have no moat, no secret sauce and neither does OpenAI.
2) People will not pay for a restricted model when free, unrestricted alternatives are comparable in quality.
3) Giant models are slowing us down as they can't be iterated upon quickly.
4) Open-Source companies are doing things in weeks with $100 and 13B params that we struggle with at $10M and 540B.
5) Training giant models from scratch not only throws away the pretraining, but also any iterative improvements that have been made on top.
6) Large models aren’t more capable in the long run.
7) Retraining models from scratch is the hard path.
8)Data quality scales better than data size.
9)Directly Competing with Open Source Is a Losing Proposition.
10) We need them more than they need us.
11) Individuals are not constrained by licenses to the same degree as corporations.
12) In the end, OpenAI doesn’t matter. They are making the same mistakes we are in their posture relative to open source, and their ability to maintain an edge is necessarily in question. Open-source alternatives can and will eventually eclipse OpenAI unless they change their stance. In this respect, at least, we can make the first move.