I am not a big fan of Large Language Models.
In fact, India should not adopt the approach of developing a LARGE language model. Instead, we should focus on SMALL language models—very specific and designed for special purposes, capable of answering complex queries and solving critical problems within a particular industry.
For example, we could create a law-specific language model or, even more precisely, a science language model, or perhaps a physics language model. If we cluster all these small language models together and, instead of building a massive knowledge graph, create a simple directory or index for these models, users could select the respective language model to get specific answers.
There will be some limitations, though.
Such models may not answer questions that are very broad or cut across multiple language models. However, if we use RAG to frame user queries in a way that breaks the query into smaller components, these components can retrieve answers from the relevant small language models. This approach provides a strategic advantage for India.
First of all, it doesn’t require very high computational bandwidth, so we can rely on indigenously built, low-cost memory and computing infrastructure. Second, there is less dependency on external data.
These models will be built using data generated in India, making them more capable of providing better answers in the Indian context. We should adopt a Digital Public Goods (DPG) approach to LLMs as well, focusing on solving real problems with resources developed indigenously.
We should work with a minimalistic approach and adhere to the principle of single responsibility. The goal should not be to create a tech monopoly but to solve specific problems in specific industries.
I believe we should avoid the trap of large language models. Even if we solve business problems using large language models, we would end up giving away more than 50% of the revenue to tech monopolies, whether they are LLM companies or hardware providers. India should carve its own path instead of falling into the trap of digital monopolies again.
@simpler_today
@aravindsrinivas @RajeevRC_X @GramPanchayatHQ @EasyGov @reliancejio @perplexity_ai @deepseek_ai
#SmallLLM #SimplerToday #AI4People
Re India training its foundation models debate: I feel like India fell into the same trap I did while running Perplexity. Thinking models are going to cost a shit ton of money to train. But India must show the world that it's capable of ISRO-like feet for AI. Elon Musk appreciated ISRO (not even Blue Origin) because he respects when people can get stuff done by not spending a lot. That's how he operates. I think that's possible for AI, given the recent achievements of DeepSeek. So, I hope India changes its stance from wanting to reuse models from open-source and instead trying to build muscle to train their models that are not just good for Indic languages but are globally competitive on all benchmarks. I'm not in a position to run a DeepSeek-like company for India, but I'm happy to help anyone obsessed enough to do it and open-source the models.