Giving LLMs tool to construct their own context is magic.
I created a knowledge graph tool, where the LLM can query it. I mean, feels like context problem is solved.
The main issue with RAG enthusiast, is that you pay an upfront cost of maintaining the graph. But I mean, reasoning becomes childs play.
Like: "Ahh, LLM is updating a line of code, the graph shows ALL dependencies, both code, data, busines logic etc".
This is AGI infra.