Really nice article summarising diff nuances with agentic memory and context windows. Lost in the Middle was something which I found interesting as that is something which doesn't strikes vey intuitively.
Until I worked on scaling products for actual load seeing numbers while reading something on system design used to bore me and I would usually rush to sol part.
Now I am more interested in those numbers part as they make more sense.
Was figuring out ways for self hosting backups. While studying on this found about UDP hole punching for solving nat traversal.
If your end application doesn't require TCP specifically then with this p2p connectivity is so easy you don't need to rely on relays or ngrok services
While building KV cache simulator I was trying to optimise how we can reduce cache invalidations as we keep summarising the context in long session for new user prompts. Interesting to learn about how StreamingLLM and attention sinks are solving this.
Was experimenting with KV caching and ran into some surprisingly simple injection style attacks for instance fuzzy prefix matching.
Feels like we’re back to the 90s, new tools, same old type of bugs.😁
Just like learning about how web works made me excited about computer science initially nowadays it has become how LLMs/ models work. Mostly from scaling and infra POV. These problems are so much diff than scaling other applications.
These pace at things are being developed is🤯
How are folks who have graduated recently positioning themselves in industry as most of work that they are assigned with is getting done by ai agents.
Should people start deepening their skills in a particular domain, if yes then what are some domains which will be relevant.
IYKYK. Like @anandesh mentioned back at our convocation the future isn’t a place where you survive doing just one job.
The real edge would be in finding problems worth solving and selling your services around them. Dont expect conventional roles to exist in long run.
Feed is completely filled with sarvam/galgotia/AIsummit/ people building cool products and here am I doing my on call duties fixing infra issues, fun life 🙃