Agent memory is a simple and powerful way to do continual learning! With the new MemAlign method from Databricks Research, we can build better LLM judges from examples of human ratings, and they scale with more data. Now in Databricks and @MLflow. databricks.com/blog/memalign…

Reliable enterprise agents require system-level reasoning when retrieving across heterogeneous knowledge sources. Traditional RAG often fails to consistently follow instructions, schemas, and constraints end to end. That’s why we’re presenting Instructed Retriever, a new retrieval architecture that propagates complete system specifications through every stage of the search pipeline. The approach delivers: - 35–50% gains in retrieval recall on instruction-following benchmarks - 70% improvements in end-to-end answer quality over simplistic RAG, and ~15% over reranking-based approaches - Strong instruction adherence with small, efficient models suitable for real-world deployment Together, these results show how system-wide instruction awareness translates directly into more accurate and efficient enterprise agents. databricks.com/blog/instruct…

Existing IR/RAG benchmarks are unrealistic: they’re often derived from easily retrievable topics, rather than grounded in solving real user problems. 🧵Introducing 𝐅𝐫𝐞𝐬𝐡𝐒𝐭𝐚𝐜𝐤, a challenging RAG benchmark on niche, recent topics. Work done during intern @databricks 🧱

How can we use small LLMs to shift more AI workloads onto our laptops and phones? In our paper and open-source code, we pair on-device LLMs (@ollama) with frontier LLMs in the cloud (@openai, @together), to solve token-intensive workloads on your 💻 at 17.5% of the cloud cost while maintaining 97.9% of the accuracy. See Gru and the Minions in action below, 🔉on please (h/t @cartesia)!

New #FeaturedCertification: LoRA Learns Less and Forgets Less Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz et al. openreview.net/forum?id=aloE… #learns #regularization #rank

The exploding curse of "99% accurate fallacy" (or the need for six sigma yield) for embodied AI in real-world settings. This may seem obvious to roboticists but worth putting into numbers. A thread 🧵 - 1/n

How can you make LLM-as-judge reliable in specialized domains? Our applied AI team developed a simple but effective approach called Grading Notes that we've been using in Databricks Assistant. We think this can help anyone doing domain-specific AI! databricks.com/blog/enhancin…

DBRX is a 16x12B MoE which was trained for 12T tokens at a final context length of 32k. I and my team have been fine tuning it and I am sure that the community is going to love this one, it's got great vibes