Excited to share our work on disentangled/abstract representations, to appear at #ICLR2025 (@iclr_conf)! We mathematically prove and experimentally demonstrate that multi-task learning leads to disentangled representations, and propose a unifying mechanism for generalization in brains and machines: parallel processing (🧵 paper below) Our work connects to the Platonic representation hypothesis, suggests why alignment across models/organisms can occur, and shows why transformers excel at constructing world models 🤖🚀

Today we launched @CallosumAI. We are building the infrastructure where heterogeneous chips & intelligence co-evolve to solve the world's hardest problems. Today we present our first results. Across four large problem spaces, we break SOTA and deliver orders-of-magnitude improvements in capabilities, cost and speed: 12× cheaper deep context. New web SOTA with open-source, 3x cheaper and faster. 2.4× cache speedups. 1,767× faster tool calling. This is the worst our infrastructure will ever be. We do it by co-evolving heterogeneous chips and multi-agent intelligence - workflows aware of their hardware, models aware of their task graph, kernels aware of their output constraints. An Intelligent System. callosum.com/blog/welcome-he…

Two of the most cracked ppl I’ve ever known building something that can truly make LLMs personalized. Congrats @ABhargava2000 and @witkowski_cam, excited to see where this goes!

If you're at #ICLR2025, and interested in how we can guarantee true out-of-distribution generalization in neural networks (extrapolation), Aman Bhargava (@ABhargava2000) and I will be presenting our work tomorrow Saturday the 26th at 3:00-5:30pm, at Hall 3 (poster number #69) We will be happy to see you there! short presentation slides: iclr.cc/virtual/2025/poster/…

If AI isn’t truly open, it will fail us. We can’t close in a black box our greatest invention yet just so that a few can freely monetize. AI needs its Linux moment, and so we started working towards it. This can only succeed if we all work together! #oumi #opensource #collaboration @rsalakhu @svlevine @larry_heck @karpathy @atalwalkar @prfsanjeevarora @tsvetshop @hhexiy @sainingxie @larry_heck @AnimaAnandkumar @JunjieHu12 @georgiagkioxari @profjoeyg @pliang279 @danqi_chen @ChrisGPotts @BillMacCartney @vinodv @tur_gokhan @dilekhakkanitur @j_foerst @gingsmith23 @kahinish @jamesjoaquin @gan3sh @ethanjb @kirbywinfield @egonzdp

Disentangled representations are widely observed, from pretrained LLMs to brains. We provide theoretical guarantees for their emergence, and experimentally confirm all of them! 🚀 For more come find @ABhargava2000 and I @unireps @NeuroAI_NeurIPS and @neur_reps workshops today!