Recursive just came out of stealth, and the team has been cooking 🔥 Our first results: an automated AI research system that can improve AI across 3 very different settings across training and GPU kernel optimization. recursive.com/articles/first…

Detailed explanation of our first results by our amazing teammate @cong_ml !

Lots of people have been asking me what we're up to at @Recursive_SI. We still can't say much quite yet, but we thought we'd share a little early demo. It turns out our automated researchers are pretty effective at performance optimization!

Look ma, no hands! tldr: Recursive's automation loop improves AIs by itself 🚀🚀🚀 more in the 👇

AI is now doing our AI research. At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method. RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time. Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three. These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench. AI is code and AI can code. The code and ideas that lead to these results were not invented by our team but by the AI system itself. To do RSI safely, we need to investigate its inventions. That's best done transparently with the community. @Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas. Link below.

Very excited to share some of our first results already!

Excited to share these preliminary results on our internal autoresearch system @Recursive_SI, where we achieve SOTA on nanochat / nanogpt speedrun / kernel benchmarks using the same underlying system without task-specific adaptations. blog: recursive.com/articles/first…

First results from Recursive on AI that improves AI! 🚀 ✨ 📈 Our automated AI research system incorporates principles from open-ended and AI-generating algorithms. 🧠 💡🌱🧬 It conducts key parts of the science loop: proposing ideas, implementing them, testing them, and picking the next ideas based on the data. 🔬 🔭 🧪⚗️ The same general system produces state-of-the-art results on three different problems (two on training language models, one on speeding up AI via kernel optimization). • NanoChat Autoresearch: 1.3x faster to reach the same loss than the best solution produced by an entire community of humans agents over months, and 1.8x faster than the initial hand-optimized solution • NanoGPT Speedrun: 3% speedup of a very efficient solution produced by entire community of humans agents over 2 years • GPU Kernel Optimization: 18% reduction in gap to theoretically optimal score on Nvidia’s SOL-ExecBench These are early tests of our system. We’re very excited for what the future holds! Post: recursive.com/articles/first… Great work by the amazing team at Recursive!

Early results from Recursive 🚀🚀 SotA results from our open-ended knowledge discovery system: 1️⃣NanoChat 5min pre-training (0.9372 bpb -> 0.9109 bpb, 2.8% lower Bits-Per-Byte than long-standing community SoTA) 2️⃣NanoGPT SpeedRun (79.7s -> 77.5s, 2.8% faster than long-standing community SoTA) 3️⃣GPU kernel optimization (Overall 7.8% better than SoTA performance in SOL- ExecBench, hosted by NVIDIA) To achieve that, our system automatically finds and combines innovations together to create better solutions than current ones carefully designed by expert humans in various domains. We have open-sourced resulting artifacts found by our system so you can check the output yourself. See a full breakdown and technical writeup: recursive.com/articles/first…

very cool results! i really believe speedruns by @kellerjordan0 @karpathy and others are a great testbed for RSI like tasks, both for evaluating and improving how models do research there are still limitations in how current speedruns are designed, and work needed on the system side to make agents efficient. seeing what models can do in math/code, i believe one missing piece to achieve the same for ai research is giving them the right environment

Excited to show results of the first steps towards automated AI research at @Recursive_SI. The same general system achieved state of the art on @NVIDIAAI's SOL-ExecBench GPU Kernel Optimization, nanoGPT Speedrun, and @karpathy's NanoChat autoresearch benchmarks.

initial results on automated ai research from our team on small scale pre-training and kernel optimization. we also open-source the corresponding artifacts. it's been great seeing all the amazing progress here in such a short time!

Recursive’s autonomous research system just set new SOTAs on @karpathy’s nanogpt nanochat and @nvidia’s kernel benchmarks!! No humans in the loop. The system discovered these gains on its own. Open-sourcing results today.

Thrilled to finally share some of the work we at @Recursive_SI has been doing since we launched!🔥 We've made a system that autonomously conducts AI research and tested it across three different settings: Small Language Model training, NanoGPT speedrunning, and kernel engineering! A bit like Karpathy's Autoresearch, but scaled up and designed to be open-ended, we can push our system to optimize models, training algorithms, and kernels in really cool ways. Blogpost: recursive.com/articles/first…