RL research focused on open-ended learning

Joined December 2024
Photos and videos
Jun 10
Exciting new research led by @lorenz_wlf to accelerate RL task generation using mech interp methods in collaboration with @GoodfireAI
Training a model to generate RL tasks not too hard, not too easy costs many solver runs per task. PROPEL predicts difficulty via a probe on its activations instead, amortizing cost and speeding up generator optimization. New open-ended RL research from @Vmax @GoodfireAI.
3
11
1,980
May 29
Exciting work on unix environment generation led by @radbadgeoffbrad!
The unix terminal is the natural interface for agents to get work done on a computer but how well can agents actually use unix? Claude Code. Codex. Devin. Every frontier agent ships as a terminal tool. With unix-ctf, Vmax is using setters and solvers to measure Unix competence.
2
13
1,260
May 20
Our designer @wwwjim has made something really special for this blog post.
1
8
1,105
Vmax retweeted
Vmax is building an open-ended learning system that generates and optimizes itself on tasks that it creates, avoiding human bias that may corrupt optimal learning curricula. In PopuLoRA, we instantiate this as co-evolving populations of LLMs performing asymmetric self-play.
21
56
289
71,101
Vmax retweeted
We are so excited to have @tensorfi joining @VmaxAI! Maxwill joins us from @Meta, where he was working RL and LLMs for recommendation. Previously, he has also worked @Tesla on the autopilot team and also in Quant finance at Kronos research. He also holds an MS in CS from Georgia Tech. Maxwill simultaneously understands pre-LLM RL fundamentals but also how to scale pipelines for RL training for modern recommendation systems. Maxwill is already levelling up our pipeline for automated environment design, pushing multiple PRs as soon as he joined. Really excited about the velocity of his contributions and excited to share more soon.
1
17
1,763
Mar 3
Welcome Geoffrey!
So excited to welcome Geoffrey Bradway as Member of Technical Staff @VmaxAI. Geoffrey is a rare catch. He was an engineer at @GoogleDeepMind, Google for Youtube and also has experience in early stage companies, having been a previous @ycombinator founder and also VP of engineering at @numerai. Fitting the Vmax DNA, he has experience with RL before it was cool (doing RL all the way back in 2014). Outside of work, Geoffrey does some really cool art with robotic drawing machines. Cannot wait to share more about what he is cooking
1
9
1,637
Vmax retweeted
PR review is one of the fast growing categories in AI for SWE, now you can benchmark agents on *real* PRs
Introducing Code Review Bench v0: codereview.withmartian.com The first independent code review benchmark. 200,000 PRs. Unbiased. Fully OSS. Updated daily. Tool performance highlights 🧵👇 Featuring: @augmentcode @baz_scm @claudeai @coderabbitai @cursor @GeminiApp @github @graphite @greptile @kilocode @OpenAIDevs @propelcode @QodoAI
1
9
1,526
Vmax retweeted
So excited to have @lorenz_wlf join @VmaxAI as a research fellow this spring! At NeurIPS last year, we caught up with Lorenz, realised how aligned he is with our research vision and invited him to join us shortly after. Lorenz comes from the @FAICDT1 programme at UCL (where I did my PhD also) and is supervised by @mircomusolesi. Previously he worked on differential privacy and personalized recommender systems at Apple and did his undergrad in mathematics and statistics at Imperial College London. Lorenz’s research focuses on RL, RLHF and modular continually learning RL agents. He has contributed to papers in ICLR, TMLR and AI STATS. So excited for him to join us and accelerate our efforts on unsupervised environment design. You can read Lorenz's research in the replies. Much more to come.
3
3
25
2,111
Vmax retweeted
22/ Reinforcement learning, but make it automated. @MavorParker & @matthewjsargent showed us how they’re generating long-horizon environments at @VmaxAI. vmax.ai/
1
2
8
1,844
Feb 2
welcome Roger!
@VmaxAI is excited to have @creus_roger joining us as a research fellow! Roger is joining us from @Mila_Quebec where he works with @pcastr and @GlenBerseth. Roger Creus Castanyer is a brilliant RL researcher working on exploration, credit assignment, and skill discovery. He is also fresh off of a NeurIPS spotlight and a recently accepted paper to ICLR, you can find more of his research in the comments. Roger is significantly accelerating our research on automated environment design - looking forward to sharing what he is cooking!
8
933
Vmax retweeted
Replying to @VmaxAI
As an initial step in this direction, we have built on top of methods like SWE-smith and BugPilot, adding to the list of repo profiles built by the swe-bench community
1
1
9
710
Vmax retweeted
This is a preview of many more tasks to come for Ares!
Replying to @joshgreaves_ml
ARES uses the Harbor task format ( @alexgshaw ). It comes with SWE-Bench Verified, TerminalBench2, SWESmith, and everything else in the Harbor ecosystem. We're also releasing 1k new JavaScript tasks with @VmaxAI ( @MavorParker @matthewjsargent ) to help the ecosystem grow.
1
5
19
1,872
Vmax retweeted
RL progress is bottlenecked by infra for training and evaluation. @VmaxAI is excited to be partnering @withmartian, generating environments for the Agentic Research and Evaluation (ARES) framework
7
30
72
8,750
Vmax retweeted
ARES uses the Harbor task format ( @alexgshaw ). It comes with SWE-Bench Verified, TerminalBench2, SWESmith, and everything else in the Harbor ecosystem. We're also releasing 1k new JavaScript tasks with @VmaxAI ( @MavorParker @matthewjsargent ) to help the ecosystem grow.
1
3
16
5,384
Vmax retweeted
Really excited for this 👀 reach out to josh!
I’m building a new RL tool for code agents. If you work in RL and want an early preview, DM me.
1
1
6
1,316
Vmax retweeted
Which pre-LLM methods matter most in the LLM era? We hosted an off the record event with @danijarh (world models for robotics), @ashvinair (RL @cursor_ai) and @nikishin_evg (reasoning @OpenAI) to share their takes @southpkcommons.
4
12
37
8,089
Vmax retweeted
The RL event @VmaxAI and @southpkcommons are organizing has a stacked panel of amazing researchers, whose work I have admired since my PhD, when RL was not as much of a hot topic as it is today. Here's a thread on our panelists 👇
1
8
33
4,806
Vmax retweeted
At @VmaxAI, we take the bitter lesson seriously. Come get your Vmax chocolate at the scaling environments workshop @SEAWorkshop
4
7
38
2,059
7 Dec 2025
Very excited to sponsor the scaling environments workshop
🚀 SEA Workshop is going LIVE TOMORROW! Join us at NeurIPS 2025 for a full day diving into the Scaling Environments for Agents featuring an incredible lineup of speakers and panelists: @egrefen @Mike_A_Merrill @mialon_gregoire @deepaknathani11 @jl_marino @syz0x1 @qhwang3 Anthony G. Cohn, Eric Sommerlade, @fredsala 📍 Upper Level Room 23ABC 🕘 08:00–17:00 Huge thanks to our sponsors: @TheInclusionAI (@AntLingAGI) @SnorkelAI @SonicjobsApp and @VmaxAI 🙌 🔥 Get ready for a day of insights and inspiring conversations!
4
8
1,180
Vmax retweeted
At Vmax, we are automating the construction of RL environments and the post-training of agents. We are hiring members of technical staff and research fellows. Come join us in SF! (link to apply in comments).
5
32
79
19,401