Excited to introduce SOLE-R1, a video-language reasoning model for zero-shot reward prediction for robot manipulation tasks! SOLE-R1 reasoning can serve as the SOLE signal for learning new tasks (completely from scratch) through online RL - i.e., robots start with random actions and learn previously unseen tasks guided only by SOLE-R1 rewards, without any demonstrations, ground-truth rewards, success indicators, or task-specific tuning. SOLE-R1 significantly outperforms strong baselines (e.g., Robometer, RoboReward, TOPReward, GPT-5, Gemini-3-Pro) in zero-shot online RL when evaluated across 40 tasks - including a real-world tabletop manipulation setting and 4 sim environments (LIBERO, ManiSkill, Meta-World, RoboSuite). We open source all models, training data, and code. Website, demos, and paper at: philip-mit.github.io/sole-r1… 🧵 (1/6)

Training diffusion & flow policies with RL is hard, but training them with behavioral cloning is easy. So what if we just train flow policies with BC, and only "do RL" at test time? We found an easy way to do this (called Q-Guided Flow), and some surprising findings👇🧵

Today, we bring SAC to RSL-RL, one of the most widely used RL frameworks in massively parallel robot learning, developed at RSL @leggedrobotics. We try to understand the long-standing performance gap between SAC and PPO, and crystallize important factors sabagian.github.io/sac_relea…

srly, at least the closest thing we have today.

Nice thread of lessons on training a reward model for real-world manipulation! Love it when authors summarize important learnings from a project 😃

Our SOTA reward model for robotic manipulation. I'm very proud of this work; it was more than a year in the making. Lessons learned below:

Great discussion on @RoboPapers about the need for more large-scale collection of authentic failures and sub-optimal-but-successful trajectories for training reward models. starts here: youtube.com/watch?v=sD_jrS_r… and more here: youtu.be/sD_jrS_rXAs?si=t-rK… Historically there has been limited demand for this type of data since it is less directly useful for imitation-style policy training, and for reward-model training it is unclear how to extract useful dense signal from these videos without hand-labeling each timestep (which is painful and slow). Robometer offers a solution to this by adding trajectory-level preference prediction (i.e., choosing which of two trajectories is more successful) as a second objective. This provides global ordering signal from failed and suboptimal trajectories, indirectly improving per-timestep progress predictions without requiring dense labels for these trajectories. It would be good to see future work explore ways of more directly deriving dense signal from the failed/sub-optimal trajectories. But overall Robometer is a real advancement and should scale nicely as more data becomes available! Congrats to @aliangdw, @yigitkkorkmaz , and @Jesse_Y_Zhang on the great work and thank you to @chris_j_paxton, @micoolcho, and @DJiafei for these great conversations!

We found that state-of-the-art VLMs (Gemini, GPT-5, etc.) fail at predicting task progress for online RL, so we built our own: SOLE-R1. SOLE-R1 is trained on 10 million images and video frames, and 4 million chain of thought traces that reason over both space and time. The result is a video-language reasoning model that can be used as a reward for online RL with no other reward signals!

The bottleneck to on-robot reinforcement learning is good, scalable reward prediction. Robometer is a massive step in that direction, and the authors have been wonderfully open as well, releasing a large dataset and continuing to improve their model post release. Thanks to @aliangdw @yigitkkorkmaz and @Jesse_Y_Zhang for joining me and @DJiafei!

Beginning to see some smoother rollouts with RLT, much better than yesterday

We're presenting PEEK at ICRA in Vienna this week 🏰 Stop by at the 9-10:30am poster session on June 3 to say hi and check out some cool VLA generalization 🤖

🚪 “Knock knock!” 👂 “Who’s there?” 🤖 “Zero-shot reward model for robot manipulation”

This was one of the better ACT rollouts we saw today. Task: Package auxiliary parts for the electronic devices into a plastic bag.

Introducing EXPO-FT – Efficient, Reliable & Open-Source VLA Finetuning! EXPO-FT unlocks π0.5 for challenging manipulation tasks: Routing string lights & inserting the power connector to illuminate them Striking pool ball into pocket Inserting flower into wine bottle (1/5)

TOPReward is now in LeRobot! Its an elegant solution using the model's own logit probabilities, and worked very well when I compared it to other VLA rewards (vid below, the red line) How to use in LeRobot: github.com/huggingface/lerob… Congrats @DJiafei @allen_ai @UW!