Stacking a tower of six cubes with VR teleop on the digital twin (cartoon sfx included lol) Feels way more natural on the real arm than in sim. But sim is great for debugging. @SHOW_ROBOTICS

Good company in that list. Heads-down on the arena: real arms, real blocks, real failures. Will show when it holds up.

CORTEX vs COGNITO Robot stacking duel in MuJoCo. Two RoArm-M3 digital twins, kinematics calibrated against real firmware (~1e-16 rad residual). Gripper's a reverse-engineered parallelogram four-bar we measured bolt by bolt. The cubes fall for real. Gravity, friction, placement noise. COGNITO's tower collapse at 6 cubes wasn't scripted - it just happened in the sim. Same seed, same crash, every time. Headless sims run in seconds. We render the interesting ones. One thing that almost killed us: default MuJoCo contacts couldn't hold a tower past 5 cubes. Stiffer contacts implicitfast integration fixed it. Physics sims lie if you don't tune them. This is the rehearsal stage. Same twin gets us VR teleoperation debugging, choreography validation before we move metal, and a shared language between code and hardware. The policies that actually race will come from VR demos on the real arms.

That was a long weekend 😅 When we picked our hardware we'd just spent a year on 3d printing and wanted something off the shelf, so we got a pair of Waveshare RoArm-M3 AI kits, sold as "@LeRobotHF integrated." Turns out the Waveshare LeRobot support is a fork that hasn't moved since mid-2025. being locked to it meant we couldn't record native LeRobot v3 datasets, plus a few things we didn't see coming: - Native v3 lands in lerobot 0.4.0 . the fork is stuck on 0.1.0, so every recording had to go through a fragile v2.1 to v3 conversion. - The fork uses the old lerobot.common API that upstream has since removed, so none of the modern train/eval tooling worked for us. - The RoArm isn't an SO-101. it talks JSON over serial through an ESP32, not the clean Feetech binary bus. that gave us a feedback flood causing stale state reads, a 115200 baud usb ceiling while the servos run at 1Mbps internally, a firmware quirk forcing a double write on the gripper, and a singularity when driving it in cartesian over VR. So we did the boring but necessary thing and ported it properly. - Wrote the RoArm-M3 as a native lerobot 0.5.2 robot plugin, recording native v3 (no more conversion step) - Reflashed the firmware 115200 to 1Mbps, state reads went 23Hz to 42Hz, teleop is smooth at 30fps now - Studied the native SO-101 stack to see what we were doing wrong, then fixed fps mismatches, loop overrun blind spots, undistort cost and inference handling - cut ~1400 lines of dead 0.1.0 code from the UI We should probably upstream the RoArm 0.5.2 LeRobot so other RoArm owners don't hit the same wall, till then hit me with a DM if you need it. Bonus: Small tour of the @SHOW_ROBOTICS workshop

Update from the @SHOW_ROBOTICS workshop - Basic VR teleoperation unlocked, still need to implement 5 sec wrist calibration (@aurel_arnold technique) - Path to first ACT stacking demo is now clear: leader-follower teleop VR teleop synthetic data in MuJoCo DAgger correction at the end - Still waiting on Dynamixel servos to build the Koch 1.1 leader and compare demo quality - RSI loop now has a name, and no it's not Jarvis 😁. It's Iris (Iterative Recursive Improvement System), talks with a neural voice when I'm in the headset

New website, check it out! showrobotics.ai

Added a sim page to the @SHOW_ROBOTICS workshop UI today. It's a work in progress. MuJoCo scene with six 25mm cubes laid out in a 2×3 grid. Two cameras (wrist front). It plays back synthetic stacking trajectories so I can watch the arm pick and place before committing GPU time to a training run. If the motion looks wrong here, no point training on it. Most of the day went into inverse kinematics and URDF partial implementation of RoArm Gripper B. Next: decompose the Gripper B mesh into separate STLs (base, jaws, linkage) and rig the six pivots so the jaws actually open and close in sim.

20 pick-and-place episodes on the RoArm M3. 95% success rate. The policy was trained a week ago. We unlocked it today by fixing the runtime. The signal: FPS counter on the eval dashboard at 12-14 while the model was trained at 20fps. Every eval was running at 60-70% of training-time inference frequency. Distribution mismatch baked in. Profiled the robot loop. send() to the servo controllers was blocking the main thread for 20-110ms per step. Refactored to an AsyncArmWorker: serial I/O on a dedicated thread, main loop latency drops to ~0ms. 20fps stable. Hardware: added a PCIe card with 4 Renesas USB controllers, cameras and arms on isolated buses. Removed the USB contention inflating send() variance. Last mile: base servo offset 3° clockwise from training calibration. Tuned, re-evaled. ACT v3 025k policy, 20 consecutive episodes at 95%. Gripper still has a residual timing quirk. Minor at this success rate, fix later. VR teleop was gated behind 70% baseline. Cleared. SmolVLA v6 (100k full finetune, unfrozen encoder) finished cooking tonight. Next on the bench.

3D realtime robot view joint angle traces just shipped.

Milestone in ! Next stacking…

The strangest thing at @show_robotics workshop right now is reading reports about the project, written by something that isn't me. An RSI loop runs in the background three time per hour. Day and night, without pause. Before each iteration it walks through its memory palace, twenty-nine markdown files where every bug, every pattern, every dead-end it has ever explored is written down, reads itself back into the present, picks up a thread, pulls. During the day the system tells me what it found. I'm placing cubes for eval trials. I'm slipping on the headset to help the system get basic VR teleop calibration. Minutes later it has built an inverse kinematics engine. It fine-tunes models. It refines datasets. It compares dry runs. Between actions the findings surface: your dataset has four outliers. Your pretrained expert is incompatible with your kinematics. Your right camera is as critical as your left. I nod. I redirect with a sentence. The loop keeps going. I'm the freaking ape touching the monolith.

Update from the @SHOW_ROBOTICS workshop: Yesterday's post: MolmoAct2 full FT. Quick honest follow-up: the arm moved but the RoArm kinematics weren't really assimilated. Too early, dataset isn't strong enough yet. So we pivoted to fix the gaps. Since then: installed @thedotmack's claude-mem plugin for persistent context across sessions, and started laying out a memory palace for the project (rooms for hardware, training, datasets, bugs). Then drafted project's RSI loop overnight (recursive self improvement). Claude code wakes up every 40 minutes and improve the project. Night results: 47 log entries, two models fine-tuned while I slept (SmolVLA v1 ACT v3), 20 bugs caught and fixed, several before they would have wasted GPU time. This morning we built a proper eval process: 10 trials per checkpoint, success/partial/fail outcomes logged in the workspace page. While I was on monkey duty (placing cubes, pressing buttons), the loop decided on its own to kick off ACT v5. So the bottleneck is now my hands on the rig. Plan from here: ACT first, then SmolVLA once SR is solid. After that we'll wire in VR teleop (Quest Pro is charged) to produce cleaner demos, then back to MolmoAct2 in a few days with a better dataset.

Update from @SHOW_ROBOTICS workshop Working toward 50-100 clean demos for a MolmoAct 2 fine-tune. Prep done today, recording tonight. Software / tooling - Split the settings page into recording / eval / hardware tabs, unified 1600px layout - Automatic cube position detection per episode (HSV camera sync, marker overlay on the workspace photo) - Quality hub now supports LeRobot v2.1 and v3 datasets; realsense depth format crash fixed Hardware / teleop - Repositioned the cameras to match MolmoAct 2's setup (top side view, wrist recorded but not trained on) - Fixed the follower gripper not tracking the leader (separate joint command in parallel) - Tuned the EMA filter on the leader's joint reads: the smoothed signal drives the follower and lands in the dataset, so lower teleop lag (~50ms → ~12ms) and much cleaner trajectories in one shot 5 episodes recorded already: 3 clean, 0 severe issues, jerk down ~15× vs the previous run.

Update from @SHOW_ROBOTICS lab Software - Workspace tracker: click-to-log placements & rollout outcomes on a snap of the workspace - Quality hub: per-episode metrics (rmse, jerk, end-var, score) with sparklines and p90 warnings - Eval screen with live outcome logging that pushes back to the tracker Hardware - Extended usb cables: no more cam shift or mid-episode disconnect - New force-control gripper servo - Marking box on the table for consistent placement zones - Troubleshooted an open ground in the workshop workspace tracker and quality hub idea straight from @muzaff3r_h: thanks man

Installing the Waveshare Gripper-B CF35-12 force-control servo on our RoArm-M3 Pro. Turned into a half-day debug at @SHOW_ROBOTICS workshop: - The different versions of the arm M1, M2, M3 each use different bus servo addresses, and the doc doesn't list them. Gripper on M3 is at bus address 17. - Stock RoArm-M3 firmware doesn't expose a scan command, so you can't even discover this from the web UI (despite the underlying General Driver firmware having T:52). - Stock firmware also has no implementation for the CF-series servo protocol, only ST3215/ST3235. The Gripper-B's force-control registers are unreachable out of the box. Fix: USB-TTL adapter straight to the servo to change its ID, then patched the firmware to add CF protocol support with current-limited control on the gripper. Capped at 300mA so the servo stops stalling when ACT (the imitation learning policy we're training) grips a cube and can't release it.

Update from @show_robotics workshop: - Moved to a new location. The lease wasn't renewed and rent was getting too high anyway. The new space is in a 2-year-old building, one long open workshop instead of several small rooms, much more practical. This is why I was quiet in April: moving all the machines was a big job. - Tests on the arms are promising. A few policies trained, hardware improvement path identified. - Ordered today: spare servos, a 4-port USB card with independent controllers (to avoid USB congestion), proper USB extenders for cable management, and a UART interface for servo programming. - Arena build starts soon.

Early checkpoint (10000 steps) 40% of the full training but this is already my highest success rate so far, you can hear in the video how excited I am 😝

New UI version, can't get enough of Claude design. Fried control board swapped, new parallel gripper installed and calibrated, cameras repositioned. Tomorrow: 50 episodes and another ACT training run. Should get a much better success rate.

While the new controllers ship, designing the stage. cyberpunk shadowbox, RGB backlit, octagonal stage. Wip.

First dataset, first training, first eval, all working. Jarvis is rough but this is the floor, only goes up from here.