To understand what it takes to build a humanoid robot with model-based control, we finetuned @physical_int 's (PI) Pi05 model for our custom use case and environment. We incurred ~$10K in hardware costs, compared to the typical ~$20K set up (DROID/ALOHA). Here are the lessons and challenges we faced building the first working prototype (shown in the video) in 3 months. Part 1: Hardware, Software, Model Selection, Custom Embodiment, Inference, Embedded Hardware, Hierarchical Planner Part 2: Model Evaluation, Data Collection, Model Training, Simulation and Teleoperation We hope sharing our experience accelerates the learning of others who are in a similar starting point.

NVIDIA showcased Newton at GTC again earlier this year. If you are familiar with MuJoCo or Isaac Sim, Newton is a new open-source, GPU-accelerated physics engine aimed at robotics and contact-rich manipulation, built with DeepMind and Disney Research. It also runs as a physics backend inside the Isaac ecosystem. I finally had time to dig through the examples, and the RJ45 plug simulation (example_contacts_rj45_plug.py) caught my attention. The latch deflects and clicks, and the cable behaves like a real 1D deformable. We've been working on cables, connectors, and other contact-heavy interactions, so seeing this as a first-class example was a pleasant surprise. I extended it to test insertion and removal, with and without pressing the latch, and visualized the result in Rerun. Pull without pressing the tab, and the latch holds the plug in place; press it, and the plug comes free. It also interested me how code-first the examples feel. We've spent real time trying to drive both MuJoCo and Isaac Sim with coding agents, and the recurring friction is being able to reason spatially to modify the simulation setup. It feels like there's real potential here. I'm curious if Newton might be better suited for agent-generated robotics simulations. If you're looking for something like this too, Newton is definitely worth a try.

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. @castformai bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

WAIRED killed it with this one, and where did you say I could get my own Casty?

Make motors go brr

If you need a domain specific model that outperforms frontier models and reduces inference cost, @googrish and the @castformai team have the best product for it.

We're in Boston this week for #RoboticsSummit, and NYC next week. Who should I meet to exchange notes and explore overlaps on vertical robotics? We're building robots for datacenters and are actively thinking about early commercialization angles, continuous post-training pipelines fed by fleet data and scalable verification for RL.

👀

The next frontier will be autonomous datacenters ran by AI and robots.

We'll be in SF next week for Data Center Expo. Do I know anyone deploying robots in industrial or similar settings? We're building robots for datacenters and are working together with our first enterprise partner. Would love to share notes and explore overlaps. Real-world RL, engineering deployment pipelines for continuously improving models, and early commercialization are top of mind.

mandatory day zero shot

Digital twins for datacenters are still hard to build in 2026 As agents control systems more autonomously, the need for high-level observability and visualization will only increase We need to know quickly how agents are running to make sure they do not go off the rails

Who’s working on continuous improvement model posttraining tools in the open source? (For AI robotics models) Sounds like the kind of thing that would benefit from co-developing with the community. Would love to meet others building their own.

Congrats on the new MolmoAct 2 release by @allen_ai! A few features that stood out for those considering this for real-world deployments: 1. YAM embodiment unlock 720h teleoperated dataset 720 hours of bimanual YAM data is a meaningful contribution. The YAM embodiment is a simple bimanual arm setup for dexterous tasks, very similar to the dual PiperX and Trossen WidowX arms. Anyone building on @physical_int 's Pi05 or similar models with a YAM-type robot now has significantly more data to fine-tune from, which should reduce the fine-tuning samples needed for a custom task, assuming the target task and environment fall within the dataset's distribution. The dataset spans household, factory, and coffee-shop settings with high object and scene variation. @cortexairobot was the data vendor. Hoping the appendix detailing the quality control protocol gets released. 2. Depth reasoning as a reproducible recipe, but only with layer-level access MolmoAct2-Think shows one way to inject depth information into the action model. Before producing an action, the model predicts a compact discrete depth representation that conditions the action expert through per-layer KV conditioning. The mechanism requires surgical access to the VLM's intermediate attention states at every layer, something only possible with fully open architectures. 3. Swappable VLM backbones for converting VLM -> VLM-ER The released training recipe effectively decouples the perception backbone from the action head. You can pick a VLM optimized for your task domain rather than accepting a generic vision encoder. Hypothetical example: for warehouse sorting where success hinges on reading tiny, cluttered, blurry SKU labels, start from a VLM fine-tuned for OCR (e.g., a custom Qwen-VL or InternVL variant) instead of a generalist web-scale VLM. Apply the MolmoAct2-ER training recipe to that backbone to produce an "OCR-VL-ER" variant, then attach a flow-matching Action Expert. The result is a bespoke VLA that inherits your perception fine-tuning, optimized for label-reading manipulation rather than generic open-world scenes. This assumes catastrophic forgetting is minimized and the backbone retains most of the baseline capabilities it had before fine-tuning. With this recipe, you can swap in domain-specific backbones (medical imaging, industrial inspection, high-res OCR) and convert them into action models entirely from open components.

Simulations are core to robotics research, but spinning up custom scenes is still tedious and has a steep learning curve We built mujoco workbench (mwb): cli agent skills for codex/claude code to scaffold and debug sim scenes from natural language

"A robotics-shaped take-off curve." Cracking distribution is as hard as the technical challenges to commercialize early, especially when customers need to see a plausible trajectory, hardware budgets are tight, and data collection needs funding. "Channel construction is the most underestimated lever in the stack." Creating the data flywheel that powers continuous improvement and eventual task reliability is what closes the gap to real-world deployments that actually create value. "Deployment-system engineering matters as much as model architecture." Similar to the systems around early LLM applications and AI voice agents, a model's capabilities are only as good as the scaffolds around it. This is where engineering depth and iteration speed compound into capability. The teams that earn the right to a first deployment reliable enough to kickstart the data flywheel will crack adoption in a new vertical.

congrats! stoked to have you building embodied autonomy here in SG