Video diffusion models learn motion indirectly through pixels. But motion itself is much lower-dimensional. We introduce 64× temporally compressed motion embeddings that directly capture scene dynamics. This enables efficient planning -> 10,000× faster than video models. 🧵👇

Another crazy CVPR 2026 world model result. “Envisioning the Future” forecasts where points in a scene will move, step by step, from a single image. No dense video needed. - 3,000x faster than video models, 10x fewer parameters, and 5x more accurate under a fixed compute budget. - An autoregressive diffusion model rolls sparse point trajectories forward through short, predictable steps, modeling uncertainty as it grows. - Why it matters: Rollouts get cheap enough to simulate thousands of futures and plan over them, hitting 78% billiard planning accuracy vs 16% for the best dense video baseline.

second runner up is a super cool paper from Björn Ommer’s group at LMU. they first encode point trajectories into a latent using a VAE. then they learn to denoise future trajectories conditioned on past. they do planning experiments on Libero by training a small decoder to convert denoised latents to robot action. I joked that this is latent stable diffusion all over again by Björn but this time with point trajectories 😄

ZipMo hands a robot its plan by daydreaming how the scene should move. No pixels, no rendering, no hour-long video to sit through. 10,000x faster than a top video model! Just a fast read on where everything's headed, and the robot runs with it. Give it a start frame and a goal in plain language. It predicts how the arm and the objects around it should move to get there. A thin policy head reads that motion plan and turns it into the next arm command. The head never sees the task, only the predicted motion. So all the reasoning lives in the motion model. The head is pure inverse dynamics, motion in, action out. On the LIBERO benchmark it replans on every new frame, and it beats the trajectory-based policy methods it lines up against: ATM, Tra-MoE, Amplify. It pulls this off by never touching pixels. It learns a compact motion space from tracked trajectories, then generates motion straight inside that space. And stranger still, compressing that space 64x makes the motion sharper, not worse. Congrats to the team, lovely work. @rmsnorm, @KoljaBauer, @StefanABaumann

You don't need a video model to predict motion. We'll talk about how to be 3000× faster at CVPR Poster 634, this morning at 10:45. Drop by and check it out!

⚠️ Standard first stages are not sufficient for safety-critical applications! The most extreme weather events are often the hardest to decode. One latent → many plausible reconstructions Deterministic decoders hide that uncertainty. Meet FREUD 🧵👇

Check out our work on how to scale NVS on internet-scale data! We provide fixes to the unsupervised NVS pipeline (RayZer) and also obtain more interpretable pose estimations while simplifying the overall setup.

💡 Training with differently noised patches increases overall image gen performance, as the model learns a better underlying representation. This holds even for plain Euler sampling, but their sampler increases the gap even more!

Cool stuff

Stop predicting motion step-by-step. Model the whole motion in a compact representation for efficient planning. 📄 Paper: arxiv.org/abs/2604.11737 💻 Models: compvis.github.io/long-term-… Joint work with @KoljaBauer, @StefanABaumann, @itsbautistam, Josh Susskind, and Björn Ommer.

Amazing work led by @rmsnorm @KoljaBauer and our collaborators at LMU, to be presented at @CVPR! Personally, I find this question of "what's the right level of abstraction for planning in physical space?" to be very intriguing. Pixels over time are very low SNR (ie. the argument behind JEPA) but motion/trajectories carries a lot on information while being extremely compressible. I believe there's a lot more to uncover from this direction. Very glad to be part of this one!

A massive step forward for AI video!

I have been saying

1️⃣x.com/neerjathakkar/status/2… Also, shoutout to two other recent works that explore how to use point tracks for world modeling. 👇...

Do we really need pixel generation to model motion? 🤔 We show how directly representing motion in a compact space enables efficient, scalable planning. 10,000× faster than video models, enabling planning and reasoning in open-world and robotics settings. Check it out ⬇️