Researching at the frontier of AI on topics of Computer Vision, Computer Graphics, Robotics, Embodied AI, and Reinforcement Learning @UCSanDiego @haosu_twitr

Joined May 2021
39 Photos and videos
Hao Su Lab retweeted
VLA models often forget their pretrained knowledge during action training, hurting generalization. 🤖Our framework unifies action & VLM training to preserve strong pretrained representations & maintain versatility, boosting generalization & robustness. gen-vla.github.io
2
7
38
5,105
Hao Su Lab retweeted
29 Sep 2025
We presented our work on studying multi-embodiment scaling at #CoRL2025 and were thrilled by the excitement around generalist cross-embodiment policies. A common question was: Will you move beyond locomotion? The answer is YES! Locomotion provides a clean starting point, but our long-term goal is to extend these ideas to more challenging domains such as manipulation, paving the way toward general cross-embodiment intelligence. Huge thanks to the amazing team for making this happen across three time zones: project co-leads @LiuDai_DL, @NicoBohlinger, Dichen Li, together with @tongzhou_mu, @ZhanxinWu0725, K. Fay, and advisors @hiskov, @Jan_R_Peters, @haosu_twitr.
20 May 2025
🧠 Can a single robot policy control many, even unseen, robot bodies? We scaled training to 1000 embodiments and found: More training bodies → better generalization to unseen ones. We call it: Embodiment Scaling Laws. A new axis for scaling. 🔗 embodiment-scaling-laws.gith… 🧵👇
13
51
10,434
Hao Su Lab retweeted
26 Sep 2025
Tongxuan’s work explores using generative models (diffusion) for state estimation and world model learning in cloth manipulation, a domain with significant visual occlusion and complex dynamics. He will give a 5-minute talk at 2:00 pm in the Simulating Robot Worlds workshop today (simulatingrobotworlds.github…), and a 1-minute spotlight presentation from 3:00–3:30 pm on Sep 28 at #CoRL.

How to capture complex environment dynamics accurately with partial observations for world modeling? 🧐 Thrilled to share our recent work on world models for robotic manipulation - UniClothDiff: Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation, accepted to #CoRL2025 🎉. We target on the challenging task of cloth manipulation, which involves partial observability due to severe self-occlusion, a high-dimensional state space, and highly non-linear dynamics. We enable robots, like humans, imagine the state of cloth through a mental model 🧠 and foresee its future state during folding! 🔗 uniclothdiff.github.io 📜 arxiv.org/abs/2503.11999
6
37
4,981
Hao Su Lab retweeted
🤖Introducing DEMO3: our new model-based RL framework for multi-stage robotic manipulation from visual inputs and sparse rewards. 🧵🔽 📜 Paper: [arxiv.org/abs/2503.01837] 🌍 Project Page: [adrialopezescoriza.github.io…] 💻 Code: [github.com/adrialopezescoriz…]
2
20
143
25,770
Hao Su Lab retweeted
📢 Introducing ManiSkill-HAB: A benchmark for low-level manipulation in home rearrangement tasks! - GPU-accelerated simulation - Extensive RL/IL baselines - Vision-based, whole-body control robot dataset All open-sourced: arth-shukla.github.io/mshab 🧵(1/5)
7
50
215
48,938
Hao Su Lab retweeted
Learning bimanual, contact-rich robot manipulation policies that generalize over diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! glide-manip.github.io 🧵1/n
1
19
74
20,877
Hao Su Lab retweeted
SIMPLER will be presented at #CoRL2024 at 4pm on Nov 8 (Section 4)! While I won't be in person due to visa constraints, @xiao_ted, @KarlPertsch, and @oier_mees will be presenting the paper and are happy to chat about it in person!
Scalable, reproducible, and reliable robotic evaluation remains an open challenge, especially in the age of generalist robot foundation models. Can *simulation* effectively predict *real-world* robot policy performance & behavior? Presenting SIMPLER!👇 simpler-env.github.io/
5
30
5,765
Hao Su Lab retweeted
2 Oct 2024
We now have an initial ManiSkill3 paper out on arXiv which you can cite, just in time for the ICLR submission deadline 😁 arxiv.org/abs/2410.00425
23 Sep 2024
for anyone using ManiSkill 3 in their research / upcoming conference deadlines, we will have a v1 preprint out some time week so you cite it! A v2 version with more experiments (mostly just running baselines data collection atm) will come out later
2
5
42
5,212
Hao Su Lab retweeted
While the Segment Anything Model (SAM) greatly improves 2D segmentation annotation efficiency, is there a foundation model that works for 3D point clouds and meshes like SAM? Introducing Point-SAM, a 3D prompt segmentation foundation model! 👇 point-sam.github.io
4
21
108
15,731
Hao Su Lab retweeted
13 Jun 2024
Join us at our first workshop on 3D Foundation Models @CVPR2024, June 18 in Summit 434, starting at 8:50AM! We have fantastic speakers to discuss the progress and prospects in 3D foundation models. Check out more details at 3dfm.github.io/
1
9
90
17,157
Hao Su Lab retweeted
🥳Excited to share: Hierarchical World Models as Visual Whole-Body Humanoid Controllers Joint work with @jyothir_s_v @vlad_is_ai @ylecun @xiaolonw @haosu_twitr Our method, Puppeteer, learns high-dim humanoid policies that look natural, in an entirely data-driven way! 🧵👇(1/n)
11
62
376
250,912
Hao Su Lab retweeted
15 May 2024
#ICRA2024 @LinghaoChen97 will present our differentiable rendering-based hand-eye calibration method, EasyHec! May 16 13:30@CC-313 (oral); May 16 16:30-18:00@ThBT03.01 (poster) It produces accurate calibration results in a fully automatic manner! ootts.github.io/easyhec/@Lin…

Hand-eye calibration is critical for sim2real in robotics. We propose EasyHeC, a differentiable-rendering-based hand-eye calibration system that is highly accurate, automatic, & convenient, thus significantly reducing sim2real gap in object manipulation! ootts.github.io/easyhec/
4
32
7,013
Hao Su Lab retweeted
9 May 2024
Don’t have a real robot/setup but want to evaluate policies trained on real world datasets? Check out SIMPLER, fast, safe, and reliable evaluation of real robot policies in sim via ManiSkill 2. The ManiSkill 3 beta will port SIMPLER over soon so stay tuned!
Scalable, reproducible, and reliable robotic evaluation remains an open challenge, especially in the age of generalist robot foundation models. Can *simulation* effectively predict *real-world* robot policy performance & behavior? Presenting SIMPLER!👇 simpler-env.github.io/
2
6
58
19,939
Hao Su Lab retweeted
Scalable, reproducible, and reliable robotic evaluation remains an open challenge, especially in the age of generalist robot foundation models. Can *simulation* effectively predict *real-world* robot policy performance & behavior? Presenting SIMPLER!👇 simpler-env.github.io/
5
24
135
86,338
TDMPC by @ncklashansen is in the new open source LeRobot library by @RemiCadene @huggingface!
Replying to @RemiCadene
LeRobot also features the Diffusion Policy, a powerful imitation learning algorithm, and TDMPC, a reinforcement learning method that includes a world model, continuously learning from its interactions with the environment. diffusion-policy.cs.columbia… yunhaifeng.com/FOWM
1
19
2,506
Hao Su Lab retweeted
2 May 2024
📢 ManiSkill 3 beta is out! Simulate everything everywhere all at once 🥯 - 18K RGBD FPS on 1 GPU, 3K on Colab! - Diverse parallel GPU sim - Tons of new robots/tasks All open-sourced: github.com/haosulab/ManiSkil… Photo: MS3 Tasks w/ scenes from AI2THOR and ReplicaCAD 🧵(1/6)
9
62
260
58,883
Hao Su Lab retweeted
27 Apr 2024
maniskill sneak peak 3: lots of new robots to use! Whether its mobile manipulation, humanoids, quadrupeds, or even tactile dextrous hands (see the shadow hand at the bottom with red tactile sensors), we have a ton of new domains being added to try out on GPU state/visual sim
3
2
44
4,734
Hao Su Lab retweeted
24 Apr 2024
Checkout DG-Mesh from @Isabella__Liu, which reconstructs time-consistent, high-quality dynamic mesh with flexible topology change from monocular videos. liuisabella.com/DG-Mesh/

Want to obtain time-consistent dynamic mesh from monocular videos? Introducing: Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Monocular Videos liuisabella.com/DG-Mesh/ We reconstruct meshes with flexible topology change and build the corresp. across meshes. 🧵(1/n)
1
4
46
5,792
Hao Su Lab retweeted
19 Apr 2024
Checkout @SarahWeii 's latest work, which presents a transformer-based model that reconstructs a high-fidelity 3D mesh from 4 (sparse) input images in less than one second! arxiv.org/abs/2404.12385

Check our MeshLRM 🌟, which achieves state-of-the-art mesh reconstruction from sparse-view inputs within 1 second! 🚀🚀🚀 Paper: arxiv.org/abs/2404.12385 Project page: sarahweiii.github.io/meshlrm…
1
4
25
4,590