Final-year CS PhD @Stanford. Prev @GoogleDeepMind @NVIDIA @MIT_CSAIL. Robotics/Manipulation

Joined January 2021
44 Photos and videos
Pinned Tweet
14 Mar 2024
Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧀! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud finger motion) for training robots with dexterous hands Everything open-sourced
25
139
620
234,740
Chen Wang retweeted
I will join Northwestern University Computer Science as an Assistant Professor in Fall 2026! I am actively recruiting PhD students and seeking collaborations in robotics, human-robot interaction, brain-computer interfaces, cognitive science, societal impact of AI & automation, and AI for art & design. Please see the recruitment announcement on my personal website, and feel free to reach out!
74
204
1,518
612,225
Chen Wang retweeted
πŸ€– Ever wondered what robots need to truly help humans around the house? 🏑 Introducing π—•π—˜π—›π—”π—©π—œπ—’π—₯ π—₯π—Όπ—―π—Όπ˜ π—¦π˜‚π—Άπ˜π—² (𝗕π—₯𝗦)β€”a comprehensive framework for mastering mobile whole-body manipulation across diverse household tasks! 🧹🫧 From taking out the trash to laying out clothes and cleaning toilets—𝗕π—₯𝗦 equips robots to handle practical, everyday activities. 🌐 Explore more: behavior-robot-suite.github.… Let's dive in! 🀿🧡
18
126
417
185,755
Chen Wang retweeted
28 Feb 2025
Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids toruowo.github.io/recipe/ TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D
11
64
308
49,498
Chen Wang retweeted
A good hand can push intelligence development. Introducing Eyesight Hand, equipped with full-hand high-res tactile sensors and proprioceptive actuators. It is compliant, agile, and powerful. Good tactile sensing makes learning more efficient and robust. Shout out to Branden!
7
45
231
23,480
Chen Wang retweeted
Introducing EgoMimic - just wear a pair of Project Aria @meta_aria smart glasses πŸ‘“ to scale up your imitation learning datasets! Check out what our robot can do. A thread belowπŸ‘‡
10
54
238
49,486
14 Oct 2024
Excited to introduce ARCap! We found that visual feedback is crucial for high-quality data collection, and AR can greatly help! We invited 20 novice users to each gather a small amount of data using only ARβ€”no robot hardware required. The combined data can successfully train a robot policy!
How can we collect high-quality robot data without teleoperation? AR can help! Introducing ARCap, a fully open-sourced AR solution for collecting cross-embodiment robot data (gripper and dex hand) directly using human hands. 🌐:stanford-tml.github.io/ARCap… πŸ“œ:arxiv.org/abs/2410.08464
2
7
65
5,955
Chen Wang retweeted
Why hand-engineer digital twins when digital cousins are free? Check out ACDC: Automated Creation of Digital Cousins πŸ‘­ for Robust Policy Learning, accepted at @corl2024! πŸŽ‰ πŸ“Έ Single image -> 🏑 Interactive scene ⏩ Fully automatic (no annotations needed!) 🦾 Robot policies deployed zero-shot in original scene 🌐: digital-cousins.github.io
11
39
162
66,800
Chen Wang retweeted
26 Sep 2024
Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing discuss: huggingface.co/papers/2409.1… We present a novel approach to synthesize dexterous motions for physically simulated hands in tasks that require coordination between the control of two hands with high temporal precision. Instead of directly learning a joint policy to control two hands, our approach performs bimanual control through cooperative learning where each hand is treated as an individual agent. The individual policies for each hand are first trained separately, and then synchronized through latent space manipulation in a centralized environment to serve as a joint policy for two-hand control. By doing so, we avoid directly performing policy learning in the joint state-action space of two hands with higher dimensions, greatly improving the overall training efficiency. We demonstrate the effectiveness of our proposed approach in the challenging guitar-playing task. The virtual guitarist trained by our approach can synthesize motions from unstructured reference data of general guitar-playing practice motions, and accurately play diverse rhythms with complex chord pressing and string picking patterns based on the input guitar tabs that do not exist in the references. Along with this paper, we provide the motion capture data that we collected as the reference for policy training.
2
68
316
26,871
29 Aug 2024
We found that the relations between keypoints are a powerful way to represent tasks. What’s more exciting is that these keypoint relations can be formulated as constraint satisfaction problems, allowing us to use off-the-shelf optimization solvers to generate complex robot actions. Check out @wenlong_huang's thread for more on how we automate the keypoint constraints generation process and how this enables robots to perform reactive, bimanual, and long-horizon tasks!
What structural task representation enables multi-stage, in-the-wild, bimanual, reactive manipulation? Introducing ReKep: LVM to label keypoints & VLM to write keypoint-based constraints, solve w/ optimization for diverse tasks, w/o task-specific training or env models. πŸ§΅πŸ‘‡
9
70
6,871
23 Aug 2024
We are hosting the dexterous manipulation workshop at CoRL this year πŸ€–! We'll dive into topics like visual & tactile perception, skill learning, and control. Don’t miss the opportunity to share your amazing works and participate! dex-manipulation.github.io/c…

23 Aug 2024
🎺 Announcing our CoRL 2024 β€œLearning Robot Fine and Dexterous Manipulation: Perception and Control” in Munich. Join us to hear from an incredible lineup of speakers! And don’t miss the opportunity to submit your work and participate! Checkout: dex-manipulation.github.io/c…
3
40
4,222
Chen Wang retweeted
Meet our AI-powered robot that’s ready to play table tennis. πŸ€–πŸ“ It’s the first agent to achieve amateur human level performance in this sport. Here’s how it works. 🧡
127
772
3,736
784,858
Chen Wang retweeted
Excited to share a new humanoid robot platform we’ve been working on. Berkeley Humanoid is a reliable and low-cost mid-scale research platform for learning-based control. We demonstrate the robot walking on various terrains and dynamic hopping with a simple RL controller.
17
77
360
80,379
Chen Wang retweeted
Try π“πžπ₯πžπ•π’π¬π’π¨π§ now without any setup: tinyurl.com/open-television It will display a 3D movie recorded during teleoperation. If you open the link with a VR device it will be a 3D movie. You can also see your hand key points if you use VR. x.com/xuxin_cheng/status/180…

Introduce Open-π“πžπ₯πžπ•π’π¬π’π¨π§πŸ€–: 
⁣ We need an intuitive and remote teleoperation interface to collect more robot data. π“πžπ₯πžπ•π’π¬π’π¨π§ lets you immersively operate a robot even if you are 3000 miles away, like in the movie 𝘈𝘷𝘒𝘡𝘒𝘳. Open-sourced!
4
12
133
9,901
Chen Wang retweeted
Our new exciting work on hand arm grasp-anything is out now. All trained in sim with RL. Direct pixels to action grasping on a 23 DoF system with hand and arm jointly coordinating to pick up objects. Fast, reactive and buttery smooth motion. Some of the coolest results (!) we’ve had recently. Led by @tylerlum23 and @martin_matak with @viktor_m81, @arthurallshire , Tucker, @robot_trainer and Karl. paper: arxiv.org/abs/2407.02274 Website: sites.google.com/view/dextra…
6
33
209
21,748
Chen Wang retweeted
We are demoing our LEAP Hand teleoperation system today (Thursday) at #rss2024 with @ManusMeta in van Hasseltzaal (the demo room behind the poster rooms in the 3rd floor). Come try and teleoperate our low-cost easy-to-build dexterous robots hands!
4
23
173
22,703
Chen Wang retweeted
15 Jul 2024
I’ve been training dogs since middle school. It’s about time I train robot dogs too πŸ˜› Introducing, UMI on Legs, an approach for scaling manipulation skills on robot dogs🐢It can toss, push heavy weights, and make your ~existing~ visuo-motor policies mobile!
13
85
445
145,880
Chen Wang retweeted
12 Jul 2024
Introducing tactile skin sim-to-real for dexterous in-hand translation! We propose a simulation model for ReSkin, a magnetic tactile sensing skin. It can simulate ternary shear and binary normal forces. More: jessicayin.github.io/tactile…
2
42
187
33,500
Chen Wang retweeted
Introducing FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes! Our system merges a library of skills with foundation models for efficient robotic feeding, tailored to user preferences. πŸ”—emprise.cs.cornell.edu/flair πŸ“ƒarxiv.org/abs/2407.07561 To Appear at RSS ’24 1/N
3
24
124
23,167
Chen Wang retweeted
Image-generation diffusion models can draw arbitrary visual-patterns. What if we finetune Stable Diffusion to πŸ–ŒοΈ draw joint actions 🦾 on RGB observations? Introducing π—šπ—˜π—‘π—œπ— π—” paper, videos, code, ckpts: genima-robot.github.io/ 🧡Thread⬇️
2
33
161
37,090
Chen Wang retweeted
8 Jul 2024
πŸ”Š Audio signals contain rich information about daily interactions. Can our robots learn from videos with sound? Introducing ManiWAV, a robotic system that learns contact-rich manipulation skills from in-the-wild audio-visual data. See thread for more details (1/4) πŸ‘‡
10
60
311
77,717