🤔 Ever wondered if simulation-based animation/avatar learnings can be applied to real humanoid in real-time? 🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large

🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform

How to start the data flywheel for human-like embodied intelligence? We think real-time teleoperating a humanoid🤖 in a whole-body manner will be a solution. The embodiment alignment allows for a seamless integration of human cognitive skills with versatile humanoid capabilities.

ABS code is now available!!!!!! Simulation training (agile policy, rocovery policy, reach-avoid prediction, ray prediction, visualizations) Real-world deployment (hardware installation, system stepup, sdk scripts) Github: github.com/LeCAR-Lab/ABS.… Website: agile-but-safe.github.io

Unfortunately, I am not at #ICRA2024, but students from LeCAR Lab at CMU will present several papers about learning and control in agile robotics. Check them out!

🚀Introducing Model-Based Diffusion (MBD), a diffusion-based traj optimization method that directly computes the score function using model info. MBD doesn't require data, but can be integrated with data of diverse quality. 🌐lecar-lab.github.io/mbd/ 🧵1/6

Diffusion models have shown strong capabilities in generating high-fidelity trajectories. However, standard diffusion processes cannot efficiently adapt to new scenarios beyond demonstrations (e.g., new robots with different dynamics). MBD (Model-Based Diffusion) is a

Introduce OmniH2O, a learning-based system for whole-body humanoid teleop and autonomy: 🦾Robust loco-mani policy 🦸Universal teleop interface: VR, verbal, RGB 🧠Autonomy via @chatgpt4o or imitation 🔗Release the first whole-body humanoid dataset omni.human2humanoid.com

OmniH2O (Omni Human2HumanOid) aims to provide a universal whole-body control interface for full-size humanoids with dexterous hands. By learning a robust whole-body loco-manipulation tracking policy and carefully designing the control interface (sparse kinematic poses), a single

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal

We introduce 𝐅𝐥𝐲𝐢𝐧𝐠 𝐂𝐚𝐥𝐥𝐢𝐠𝐫𝐚𝐩𝐡𝐞𝐫, an aerial manipulation system that can draw various calligraphy artworks: 🎯Contact-aware trajectory planning and hybrid control ✏️Intuitive user interface and novel end-effector design 🧑‍🎨UAM can draw letters with changing

This is the most artistic project I've ever worked on🎨 Our aerial manipulation can achieve precise contact force and motion planning and tracking, which enables the calligraphy demos in the video. Beyond calligraphy, it can be applied in many high-altitude tasks such as

The CMU LeCAR Lab will present several papers at #RSS2024 this week, covering topics from learning & control & optimization, safe autonomy, to humanoids. Check them out!

Thanks AK! Finally, robot can do continuous, agile, autonomous, adaptive jumping over stair and stepping stone Key idea: combine the pros of model-free RL and model-based control. RL (for CoM refs) + QP (for GRF) + WBC (for torque) Open-sourced: yxyang.github.io/jumping_cod/

🎉 Diffusion-style annealing + sampling-based MPC can surpass RL, and seamlessly adapt to task parameters, all 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴-𝗳𝗿𝗲𝗲! We open sourced DIAL-MPC, the first training-free method for whole-body torque control using full-order dynamics 🧵 lecar-lab.github.io/dial-mpc/

H2O (👉human2humanoid.com) and OmniH2O (👉omni.human2humanoid.com) are open-sourced! Check out our fully open-source code: github.com/LeCAR-Lab/huma…, featuring simulation training, motion data retargeting, and real-world deployment. Have fun with your humanoids!

I've been working on dynamics model learning in robotics for >6 years (from the Neural-Lander paper). Training a small DNN + some regularization + some online weight adaptation have proven to be effective for specific robots in specific tasks. LLMs have shown the power of

🎙️I gave a talk "Building Generalist Robots with Agility via Learning and Control: Humanoids and Beyond" at the CMU RI Seminar and Michigan AI Symposium. It covers LeCAR Lab at CMU's research and my thoughts on building agile generalist robots. Recording: youtu.be/Uym3Tr6t5TM

🦾How can humanoids unlock real strength for heavy-duty loco-manipulation? Meet FALCON🦅: Learning Force-Adaptive Humanoid Loco-Manipulation. 🌐: lecar-lab.github.io/falcon-humanoi… See the details below👇:

On my way ✈️ to ATL for IEEE ICRA! LeCAR Lab at CMU will present 8 conference papers (including DIAL-MPC as the Best Paper Finalist) and one RA-L paper. Details: lecar-lab.github.io Hope to meet old & new friends and chat about building generalist 🤖 with agility 🚀