An autoregressive diffusion model with a hybrid explicit-root/latent-body representation generates real-time, controllable 3D human motion from text and spatial constraints.
hub
Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills
45 Pith papers cite this work, alongside 2 external citations. Polarity classification is still indexing.
45
Pith papers citing it
2
external citations · Pith
abstract
Humanoid robots hold the potential for unparalleled versatility in performing human-like, whole-body skills. However, achieving agile and coordinated whole-body motions remains a significant challenge due to the dynamics mismatch between simulation and the real world. Existing approaches, such as system identification (SysID) and domain randomization (DR) methods, often rely on labor-intensive parameter tuning or result in overly conservative policies that sacrifice agility. In this paper, we present ASAP (Aligning Simulation and Real-World Physics), a two-stage framework designed to tackle the dynamics mismatch and enable agile humanoid whole-body skills. In the first stage, we pre-train motion tracking policies in simulation using retargeted human motion data. In the second stage, we deploy the policies in the real world and collect real-world data to train a delta (residual) action model that compensates for the dynamics mismatch. Then, ASAP fine-tunes pre-trained policies with the delta action model integrated into the simulator to align effectively with real-world dynamics. We evaluate ASAP across three transfer scenarios: IsaacGym to IsaacSim, IsaacGym to Genesis, and IsaacGym to the real-world Unitree G1 humanoid robot. Our approach significantly improves agility and whole-body coordination across various dynamic motions, reducing tracking error compared to SysID, DR, and delta dynamics learning baselines. ASAP enables highly agile motions that were previously difficult to achieve, demonstrating the potential of delta action learning in bridging simulation and real-world dynamics. These results suggest a promising sim-to-real direction for developing more expressive and agile humanoids.
hub tools
citation-role summary
background 1
method 1
other 1
citation-polarity summary
representative citing papers
DeVI enables zero-shot physically plausible dexterous control by imitating synthetic videos via a hybrid 3D-human plus 2D-object tracking reward.
BeyondMimic combines compact motion tracking with a unified guided latent diffusion model to master diverse agile behaviors from human demos and solve unseen downstream tasks via test-time classifier guidance.
A multi-source 16,074-clip quadruped motion library plus a flow-matching generalist tracker shows empirical data scaling and zero-shot unseen tracking, integrated with all-terrain locomotion and real-robot deployment.
FADA is a three-stage Planner-IDM method that achieves few-shot domain adaptation for humanoid control by distilling an oracle policy then finetuning only the IDM on short target-domain rollouts via supervised learning.
SceneBot conditions a humanoid tracking policy on motion references and contact labels, using reconstructed scene-interaction data to unify free-space locomotion with contact-rich manipulation and terrain tasks.
PressMimic fuses RGB and pressure for pose estimation via FRAPPE++ and uses pressure signals in RL policy PSP, backed by the MotionPRO dataset, to achieve physically consistent humanoid motion imitation.
OmniContact introduces contact flow as a shared representation of body trajectories and contact signals to learn and chain loco-manipulation meta-skills, reporting 98.7% success on box carrying and 76.5% on push-stack tasks.
Stubborn introduces a unified RL framework with yaw-aligned representation, Bernoulli probabilistic termination, and adaptive sampling for robust humanoid motion tracking and fall recovery.
FAWAM integrates force signals into perception, prediction, and closed-loop correction, raising success rates 36% over vision baselines in contact-rich manipulation tasks.
EgoPriMo learns a unified egocentric motion prior with a Triple-stream DiT model that supports reconstruction, generation, and forecasting of SMPL motions from egocentric views and text, outperforming prior methods and transferable to humanoid controllers.
Perceptive BFM grounds human motion priors in robot terrain perception via terrain-conformal reference synthesis and teacher-student transfer from adapted to raw-reference tracking.
MPC-based retargeting framework enables cross-morphology whole-body teleoperation from a single XR device via dynamic feasibility optimization, state synchronization, and SLAM feedback, with reported gains in simulation and real-world tests.
A multi-condition latent diffusion model transfers human motion styles to diverse humanoid robot contents with physics regularizations, achieving 96% success in real-robot trials on Unitree G1.
UniLab is a CPU/GPU heterogeneous system for robot RL training using MuJoCoUni and MotrixSim backends that reports 3-10x end-to-end efficiency improvements and cross-platform compatibility beyond CUDA.
Imagine2Real enables zero-shot humanoid-object interaction by unifying motions as 4D point trajectories, tracking only base/hands/object keypoints inside a BFM latent space, and training with progressive simple rewards for mocap deployment.
LineRides enables commandable bicycle robot stunts via line-guided RL that uses spatial guidelines, a tracking margin for feasibility, distance-based progress, and sparse key-orientations.
BifrostUMI enables robot-free human demonstration capture via VR and wrist cameras to train visuomotor policies that predict keypoint trajectories for transfer to humanoid whole-body control through retargeting.
VOFA combines a depth-image visuomotor policy with a force-adaptive whole-body controller to push objects of unknown mass to arbitrary goals on a humanoid.
A weightlessness mechanism enables humanoid robots to dynamically relax joints for stable, contact-rich motions across diverse environments without task-specific tuning.
Differentiable simulation enables torque-sensor-free actuator model identification from trajectory data, achieving 1.88x better position tracking than a stand-trained baseline and 46% longer travel in downstream locomotion policies.
RoSHI is a hybrid wearable that combines sparse IMUs and egocentric SLAM to capture accurate full-body 3D pose and shape data in natural environments for robot learning.
HAIC enables robust humanoid interactions with underactuated objects by predicting their dynamics from proprioceptive history and using a world model for adaptive control.
HUSKY combines humanoid-skateboard dynamics modeling with adversarial motion priors and physics-guided lean-to-steer strategies to achieve real-world stable skateboarding on a humanoid robot.
citing papers explorer
-
ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation
An autoregressive diffusion model with a hybrid explicit-root/latent-body representation generates real-time, controllable 3D human motion from text and spatial constraints.
-
DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitation
DeVI enables zero-shot physically plausible dexterous control by imitating synthetic videos via a hybrid 3D-human plus 2D-object tracking reward.
-
BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion
BeyondMimic combines compact motion tracking with a unified guided latent diffusion model to master diverse agile behaviors from human demos and solve unseen downstream tasks via test-time classifier guidance.
-
Behavior Foundations for Quadruped Robots: ABot-C0 Technical Report
A multi-source 16,074-clip quadruped motion library plus a flow-matching generalist tracker shows empirical data scaling and zero-shot unseen tracking, integrated with all-terrain locomotion and real-robot deployment.
-
FADA: Few-Shot Domain Adaptation via Dynamics Alignment for Humanoid Control
FADA is a three-stage Planner-IDM method that achieves few-shot domain adaptation for humanoid control by distilling an oracle policy then finetuning only the IDM on short target-domain rollouts via supervised learning.
-
SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interaction
SceneBot conditions a humanoid tracking policy on motion references and contact labels, using reconstructed scene-interaction data to unify free-space locomotion with contact-rich manipulation and terrain tasks.
-
PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation
PressMimic fuses RGB and pressure for pose estimation via FRAPPE++ and uses pressure signals in RL policy PSP, backed by the MotionPRO dataset, to achieve physically consistent humanoid motion imitation.
-
OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation
OmniContact introduces contact flow as a shared representation of body trajectories and contact signals to learn and chain loco-manipulation meta-skills, reporting 98.7% success on box carrying and 76.5% on push-stack tasks.
-
Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoids
Stubborn introduces a unified RL framework with yaw-aligned representation, Bernoulli probabilistic termination, and adaptive sampling for robust humanoid motion tracking and fall recovery.
-
FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation
FAWAM integrates force signals into perception, prediction, and closed-loop correction, raising success rates 36% over vision baselines in contact-rich manipulation tasks.
-
EgoPriMo: Egocentric Motion Generation for Interactive Humanoid Control
EgoPriMo learns a unified egocentric motion prior with a Triple-stream DiT model that supports reconstruction, generation, and forecasting of SMPL motions from egocentric views and text, outperforming prior methods and transferable to humanoid controllers.
-
Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain
Perceptive BFM grounds human motion priors in robot terrain perception via terrain-conformal reference synthesis and teacher-student transfer from adapted to raw-reference tracking.
-
X-OP: Cross-Morphology Whole-Body Teleoperation via MPC Retargeting
MPC-based retargeting framework enables cross-morphology whole-body teleoperation from a single XR device via dynamic feasibility optimization, state synchronization, and SLAM feedback, with reported gains in simulation and real-world tests.
-
Bionic Human-Motion Style Transfer for Physically Executable Whole-Body Control of Humanoid Robots
A multi-condition latent diffusion model transfers human motion styles to diverse humanoid robot contents with physics regularizations, achieving 96% success in real-robot trials on Unitree G1.
-
UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms
UniLab is a CPU/GPU heterogeneous system for robot RL training using MuJoCoUni and MotrixSim backends that reports 3-10x end-to-end efficiency improvements and cross-platform compatibility beyond CUDA.
-
Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors
Imagine2Real enables zero-shot humanoid-object interaction by unifying motions as 4D point trajectories, tracking only base/hands/object keypoints inside a BFM latent space, and training with progressive simple rewards for mocap deployment.
-
LineRides: Line-Guided Reinforcement Learning for Bicycle Robot Stunts
LineRides enables commandable bicycle robot stunts via line-guided RL that uses spatial guidelines, a tracking margin for feasibility, distance-based progress, and sparse key-orientations.
-
BifrostUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation
BifrostUMI enables robot-free human demonstration capture via VR and wrist cameras to train visuomotor policies that predict keypoint trajectories for transfer to humanoid whole-body control through retargeting.
-
VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids
VOFA combines a depth-image visuomotor policy with a force-adaptive whole-body controller to push objects of unknown mass to arbitrary goals on a humanoid.
-
Learn Weightlessness: Imitate Non-Self-Stabilizing Motions on Humanoid Robot
A weightlessness mechanism enables humanoid robots to dynamically relax joints for stable, contact-rich motions across diverse environments without task-specific tuning.
-
Trajectory-based actuator identification via differentiable simulation
Differentiable simulation enables torque-sensor-free actuator model identification from trajectory data, achieving 1.88x better position tracking than a stand-trained baseline and 46% longer travel in downstream locomotion policies.
-
RoSHI: A Versatile Robot-oriented Suit for Human Data In-the-Wild
RoSHI is a hybrid wearable that combines sparse IMUs and egocentric SLAM to capture accurate full-body 3D pose and shape data in natural environments for robot learning.
-
HAIC: Humanoid Agile Object Interaction Control via Dynamics-Aware World Model
HAIC enables robust humanoid interactions with underactuated objects by predicting their dynamics from proprioceptive history and using a world model for adaptive control.
-
HUSKY: Humanoid Skateboarding System via Physics-Aware Whole-Body Control
HUSKY combines humanoid-skateboard dynamics modeling with adversarial motion priors and physics-guided lean-to-steer strategies to achieve real-world stable skateboarding on a humanoid robot.
-
Commanding Humanoid by Free-form Language: A Large Language Action Model with Unified Motion Vocabulary
Humanoid-LLA converts unconstrained natural language commands into stable whole-body motions for humanoid robots using a unified motion vocabulary and two-stage supervised-plus-reinforcement fine-tuning.
-
SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control
Scaling motion tracking models along size, data volume, and compute produces a foundation model for natural, robust humanoid whole-body control with downstream uses in kinematic planning and vision-language-action models.
-
ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments
Dual-expert RL plus difficulty-aware multi-teacher distillation improves physics-based human–scene interaction imitation under complex 3D geometry versus prior single-policy baselines.
-
HiL-ResRL: A Model-Agnostic Finetuning Adapter via Human-in-the-loop Residual Reinforcement Learning
HiL-ResRL trains a model-agnostic residual policy on VLA actions using human-guided online RL, achieving over 95% success rate after 1.5 hours of real-robot training.
-
VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands
VAIC distills a teacher policy into a vision-and-proprioception student policy using recurrent adaptation and decoupled commands, enabling diverse real-robot tasks like box carrying and skateboarding that outperform baselines.
-
QuadVerse: An Integrated Framework Aligning Visual-Physical Reality for Quadruped Simulation
QuadVerse integrates 3D Gaussian Splatting scene reconstruction, friction calibration via trajectory search, and a residual dynamics compensator to improve quadruped simulation fidelity and enable zero-shot policy transfer.
-
TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation
TAM is a policy-agnostic torque adaptation module trained in randomized simulation that improves zero-shot real-robot performance on dynamic manipulation tasks compared to system identification and RMA baselines.
-
M3imic: Learning a Versatile Whole-Body Controller for Multimodal Motion Mimicking
M3imic unifies heterogeneous motion modalities via encoders into a shared latent space for a single RL-trained whole-body controller achieving high sim success and sim-to-real transfer on Unitree G1.
-
Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking
Humanoid-GPT is a causal Transformer pre-trained on a unified billion-scale motion dataset that tracks dynamic behaviors with zero-shot generalization to unseen motions and tasks.
-
SPRINT: Efficient Spectral Priors for Humanoid Athletic Sprints
SPRINT generates sprint trajectories for humanoids via spectral priors from five human motion sequences, achieving 6 m/s peak velocity with zero-shot sim-to-real transfer on Unitree G1.
-
Learning to Balance Motor Thermal Safety and Quadrupedal Locomotion Performance with Residual Policy
A thermal residual policy on a pre-trained quadruped locomotion controller prevents motor overheating under payload while preserving performance, lasting over 13 minutes on a Unitree A1 versus ~5 minutes for the nominal policy.
-
ParkourFormer: Integrating Predictive Supervision and Sequence Modeling into Parkour Locomotion
ParkourFormer achieves 93.85% average success on multi-terrain humanoid parkour by fusing Transformer sequence modeling with supervised future-state prediction.
-
MuGen: Multi-Skill Generative Locomotion Controller for Humanoid Robots
MuGen learns a generative latent representation of multi-skill humanoid locomotion from heterogeneous human data using VQ-VAEs and RL, then distills a deployable policy that tracks unseen motions and reuses the latent space.
-
Switch: Learning Agile Skills Switching for Humanoid Robots
Switch enables humanoid robots to perform agile, seamless transitions between locomotion skills via a kinematic skill graph, DRL tracking policy, and real-time graph-search scheduler.
-
UniCon: A Unified System for Efficient Robot Learning Transfers
UniCon standardizes states and control logic into modular execution graphs for efficient transfer of learning controllers across heterogeneous robots, with lower latency than ROS.
-
Learning Agile Striker Skills for Humanoid Soccer Robots from Noisy Sensory Input
A four-stage RL system with teacher-student distillation and online constrained adaptation enables humanoid robots to achieve robust ball-kicking accuracy under noisy perception in simulation and on physical hardware.
-
One-shot Adaptation of Humanoid Whole-body Motion with Walking Priors
A one-shot adaptation technique for humanoid whole-body motion that computes order-preserving optimal transport distances between walking and target sequences, interpolates geodesic intermediate poses, optimizes for collision-free retargeting, and adapts via reinforcement learning.
-
Natural Human Motion Recovery by Aligning High-Order Temporal Dynamics from Monocular Videos
HTD-Refine uses a temporal transformer (PVA-Net) to predict high-order dynamics and refines HMR outputs via optimization for more natural motion.
-
RPG: Robust Policy Gating for Smooth Multi-Skill Transitions in Humanoid Fighting
RPG trains a unified humanoid robot policy using motion and temporal randomization to achieve smooth, stable transitions between fighting skills and locomotion.
-
A Tutorial on World Models and Physical AI
A tutorial that unifies explicit and implicit world models through shared predictive structure for applications in physical AI such as robotics.
- GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning