Real-World Humanoid Locomotion with Reinforcement Learning
Reviewed by Pithpith:VHLLZOWDopen to challenge →
read the original abstract
Humanoid robots that can autonomously operate in diverse environments have the potential to help address labour shortages in factories, assist elderly at homes, and colonize new planets. While classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesize that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in-context, without updating its weights. We train our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deploy it to the real world zero-shot. Our controller can walk over various outdoor terrains, is robust to external disturbances, and can adapt in context.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
The Informational Cost of Agency: A Bounded Measure of Interaction Efficiency for Deployed Reinforcement Learning
Bipredictability P measures shared predictability in RL interaction loops with a universal upper bound of 0.5 that agency suppresses to 0.33, operationalized via an Information Digital Twin for faster detection of cou...
-
Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots
A single learned controller called MHC enables real humanoid robots to execute diverse whole-body behaviors from multi-modal inputs via masked target trajectories.
-
MuGen: Multi-Skill Generative Locomotion Controller for Humanoid Robots
MuGen learns a generative latent representation of multi-skill humanoid locomotion from heterogeneous human data using VQ-VAEs and RL, then distills a deployable policy that tracks unseen motions and reuses the latent space.
-
The Informational Cost of Agency: A Bounded Measure of Interaction Efficiency for Deployed Reinforcement Learning
Introduces Bipredictability P with a provable bound P ≤ 0.5 from entropy subadditivity, showing responsive agency imposes an informational cost by suppressing P to ~0.33, validated across RL agents and other systems, ...
-
No More Marching: Learning Humanoid Locomotion for Short-Range SE(2) Targets
Reinforcement learning with a constellation-based reward enables direct, efficient humanoid locomotion to short-range SE(2) targets, outperforming velocity-tracking baselines in simulation and transferring to hardware.
-
TACT-ful: Multi-Channel Terrain Affordance and Compliance Training for Payload-Robust Perceptive Humanoid Locomotion
A multi-channel terrain affordance reward combined with lower-body compliance training via virtual wrenches enables end-to-end PPO-trained humanoid policies to walk at 1 m/s on 0.2 m risers with improved payload robustness.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.