Pith. sign in

REVIEW 4 major objections 5 minor 19 cited by

Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Split control lets a humanoid walk and manipulate at once

desk verdict A genuinely new decoupled-control architecture with promising sim results, but the evaluation protocol correlates the motion prior with the locomotion command, leaving the teleoperation claim under-supported. read the letter →

arxiv 2412.07773 v2 pith:ZYXDVU5L submitted 2024-12-10 cs.RO cs.AIcs.LG

Chenhao Lu , Xuxin Cheng , Jialong Li , Shiqi Yang , Mazeyu Ji , Chengjing Yuan , Ge Yang , Sha Yi
show 1 more author
Xiaolong Wang
This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords humanoidwhole-bodycontrolloco-manipulationpredictivemotionpriorsconditionalvariationalautoencoderreinforcementlearningteleoperationinversekinematicsretargeting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Humanoid robots that can walk and use their arms at the same time usually sacrifice one ability for the other: end-to-end RL whole-body policies are robust but imprecise with high-DoF arms. This paper tries to get both by splitting the problem: the arms are controlled by inverse kinematics and motion retargeting for precision, and a reinforcement-learned policy controls only the legs. The two halves are tied together by a Predictive Motion Prior, a CVAE latent vector that tells the leg policy what the upper body is about to do. The authors report that this prior is what keeps the robot stable under arm motion, and that the system tracks upper-body targets with roughly 25–40% lower error than whole-body RL baselines. The payoff is a teleoperation setup in which one person drives the robot with pedals and gestures while it walks through real tasks like opening a fridge and riding an elevator.

What carries the argument

The central object is the Predictive Motion Prior (PMP), a 64-dimensional latent vector sampled from a conditional variational autoencoder. The CVAE has a prior network, an encoder, and a decoder, each a three-layer MLP; given a one-second window of past upper-body joint angles, the prior network produces a distribution over the latent vector that the decoder turns into a prediction of the next second of upper-body motion. The locomotion policy receives this latent vector as an auxiliary observation on top of proprioception and gait signals. During training, upper-body target joints are set directly from retargeted human motion, scaled by a curriculum factor alpha that starts low and increases when an episode is survived; the leg policy is trained with PPO on lower-body rewards.

What would settle it

Have a teleoperator or script drive the arms through movements far outside the retargeted human-motion distribution, such as fast asymmetric reaches or high-frequency oscillations, while the robot walks, and compare projected-gravity error or fall rate with and without the motion prior; if the prior gives no stability benefit on those trajectories, the paper's central claim is contradicted.

Watch

Extended reading notes

Core claim

The paper claims that precise arm control and robust walking are not competing objectives that must be traded inside one RL policy; they can be decoupled and reconnected by a learned representation. Upper-body motion is commanded directly through inverse kinematics and retargeting, giving the arms exact joint targets, while a lower-body RL policy trained with PPO handles velocity, orientation, and height tracking. The bridge is the Predictive Motion Prior: a CVAE trained on retargeted human motion produces a latent code that predicts the near-future upper-body motion, and this latent code is given to the leg policy as an observation. In simulation, this configuration reduces upper-body joint-position error by roughly 25–40% relative to whole-body RL baselines while keeping stability metrics comparable, and on the real H1 robot the system walks while opening a fridge, riding an elevator, and carrying objects.

Load-bearing premise

The key assumption is that the learned motion prior, trained on retargeted human motion, continues to produce useful predictions when a teleoperator drives the arms in ways the training data did not include.

Editorial extensions

If this is right

  • Operators can command locomotion with pedals or a joystick and arms with a VR headset, without attending to balance.
  • High-DoF arms with dexterous hands can be used while the robot is walking, enabling tasks such as fridge-opening, elevator-riding, and object carrying.
  • RL training is simplified because the policy only controls the twelve lower-body joints, while precise arm control comes from IK and retargeting.
  • The motion prior and curriculum improve recovery from pushes, especially while standing, cutting recovery time in the reported tests.
  • The approach transfers to a second humanoid platform (GR1) in simulation, indicating the control structure is not specific to one robot.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The real-world demos show teleoperators adapting to the robot; a sterner test of the prior would separate the prior's contribution from the operator's own corrective behavior by using scripted arm trajectories.
  • The same decoupling recipe could extend to other mobile manipulators: any system with a precise arm controller and a learned base policy could condition the base on a predictive latent of arm motion.
  • Because the leg policy is driven by velocity commands, the approach could be paired with navigation stacks or learned task planners, not just a human teleoperator, to keep balance during planned arm movements.
  • If the prior generalizes poorly to novel arm motion, a useful comparison would be to condition the leg policy directly on IK targets or on a short history of arm joint commands, testing whether the latent code itself or the predictive information is doing the stabilizing work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a decoupled whole-body control architecture for humanoid robots: upper-body manipulation is controlled by inverse kinematics (IK) and motion retargeting, while a reinforcement learning policy controls the lower body for locomotion. To make the lower-body policy aware of upper-body dynamics, the authors introduce a Predictive Motion Prior (PMP), a CVAE latent representation of future upper-body motion that is fed as an observation to the locomotion policy. Experiments in simulation compare PMP against whole-body RL baselines (ExBody and variants) on H1 and GR1, and real-world teleoperation demos show the H1 robot performing tasks such as opening a fridge and using an elevator while walking. The paper claims PMP improves stability and maintains high manipulation precision relative to RL-based whole-body control.

Significance. The decoupling idea is practically attractive: it sidesteps the difficulty of high-DoF arm control in RL while retaining learned robustness for the legs. If the PMP benefit is real, the system would be a notable step toward practical humanoid teleoperation. The paper has clear strengths: a clean three-stage pipeline, a learned representation with a genuine reconstruction objective (so the evaluation is not circular), comparisons across two robot platforms, and real-world teleoperation demos. However, the current evidence has a load-bearing gap: the simulation evaluation couples the locomotion command to the same mocap sequence that generates the upper-body motion, which may inflate the benefit of a predictive prior. The H1 velocity-tracking results are also weaker than the baselines despite the text claiming comparability, and the GR1 baseline is a reimplementation without details. These issues make the central claim plausible but not yet fully supported.

major comments (4)
  1. [Section IV-A, Table II, Figure 3] The evaluation protocol couples the upper-body motion and the locomotion command because both are derived from the same mocap recording, as stated in the captions of Table II and Figure 3: 'the locomotion commands are from the base state of the corresponding motion.' Since the CVAE prior z_t is trained to predict future upper-body motion from past motion, it is informative about both the upcoming upper-body motion and the correlated velocity command. In teleoperation (Section IV-B), the operator's arm motions and the pedal/joystick command are independent, so any pairing of upper-body trajectory and velocity command can occur. The robustness tests in Figure 3 vary only playback speed and push velocity, leaving the command-motion association intact. This is a load-bearing gap: the measured stability improvement of PMP over 'w/o motion prior' (e.g., Eg 1.301 vs 1.585 on H1 in Table II) may not transfer to teleoperation. Please add an evaluation that decouples the command from the motion sequence (e.g., randomly pairing upper-body trajectories with commands from different mocap clips) and report whether the PMP benefit persists.
  2. [Section IV-A, Table II(a)] The claim that PMP 'achieves performance metrics comparable to the best baseline' is not supported for linear velocity tracking on H1: Ours (PMP) reports Evel = 0.01296, while Exbody and Exbody (Whole) report 0.00784 and 0.00782, respectively—roughly 65% worse. Additionally, no standard deviations or confidence intervals are reported for any metric in Table II or Figure 3, so the statistical significance of the differences (including the Eg improvement) cannot be assessed. Please report variance across seeds and across motion sequences.
  3. [Section IV-A] The GR1 baseline 'Exbody (Reimplementation)' is stated to be a reimplementation, but no details are given about its fidelity to the original ExBody method, hyperparameters, reward calibration, or training procedure, and the code is not provided. The GR1 results also show only a small improvement of PMP over 'Ours w/o motion prior' in Eg (0.420 vs 0.442), which weakens the cross-robot generality claim. Please either use the original ExBody implementation or provide reimplementation details and multiple random seeds to demonstrate that the baseline is fair.
  4. [Section IV-C, Figure 1] The central claim that PMP enables precise manipulation while walking in the real world is supported only by qualitative demos and by push-robustness tests in Table III that do not involve manipulation. Table IV measures arm tracking error but is not tied to locomotion state and does not compare PMP with 'w/o motion prior'. Provide a real-world comparison that isolates the effect of the motion prior during teleoperation, for example by measuring stability or task success with and without PMP under operator-generated arm motions and independent locomotion commands.
minor comments (5)
  1. [Section III-A] The notation M 0_t and M 1_t is introduced as 'two consecutive motion sequence'; please fix the grammar and define the superscripts explicitly (past vs. future windows) to avoid ambiguity.
  2. [Figure 3] The axes labels 'Push Velocity' and 'Motion Speed' lack units and the tested ranges are not stated in the caption; please specify the units and ranges so that the disturbance levels are reproducible.
  3. [Section IV-B] The unified teleoperation setup with pedals is described, but the mapping from pedal inputs to the goal space ⟨v, rpy, h⟩ is not specified; clarify how the pedals generate linear/angular velocity commands and whether orientation/height are also controlled.
  4. [Table I] The table uses the abbreviation 'PrManip' without defining it in the caption; please define it as 'precise manipulation' and note the DoF counts for arms are not the same as the total number of upper-body joints reported elsewhere.
  5. [References] Reference [8] is cited as 'Learning deep sensorimotor policies for vision-based autonomous drone racing,' which appears unrelated to the context of legged locomotion and whole-body control; please verify and replace or reposition the citation.

Circularity Check

1 steps flagged · score 6.0 of 10

Simulation evaluation ties the locomotion command to the same mocap sequence as the upper-body motion, so the reported PMP stability benefit may be a command-leak artifact rather than a transferable robustness improvement.

  1. other [Section IV-A (Table II caption and Fig. 3 caption), with the training protocol in Section III-B]
    "We sample 5 trajectories for each motion from the dataset in simulation and report their mean episode metrics. The locomotion commands are from the base state of the corresponding motion."

    Section III-B trains the lower-body policy by sampling an upper-body sequence M_t from the retargeted dataset M and setting it as the PD target; Section IV-A evaluates with the locomotion command taken "from the base state of the corresponding motion." Thus at every simulated timestep the same recorded sequence determines both q^upper_t and v_t. The CVAE prior z_t (Eqs. 1-2) is trained to encode the future upper-body frames M^1_t of that same sequence, so z_t is also predictive of the command derived from that sequence. The reported PMP stability advantage (Table II: Eg 1.301 vs 1.585 for H1; Fig. 3) can therefore be produced by the policy reading v_t out of z_t rather than by more robust balance. In deployment (Sec.

full rationale

The CVAE itself is a legitimate learned representation: its training objective is reconstruction of future upper-body motion, not stability, and no parameter is fitted directly to the stability metric. The real-world teleoperation results are qualitative demonstrations and do not rest on a circular derivation. The main circularity-adjacent defect is that the simulation protocol supplying the quantitative evidence for the central stability claim couples the motion prior's input (upper-body motion) to the locomotion command derived from the same dataset sequence. This makes the PMP-versus-no-prior comparison in simulation a test of in-distribution command correlation rather than of the deployment scenario where arm motion and velocity commands are independent. That is a construction-level confound in one of the paper's central quantitative predictions, so the score is 6 rather than higher; the method still has independent content in its IK/retargeting upper-body control, RL lower-body policy, and system integration. Self-citations to ExBody for retargeting and reward design are methodological and not load-bearing for the core claim.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

This ledger records the data, modeling, and generalization assumptions the central claim rests on. The paper introduces no new physical entities; the PMP latent is a learned representation with internal evidence (reconstruction loss and ablations). The main free parameters are the CVAE architecture choices and the unstated RL reward and PD gains, which a replication would need to reconstruct.

free parameters (5)
  • latent dimension H = 64
    Size of the CVAE motion prior latent vector; a design choice that shapes the information passed to the locomotion policy, stated without search or justification in Section III-A.
  • time window W = 50 (H1), 100 (GR1)
    Window length for past and future motion sequences, set to correspond to 1 second of real time at each robot's control frequency (Section III-A); affects how much future context the prior provides.
  • curriculum difficulty factor alpha_i = Per-motion, updated by +0.05 on 90% survival, -0.01 otherwise
    Adaptive difficulty scaling of upper-body motion amplitude during RL training (Eq. 3-4); a free per-motion parameter that heavily influences what the policy learns.
  • reward weights = unstated
    The RL reward follows ExBody with additional gait periodicity rewards, but the specific weights and reward terms are not provided (Section III-B), leaving them effectively free parameters for any replication.
  • PD gains = unstated
    Joint PD gains for upper and lower body are not reported; they are critical for the direct upper-body tracking claim and affect lower-body stiffness.
assumptions (5)
  • domain assumption The retargeted human motion dataset M is sufficiently diverse and representative of the upper-body motions needed for the target tasks.
    Introduced in Section III, the CVAE and RL policy both depend on this dataset; if teleoperation motions differ from it, the prior and curriculum may not help.
  • domain assumption Decoupling upper and lower body, with upper-body motions treated as external disturbances, is sufficient; the motion prior and current proprioception capture all relevant coupling.
    Core modeling choice in Sections II and III; the lower-body policy never sees upper-body torques or true interaction forces, only joint positions, velocities, and the prior.
  • domain assumption The CVAE prior model R generalizes to teleoperator-generated arm motions not seen in training.
    Used at deployment (Section III-A); without this, the lower-body policy obtains unreliable predictions and may lose the stability benefit.
  • domain assumption The sim-to-real gap for the lower-body RL policy is small enough that the trained controller transfers to the real H1 without additional mechanism described.
    Real-world results in Section IV-C rely on this; the paper does not describe domain randomization or system identification.
  • standard math PPO with the stated reward and curriculum converges to a policy that generalizes across the motion dataset.
    Standard RL assumption invoked in Section III-B.
invented entities (1)
  • PMP latent vector z_t
    purpose: Compact representation of predicted future upper-body motion, fed as an observation to the lower-body RL policy to anticipate balance disturbances.
    The latent is a learned representation, not a physical entity. Its evidence base is internal: the CVAE is trained with reconstruction and KL losses on a retargeted motion dataset, and the ablation without the motion prior degrades stability metrics. There is no externally falsifiable handle outside the paper's own evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control." pith.science (2026). https://pith.science/paper/ZYXDVU5L

@misc{pith2026241207773,
  author       = {Pith},
  title        = {Pith review of: Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZYXDVU5L}},
  note         = {Machine review of arXiv:2412.07773}
}
read the original abstract

Humanoid robots require both robust lower-body locomotion and precise upper-body manipulation. While recent Reinforcement Learning (RL) approaches provide whole-body loco-manipulation policies, they lack precise manipulation with high DoF arms. In this paper, we propose decoupling upper-body control from locomotion, using inverse kinematics (IK) and motion retargeting for precise manipulation, while RL focuses on robust lower-body locomotion. We introduce PMP (Predictive Motion Priors), trained with Conditional Variational Autoencoder (CVAE) to effectively represent upper-body motions. The locomotion policy is trained conditioned on this upper-body motion representation, ensuring that the system remains robust with both manipulation and locomotion. We show that CVAE features are crucial for stability and robustness, and significantly outperforms RL-based whole-body control in precise manipulation. With precise upper-body motion and robust lower-body locomotion control, operators can remotely control the humanoid to walk around and explore different environments, while performing diverse manipulation tasks.

Figures

Figures reproduced from arXiv: 2412.07773 by the authors.

Figure 1
Figure 1. Humanoid robot doing whole-body tasks that require both precise manipulation and robust locomotion. The robot [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The training pipeline consists of three stages: (a) preprocessing of the motion dataset by mapping local rotation, (b) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Evaluation for precision (Top) and stability (Bottom) under disturbance (lower is better for all figures). We sample [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Left: A unified teleoperation setup using Apple [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Robustness testing. Left: robot being pushed while [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation

    cs.RO 2025-10 conditional novelty 6.0 of 10

    A 10,300-demonstration, 260-task multimodal humanoid manipulation dataset with baseline policy evaluations and a cloud evaluation platform.

  2. TOP: Time Optimization Policy for Stable and Accurate Standing Manipulation with Humanoid Robots

    cs.RO 2025-08 conditional novelty 6.0 of 10

    A reinforcement-learned time optimization policy that adaptively slows upper-body motion clips improves stability and precision of humanoid standing manipulation at a modest time cost.

  3. Robot Drummer: Learning Rhythmic Skills for Humanoid Drumming

    cs.RO 2025-07 conditional novelty 6.0 of 10

    A simulated Unitree G1 humanoid learns to drum dozens of popular songs from MIDI with high F1 scores using a Rhythmic Contact Chain and temporal decomposition.

  4. Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining

    cs.RO 2025-06 conditional novelty 6.0 of 10

    Pretraining a modular transformer policy on human demonstrations then finetuning on a small robot dataset improves success on six real quadruped manipulation tasks, including out-of-distribution objects.

  5. GMT: General Motion Tracking for Humanoid Whole-Body Control

    cs.RO 2025-06 conditional novelty 6.0 of 10

    GMT trains a single unified humanoid policy using adaptive sampling and mixture-of-experts, achieving lower tracking errors than a re-implemented ExBody2 across diverse whole-body motions.

  6. Versatile Loco-Manipulation through Flexible Interlimb Coordination

    cs.RO 2025-06 conditional novelty 6.0 of 10

    ReLIC lets a robot dog dynamically reassign its legs between walking and manipulating, achieving 78.9% average success across 12 real-world loco-manipulation tasks.

  7. Hold My Beer: Learning Gentle Humanoid Locomotion and End-Effector Stabilization Control

    cs.RO 2025-05 conditional novelty 6.0 of 10

    A slow-fast two-agent reinforcement learning architecture with separate upper- and lower-body policies reduces end-effector shaking during humanoid locomotion.

  8. Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning

    cs.RO 2025-05 conditional novelty 6.0 of 10

    PDC trains a single egocentric-vision policy that lets a simulated humanoid search for, grasp, and place objects and open drawers without privileged state information.

  9. TWIST: Teleoperated Whole-Body Imitation System

    cs.RO 2025-05 conditional novelty 6.0 of 10

    Human MoCap drives a Unitree G1 humanoid in real time through a single teacher-student RL+BC controller that transfers zero-shot from simulation.

  10. A Unified and General Humanoid Whole-Body Controller for Versatile Locomotion

    cs.RO 2025-02 conditional novelty 6.0 of 10

    A single RL policy controls walking, jumping, and standing gaits of a humanoid with tunable foot and posture parameters, plus a separate policy for hopping, and supports real-time upper-body intervention for loco-mani...

  11. Towards Miniature Humanoid Tele-Loco-Manipulation Using Virtual Reality and Reinforcement Learning

    cs.RO 2026-07 conditional novelty 5.0 of 10

    A VR-teleoperated, reinforcement-learning-balanced control stack lets a miniature ROBOTIS OP3 humanoid walk and manipulate objects simultaneously.

  12. REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

    cs.RO 2026-03 conditional novelty 5.0 of 10

    Jointly fine-tuning a diffusion-policy high-level planner and an RL low-level controller raises humanoid loco-manipulation success rates from ~50-70% to >90% in simulation.

  13. Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots

    cs.RO 2025-07 conditional novelty 5.0 of 10

    A humanoid-specific multimodal occupancy perception system with a new dataset, sensor layout, and a fusion network that claims state-of-the-art results on its own benchmark.

  14. EMP: Executable Motion Prior for Humanoid Robot Standing Upper-body Motion Imitation

    cs.RO 2025-07 conditional novelty 5.0 of 10

    A state-conditioned executable motion prior network modifies upper-body motion targets so a humanoid can imitate human gestures while maintaining balance.

  15. AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation

    cs.RO 2025-07 conditional novelty 5.0 of 10

    AC-DiT adds mobility-to-body conditioning and perception-aware 2D/3D weighting to a diffusion transformer, improving success rates on simulated and real-world mobile manipulation tasks.

  16. SMAP: Self-supervised Motion Adaptation for Physically Plausible Humanoid Whole-body Control

    cs.RO 2025-05 conditional novelty 5.0 of 10

    SMAP uses a vector-quantized periodic autoencoder to adapt human motion into physically plausible humanoid motion, then distills an RL teacher policy into a student policy for whole-body control.

  17. AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control

    cs.RO 2025-05 conditional novelty 5.0 of 10

    A hybrid trajectory-optimization and RL framework lets a humanoid robot flex its torso and legs to reach and manipulate objects beyond the range of prior controllers.

  18. Embrace Collisions: Humanoid Shadowing for Deployable Contact-Agnostics Motions

    cs.RO 2025-02 conditional novelty 5.0 of 10

    A whole-body reinforcement-learning controller, trained only in simulation, lets a Unitree G1 humanoid perform extreme contact-agnostic motions such as getting up from the ground and breaking-dance moves in the real world.

  19. ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills

    cs.RO 2025-02 conditional novelty 5.0 of 10

    ASAP trains a residual action model on real-world rollouts and fine-tunes simulation policies through it, reducing humanoid whole-body motion tracking error in sim-to-real transfer.

Reference graph

Works this paper leans on

47 extracted references · 29 canonical work pages · cited by 19 Pith papers

  1. [1]

    Humanplus: Humanoid shadowing and imitation from humans,

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Humanoid shadowing and imitation from humans,” 2024. [Online]. Available: https://arxiv.org/abs/2406.10454 1, 2, 4, 6

    arXiv 2024

  2. [2]

    Expressive whole-body control for humanoid robots,

    X. Cheng, Y . Ji, J. Chen, R. Yang, G. Yang, and X. Wang, “Expressive whole-body control for humanoid robots,” 2024. [Online]. Available: https://arxiv.org/abs/2402.16796 1, 2, 4, 6

    arXiv 2024

  3. [3]

    Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning,

    T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. Kitani, C. Liu, and G. Shi, “Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning,” 2024. [Online]. Available: https://arxiv.org/abs/2406.08858 1, 2, 4, 6

    arXiv 2024

  4. [4]

    Amass: Archive of motion capture as surface shapes,

    N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black, “Amass: Archive of motion capture as surface shapes,” in The IEEE International Conference on Computer Vision (ICCV) , Oct

    work page

  5. [5]

    Open-television: Teleoperation with immersive active visual feedback,

    X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang, “Open-television: Teleoperation with immersive active visual feedback,” CoRL, 2024. 1, 4, 5, 6

    work page 2024

  6. [6]

    Using apple vision pro to train and control robots,

    Y . Park and P. Agrawal, “Using apple vision pro to train and control robots,” 2024. [Online]. Available: https://github.com/Improbable-AI/ VisionProTeleop 1

    work page 2024

  7. [7]

    Visual whole-body control for legged loco-manipulation,

    M. Liu, Z. Chen, X. Cheng, Y . Ji, R. Qiu, R. Yang, and X. Wang, “Visual whole-body control for legged loco-manipulation,” CoRL,

    work page

  8. [8]

    Learning deep sensorimotor policies for vision-based autonomous drone racing,

    J. Fu, Y . Song, Y . Wu, F. Yu, and D. Scaramuzza, “Learning deep sensorimotor policies for vision-based autonomous drone racing,”

    work page

Show all 47 references
  1. [9]

    Humor: 3d human motion model for robust pose estimation,

    D. Rempe, T. Birdal, A. Hertzmann, J. Yang, S. Sridhar, and L. J. Guibas, “Humor: 3d human motion model for robust pose estimation,” in International Conference on Computer Vision (ICCV) , 2021. 3

    2021

  2. [10]

    Universal humanoid motion representations for physics-based control,

    Z. Luo, J. Cao, J. Merel, A. Winkler, J. Huang, K. Kitani, and W. Xu, “Universal humanoid motion representations for physics-based control,” arXiv preprint arXiv:2310.04582 , 2023. 3

    2023 arXiv

  3. [11]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. 3

    2017 arXiv

  4. [12]

    Whole-body humanoid robot locomotion with human reference,

    Q. Zhang, P. Cui, D. Yan, J. Sun, Y . Duan, A. Zhang, and R. Xu, “Whole-body humanoid robot locomotion with human reference,” arXiv preprint arXiv:2402.18294 , 2024. 4

    2024 arXiv

  5. [13]

    Walk these ways: Tuning robot control for generalization with multiplicity of behavior,

    G. B. Margolis and P. Agrawal, “Walk these ways: Tuning robot control for generalization with multiplicity of behavior,” in Conference on Robot Learning . PMLR, 2023, pp. 22–31. 4

    2023

  6. [14]

    Dynamic walk of a biped,

    H. Miura and I. Shimoyama, “Dynamic walk of a biped,” IJRR, 1984. 6

    1984

  7. [15]

    Simbicon: Simple biped locomotion control,

    K. Yin, K. Loken, and M. Van de Panne, “Simbicon: Simple biped locomotion control,” ACM Transactions on Graphics , 2007. 6

    2007

  8. [16]

    Anymal-a highly mobile and dynamic quadrupedal robot,

    M. Hutter, C. Gehring, D. Jud, A. Lauber, C. D. Bellicoso, V . Tsounis, J. Hwangbo, K. Bodie, P. Fankhauser, M. Bloesch et al. , “Anymal-a highly mobile and dynamic quadrupedal robot,” in IROS, 2016. 6

    2016

  9. [17]

    Whole-body control of humanoid robots,

    F. L. Moro and L. Sentis, “Whole-body control of humanoid robots,” Humanoid Robotics: A reference, Springer , Dordrecht , 2019. 6

    2019

  10. [18]

    Whole body humanoid control from human motion descriptors,

    B. Dariush, M. Gienger, B. Jian, C. Goerick, and K. Fujimura, “Whole body humanoid control from human motion descriptors,” in2008 IEEE International Conference on Robotics and Automation . IEEE, 2008, pp. 2677–2684. 6

    2008

  11. [19]

    The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation,

    S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa, “The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation,” in Proceedings 2001 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems. Expanding the Societal Rol...

    2001

  12. [20]

    Hybrid zero dynamics of planar biped walkers,

    E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek, “Hybrid zero dynamics of planar biped walkers,” IEEE transactions on automatic control, vol. 48, no. 1, pp. 42–56, 2003. 6

    2003

  13. [21]

    Rapid locomotion via reinforcement learning,

    G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal, “Rapid locomotion via reinforcement learning,” arXiv preprint arXiv:2205.02824, 2022. 6

    2022 arXiv

  14. [22]

    Rma: Rapid motor adaptation for legged robots,

    A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,” arXiv preprint arXiv:2107.04034 , 2021. 6

    2021 arXiv

  15. [23]

    Minimizing energy consumption leads to the emergence of gaits in legged robots,

    Z. Fu, A. Kumar, J. Malik, and D. Pathak, “Minimizing energy consumption leads to the emergence of gaits in legged robots,” Conference on Robot Learning (CoRL) , 2021. 6

    2021

  16. [24]

    Adversarial motion priors make good substitutes for complex reward functions. 2022 ieee,

    A. Escontrela, X. B. Peng, W. Yu, T. Zhang, A. Iscen, K. Goldberg, and P. Abbeel, “Adversarial motion priors make good substitutes for complex reward functions. 2022 ieee,” in International Conference on Intelligent Robots and Systems (IROS) , vol. 2, 2022. 6

    2022

  17. [25]

    Reinforcement learning for robust parameterized locomotion control of bipedal robots,

    Z. Li, X. Cheng, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Reinforcement learning for robust parameterized locomotion control of bipedal robots,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 2811–2817. 6

    2021

  18. [26]

    Blind bipedal stair traversal via sim-to-real reinforcement learning,

    J. Siekmann, K. Green, J. Warila, A. Fern, and J. Hurst, “Blind bipedal stair traversal via sim-to-real reinforcement learning,” arXiv preprint arXiv:2105.08328, 2021. 6

    2021 arXiv

  19. [27]

    Extreme parkour with legged robots,

    X. Cheng, K. Shi, A. Agarwal, and D. Pathak, “Extreme parkour with legged robots,” 2023. [Online]. Available: https: //arxiv.org/abs/2309.14341 6

    2023 arXiv

  20. [28]

    Legs as manipulator: Pushing quadrupedal agility beyond locomotion,

    X. Cheng, A. Kumar, and D. Pathak, “Legs as manipulator: Pushing quadrupedal agility beyond locomotion,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023. 6

    2023

  21. [29]

    Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,

    H. Ha, Y . Gao, Z. Fu, J. Tan, and S. Song, “Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers,” arXiv preprint arXiv:2407.10353 , 2024. 6

    2024 arXiv

  22. [30]

    Learning human-to-humanoid real-time whole-body teleoperation,

    T. He, Z. Luo, W. Xiao, C. Zhang, K. Kitani, C. Liu, and G. Shi, “Learning human-to-humanoid real-time whole-body teleoperation,” arXiv preprint arXiv:2403.04436 , 2024. 6

    2024 arXiv

  23. [31]

    Wococo: Learning whole- body humanoid control with sequential contacts,

    C. Zhang, W. Xiao, T. He, and G. Shi, “Wococo: Learning whole- body humanoid control with sequential contacts,” arXiv e-prints , pp. arXiv–2406, 2024. 6

    2024

  24. [32]

    Locoman: Advancing versatile quadrupedal dexterity with lightweight loco-manipulators,

    C. Lin, X. Liu, Y . Yang, Y . Niu, W. Yu, T. Zhang, J. Tan, B. Boots, and D. Zhao, “Locoman: Advancing versatile quadrupedal dexterity with lightweight loco-manipulators,” arXiv preprint arXiv:2403.18197,

    arXiv

  25. [33]

    Dribblebot: Dynamic legged manipulation in the wild,

    Y . Ji, G. B. Margolis, and P. Agrawal, “Dribblebot: Dynamic legged manipulation in the wild,” arXiv preprint arXiv:2304.01159 , 2023. 6

    2023 arXiv

  26. [34]

    Learn- ing whole-body manipulation for quadrupedal robot,

    S. Jeon, M. Jung, S. Choi, B. Kim, and J. Hwangbo, “Learn- ing whole-body manipulation for quadrupedal robot,” arXiv preprint arXiv:2308.16820, 2023. 6

    2023 arXiv

  27. [35]

    Learning human-to-humanoid real-time whole-body teleoperation,

    T. He, Z. Luo, W. Xiao, C. Zhang, K. Kitani, C. Liu, and G. Shi, “Learning human-to-humanoid real-time whole-body teleoperation,” in arXiv, 2024. 6

    2024

  28. [36]

    Combining learning-based locomotion policy with model-based manipulation for legged mobile manipulators,

    Y . Ma, F. Farshidian, T. Miki, J. Lee, and M. Hutter, “Combining learning-based locomotion policy with model-based manipulation for legged mobile manipulators,” RA-L, 2022. 6

    2022

  29. [37]

    Roboduet: A framework affording mobile-manipulation and cross- embodiment,

    G. Pan, Q. Ben, Z. Yuan, G. Jiang, Y . Ji, J. Pang, H. Liu, and H. Xu, “Roboduet: A framework affording mobile-manipulation and cross- embodiment,” arXiv preprint arXiv:2403.17367 , 2024. 6

    2024 arXiv

  30. [38]

    Deep imitation learning for humanoid loco-manipulation through human teleoperation,

    M. Seo, S. Han, K. Sim, S. H. Bang, C. Gonzalez, L. Sentis, and Y . Zhu, “Deep imitation learning for humanoid loco-manipulation through human teleoperation,” in 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids) . IEEE, 2023, pp. 1–8. 6

    2023

  31. [39]

    Sim-to-real learning for humanoid box loco-manipulation,

    J. Dao, H. Duan, and A. Fern, “Sim-to-real learning for humanoid box loco-manipulation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 16 930–16 936. 6

    2024

  32. [40]

    Amp: Adversarial motion priors for stylized physics-based character con- trol,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character con- trol,” ACM Transactions on Graphics (ToG) , vol. 40, no. 4, pp. 1–20,

  33. [41]

    Ase: Large- scale reusable adversarial skill embeddings for physically simulated characters,

    X. B. Peng, Y . Guo, L. Halper, S. Levine, and S. Fidler, “Ase: Large- scale reusable adversarial skill embeddings for physically simulated characters,” ACM Trans. Graph. , vol. 41, no. 4, Jul. 2022. 6

    2022

  34. [43]

    Synthesizing physical character-scene interactions,

    M. Hassan, Y . Guo, T. Wang, M. Black, S. Fidler, and X. B. Peng, “Synthesizing physical character-scene interactions,” 2023. [Online]. Available: https://doi.org/10.1145/3588432.3591525 6

    2023

  35. [44]

    Universal humanoid motion representations for physics-based control,

    Z. Luo, J. Cao, J. Merel, A. Winkler, J. Huang, K. M. Kitani, and W. Xu, “Universal humanoid motion representations for physics-based control,” in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https: //openreview.net/forum?id=OrO...

    2024

  36. [45]

    Character controllers using motion vaes,

    H. Y . Ling, F. Zinno, G. Cheng, and M. Van De Panne, “Character controllers using motion vaes,” ACM Transactions on Graphics (TOG), vol. 39, no. 4, pp. 40–1, 2020. 6

    2020

  37. [46]

    Learning physically simulated tennis skills from broadcast videos,

    H. Zhang, Y . Yuan, V . Makoviychuk, Y . Guo, S. Fidler, X. B. Peng, and K. Fatahalian, “Learning physically simulated tennis skills from broadcast videos,” ACM Trans. Graph. , vol. 42, no. 4, jul 2023. [Online]. Available: https://doi.org/10.1145/3592408 6

    2023 doi

  38. [47]

    Strategy and skill learning for physics-based table tennis animation,

    J. Wang, J. Hodgins, and J. Won, “Strategy and skill learning for physics-based table tennis animation,” in ACM SIGGRAPH 2024 Conference Papers, 2024, pp. 1–11. 6

    2024

  39. [2019]

    Available: https://amass.is.tue.mpg.de 1

    [Online]. Available: https://amass.is.tue.mpg.de 1

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.