Pith. sign in

REVIEW 4 major objections 4 minor 47 references

MASH: Cooperative-Heterogeneous Multi-Agent Reinforcement Learning for Single Humanoid Robot Locomotion

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Treating each limb as an independent learning agent produces faster, smoother humanoid walking than training the whole body as one agent.

desk verdict Plausible extension of limb-as-agent MARL to humanoids, with a nice hardware demo, but the central comparison to single-agent PPO is confounded by an unspecified baseline and a privileged critic. read the letter →

arxiv 2508.10423 v1 pith:XZRHKSZS submitted 2025-08-14 cs.RO cs.AIcs.SYeess.SY

Qi Liu , Xiaopeng Zhang , Mingshan Tan , Shuaikang Ma , Jinliang Ding , Yanjie Li This is my paper · ORCID
classification cs.ROcs.AIcs.SYeess.SY
keywords humanoidlocomotionmulti-agentreinforcementlearningMAPPOcentralizedtrainingdecentralizedexecutioncooperativeagentssim-to-realtransferwhole-bodycontrolgaitcoordination
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most humanoid locomotion policies are trained as a single reinforcement-learning agent that controls the whole body. MASH instead treats the two legs and two arms as four independent agents that share a global critic, and is trained with multi-agent PPO under centralized training with decentralized execution. The paper claims this accelerates convergence, raises asymptotic reward, and improves limb coordination compared with single-agent PPO, and that the resulting policy transfers to a physical robot. The point, if right, is that decomposing one robot into cooperating limb-agents can be a better inductive bias for whole-body control than monolithic policies.

What carries the argument

The load-bearing object is the decPOMDP reformulation of a single body: each limb is an agent, with shared-parameter actor networks across symmetric limbs, a global critic under centralized training with decentralized execution (CTDE), and a temporal director that injects gait phase $T_i(t) = \sin(2\pi(kt + \Delta_i))$. The shared actors exploit left–right symmetry; the global critic provides a cooperative value signal; the phase offsets synchronize stepping and arm swing.

What would settle it

Train the single-agent PPO baseline with the same 106-dimensional privileged observations used by MASH's critic (and the same reward), then compare convergence and deployment metrics; if the gap collapses, the multi-agent design is not the cause. A second check is to strip MASH's global critic down to per-limb local critics and see whether the coordination metrics hold.

Watch

Extended reading notes

Core claim

The central claim is that the MDP for a single humanoid can be reformulated as a decPOMDP whose agents are the limbs, and that MAPPO with a shared global critic learns a better walking policy than single-agent PPO. Each leg agent observes joint positions, velocities, previous action, timing phase, torso orientation, and its own one-hot ID (32 dimensions); each arm agent observes a 26-dimensional analogue; and the critic sees a 106-dimensional global state including forces, friction, mass, and contact masks. Two shared-parameter actors—one for legs, one for arms—are trained with a shared reward and a temporal director $T_i(t) = \sin(2\pi(kt + \Delta_i))$ that encodes gait phase. Experiments r

Load-bearing premise

The single-agent PPO baseline is given the same reward and equivalent information, so the reported gains come from the multi-agent decomposition rather than from MASH's critic seeing privileged global state.

Editorial extensions

If this is right

  • MARL becomes a viable paradigm for single-robot control, not just multi-robot teams.
  • Shared-parameter actors with a phase-offset temporal director can encode gait symmetry and inter-limb coordination directly into the policy architecture.
  • Domain randomization combined with MAPPO supports sim-to-real transfer for humanoid walking.
  • Faster convergence means lower sample complexity for high-dimensional humanoid control tasks.
  • The same decomposition could apply to other robot configurations, as the paper states as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported advantage may partly reflect information asymmetry: the single-agent PPO baseline likely lacked the global critic's privileged state, so the gain could come from extra observations rather than from multi-agent credit assignment.
  • A cleaner test of the cooperation story is to ablate the global critic: if per-limb local critics perform as well, the symmetry and parameter sharing may be doing the work, not the centralized value function.
  • The temporal director is effectively a gait rhythm prior; with more limbs or DOF, learning the phase offsets rather than hand-setting them could be a natural extension.
  • If the limb-decomposition result holds across robots, it suggests whole-body loco-manipulation—where arms already have distinct roles—could inherit the same architecture without retraining from scratch.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes MASH, a cooperative-heterogeneous MARL framework for a single humanoid robot. Each limb (two legs, two arms) is treated as an independent agent, with parameter sharing across symmetric limbs and a centralized global critic under the CTDE paradigm. The method is evaluated in Isaac Gym against a single-agent PPO baseline on bipedal and arm-swing walking, with metrics for convergence time, action smoothness, torso stability, and limb coordination, followed by a real-world deployment demonstrating a periodic knee trajectory. The central claim is that modeling the locomotion problem as cooperative MARL accelerates training convergence and improves whole-body coordination relative to conventional single-agent RL.

Significance. If the empirical comparison were clean, the paper would make a useful contribution by extending the recent idea of applying MARL to a single legged robot from quadrupeds (MASQ) to humanoids, with hardware validation. The real-world deployment with domain randomization is a concrete strength. However, the current evidence does not yet support the central claim: the single-agent PPO baseline is underspecified and likely lacks the privileged global information available to MASH's critic, the results are single-seed, and several evaluation metrics overlap with the reward terms used for training. These are fixable within the scope of a revision, so the paper is worth pursuing, but the comparison must be made rigorous before the claimed advantage can be accepted.

major comments (4)
  1. [§V-A3 and §IV-A] The single-agent PPO baseline is described only as 'conventional single-agent PPO'; its observation space, critic inputs, network architecture, and hyperparameters are not given. Meanwhile, MASH's critic takes a 106-dimensional privileged global state that includes external disturbance forces f_t, external torques τ_t, friction coefficient μ_t, body mass m_t, and stance/contact masks. If the baseline uses actor-only observations and a critic without these privileged quantities, the improvements in Fig. 4 and Table II may be entirely due to information asymmetry rather than the MARL decomposition. This is the load-bearing comparison for the paper's central claim. Please specify the baseline completely and add an ablation: single-agent PPO with the same privileged critic input and the same observation preprocessing as MASH's actor, plus full hyperparameter tables for both methods.
  2. [§V-B, Fig. 4, Table II] All results appear to come from a single training run per condition. RL locomotion is high-variance, and the reported convergence-time differences (~1306 vs ~1661 and ~1017 vs ~1238 iterations) are modest; without multiple seeds and error bars these differences cannot be distinguished from stochasticity. Report at least five independent seeds with mean ± std (or box plots) for convergence time and deployment metrics, and state the number of seeds in the figure captions and table notes.
  3. [§V-A2 and Table I] The evaluation metrics are not independent of the training objective. T_conv is defined directly from the training reward curve, S_action matches the 'Action Smoothness 1' reward term (Table I), S_torso combines quantities penalized by the Orientation and Base Height rewards, and C_limb is shaped by the temporal director of Eq. (12). Table II may therefore be measuring reward-shaping effects rather than genuine behavioral superiority. Please report task-level metrics that are not directly part of the reward, such as forward walking speed versus command, energy consumption, push-recovery success rate, or a holdout validation curve, and define convergence using such a validation measure rather than the training return.
  4. [§IV-A and §IV-B] The agent decomposition is not fully specified. The text states that each limb is an independent agent, but the actor network receives concatenated observations from 'two agents' for the legs and 'two agents' for the arms, and the critic outputs a value V_t ∈ R^4. Equations (10)–(11) show shared policies across left/right limbs, so it is unclear whether there are four agents with two shared policies, or two groups. This matters for interpreting the MARL contribution and for reproducing the baseline comparison. Clarify the exact number of policies, how agent IDs are used, and what the four value outputs represent. Also state how the single-agent PPO baseline's action space and actor/critic sizes were matched.
minor comments (4)
  1. [§IV-C] The sentence 'To enhance sim-to-real transfer, we employ domain randomization.' is duplicated verbatim in the same paragraph.
  2. [§V-A2, Eq. (13)] The text says S_action is the squared L2 norm of the second-order difference, but Eq. (13) uses (a_{i,t+1} - a_{i,t})^2, which is a first-order difference. Please align the definition and formula.
  3. [Fig. 5] The y-axis label 'Referece position' is misspelled; also clarify the units and time/step axis for the hip pitch trajectory.
  4. [§V-A3] The episode length is stated as 48 steps; please state the control frequency and physical time per episode, since 48 steps may be less than one full stride and affects the interpretation of the gait metrics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MASH's derivation and comparisons are self-contained; the baseline information-asymmetry concern is an experimental fairness issue, not a circular reduction.

full rationale

The paper's central claim is that reformulating single-humanoid locomotion as cooperative-heterogeneous MARL (decPOMDP in Sec. IV-A) accelerates convergence and improves coordination relative to single-agent PPO. No derivation step reduces by construction to its own inputs. The MAPPO objective (Eq. 7) is the standard multi-agent PPO objective, and the policy parameterization (Eqs. 10-11) is a factored shared-parameter actor; neither equation is defined in terms of the reported outcomes. The evaluation metrics (TConv, Saction, Storso, Climb) overlap with reward terms in Table I, but the single-agent baseline is trained with the same reward, so this is not a circularity—it is at most a reward-overfitting or metric-design concern. The definition of TConv from the training reward curve is a measure-construction issue, not a prediction fitted from data and then renamed. The self-citations ([2], [23], [24], [41]) are prior related work and are not load-bearing: [24] is the quadruped MARL analogue, but the humanoid results here are newly reported and do not rely on an unverified uniqueness theorem or ansatz smuggled via citation. The strongest genuine weakness is that Section V-A3 does not specify the baseline's observation space or critic inputs, while MASH's critic uses a privileged 106-dimensional global state (Sec. IV-A); if the baseline lacks this information, the speedups in Fig. 4 and Table II could stem from information asymmetry rather than from the MARL decomposition. That is an experimental fairness and correctness concern, not a circular derivation, and therefore does not raise the circularity score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's contribution rests on the limb-as-agent modeling choice, the shared-critic architecture, and an implicitly fair baseline. The reward shaping dominates the learning objective, and the model's 'agents' are an abstraction rather than new physical entities.

free parameters (3)
  • Reward scaling coefficients (16 terms, Table I) = e.g., 3.5, 1.5, 1.4, -2e-3, -5e-4, -1e-7, 2.0, 2.0, 1.2, 1.0, -1.0, -5e-2, 0.2, -0.1, -0.1, -2e-4
    Hand-tuned weights shape the optimization objective and are decisive for the learned gait; no sensitivity analysis is provided.
  • Temporal director parameters k and phase offsets Delta_i = not reported
    Eq. (12) uses a gait-cycle scaling and per-limb phase offsets that coordinate stepping; values are not given, though they are required to reproduce the gait sequencing.
  • Evaluation metric weights w_h, w_theta = not reported
    Storso in Eq. (14) weights height and orientation variance, but the weight values are never specified, making the reported torso-stability score uninterpretable.
assumptions (4)
  • domain assumption Dec-POMDP modeling of a single humanoid robot with each limb as an independent agent is a valid abstraction.
    Introduced in Section IV-A; limbs are mechanically coupled, so treating them as independent decision-makers is a modeling choice, not a physical fact.
  • domain assumption The shared-parameter actor network with concatenated per-limb observations implements independent agents rather than a single monolithic policy.
    Section IV-B claims independence while using one network per limb pair that outputs all actions in one forward pass; this equivalence is asserted, not demonstrated.
  • domain assumption The single-agent PPO baseline is trained with comparable information and thus provides a fair control condition.
    Section V-A3 simply names 'conventional single-agent PPO' without specifying its observation space or critic state; if the baseline lacks privileged global observations, the comparison is invalid.
  • domain assumption Isaac Gym with the listed domain randomization is an adequate proxy for the physical BanXing robot.
    Section IV-C and Table III list randomized parameters; no quantitative sim-to-sim results are shown despite a MuJoCo transfer being mentioned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MASH: Cooperative-Heterogeneous Multi-Agent Reinforcement Learning for Single Humanoid Robot Locomotion." pith.science (2026). https://pith.science/paper/XZRHKSZS

@misc{pith2026250810423,
  author       = {Pith},
  title        = {Pith review of: MASH: Cooperative-Heterogeneous Multi-Agent Reinforcement Learning for Single Humanoid Robot Locomotion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XZRHKSZS}},
  note         = {Machine review of arXiv:2508.10423}
}
read the original abstract

This paper proposes a novel method to enhance locomotion for a single humanoid robot through cooperative-heterogeneous multi-agent deep reinforcement learning (MARL). While most existing methods typically employ single-agent reinforcement learning algorithms for a single humanoid robot or MARL algorithms for multi-robot system tasks, we propose a distinct paradigm: applying cooperative-heterogeneous MARL to optimize locomotion for a single humanoid robot. The proposed method, multi-agent reinforcement learning for single humanoid locomotion (MASH), treats each limb (legs and arms) as an independent agent that explores the robot's action space while sharing a global critic for cooperative learning. Experiments demonstrate that MASH accelerates training convergence and improves whole-body cooperation ability, outperforming conventional single-agent reinforcement learning methods. This work advances the integration of MARL into single-humanoid-robot control, offering new insights into efficient locomotion strategies.

Figures

Figures reproduced from arXiv: 2508.10423 by the authors.

Figure 1
Figure 1. MARL model for a single humanoid robot’s locomotion [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The framework of MASH This paper proposes MASH, which applies the MARL algorithm to treat different parts of a single humanoid robot as independent agents, trained collaboratively using shared rewards. Specifically, this paper uses multi-agent PPO [45] (MAPPO) to solve the modeled multi-agent problem. MAPPO optimizes the following objective function in a multi-agent system: L CLIP MAPPO(θ) = Xn i=1 Et h min  r i t … view at source ↗
Figure 3
Figure 3. Simulation experiments of MASH A. Experiment Setup 1) Task Description: In the simulation environment, we train a single humanoid robot to walk forward on flat terrain in a fast and stable manner. This task serves as a standard benchmark in humanoid robot control, assessing the robot’s capabilities in terms of speed and balance. Our training pipeline begins with upright locomotion training in Isaac Gym, which includ… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Reward growth trends for (a) leg training and (b) whole-body training, comparing MASH with the Single Agent PPO baseline. The smoothed reward [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparison of hip pitch joint trajectories. The plots show the tracking performance of our MASH controller versus a Single-agent PPO baseline [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Stable walking gait on the physical humanoid robot using a policy trained with MASH. The stroboscopic images (top) and corresponding knee joint [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 39 canonical work pages

  1. [24]

    MASQ: Multi-Agent Reinforcement Learning for Single Quadruped Robot Locomotion

    Q. Liu, J. Guo, S. Lin, S. Ma, J. Zhu, and Y . Li, “Masq: Multi-agent reinforcement learning for single quadruped robot locomotion,” arXiv preprint arXiv:2408.13759, 2024

    work page Pith review arXiv 2024

  2. [1]

    HOVER: Versatile neural whole-body controller for humanoid robots,

    T. He, W. Xiao, T. Lin, Z. Luo, Z. Xu, Z. Jiang, J. Kautz, C. Liu, G. Shi, X. Wang et al. , “HOVER: Versatile neural whole-body controller for humanoid robots,” in 2025 IEEE International Conference on Robotics and Automation (ICRA) , 2025

    work page 2025

  3. [2]

    Distributional policy gradient with distributional value function,

    Q. Liu, Y . Li, X. Shi, K. Lin, Y . Liu, and Y . Lou, “Distributional policy gradient with distributional value function,”IEEE Transactions on Neural Networks and Learning Systems , vol. 36, no. 4, pp. 6556–6568, 2025

    work page 2025

  4. [3]

    Walk these ways: Tuning robot control for generalization with multiplicity of behavior,

    G. B. Margolis and P. Agrawal, “Walk these ways: Tuning robot control for generalization with multiplicity of behavior,” in Conference on Robot Learning. PMLR, 2023, pp. 22–31

    work page 2023

  5. [4]

    Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning,

    I. M. Aswin Nahrendra, B. Yu, and H. Myung, “Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023, pp. 5078–5084

    work page 2023

  6. [5]

    Biped dynamic walking using reinforcement learning,

    H. Benbrahim and J. A. Franklin, “Biped dynamic walking using reinforcement learning,” Robotics and Autonomous Systems, vol. 22, no. 3-4, pp. 283–302, 1997

    work page 1997

  7. [6]

    Learning vision-based bipedal locomotion for challeng- ing terrain,

    H. Duan, B. Pandit, M. S. Gadde, B. Van Marum, J. Dao, C. Kim, and A. Fern, “Learning vision-based bipedal locomotion for challeng- ing terrain,” in 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 56–62

    work page 2024

  8. [7]

    Motor anomaly detection for unmanned aerial vehicles using reinforcement learning,

    H. Lu, Y . Li, S. Mu, D. Wang, H. Kim, and S. Serikawa, “Motor anomaly detection for unmanned aerial vehicles using reinforcement learning,” IEEE Internet of Things Journal , vol. 5, no. 4, pp. 2315–2322, 2018

    work page 2018

Show all 47 references
  1. [8]

    Optimization-based control for dynamic legged robots,

    P. M. Wensing, M. Posa, Y . Hu, A. Escande, N. Mansard, and A. D. Prete, “Optimization-based control for dynamic legged robots,” IEEE Transactions on Robotics , vol. 40, pp. 43–63, 2024

    2024

  2. [9]

    Versatile multicontact planning and control for legged loco-manipulation,

    J.-P. Sleiman, F. Farshidian, and M. Hutter, “Versatile multicontact planning and control for legged loco-manipulation,” Science Robotics , vol. 8, no. 81, p. eadg5014, 2023

    2023

  3. [10]

    Combining trajectory optimization, supervised machine learning, and model structure for mitigating the curse of dimensionality in the control of bipedal robots,

    X. Da and J. Grizzle, “Combining trajectory optimization, supervised machine learning, and model structure for mitigating the curse of dimensionality in the control of bipedal robots,” The International Journal of Robotics Research , vol. 38, no. 9, pp. 1063–1097, 2019

    2019

  4. [11]

    Real-world humanoid locomotion with reinforcement learning,

    I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath, “Real-world humanoid locomotion with reinforcement learning,” Sci- ence Robotics, vol. 9, no. 89, p. eadi9579, 2024

    2024

  5. [12]

    Not only rewards but also constraints: Applications on legged robot locomotion,

    Y . Kim, H. Oh, J. Lee, J. Choi, G. Ji, M. Jung, D. Youm, and J. Hwangbo, “Not only rewards but also constraints: Applications on legged robot locomotion,” IEEE Transactions on Robotics , 2024

    2024

  6. [13]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,” The International Journal of Robotics Research , p. 02783649241273668, 2023

    2023

  7. [14]

    Learning-based legged locomotion: State of the art and future perspectives,

    S. Ha, J. Lee, M. van de Panne, Z. Xie, W. Yu, and M. Khadiv, “Learning-based legged locomotion: State of the art and future perspectives,” The International Journal of Robotics Research , p. 02783649241312698, 2024

    2024

  8. [15]

    Learning whole-body loco-manipulation for omni-directional task space pose tracking with a wheeled-quadrupedal-manipulator,

    K. Jiang, Z. Fu, J. Guo, W. Zhang, and H. Chen, “Learning whole-body loco-manipulation for omni-directional task space pose tracking with a wheeled-quadrupedal-manipulator,” IEEE Robotics and Automation Letters, 2024

    2024

  9. [16]

    Learning agile soccer skills for a bipedal robot with deep reinforcement learning,

    T. Haarnoja, B. Moran, G. Lever, S. H. Huang, D. Tirumala, J. Humplik, M. Wulfmeier, S. Tunyasuvunakool, N. Y . Siegel, R. Hafner et al. , “Learning agile soccer skills for a bipedal robot with deep reinforcement learning,” Science Robotics, vol. 9, no. 89, p. eadi8022, 2024

    2024

  10. [17]

    Visual whole-body control for legged loco-manipulation,

    M. Liu, Z. Chen, X. Cheng, Y . Ji, R. Qiu, R. Yang, and X. Wang, “Visual whole-body control for legged loco-manipulation,” in Conference on Robot Learning, 2024

    2024

  11. [18]

    Teleoperation of humanoid robots: A survey,

    K. Darvish, L. Penco, J. Ramos, R. Cisneros, J. Pratt, E. Yoshida, S. Ivaldi, and D. Pucci, “Teleoperation of humanoid robots: A survey,” IEEE Transactions on Robotics , vol. 39, no. 3, pp. 1706–1727, 2023

    2023

  12. [19]

    Sim-to-real robotic sketching using behavior cloning and reinforcement learning,

    B. Jia and D. Manocha, “Sim-to-real robotic sketching using behavior cloning and reinforcement learning,” in 2024 IEEE International Con- ference on Robotics and Automation (ICRA) , 2024, pp. 18 272–18 278

    2024

  13. [20]

    A composite control strategy for quadruped robot by integrating reinforcement learning and model-based control,

    S. Lyu, H. Zhao, and D. Wang, “A composite control strategy for quadruped robot by integrating reinforcement learning and model-based control,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2023, pp. 751–758

    2023

  14. [21]

    Towards human-level bimanual dexterous manipulation with reinforcement learning,

    Y . Chen, T. Wu, S. Wang, X. Feng, J. Jiang, Z. Lu, S. McAleer, H. Dong, S.-C. Zhu, and Y . Yang, “Towards human-level bimanual dexterous manipulation with reinforcement learning,” in Advances in Neural Information Processing Systems , vol. 35, 2022, pp. 5150–5163

    2022

  15. [22]

    Monotonic value function factorisation for deep multi- agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Monotonic value function factorisation for deep multi- agent reinforcement learning,” Journal of Machine Learning Research , vol. 21, no. 178, pp. 1–51, 2020

    2020

  16. [23]

    Data efficient deep reinforcement learning with action-ranked temporal difference learning,

    Q. Liu, Y . Li, Y . Liu, K. Lin, J. Gao, and Y . Lou, “Data efficient deep reinforcement learning with action-ranked temporal difference learning,” IEEE Transactions on Emerging Topics in Computational Intelligence , vol. 8, no. 4, pp. 2949–2961, 2024

    2024

  17. [25]

    Expressive whole-body control for humanoid robots,

    X. Cheng, Y . Ji, J. Chen, R. Yang, G. Yang, and X. Wang, “Expressive whole-body control for humanoid robots,” in Robotics Science and Systems, 2024

    2024

  18. [26]

    Mobile-television: Predictive motion priors for hu- manoid whole-body control,

    C. Lu, X. Cheng, J. Li, S. Yang, M. Ji, C. Yuan, G. Yang, S. Yi, and X. Wang, “Mobile-television: Predictive motion priors for hu- manoid whole-body control,” in 2025 IEEE International Conference on Robotics and Automation (ICRA) , 2025

    2025

  19. [27]

    Wococo: Learning whole-body humanoid control with sequential contacts,

    C. Zhang, W. Xiao, T. He, and G. Shi, “Wococo: Learning whole-body humanoid control with sequential contacts,” in Conference on Robot Learning, 2024

    2024

  20. [28]

    Learning human-to-humanoid real-time whole-body teleoperation,

    T. He, Z. Luo, W. Xiao, C. Zhang, K. Kitani, C. Liu, and G. Shi, “Learning human-to-humanoid real-time whole-body teleoperation,” in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024, pp. 8944–8951

    2024

  21. [29]

    Humanplus: Humanoid shadowing and imitation from humans,

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Humanoid shadowing and imitation from humans,” in Conference on Robot Learning, 2024

    2024

  22. [30]

    Okami: Teaching humanoid robots manipulation skills through single video imitation,

    J. Li, Y . Zhu, Y . Xie, Z. Jiang, M. Seo, G. Pavlakos, and Y . Zhu, “Okami: Teaching humanoid robots manipulation skills through single video imitation,” in Conference on Robot Learning , 2024. 10

    2024

  23. [31]

    OmniH2O: Universal and dexterous human-to- humanoid whole-body teleoperation and learning,

    T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. M. Kitani, C. Liu, and G. Shi, “OmniH2O: Universal and dexterous human-to- humanoid whole-body teleoperation and learning,” in Conference on Robot Learning, 2024

    2024

  24. [32]

    Perpetual humanoid control for real-time simulated avatars,

    Z. Luo, J. Cao, K. Kitani, W. Xu et al. , “Perpetual humanoid control for real-time simulated avatars,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 10 895–10 904

    2023

  25. [33]

    Robust and versatile bipedal jumping control through reinforcement learning,

    Z. Li, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Robust and versatile bipedal jumping control through reinforcement learning,” in Robotics Science and Systems , 2023

    2023

  26. [34]

    Feedback control for cassie with deep reinforcement learning,

    Z. Xie, G. Berseth, P. Clary, J. Hurst, and M. Van de Panne, “Feedback control for cassie with deep reinforcement learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 1241–1246

    2018

  27. [35]

    Sim-to-real learning of all common bipedal gaits via periodic reward composition,

    J. Siekmann, Y . Godse, A. Fern, and J. Hurst, “Sim-to-real learning of all common bipedal gaits via periodic reward composition,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 7309–7315

    2021

  28. [36]

    Amp: Adversarial motion priors for stylized physics-based character control,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character control,” ACM Transactions on Graphics (ToG) , vol. 40, no. 4, pp. 1–20, 2021

    2021

  29. [37]

    Advancing humanoid locomotion: Mastering challenging terrains with denoising world model learning,

    X. Gu, Y .-J. Wang, X. Zhu, C. Shi, Y . Guo, Y . Liu, and J. Chen, “Advancing humanoid locomotion: Mastering challenging terrains with denoising world model learning,” arXiv e-prints, pp. arXiv–2408, 2024

    2024

  30. [38]

    Reinforcement learning for swarm robotics: An overview of applications, algorithms and simulators,

    M.-A. Blais and M. A. Akhloufi, “Reinforcement learning for swarm robotics: An overview of applications, algorithms and simulators,” Cognitive Robotics, vol. 3, pp. 226–256, 2023

    2023

  31. [39]

    Smarts: An open-source scalable multi-agent rl training school for autonomous driving,

    M. Zhou, J. Luo, J. Villella, Y . Yang, D. Rusu, J. Miao, W. Zhang, M. Alban, I. Fadakar, Z. Chen et al., “Smarts: An open-source scalable multi-agent rl training school for autonomous driving,” in Conference on robot learning . PMLR, 2021, pp. 264–285

    2021

  32. [40]

    Optimal tethered-uav deployment in a2g communication networks: Multi-agent q-learning approach,

    S. Lim, H. Yu, and H. Lee, “Optimal tethered-uav deployment in a2g communication networks: Multi-agent q-learning approach,” IEEE Internet of Things Journal , vol. 9, no. 19, pp. 18 539–18 549, 2022

    2022

  33. [41]

    Multi- agent target assignment and path finding for intelligent warehouse: A cooperative multi-agent deep reinforcement learning perspective,

    Q. Liu, J. Gao, D. Zhu, Z. Qiao, P. Chen, J. Guo, and Y . Li, “Multi- agent target assignment and path finding for intelligent warehouse: A cooperative multi-agent deep reinforcement learning perspective,” arXiv preprint arXiv:2408.13750, 2024

    2024 arXiv

  34. [42]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT press, 2018

    2018

  35. [43]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  36. [44]

    Pomdps for robotic tasks with mixed observability

    S. C. Ong, S. W. Png, D. Hsu, and W. S. Lee, “Pomdps for robotic tasks with mixed observability.” in Robotics: Science and Systems , vol. 5, 2009, p. 4

    2009

  37. [45]

    The surprising effectiveness of ppo in cooperative multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of ppo in cooperative multi-agent games,” in Advances in Neural Information Processing Systems , vol. 35, 2022, pp. 24 611–24 624

    2022

  38. [46]

    Stabilising experience replay for deep multi-agent rein- forcement learning,

    J. Foerster, N. Nardelli, G. Farquhar, T. Afouras, P. H. Torr, P. Kohli, and S. Whiteson, “Stabilising experience replay for deep multi-agent rein- forcement learning,” in International Conference on Machine Learning . PMLR, 2017, pp. 1146–1155

    2017

  39. [47]

    Isaac gym: High performance gpu based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa et al. , “Isaac gym: High performance gpu based physics simulation for robot learning,” in Thirty- fifth Conference on Neural Information Processing Systems Datas...

    2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.