REVIEW 3 major objections 4 minor 33 references
Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Multi-Loco claims that one shared policy, a frozen cross-embodiment diffusion prior plus a shared residual reinforcement-learning policy, can control four morphologically different legged robots and outperform per-robot PPO baselines by…
desk verdict Mask convention in the core objective is self-contradictory (Eq. 2 vs. Algorithm 1 and A.2), and without code the headline results are unverifiable; the architecture itself is worth a referee's time. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the masked denoising score-matching objective of Equation (2), which is a training loss that applies the diffusion reconstruction error only on the valid dimensions of a zero-padded action vector, using a binary mask $b$ with 1 for real action dimensions and 0 for padding. This lets a single EDM/DiT denoiser learn the action distribution of every embodiment from one shared dataset. The second component is a residual PPO policy with one shared actor and one critic per robot; the actor outputs an additive correction $\Delta a$ to the diffusion prior, and the per-robot critics stabilize optimization across conflicting reward scales. The mask defines which data the prior learns from, and the residual defines which task-specific corrections survive online.
What would settle it
Compare the mask operations in the implementation with the two conventions in the paper: if the training loss is $((1-b)\odot(D_\theta(\cdot)-a_0))^2$ and inference applies $(1-b)\cdot a_i$, as Algorithm 1 and Section A.2 state, then valid action dimensions are never trained and are dropped at deployment; re-running with the Equation (2) convention (loss weighted by $b$, inference zeroing only padded dimensions) would either reproduce the reported 10.35% average gain or show the mask flip was responsible.
Extended reading notes
Core claim
The central claim, stated on the paper's own terms, is that morphology-agnostic representation is enough to unify legged locomotion across a point-foot biped, a wheeled biped, a humanoid, and a quadruped, without explicit morphology descriptors, joint information, or per-robot encoders. This is achieved by zero-padding all observations and actions into common spaces and training one EDM denoiser with masked denoising score matching, so the diffusion loss acts only on each robot's valid action dimensions. The denoiser's output is then added to a residual action predicted by a single PPO actor shared across all embodiments, while a separate critic per robot provides the advantage estimates. The authors report that the full method reaches 113.57% of the baseline's average return, that cross-robot diffusion training beats single-robot diffusion training by 17.96%, and that the same policy runs in real time at 50 Hz on hardware.
Load-bearing premise
The load-bearing premise is that the binary mask used in training and inference matches Equation (2), where the mask marks valid action dimensions; the paper's own Algorithm 1 and Section A.2 apply the complement mask $(1-b)$, so if those describe the implementation, the diffusion model would train on padded dimensions only and would zero out the real actions at deployment.
Editorial extensions
If this is right
- A single shared policy can replace per-robot control stacks on four different legged morphologies, so locomotion data collected on any of them can contribute to one controller.
- Cross-embodiment diffusion training transfers terrain skills: the wheeled biped acquires leg-lifting behavior on discrete terrain that its own single-robot diffusion model and RL baseline never produced.
- The diffusion prior alone reaches 87.78% of the baseline return, and the residual RL lifts this to 113.57%, showing the generative prior and the online policy are complementary rather than redundant.
- The same unified policy transfers zero-shot to a previously unseen quadruped morphology and to physical hardware on grass, slopes, stairs, and gravel paths.
Reading between the lines
- Editorial inference: the padding-plus-mask recipe could generalize beyond legs to any control task with heterogeneous action spaces, such as manipulators with different numbers of fingers, provided the mask convention between training and inference is kept consistent.
- Editorial inference: the paper's finding that reducing humanoid data degrades wheeled-biped performance suggests high-dimensional whole-body datasets act as a general prior for lower-dimensional locomotion; one could test whether adding more humanoid data monotonically improves wheeled-biped terrain metrics.
- Editorial inference: a clean ablation that freezes or removes the residual policy at deployment would isolate how much of the reported sim2real robustness comes from the diffusion prior versus the RL refinement.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Multi-Loco, a framework for multi-embodiment legged locomotion that combines a frozen, cross-embodiment diffusion prior (trained with masked denoising score matching) with a shared residual policy trained by multi-critic PPO. The action space is unified across four robot types by zero-padding, and the claimed contribution is that the hybrid policy outperforms per-robot PPO baselines by 10.35% average return, with gains up to 13.57% on a wheeled biped, while also transferring zero-shot to a Unitree Go2 and to real hardware. The manuscript reports simulation and real-world experiments, hyperparameter tables, and several ablations of dataset size, sampling steps, and dataset composition.
Significance. If the results are verified, the paper would make a useful contribution to cross-embodiment locomotion: it demonstrates a concrete architecture for combining a morphology-agnostic diffusion prior with a shared residual RL policy, and it reports real-world deployment on four distinct platforms plus zero-shot transfer to an unseen quadruped. The authors provide substantial experimental detail, including reward tables, network configurations, and an appendix with ablations. However, the central quantitative claims currently lack the necessary supporting evidence: the mask-convention inconsistency directly affects the correctness of the diffusion prior, the headline return numbers are not backed by a numeric table with errors, and the diffusion prior is trained on rollouts from the very baseline it is compared against. These issues are fixable, but they are load-bearing rather than cosmetic.
major comments (3)
- [Section 2.2 / Algorithm 1 / Section A.2] The mask convention is self-contradictory in a way that directly affects the validity of the diffusion prior. Section 2.1 defines b with 1 for valid action dimensions and 0 for padded dimensions, and Eq. (2) multiplies the denoising loss by b, so only valid dimensions are supervised. In contrast, Algorithm 1 line 26 and Section A.2 compute L = ((1-b)*(D_theta(...)-a0))^2, and Algorithm 1 line 38 sets a'_i = (1-b)*a_i during inference. Under the Algorithm 1 convention, the network is trained only on padded dimensions and the valid dimensions are zeroed at inference, the exact complement of Eq. (2). Since no code or data is released, a reader cannot determine which convention the reported experiments actually used; this is load-bearing because the central 10.35% and 13.57% claims depend on the frozen diffusion prior supplying useful cross-embodiment actions. The authors must correct the inconsistency and, ideally, release code or detailed pseudocode that matches the implementation.
- [Section 3.2, Table 2, Figure 3] The headline average return improvement is not supported by a numeric table. Table 2 reports only MEL, LVT, and AVT, while the claimed 10.35% average improvement and per-robot gains (12.49%, 13.57%, 4.38%, 10.97%) appear only in the caption of Figure 3, which is a bar chart without error bars or numerical values. For the quadruped, Table 2 shows CR-DP+RA slightly underperforming the RL baseline on all three reported metrics (MEL 19.60 vs. 19.79, LVT 5.42 vs. 5.45, AVT 4.32 vs. 4.34), which is difficult to reconcile with a claimed 4.38% return gain unless average return captures a different quantity. Please provide a table of Average Return with means and standard deviations across seeds for every method and embodiment.
- [Section 3.2, Evaluation Setup] The diffusion prior is not an external benchmark because it is distilled from rollouts of the same RL baselines it is later compared against. The text states: 'Our evaluation protocol employs policies trained via baseline RL to generate rollout data for diffusion policy (DP) training.' This makes the SR-DP/CR-DP comparisons informative only about behavioral cloning from the baseline, not about whether the diffusion prior provides knowledge beyond the baseline. The final residual policy is trained with environment rewards, which gives some independent grounding, but the contribution of the diffusion prior itself is not isolated: there is no ablation of a shared residual policy without the diffusion prior (e.g., a Gaussian-prior residual PPO with the same multi-critic architecture and reward weights). Adding such an ablation would clarify whether the reported gains come from the diffusion prior, the residual RL, or the larger shared architecture.
minor comments (4)
- [Section A.2] The sentence 'we zero-pad observation and action vectors to match the largest dimension across all robot morphologies (observations: 68D, actions: 20D) and the dimension configurations of .' is incomplete; it ends with 'of .' and appears to reference a table or equation that is not included.
- [Figure 3 caption] The caption labels the full method as 'SR-DP+RA (Ours)', but the method introduced in Section 3.2 is 'CR-DP with Residual Adaptation (CR-DP+RA)'. This likely a typo and should be corrected to avoid confusion.
- [Algorithm 1 / Table 3] There are inconsistent hyperparameter values: Algorithm 1 line 5 specifies a batch size of 512 in the initialization block, while Table 3 lists 'Batch Size 8192'. Similarly, Algorithm 1's training procedure is written as 'Epochs=400K' while Table 3 reports 'Number of Epochs 500'. Please align these values.
- [Section 2.3, Eq. (4)] The actor loss in Eq. (4) is written as a weighted log-probability objective, but it omits the importance-sampling ratio and clipping that characterize the PPO update described in the text. A reader implementing from Eq. (4) would not reproduce standard PPO; please either present the full clipped surrogate loss or explain the simplification.
Circularity Check
No circular derivation: the reported gains are empirical measurements and the residual RL stage provides independent grounding.
full rationale
Multi-Loco does not present a derivation chain in which an output is defined as its input. The central comparison (CR-DP+RA vs RL baseline) is empirical: although the diffusion prior is trained on rollouts generated by the baseline RL policies (Section 3.2: 'Our evaluation protocol employs policies trained via baseline RL to generate rollout data for diffusion policy (DP) training'), the final action is a_prior + Delta and Delta is optimized with environment rewards, so the reported 10.35% improvement is not equal to the baseline by construction. This training-data overlap is a benchmarking confound, not a circular reduction. Likewise, CR-DP vs SR-DP is confounded by data volume (four datasets vs one), not by definition. The paper contains self-citations (e.g., [9], [28], [29]) but none is load-bearing for the main claim. A separate, non-circular reproducibility problem is the mask-convention contradiction: Eq. (2) multiplies the loss by b (valid dimensions), while Algorithm 1 line 26 and Section A.2 use (1-b) and inference zeros valid dimensions; this makes the experimental results unverifiable from the manuscript alone, but it is an internal inconsistency, not a reduction of the prediction to its inputs. No uniqueness arguments are imported from prior author work. Therefore no circularity is established.
Assumptions & free parameters
free parameters (5)
- Residual penalty coefficient alpha =
0.2 (PPO Residual Coeff)
- Per-robot reward weights =
Tables 8 and 9, e.g. velocity gains 4.0/2.0/2.0/6.0 across robots
- Desired base height for a1 and go2 =
Not numerically reported, Appendix A.6
- Diffusion sampling steps =
5
- Dataset size cap per robot =
2,048,000 samples
assumptions (4)
- standard math EDM and score-matching theory correctly model the multimodal action distribution, and the 5-step Euler ODE solver produces accurate actions.
- domain assumption Rollouts from per-robot RL baseline policies are sufficient and representative training data for the diffusion prior.
- domain assumption Zero-padded fixed-size observation and action vectors preserve enough information for one shared network to infer each robot's embodiment and dynamics without explicit morphology descriptors.
- ad hoc to paper A multi-critic PPO with one shared actor resolves optimization conflicts across embodiments better than alternatives.
Cite this review
Pith. "Pith review of Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion." pith.science (2026). https://pith.science/paper/D34DVYO2
@misc{pith2026250611470,
author = {Pith},
title = {Pith review of: Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion},
year = {2026},
howpublished = {\url{https://pith.science/paper/D34DVYO2}},
note = {Machine review of arXiv:2506.11470}
}
read the original abstract
Generalizing locomotion policies across diverse legged robots with varying morphologies is a key challenge due to differences in observation/action dimensions and system dynamics. In this work, we propose Multi-Loco, a novel unified framework combining a morphology-agnostic generative diffusion model with a lightweight residual policy optimized via reinforcement learning (RL). The diffusion model captures morphology-invariant locomotion patterns from diverse cross-embodiment datasets, improving generalization and robustness. The residual policy is shared across all embodiments and refines the actions generated by the diffusion model, enhancing task-aware performance and robustness for real-world deployment. We evaluated our method with a rich library of four legged robots in both simulation and real-world experiments. Compared to a standard RL framework with PPO, our approach -- replacing the Gaussian policy with a diffusion model and residual term -- achieves a 10.35% average return improvement, with gains up to 13.57% in wheeled-biped locomotion tasks. These results highlight the benefits of cross-embodiment data and composite generative architectures in learning robust, generalized locomotion skills.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter. Learn- ing agile and dynamic motor skills for legged robots.Science Robotics, 4(26):eaau5872, 2019
work page 2019
-
[3]
J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter. Learning quadrupedal locomo- tion over challenging terrain.Science robotics, 5(47):eabc5986, 2020
2020
-
[4]
T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter. Learning robust percep- tive locomotion for quadrupedal robots in the wild.Science robotics, 7(62):eabk2822, 2022
work page 2022
-
[5]
J. Yang, D. Sadigh, and C. Finn. Polybot: Training one policy across robots while embracing variability.arXiv preprint arXiv:2307.03719, 2023
arXiv 2023
-
[6]
M. Xu, Z. Xu, C. Chi, M. Veloso, and S. Song. Xskill: Cross embodiment skill discovery. In Conference on Robot Learning, pages 3536–3555. PMLR, 2023
work page 2023
-
[7]
O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024
arXiv 2024
- [8]
Show all 33 references
-
[9]
H. Wang, H. Luo, W. Zhang, and H. Chen. Cts: Concurrent teacher-student reinforcement learning for legged locomotion.IEEE Robotics and Automation Letters, 2024
2024
-
[10]
Cheng, K
X. Cheng, K. Shi, A. Agarwal, and D. Pathak. Extreme parkour with legged robots. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 11443–11450. IEEE, 2024. 9
2024
-
[11]
Hoeller, N
D. Hoeller, N. Rudin, D. Sako, and M. Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots.Science Robotics, 9(88):eadi7566, 2024
2024
-
[12]
J. Lee, M. Bjelonic, A. Reske, L. Wellhausen, T. Miki, and M. Hutter. Learning robust au- tonomous navigation and locomotion for wheeled-legged robots.Science Robotics, 9(89): eadi9641, 2024
2024
-
[13]
T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. Kitani, C. Liu, and G. Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning.arXiv preprint arXiv:2406.08858, 2024
2024 arXiv
-
[14]
T. He, W. Xiao, T. Lin, Z. Luo, Z. Xu, Z. Jiang, J. Kautz, C. Liu, G. Shi, X. Wang, et al. Hover: Versatile neural whole-body controller for humanoid robots.arXiv preprint arXiv:2410.21229, 2024
2024 arXiv
-
[15]
Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based gen- erative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020
2011 arXiv
-
[16]
J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020
2010 arXiv
-
[17]
C. Lu, Y . Zhou, F. Bao, J. Chen, C. Li, and J. Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787, 2022
2022
-
[18]
Karras, M
T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022
2022
-
[19]
C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, page 02783649241273668, 2023
2023
-
[20]
Huang, Y
X. Huang, Y . Chi, R. Wang, Z. Li, X. B. Peng, S. Shao, B. Nikolic, and K. Sreenath. Diffuse- loco: Real-time legged locomotion control with diffusion from offline datasets.arXiv preprint arXiv:2404.19264, 2024
2024 arXiv
-
[21]
G. Feng, H. Zhang, Z. Li, X. B. Peng, B. Basireddy, L. Yue, Z. Song, L. Yang, Y . Liu, K. Sreenath, et al. Genloco: Generalized locomotion controllers for quadrupedal robots. In Conference on Robot Learning, pages 1893–1903. PMLR, 2023
1903
-
[22]
Shafiee, G
M. Shafiee, G. Bellegarda, and A. Ijspeert. Manyquadrupeds: Learning a single locomotion policy for diverse quadruped robots. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 3471–3477. IEEE, 2024
2024
-
[23]
Z. Luo, Y . Dong, X. Li, R. Huang, Z. Shu, E. Xiao, and P. Lu. Moral: Learning morphologically adaptive locomotion controller for quadrupedal robots on challenging terrains.IEEE Robotics and Automation Letters, 2024
2024
-
[24]
Bohlinger, G
N. Bohlinger, G. Czechmanowski, M. Krupka, P. Kicki, K. Walas, J. Peters, and D. Tateo. One policy to run them all: an end-to-end learning approach to multi-embodiment locomotion. arXiv preprint arXiv:2409.06366, 2024
2024
-
[25]
Peebles and S
W. Peebles and S. Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023
2023
-
[26]
Mysore, G
S. Mysore, G. Cheng, Y . Zhao, K. Saenko, and M. Wu. Multi-critic actor learning: Teaching rl policies to act with style. InInternational Conference on Learning Representations, 2022. 10
2022
-
[27]
P. Xu, X. Shang, V . Zordan, and I. Karamouzas. Composite motion learning with task control. ACM Transactions on Graphics (TOG), 42(4):1–16, 2023
2023
-
[28]
Huang, J
T. Huang, J. Ren, H. Wang, Z. Wang, Q. Ben, M. Wen, X. Chen, J. Li, and J. Pang. Learning humanoid standing-up control across diverse postures.arXiv preprint arXiv:2502.08378, 2025
2025 arXiv
-
[29]
H. Wang, Z. Wang, J. Ren, Q. Ben, T. Huang, W. Zhang, and J. Pang. Beamdojo: Learning agile humanoid locomotion on sparse footholds.arXiv preprint arXiv:2502.10363, 2025
2025 arXiv
-
[30]
Rudin, D
N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. InConference on Robot Learning, pages 91–100. PMLR, 2022
2022
-
[31]
Ross and D
S. Ross and D. Bagnell. Efficient reductions for imitation learning. InProceedings of the thir- teenth international conference on artificial intelligence and statistics, pages 661–668. JMLR Workshop and Conference Proceedings, 2010
2010
-
[32]
D. Picard. Torch. manual seed (3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision.arXiv preprint arXiv:2109.08203, 2021
2021 arXiv
-
[2024]
elu", "elu
URLhttps://arxiv.org/abs/2406.09509. 11 Appendix Experiment Videos We conducted comprehensive real-world evaluations of our framework across four distinct legged robotic platforms. For detailed empirical validation, we encourage readers to view the supplemen- tary video. As de...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.