Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Dynamic Legged Ball Manipulation on Rugged Terrains with Hierarchical Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A hierarchical RL policy that switches between dribbling and locomotion skills lets a quadruped dribble a ball across rugged terrain, beating single-policy baselines.

desk verdict A useful hierarchical RL system for rugged-terrain dribbling, but the main quantitative claim is undercut by a success metric that doesn't require ball control. read the letter →

arxiv 2504.14989 v1 pith:DHDZBN4T submitted 2025-04-21 cs.RO

Dongjie Zhu , Zhuo Yang , Tianhang Wu , Luzhou Ge , Xuesong Li , Qi Liu , Xiang Li This is my paper · ORCID
classification cs.RO
keywords hierarchicalreinforcementlearningquadrupedloco-manipulationballdribblingroughterrainnavigationskillswitchingmixeddiscrete-continuousactionspacessim-to-realtransferdynamicskill-focusedpolicyoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that dynamic ball manipulation on rugged terrain is achievable by a quadruped robot through hierarchical reinforcement learning rather than end-to-end single-policy training. The claim is that a high-level policy, running at 10 Hz and reading the robot's own sensors plus the ball's position, can switch among pre-trained low-level skills — two dribbling skills and two locomotion skills — to keep the ball moving while traversing ramps, rough ground, stairs, and flat terrain. To make this work, the paper introduces Dynamic Skill-Focused Policy Optimization (DSF-PO), a modification of the PPO surrogate loss that weights gradients by the probability of selecting the active skill and masks out command dimensions that are irrelevant to that skill. Simulation and real-world experiments are presented as evidence that this framework outperforms existing dribbling methods on rugged terrain. A sympathetic reader would care because legged loco-manipulation is currently mostly confined to flat ground, and the proposed decomposition is a route to extending it to realistic outdoor environments.

What carries the argument

The central machinery is the DSF-PO surrogate loss, a PPO objective adapted to mixed discrete-continuous action spaces. In standard PPO the importance ratio is $\pi_\theta(a_t|s_t)/\pi_{\theta_{\mathrm{old}}}(a_t|s_t)$; DSF-PO rewrites this as a skill ratio multiplied by command ratios, with each command ratio raised to the skill-focus weight $w_k(s_t)=\pi^d_\theta(k|s_t)$ and gated by an indicator so only the command block belonging to the selected skill enters the gradient. The effect is that the optimizer pushes the policy toward higher-probability skills and updates only the command dimensions that those skills actually consume. The second load-bearing piece is the hierarchical execution loop: the high-level actor runs at 10 Hz, the low-level skills run at 50 Hz producing joint targets for PD control, and during high-level training the low-level skills and the context-aided estimator are frozen.

What would settle it

Re-run the terrain traversability evaluation with success redefined as the ball crossing the same boundary point as the robot, and count episodes in which the ball is abandoned; if the reported advantage over the baselines disappears under this ball-focused criterion, the central claim of superior ball manipulation is not supported.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that ball dribbling and terrain traversal need not be learned as one monolithic policy. The contribution is a two-level architecture: four low-level skills are trained end-to-end in simulation — two dribbling skills that apply kicks of different amplitudes and two locomotion skills for fast flat movement and rugged-terrain traversal — and a high-level actor outputs a discrete skill index and continuous low-level commands at 10 Hz. The high-level policy is trained with DSF-PO, whose importance ratio separates the discrete skill-selection ratio from the continuous command ratio and weights each command block by the skill focus weight, so that inactive command dimensions (for example locomotion commands while dribbling) do not produce gradient noise. The paper reports that the resulting policy reaches higher cumulative reward and longer episodes than standard PPO in training, reaches the terrain boundary more often than two prior dribbling systems on ramp-up, ramp-down, rough, and stair-descent terrain, and transfers zero-shot to a physical quadruped that dribbles a ball across indoor ramps, stairs, and outdoor gravel.

Load-bearing premise

The load-bearing premise is that the success metric in the main simulation comparison — the robot reaching the terrain boundary — is a valid measure of ball manipulation, even though it does not require the ball to be transported or controlled.

Editorial extensions

If this is right

  • If the hierarchical decomposition is correct, long-horizon loco-manipulation problems can be solved by training reusable low-level skills once and learning only a switch policy on top.
  • Because DSF-PO is a loss-only change, it should accelerate learning in any RL setting where a discrete selector gating continuous commands makes much of the action space irrelevant at any given step.
  • The zero-shot real-world transfer reported here implies that a simulator-trained skill library plus an onboard ball detector may be enough to deploy ball manipulation on terrain types not seen during training.
  • The reported success on stair descent and gravel suggests that legged object transport is not limited to flat indoor floors, which would matter for deliveries, search-and-rescue, and robot soccer on outdoor fields.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stricter test of the paper's central claim would track whether the ball crosses the terrain boundary alongside the robot; the simulation comparison as reported counts only the robot reaching the boundary, so it does not directly verify ball transport.
  • Because DSF-PO only changes the loss objective, it could be transplanted to other hierarchical skill-switching tasks with a discrete selector and continuous command heads, even outside legged locomotion; the paper does not test this transfer.
  • The reported strategy for stair descents — kicking the ball down before following — suggests the high-level policy learns a tacit sequence of manipulation-then-locomotion; a natural extension would randomize the ball's starting position and test whether this sequencing persists.
  • On downhill terrain the ball's velocity is heavily influenced by gravity rather than by robot contact, so a follow-up ablation separating robot-induced ball motion from terrain-induced motion would clarify how much of the observed performance is active manipulation versus passive rolling.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes a hierarchical reinforcement learning (RL) framework for dynamic legged ball manipulation on rugged terrain. A high-level policy, trained with a proposed Dynamic Skill-Focused Policy Optimization (DSF-PO), selects among four pre-trained low-level skills (two dribbling, two locomotion) and issues skill-specific commands. The framework is trained in Isaac Gym and transferred zero-shot to a Unitree Go2. The central empirical claim is that the proposed method outperforms DribbleBot and DexDribbler in ball manipulation on rugged terrain, based on simulation results in Table II and real-world results in Table III.

Significance. If the central claim were established, the work would be a useful contribution to learning-based loco-manipulation: it addresses a mixed discrete-continuous hierarchical action space, provides a skill-switching mechanism, and demonstrates real-world deployment. The paper also includes a clear low-level skill decomposition and a plausible curriculum design, and the authors are transparent that baseline real-world reproduction was infeasible. However, the quantitative evidence for the headline outperformance claim is not adequate: the main simulation metric does not actually measure ball manipulation, and the real-world comparison is not controlled. These issues are load-bearing because the abstract and conclusion explicitly claim superiority over baselines in dynamic ball manipulation.

major comments (3)
  1. [Section VI.A.2, Table II] The simulation success metric is defined as "the robot reaching the terrain boundary," with no requirement on the ball's final position, trajectory, or continued contact. A policy that abandons the ball and walks to the boundary would be counted as successful, so Table II, titled "ball dribbling success rates," does not measure ball manipulation. This metric is the basis for the central claim that the proposed method outperforms DribbleBot and DexDribbler in dynamic ball manipulation across rugged terrains. The evaluation should use a ball-centric success criterion, such as the ball crossing the boundary while remaining within a distance threshold of the robot, and this metric should be reported consistently for all compared methods.
  2. [Section VI.B, Table III] The real-world comparison is not controlled. The baseline numbers are taken from published results [5], [6] rather than measured in the same environment, on the same robot hardware, or under the same task definition; the text explicitly states that reproducing them is infeasible. With only five trials per condition and no statistical analysis, Table III cannot substantiate the claim that the proposed method achieves the highest success rate. The authors should either reproduce baselines under matched conditions or report absolute success counts without comparative claims.
  3. [Section IV, Eqs. (5)-(7)] The derivation of DSF-PO is not self-consistent. According to the policy decomposition in Eq. (4), the likelihood of the sampled command for skill d is N(c_d; mu_d, Sigma_d), so the standard importance ratio for that command should be N(c_d; mu_d)/N_old(c_d; mu_d_old). Eq. (5)/(10) instead raises this ratio to the power w_k(s), and Eq. (7) is admitted in the text to be only "partially correct" as an intuitive understanding. Without a corrected derivation from a well-defined objective, the claimed property of suppressing gradients from inactive skills is not established. The Fig. 3 ablation, which only reports total reward and episode length, does not by itself validate the loss formulation.
minor comments (5)
  1. [Section VI.A.3, Fig. 4] The cross-terrain demonstration is a single trajectory with no repeated trials, no baseline comparison, and no explicitly defined success criterion; the text states that the robot "successfully completes the task" but does not specify how completion is measured.
  2. [Fig. 3] The shaded regions are described as standard deviations over multiple runs, but the number of random seeds is not stated; this should be specified for reproducibility.
  3. [Eq. (10)] The notation C^k and I(c_t^k in C^k) is introduced in Eq. (10) but not defined before first use; clarify how the shared command subsets for dribbling skills are handled.
  4. [Section III.B.2 and Section IV] The description of the five-dimensional command output as a concatenation for dribbling and locomotion skills is not fully aligned with the per-skill distribution formulation in Section IV; the relationship between the shared command subsets and the per-skill normal distributions should be made precise.
  5. [Table III] The "-" entries for stair descent indicate that the baseline methods were not evaluated on that terrain, so the statement that the proposed method achieves the highest success rate "across all four terrains" is misleading; this limitation should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained and the main caveat is a metric-validity concern, not circular reasoning.

full rationale

The paper is an empirical reinforcement-learning system paper rather than a first-principles derivation, and its claimed results do not reduce to their own inputs. Low-level dribbling and locomotion skills are trained with PPO, and the high-level skill-switching policy is trained with a modified PPO surrogate loss (DSF-PO); the importance ratios in Eqs. (5), (10), and (11) follow from the policy decomposition in Eq. (4) by the standard PPO construction, rather than by assuming the desired conclusion. No parameter is fitted to a subset of the data and then renamed a prediction, and no load-bearing argument depends on a self-citation: the cited prior works (DribbleBot, DexDribbler, and the locomotion works) are external baselines or inspirations. Two concerns appear in the paper, but they are evaluation-validity or rigor issues, not circularity. First, Table II reports 'ball dribbling success rates' while Section VI.A.2 defines success as 'the robot reaching the terrain boundary,' so the metric does not independently verify ball transport or control; this undermines the empirical comparison, but it is a construct-validity problem rather than a derivation that is circular by construction. Second, Eq. (7) is explicitly called 'partially correct but intuitive,' meaning DSF-PO's gradient intuition is not rigorously derived; again, this is an intellectual-honesty concern about a proof, not a circular step. Because the method is not defined in terms of its measured outcomes and the baselines are external, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-tuned reward weights and PD gains, an unvalidated surrogate loss (DSF-PO), the competence of pre-trained low-level skills, and a simulation success metric that does not require ball control. These should be justified or made available.

free parameters (4)
  • Reward weights in Table I = Given in Table I (e.g., -5.0, 4.0, 8.0, 1.0)
    Hand-chosen weights for the five reward terms directly shape the high-level policy's behavior; no tuning procedure or sensitivity analysis is reported.
  • PD gains for low-level skills = kp=20, kd=0.5 for low-level policies; kp=40, kd=1.0 for locomotion skill πL4
    Selected manually and inherited from prior work; changes the dynamic response of the robot and affects dribbling behavior.
  • Curriculum thresholds d_th1, d_th2, d_max = not specified numerically
    Thresholds used in the curriculum reward terms in Table I; without numerical values the training cannot be replicated.
  • Terrain difficulty values for evaluation = slope=0.1, rough elevation difference=0.1 m, stair height=0.05 m, stair width=0.5 m
    Chosen for experiments; no justification that these values represent rugged terrain generally.
assumptions (4)
  • domain assumption The four pre-trained low-level skills (dribbling πL1,πL2 and locomotion πL3,πL4) are competent in both simulation and the real world, and can be used zero-shot after freezing.
    The high-level policy can only select among these; the whole system's performance is bounded by these skills, yet only video evidence is provided for their individual quality.
  • ad hoc to paper DSF-PO's modified PPO surrogate loss is a valid way to optimize the hierarchical policy.
    No proof or derivation is given; Eq. 7 is labeled 'partially correct but intuitive', and the surrogate is not shown to be an unbiased estimator.
  • ad hoc to paper Reaching the terrain boundary is a sufficient success criterion for ball dribbling in simulation.
    Used in Table II; does not require ball transport or control, so the reported success rates may not reflect dribbling ability.
  • domain assumption The real-world ball position from YOLO and Kalman filtering is accurate enough for the frozen high-level policy to make good skill choices.
    No localization error statistics are given, and the high-level policy was trained on ground-truth ball positions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic Legged Ball Manipulation on Rugged Terrains with Hierarchical Reinforcement Learning." pith.science (2026). https://pith.science/paper/DHDZBN4T

@misc{pith2026250414989,
  author       = {Pith},
  title        = {Pith review of: Dynamic Legged Ball Manipulation on Rugged Terrains with Hierarchical Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHDZBN4T}},
  note         = {Machine review of arXiv:2504.14989}
}
read the original abstract

Advancing the dynamic loco-manipulation capabilities of quadruped robots in complex terrains is crucial for performing diverse tasks. Specifically, dynamic ball manipulation in rugged environments presents two key challenges. The first is coordinating distinct motion modalities to integrate terrain traversal and ball control seamlessly. The second is overcoming sparse rewards in end-to-end deep reinforcement learning, which impedes efficient policy convergence. To address these challenges, we propose a hierarchical reinforcement learning framework. A high-level policy, informed by proprioceptive data and ball position, adaptively switches between pre-trained low-level skills such as ball dribbling and rough terrain navigation. We further propose Dynamic Skill-Focused Policy Optimization to suppress gradients from inactive skills and enhance critical skill learning. Both simulation and real-world experiments validate that our methods outperform baseline approaches in dynamic ball manipulation across rugged terrains, highlighting its effectiveness in challenging environments. Videos are on our website: dribble-hrl.github.io.

Figures

Figures reproduced from arXiv: 2504.14989 by the authors.

Figure 1
Figure 1. Demonstration of legged ball dribbling with hierarchical framework. The high-level policy selects and coordinates pre￾trained dribbling and locomotion skills for dynamic ball manipula￾tion, optimized through deep RL. We deploy the trained policy in the real world via zero-shot transfer, enabling the robot to perform cross-terrain dribbling. term and a context-aided estimator to improve the robot’s ball control capab… view at source ↗
Figure 2
Figure 2. Proposed hierarchical framework. The figure illustrates that when the high-level actor outputs dt = 1, only π L 1 is activated, with the first two dimensions of c L t provided as input. The context-aided estimator network ϕ and all low-level skills π L are frozen during training. 2) Locomotion Skills: Cross-terrain loco-manipulation places significant demands on the quadruped robot’s locomo￾tion capabilities, requir… view at source ↗
Figure 3
Figure 3. Training curves of PPO with DSF-PO compared to standard PPO. The shaded regions indicate the standard deviation over multiple runs [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Cross-terrain dribbling performance evaluation. (a) A trajectory schematic of the ball dribbling across five terrains in sequence: stair descent, ramp-down, rough terrain, ramp-up, and flat ground. Each terrain measures 10 m per side. (b) Visualization of the invocatio…
Figure 5
Figure 5. Figure 5: Usage frequency of low-level skills across different terrains. The numbers represent the proportion of each low-level skill’s usage frequency on a given terrain [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams

    cs.RO 2025-05 conditional novelty 6.0 of 10

    Hierarchical MARL with fictitious self-play trains quadruped soccer teams in simulation and transfers them zero-shot to real robots, enabling onboard, decentralized 1v1 and 2v1 soccer with emergent passing and role al...

Reference graph

Works this paper leans on

38 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [5]

    Dribblebot: Dynamic legged manipulation in the wild,

    Y . Ji, G. B. Margolis, and P. Agrawal, “Dribblebot: Dynamic legged manipulation in the wild,” in ICRA, 2023

    work page 2023

  2. [6]

    Dexdribbler: Learning dexterous soccer manipulation via dynamic supervision,

    Y . Hu, K. Wen, and F. Yu, “Dexdribbler: Learning dexterous soccer manipulation via dynamic supervision,” in IROS, 2024

    work page 2024

  3. [1]

    Rapid locomotion via reinforcement learning,

    G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal, “Rapid locomotion via reinforcement learning,” The International Journal of Robotics Research, 2024

    work page 2024

  4. [2]

    Robot parkour learning,

    Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao, “Robot parkour learning,” in CoRL, 2023

    work page 2023

  5. [3]

    Hierarchical reinforcement learning for precise soccer shooting skills using a quadrupedal robot,

    Y . Ji, Z. Li, Y . Sun, X. B. Peng, S. Levine, G. Berseth, and K. Sreenath, “Hierarchical reinforcement learning for precise soccer shooting skills using a quadrupedal robot,” in IROS, 2022

    work page 2022

  6. [4]

    Learning-based legged locomotion: State of the art and future per- spectives,

    S. Ha, J. Lee, M. van de Panne, Z. Xie, W. Yu, and M. Khadiv, “Learning-based legged locomotion: State of the art and future per- spectives,” The International Journal of Robotics Research , 2024

    work page 2024

  7. [7]

    Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning,

    I. M. A. Nahrendra, B. Yu, and H. Myung, “Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning,” in ICRA, 2023

    work page 2023

  8. [8]

    Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,

    G. Ji, J. Mun, H. Kim, and J. Hwangbo, “Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion,” IEEE Robotics and Automation Letters , 2022

    work page 2022

Show all 38 references
  1. [9]

    Not only rewards but also constraints: Applications on legged robot locomotion,

    Y . Kim, H. Oh, J. Lee, J. Choi, G. Ji, M. Jung, D. Youm, and J. Hwangbo, “Not only rewards but also constraints: Applications on legged robot locomotion,” IEEE Transactions on Robotics , 2024

    2024

  2. [10]

    Learning agile and dynamic motor skills for legged robots,

    J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,” Science Robotics , 2019

    2019

  3. [11]

    Learning quadrupedal locomotion over challenging terrain,

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,” Science Robotics, 2020

    2020

  4. [12]

    Learning to walk in minutes using massively parallel deep reinforcement learning,

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in CoRL, 2022

    2022

  5. [13]

    Learning robust perceptive locomotion for quadrupedal robots in the wild,

    T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,” Science Robotics , 2022

    2022

  6. [14]

    Walk these ways: Tuning robot control for generalization with multiplicity of behavior,

    G. B. Margolis and P. Agrawal, “Walk these ways: Tuning robot control for generalization with multiplicity of behavior,” in CoRL, 2023

    2023

  7. [15]

    Rma: Rapid motor adaptation for legged robots,

    A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,” in Robotics: Science and Systems , 2021

    2021

  8. [16]

    Pedipulate: Enabling manipulation skills using a quadruped robot’s leg,

    P. Arm, M. Mittal, H. Kolvenbach, and M. Hutter, “Pedipulate: Enabling manipulation skills using a quadruped robot’s leg,” in ICRA, 2024

    2024

  9. [17]

    Legs as manipulator: Pushing quadrupedal agility beyond locomotion,

    X. Cheng, A. Kumar, and D. Pathak, “Legs as manipulator: Pushing quadrupedal agility beyond locomotion,” in ICRA, 2023

    2023

  10. [18]

    Creating a dynamic quadrupedal robotic goalkeeper with reinforcement learning,

    X. Huang, Z. Li, Y . Xiang, Y . Ni, Y . Chi, Y . Li, L. Yang, X. B. Peng, and K. Sreenath, “Creating a dynamic quadrupedal robotic goalkeeper with reinforcement learning,” in IROS, 2023

    2023

  11. [19]

    Learning visual quadrupedal loco-manipulation from demonstrations,

    Z. He, K. Lei, Y . Ze, K. Sreenath, Z. Li, and H. Xu, “Learning visual quadrupedal loco-manipulation from demonstrations,” in IROS, 2024

    2024

  12. [20]

    Hilma-res: A general hierarchical framework via residual rl for combining quadrupedal locomotion and manipulation,

    X. Huang, Q. Liao, Y . Ni, Z. Li, L. Smith, S. Levine, X. B. Peng, and K. Sreenath, “Hilma-res: A general hierarchical framework via residual rl for combining quadrupedal locomotion and manipulation,” in IROS, 2024

    2024

  13. [21]

    Precup, Temporal abstraction in reinforcement learning

    D. Precup, Temporal abstraction in reinforcement learning . Univer- sity of Massachusetts Amherst, 2000

    2000

  14. [22]

    Near-optimal representa- tion learning for hierarchical reinforcement learning,

    O. Nachum, S. Gu, H. Lee, and S. Levine, “Near-optimal representa- tion learning for hierarchical reinforcement learning,” in ICLR, 2019

    2019

  15. [23]

    Hierarchical reinforce- ment learning: A survey and open research challenges,

    M. Hutsebaut-Buysse, K. Mets, and S. Latré, “Hierarchical reinforce- ment learning: A survey and open research challenges,” Machine Learning and Knowledge Extraction , 2022

    2022

  16. [24]

    Reinforcement learning in finite mdps: Pac analysis

    A. L. Strehl, L. Li, and M. L. Littman, “Reinforcement learning in finite mdps: Pac analysis.” Journal of Machine Learning Research , 2009

    2009

  17. [25]

    Minimax regret bounds for reinforcement learning,

    M. G. Azar, I. Osband, and R. Munos, “Minimax regret bounds for reinforcement learning,” in ICML, 2017

    2017

  18. [26]

    Why does hierarchy (sometimes) work so well in reinforcement learning?

    O. Nachum, H. Tang, X. Lu, S. Gu, H. Lee, and S. Levine, “Why does hierarchy (sometimes) work so well in reinforcement learning?” arXiv preprint arXiv:1909.10618 , 2019

    1909 arXiv

  19. [27]

    Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning,

    X. B. Peng, G. Berseth, K. Yin, and M. Van De Panne, “Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning,” ACM Trans. Graph. , 2017

    2017

  20. [28]

    Hierarchical reinforcement learning for quadruped locomotion,

    D. Jain, A. Iscen, and K. Caluwaerts, “Hierarchical reinforcement learning for quadruped locomotion,” in IROS, 2019

    2019

  21. [29]

    A hierarchical framework for quadruped omnidirectional locomotion based on reinforcement learning,

    W. Tan, X. Fang, W. Zhang, R. Song, T. Chen, Y . Zheng, and Y . Li, “A hierarchical framework for quadruped omnidirectional locomotion based on reinforcement learning,” IEEE Transactions on Automation Science and Engineering , 2023

    2023

  22. [30]

    Hierarchical skills for efficient exploration,

    J. Gehring, G. Synnaeve, A. Krause, and N. Usunier, “Hierarchical skills for efficient exploration,” in NeurIPS, 2021

    2021

  23. [31]

    Hi- erarchical reinforcement learning with universal policies for multistep robotic manipulation,

    X. Yang, Z. Ji, J. Wu, Y .-K. Lai, C. Wei, G. Liu, and R. Setchi, “Hi- erarchical reinforcement learning with universal policies for multistep robotic manipulation,” IEEE Transactions on Neural Networks and Learning Systems , 2021

    2021

  24. [32]

    Cascaded compositional residual learning for complex interactive behaviors,

    K. N. Kumar, I. Essa, and S. Ha, “Cascaded compositional residual learning for complex interactive behaviors,” IEEE Robotics and Au- tomation Letters , 2023

    2023

  25. [33]

    Asc: Adaptive skill coordination for robotic mobile manipulation,

    N. Yokoyama, A. Clegg, J. Truong, E. Undersander, T.-Y . Yang, S. Ar- naud, S. Ha, D. Batra, and A. Rai, “Asc: Adaptive skill coordination for robotic mobile manipulation,” IEEE Robotics and Automation Letters , 2023

    2023

  26. [34]

    SLR: Learning quadruped locomotion without privileged information,

    S. Chen, Z. Wan, S. Yan, C. Zhang, W. Zhang, Q. Li, D. Zhang, and F. U. D. Farrukh, “SLR: Learning quadruped locomotion without privileged information,” in CoRL, 2024

    2024

  27. [35]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  28. [36]

    Policy gradi- ent methods for reinforcement learning with function approximation,

    R. S. Sutton, D. McAllester, S. Singh, and Y . Mansour, “Policy gradi- ent methods for reinforcement learning with function approximation,” in NeurIPS, 1999

    1999

  29. [37]

    Asymmetric actor critic for image-based robot learning,

    L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, “Asymmetric actor critic for image-based robot learning,” in Robotics: Science and Systems , 2018

    2018

  30. [38]

    Ultralytics yolo11,

    G. Jocher and J. Qiu, “Ultralytics yolo11,” 2024. [Online]. Available: https://github.com/ultralytics/ultralytics

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.