Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Unsupervised Skill Discovery as Exploration for Learning Agile Locomotion

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A simple task reward plus unsupervised skill discovery can train a quadruped to leap, climb, crawl, and execute a wall-jump.

desk verdict The combination works and the wall-jump is real, but the 'autonomous exploration' framing oversells it—you still have to hand-pick which state dimensions matter. read the letter →

arxiv 2508.08982 v1 pith:MES3EWZX submitted 2025-08-12 cs.RO cs.AIcs.LG

Seungeun Rho , Kartik Garg , Morgan Byrd , Sehoon Ha This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords UnsupervisedSkillDiscoveryExplorationLeggedLocomotionReinforcementLearningQuadrupedRobotParkourSim-to-RealTransferBi-levelOptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper demonstrates that a quadruped can learn four agile obstacle-crossing behaviors—leaping, climbing, crawling, and a wall-jump—using only a simple task reward plus an unsupervised skill-discovery reward, with no demonstrations, curricula, or careful reward shaping. The key claim is that diversity-based exploration, rather than state-level novelty, is what lets the policy find the required heights, velocities, and orientations. A learned balancing parameter automatically schedules exploration: high diversity pressure early, more task focus later. If correct, this removes a significant human-engineering bottleneck in agile locomotion and points to skill discovery as a generic exploration primitive for robot learning.

What carries the argument

The load-bearing mechanism is the combination of a skill-conditioned policy $\pi_\theta(a|s,z)$ with a learnable balancing parameter $\lambda$ that weights a diversity reward against a task reward. The diversity reward is taken from METRA, $r^{\mathrm{div}} = (\phi(s_{t+1})-\phi(s_t))^\top z$, with $\phi$ a state representation constrained to be 1-Lipschitz, so maximizing it encourages different latent skills $z$ to visit different states. The balancing parameter is updated by a bi-level gradient rule, Equation (3), which increases $\lambda$ when the task-advantage gradient and diversity-advantage gradient agree and decreases it when they conflict. What makes the scheme practical is 'positive collapse': because all skills share one policy network, once task rewards select a successful behavior, an increasing fraction of randomly sampled skills adopts it, so no skill-selection mechanism is needed at deployment.

What would settle it

Give SDAX a new obstacle whose solution depends on a state dimension absent from the manual list—say, a narrow diagonal gap that requires lateral roll when only base height and forward velocity are specified—and train several seeds with the same budget. If no skill vector consistently solves the task, the claim that SDAX reduces human engineering effort for arbitrary obstacles is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that the SDAX framework—a skill-conditioned policy trained on $r^{\mathrm{task}} + \lambda r^{\mathrm{div}}$ with $\lambda$ adjusted by a bi-level gradient rule—can learn a quadruped's agile obstacle-crossing skills, including leaping a 48 cm gap, climbing a 25 cm platform, crawling under a 29 cm obstacle, and performing a wall-jump, without demonstration data or a manually designed curriculum. The task reward is deliberately simple (forward velocity tracking plus mild penalty terms), and the diversity reward comes from unsupervised skill discovery, either METRA or DIAYN. The adaptive $\lambda$ is what makes this work: it starts high to encourage broad exploration and then decreases as task reward becomes the dominant signal, outperforming all fixed values of $\lambda$ tested on the leap task. After a short fine-tuning phase with observation noise and domain randomization, the learned policies transfer to a real Unitree A1 robot.

Load-bearing premise

SDAX's success rests on a human first specifying which sub-dimensions of the robot state the diversity reward should explore; if a task needs exploration along a dimension the designer did not list, the framework has no mechanism to discover that dimension.

Editorial extensions

If this is right

  • On the three obstacle tasks, SDAX with METRA learns to clear all obstacles while the task-only baseline does not, showing diversity reward is necessary for these skills.
  • Adaptively training $\lambda$ beats every fixed value tested (0.01, 0.1, 1, 10) on leaping, with better sample efficiency and final performance.
  • As training progresses, the fraction of random skill vectors that solve each task rises (for leaping, from roughly 43% to 97%), so deployment needs no skill-selection module.
  • The same framework, with the robot's roll, pitch, and yaw added to the diversity objective, learns a wall-jump that a task-only policy cannot achieve.
  • After fine-tuning with observation noise and domain randomization, the learned policies transfer to real hardware for leaping, climbing, and crawling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The manual sub-dimension requirement means SDAX is best framed as replacing reward or curriculum engineering with a cheaper but still human-provided 'exploration dimension' choice; an obvious extension is to learn these dimensions automatically from the task reward itself.
  • Editorial inference: Because the benefit of METRA over DIAYN is attributed to non-saturating diversity, SDAX should benefit from any future skill-discovery method with a non-saturating metric, making the framework modular.
  • Editorial inference: The positive-collapse effect suggests SDAX could be adapted to multi-task or open-ended settings where a single shared policy is fine-tuned on successive obstacles, with the diversity reward re-seeding exploration each time.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SDAX, a framework for legged locomotion that combines a task reward with an unsupervised skill-discovery diversity reward (METRA or DIAYN) and learns a scalar balancing parameter λ through a bi-level surrogate gradient (Eq. 3). The policy is skill-conditioned and trained with PPO. Experiments on a Unitree A1 in Isaac Gym show that SDAX with METRA solves leaping, climbing, crawling, and a wall-jump, outperforms fixed-λ variants on the leap task, and transfers to real hardware for leap, climb, and crawl. The paper also reports a 'positive collapse' phenomenon in which a growing fraction of sampled skill vectors solve the task.

Significance. If the results hold, SDAX provides a simple exploration recipe that can replace curricula or demonstrations for several agile locomotion skills, and the multi-seed training curves and real-robot deployment are genuine strengths. The method's reliance on manually chosen state sub-dimensions for the diversity reward and a human-specified guideline for the wall-jump, however, materially limits the claimed reduction in human engineering effort.

major comments (4)
  1. [Appendix A, Eq. (3) and Eq. (7)] The derivation of the balancing-parameter gradient is not valid as written. In Eq. (6), A^{task+div} is treated as a constant when differentiating the product A^{task+div}∇θ log πθ(a|s,z) with respect to λ, but both A^{task} and A^{div} depend on λ through the policy, and the derivative of ∇θ J^{task} with respect to λ is omitted. The derivation also drops expectation signs and treats single-sample advantages as exact gradient estimates. Since the λ update is contribution (2) and is used in Algorithm 1 (line 17), please supply a correct derivation or explicitly state that Eq. (3) is a heuristic surrogate, and justify it empirically, for example by comparing against alternative λ-update rules on the same tasks.
  2. [Section 6 and Section 4.1] The central claim of autonomous exploration is not established for tasks whose relevant behavioral dimension is not known in advance. The diversity reward r_div = (φ(s')−φ(s))^T z only rewards change along the state features provided to φ, and the paper reports that for each task a human chose the sub-dimensions: base height for climb/crawl, forward velocity for leap, and base orientation for wall-jump. Section 6 concedes that 'effective training requires manual specification of sub-dimensions of the state space to guide exploration.' If the correct dimension is omitted, λ has no mechanism to increase exploration along it, and SDAX reduces to task-only training, which the paper itself shows fails on the wall-jump (Fig. 9b). Please either add a mechanism for discovering or expanding the explored dimensions, or temper the claim that SDAX 'autonomously exploring a diverse set of strategies' reduces human engineering for novel tasks.
  3. [Section 4.4 and Appendix D.1] The wall-jump experiment uses a human-specified guideline reward as r_task, consisting of a sequence of user-defined waypoints. This is a form of task-specific reward engineering, which the Introduction lists among the techniques SDAX aims to reduce. The wall-jump result should be framed as SDAX combined with a hand-designed task reward, and the paper should report how sensitive the result is to the guideline shape, such as the number of waypoints or their spacing.
  4. [Section 4.2] The adaptive-λ comparison against fixed λ values is conducted only on the leaping task. Since the paper claims that the learned λ dynamically balances exploration and exploitation and that 'our adaptive method outperforms all fixed-value experiments,' the claim is stronger than the evidence. Please run the fixed-λ comparison on at least one additional task (e.g., climb or crawl) or present the λ trajectories for all tasks to support the general mechanism.
minor comments (5)
  1. [Section 3.2] There is a typo: 'straightfoward' should be 'straightforward.'
  2. [Table 3] The observation table lists 'Sampled skill' with dimension 1, but the text says z is sampled from N(0,I) and is used in the diversity reward as a vector. Please clarify the skill dimension used in experiments and whether z is scalar or vector.
  3. [Section 4.1] The Div-only baseline is given the robot's base x position as an additional input to the skill discovery algorithm, while the other methods are not described as receiving this input. Please state why this extra input is needed for Div-only and how it affects the comparison.
  4. [Section 4.5] The real-robot experiments are reported qualitatively via figures and video. To support the transfer claim, please report the number of trials and the success count for each hardware experiment.
  5. [Appendix A] The derivation defines θ′ as the parameter after a single gradient step, but Algorithm 1 uses PPO with multiple epochs. Please clarify the exact update used when computing the surrogate gradient in practice.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SDAX's lambda update is an explicit bi-level optimization, the diversity reward comes from external skill-discovery methods, and the performance claims are validated by comparisons and real-hardware transfer.

full rationale

The paper's central mechanism is a bi-level optimization in which the policy maximizes r_task + lambda * r_div and the balancing parameter lambda is trained to maximize only the task return (Eq. 1-3). The derivation of Eq. 3 in Appendix A uses the policy gradient theorem and a one-step expansion of the policy update; it does not assume the result it claims to explain. The diversity reward is taken from METRA/DIAYN, which are external methods (Eq. 4 in the paper cites Park et al. for the Wasserstein Dependency Measure formulation), so SDAX does not define its own exploration signal in terms of task success. The adaptive-lambda claim is tested empirically against fixed lambda values, and the locomotion results are compared with task-only, diversity-only, RND, and DIAYN baselines, plus real-robot deployment. The only author self-citation is reference [22] (LGSD), which appears in the related-work taxonomy and is not load-bearing for any derivation. The acknowledged limitation in Section 6, that effective training requires manual specification of state sub-dimensions, concerns the breadth of the claimed reduction in human engineering effort, not circularity: specifying base height or orientation as an input to the diversity objective does not by itself produce crawling or wall-jumping, and the paper shows task-only training fails. No equation in the paper reduces a prediction to a fitted parameter or to its own inputs by construction, so no circular step can be exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central method depends on a hand-chosen lambda initialization, hand-picked exploration sub-dimensions, and a human-drawn guideline for the wall-jump task. It adds no new physical entities, but it does rely on several domain assumptions and one paper-specific gradient approximation.

free parameters (3)
  • Initial balancing parameter lambda_0 = 10.0
    The algorithm learns lambda, but its starting value is chosen by hand; Fig. 5(b) shows it starts at 10.0 and decreases. The training dynamics depend on this initialization.
  • Manual exploration sub-dimensions for diversity reward = per-task choices: base height for climb and crawl, forward velocity for leap, base orientation for wall-jump
    These choices encode human domain knowledge about what to explore, and Section 6 states that effective training requires this manual specification.
  • Wall-jump guideline waypoints = not provided numerically
    A human-drawn sequence of 3D points defines the task reward for wall-jump. Without these coordinates, the wall-jump result is not fully reproducible from the text alone.
assumptions (4)
  • domain assumption The MDP abstraction with joint-position PD targets adequately models the Unitree A1 in Isaac Gym and on hardware.
    Invoked in Sec. 3.1. If the simulation or PD model is not faithful, the learned behaviors may not transfer, though domain randomization partially addresses this.
  • domain assumption METRA's Wasserstein dependency measure, optimized via dual gradient descent, provides a stable and useful diversity reward when restricted to manually chosen sub-dimensions.
    Invoked in Sec. 3.3 and Sec. 4.1. If the diversity reward saturates or decorrelates from task progress, exploration could fail to produce useful behaviors.
  • ad hoc to paper The one-step surrogate gradient in Eq. (3) and Appendix A accurately estimates the effect of lambda on task return.
    Eq. (7) computes lambda updates from sample advantages and one gradient step. The derivation drops expectations and higher-order terms, so this is a paper-specific approximation rather than a theorem.
  • domain assumption Observation noise and domain randomization parameters are sufficient for sim-to-real transfer.
    Used in Sec. 4.5. Hardware results are qualitative, so the sufficiency of these ranges is supported only by the reported deployments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unsupervised Skill Discovery as Exploration for Learning Agile Locomotion." pith.science (2026). https://pith.science/paper/MES3EWZX

@misc{pith2026250808982,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Skill Discovery as Exploration for Learning Agile Locomotion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MES3EWZX}},
  note         = {Machine review of arXiv:2508.08982}
}
read the original abstract

Exploration is crucial for enabling legged robots to learn agile locomotion behaviors that can overcome diverse obstacles. However, such exploration is inherently challenging, and we often rely on extensive reward engineering, expert demonstrations, or curriculum learning - all of which limit generalizability. In this work, we propose Skill Discovery as Exploration (SDAX), a novel learning framework that significantly reduces human engineering effort. SDAX leverages unsupervised skill discovery to autonomously acquire a diverse repertoire of skills for overcoming obstacles. To dynamically regulate the level of exploration during training, SDAX employs a bi-level optimization process that autonomously adjusts the degree of exploration. We demonstrate that SDAX enables quadrupedal robots to acquire highly agile behaviors including crawling, climbing, leaping, and executing complex maneuvers such as jumping off vertical walls. Finally, we deploy the learned policy on real hardware, validating its successful transfer to the real world.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 12 canonical work pages

  1. [1]

    J. Tan, T. Zhang, E. Coumans, A. Iscen, Y . Bai, D. Hafner, S. Bohez, and V . Vanhoucke. Sim- to-real: Learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804.10332 , 2018

    arXiv 2018

  2. [3]

    Hwangbo, J

    J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter. Learn- ing agile and dynamic motor skills for legged robots. Science Robotics, 4(26):eaau5872, 2019

    2019

  3. [4]

    Z. Xie, X. Da, M. Van de Panne, B. Babich, and A. Garg. Dynamics randomization revisited: A case study for quadrupedal locomotion. pages 4955–4961, 2021

    work page 2021

  4. [5]

    X. Song, Y . Yang, K. Choromanski, K. Caluwaerts, W. Gao, C. Finn, and J. Tan. Rapidly adaptable legged robots via evolutionary meta-learning. pages 3769–3776, 2020

    work page 2020

  5. [6]

    Haarnoja, S

    T. Haarnoja, S. Ha, A. Zhou, J. Tan, G. Tucker, and S. Levine. Learning to walk via deep reinforcement learning. arXiv preprint arXiv:1812.11103, 2018

    arXiv 2018

  6. [7]

    Smith, J

    L. Smith, J. C. Kew, T. Li, L. Luu, X. B. Peng, S. Ha, J. Tan, and S. Levine. Learning and adapting agile locomotion skills by transferring experience. arXiv preprint arXiv:2304.09834, 2023

    arXiv 2023

  7. [8]

    S. Luo, S. Li, R. Yu, Z. Wang, J. Wu, and Q. Zhu. Pie: Parkour with implicit-explicit learning framework for legged robots. IEEE Robotics and Automation Letters , 2024

    work page 2024

  8. [9]

    Zhuang, Z

    Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao. Robot parkour learning. In Conference on Robot Learning (CoRL) , 2023

    2023

Show all 42 references
  1. [10]

    Cheng, K

    X. Cheng, K. Shi, A. Agarwal, and D. Pathak. Extreme parkour with legged robots. arXiv preprint arXiv:2309.14341, 2023

    2023 arXiv

  2. [11]

    Y . Yang, G. Shi, X. Meng, W. Yu, T. Zhang, J. Tan, and B. Boots. Cajun: Continuous adaptive jumping using a learned centroidal controller. arXiv preprint arXiv:2306.09557, 2023

    2023 arXiv

  3. [12]

    Bogdanovic, M

    M. Bogdanovic, M. Khadiv, and L. Righetti. Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization. Frontiers in Robotics and AI , 9:854212, 2022

    2022

  4. [13]

    Kilinc and G

    O. Kilinc and G. Montana. Reinforcement learning for robotic manipulation using simulated locomotion demonstrations. Machine Learning, pages 1–22, 2022

    2022

  5. [14]

    C. Li, M. Vlastelica, S. Blaes, J. Frey, F. Grimminger, and G. Martius. Learning agile skills via adversarial imitation of rough partial demonstrations. In Conference on Robot Learning , pages 342–352. PMLR, 2023

    2023

  6. [15]

    Z. He, K. Lei, Y . Ze, K. Sreenath, Z. Li, and H. Xu. Learning visual quadrupedal loco- manipulation from demonstrations. arXiv preprint arXiv:2403.20328, 2024. 10

    2024 arXiv

  7. [16]

    Kumar, Z

    A. Kumar, Z. Fu, D. Pathak, and J. Malik. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021

    2021 arXiv

  8. [17]

    R. S. Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018

    2018

  9. [18]

    Eysenbach, A

    B. Eysenbach, A. Gupta, J. Ibarz, and S. Levine. Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv:1802.06070, 2018

    2018 arXiv

  10. [19]

    Gregor, D

    K. Gregor, D. J. Rezende, and D. Wierstra. Variational intrinsic control. arXiv preprint arXiv:1611.07507, 2016

    2016 arXiv

  11. [20]

    S. Park, J. Choi, J. Kim, H. Lee, and G. Kim. Lipschitz-constrained unsupervised skill discov- ery. arXiv preprint arXiv:2202.00914, 2022

    2022 arXiv

  12. [21]

    S. Park, O. Rybkin, and S. Levine. Metra: Scalable unsupervised rl with metric-aware abstrac- tion. arXiv preprint arXiv:2310.08887, 2023

    2023 arXiv

  13. [22]

    S. Rho, L. Smith, T. Li, S. Levine, X. B. Peng, and S. Ha. Language guided skill discovery. arXiv preprint arXiv:2406.06615, 2024

    2024 arXiv

  14. [23]

    G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal. Rapid locomotion via rein- forcement learning. arXiv preprint arXiv:2205.02824, 2022

    2022 arXiv

  15. [24]

    Z. Fu, A. Kumar, J. Malik, and D. Pathak. Minimizing energy consumption leads to the emer- gence of gaits in legged robots. arXiv preprint arXiv:2111.01674, 2021

    2021 arXiv

  16. [25]

    Z. Li, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath. Robust and ver- satile bipedal jumping control through multi-task reinforcement learning. arXiv preprint arXiv:2302.09450, 2023

    2023 arXiv

  17. [26]

    Y . Yang, X. Meng, W. Yu, T. Zhang, J. Tan, and B. Boots. Continuous versatile jumping using learned action residuals. In Learning for Dynamics and Control Conference , pages 770–782. PMLR, 2023

    2023

  18. [27]

    Rudin, D

    N. Rudin, D. Hoeller, M. Bjelonic, and M. Hutter. Advanced skills by learning locomotion and local navigation end-to-end. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2497–2503. IEEE, 2022

    2022

  19. [28]

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter. Learning quadrupedal locomo- tion over challenging terrain. Science robotics, 5(47):eabc5986, 2020

    2020

  20. [29]

    Cheng, M

    J. Cheng, M. Vlastelica, P. Kolev, C. Li, and G. Martius. Learning diverse skills for local nav- igation under multi-constraint optimality. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5083–5089. IEEE, 2024

    2024

  21. [30]

    Atanassov, W

    V . Atanassov, W. Yu, A. L. Mitchell, M. N. Finean, and I. Havoutis. Constrained skill dis- covery: Quadruped locomotion with unsupervised reinforcement learning. arXiv preprint arXiv:2410.07877, 2024

    2024 arXiv

  22. [31]

    Zheng, J

    Z. Zheng, J. Oh, and S. Singh. On learning intrinsic rewards for policy gradient methods. Advances in Neural Information Processing Systems , 31, 2018

    2018

  23. [32]

    Ozair, C

    S. Ozair, C. Lynch, Y . Bengio, A. Van den Oord, S. Levine, and P. Sermanet. Wasserstein dependency measure for representation learning. Advances in Neural Information Processing Systems, 32, 2019

    2019

  24. [33]

    Makoviychuk, L

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac gym: High performance gpu-based physics simula- tion for robot learning, 2021. 11

    2021

  25. [34]

    Rudin, D

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. InConference on Robot Learning, pages 91–100. PMLR, 2022

    2022

  26. [35]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  27. [36]

    Burda, H

    Y . Burda, H. Edwards, A. Storkey, and O. Klimov. Exploration by random network distillation. arXiv preprint arXiv:1810.12894, 2018

    2018 arXiv

  28. [37]

    Z. Tang, D. Kim, and S. Ha. Learning agile motor skills on quadrupedal robots using curricu- lum learning. In International Conference on Robot Intelligence Technology and Applications, volume 3, 2021

    2021

  29. [38]

    J. Gu, S. Kirmani, P. Wohlhart, Y . Lu, M. G. Arenas, K. Rao, W. Yu, C. Fu, K. Gopalakrishnan, Z. Xu, et al. Rt-trajectory: Robotic task generalization via hindsight trajectory sketches. arXiv preprint arXiv:2311.01977, 2023

    2023 arXiv

  30. [39]

    Tobin, R

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) , pages 23–30. IEEE, 2017

    2017

  31. [40]

    R. S. Sutton, D. McAllester, S. Singh, and Y . Mansour. Policy gradient methods for rein- forcement learning with function approximation. Advances in neural information processing systems, 12, 1999

    1999

  32. [41]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  33. [42]

    Schulman, P

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015

    2015 arXiv

  34. [43]

    Clevert, T

    D.-A. Clevert, T. Unterthiner, and S. Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289, 2015. 12 A Proof of Equation 3 We begin with the Equation (1), which is the decomposition of∇λJ task using the chain r...

    2015 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.