Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Constraints as Rewards: Reinforcement Learning for Robots without Reward Functions

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Setting the reward to zero and expressing the task as constraints lets Lagrange multipliers balance objectives automatically, and a six-wheeled robot learns to stand up where hand-designed reward functions fail.

desk verdict Real robot result, but the theory as written is wrong: Eq (7) is unbounded and Algorithm 1 minimizes over λ rather than maximizing, so the automatic-weighting claim doesn't follow. read the letter →

arxiv 2501.04228 v2 pith:VRWAL5XZ submitted 2025-01-08 cs.RO cs.AIcs.LG

Yu Ishihara , Noriaki Takasugi , Kotaro Kawakami , Masaya Kinoshita , Kazumi Aoyama This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords constraintsasrewardsrewardengineeringreinforcementlearningwithLagrangianmethodQRSACleggedrobotstandingupTachyon3multi-objectivebalancing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Constraints as Rewards (CaR) is a proposal for training robots without writing a reward function: the task is described by constraint functions, the reward is set to zero, and the Lagrange multipliers serve as the weights that balance the task objectives. The paper argues this removes the trial-and-error of reward engineering because the weights adjust automatically during training instead of being tuned by hand. To make the idea practical, it introduces QRSAC-Lagrangian, a quantile-based actor-critic that alternates policy updates with multiplier updates, along with four reusable constraint templates covering timestep and episode requirements. It applies the method to the standing-up task of a six-wheeled telescopic-legged robot, Tachyon 3, and reports that the learned policy stands the robot up in simulation, on unfamiliar terrain, and on the physical robot, while several manually designed reward functions fail to learn the task. The paper's central claim is that, for tasks expressible as constraints, reward engineering can be replaced by constraint design with automatically balanced weights.

What carries the argument

The load-bearing object is the Lagrangian objective with zero reward, $L(\pi, \lambda) = \sum_{m=1}^{M}\lambda_m \mathbb{E}_\pi\left[\sum_{t=0}^{T}\gamma^t g_m(s_t,a_t)\right]$, maximized with respect to both the policy and the multiplier vector; the multipliers are the weights that balance task objectives. Carrying the practical implementation is QRSAC-Lagrangian, an extension of QRSAC whose quantile critic estimates the distribution of Q values rather than a point estimate, which the authors argue is more stable when the target distribution shifts as the multipliers update. The four constraint templates are timestep probability constraints, timestep value constraints, episode probability constraints, and episode value constraints, each written so that the requirement $\mathbb{E}\left[\sum_t \gamma^t g(s_t,a_t)\right] \ge 0$ has an intuitive interpretation in terms of the task.

What would settle it

Run the same training while logging each constraint weight $\lambda_m$ and the Lagrangian value $L$; if the multiplier update lowers $L$ whenever a constraint is violated, then the update is moving opposite to the max-over-$\lambda$ objective in Eq. (7), and the stated automatic-balancing justification is not what the algorithm implements.

Watch

Extended reading notes

Core claim

The paper's central claim is that reward engineering can be sidestepped by replacing the reward function with a set of constraint functions and setting the reward to zero. The resulting objective, $\max_{\lambda_m > 0, \pi} \sum_{m=1}^{M} \lambda_m \mathbb{E}_\pi\left[\sum_{t=0}^{T}\gamma^t g_m(s_t,a_t)\right]$, is a Lagrange dual form of a constrained reinforcement learning problem, and the multipliers act as automatically tuned weights among the objectives. To solve this objective, the paper proposes QRSAC-Lagrangian, an extension of QRSAC whose quantile critic estimates the distribution of Q values rather than a point estimate, arguing that this copes with the shifting distribution of target values as multipliers update. In experiments on Tachyon 3, the method is reported to learn the standing-up motion from arbitrary initial poses in simulation, generalize to rough and sloped terrain, and transfer to the physical robot, while five manually designed reward functions and two of the three comparison algorithms fail to learn the task.

Load-bearing premise

The method assumes that repeatedly alternating between improving the policy and adjusting the constraint weights converges to the balanced solution the constrained problem asks for, even though the optimization problem is not convex and the weight update in the pseudocode moves opposite to the direction the equations prescribe.

Editorial extensions

If this is right

  • For tasks expressible as constraints, designers can skip hand-tuning reward weights; the multiplier updates supply the balance during training.
  • QRSAC-Lagrangian is reported to converge faster and more stably than SAC-Lagrangian, PPO-Lagrangian, and CaT on the standing-up task and on an inverted pendulum benchmark.
  • A policy trained only in a flat simulation stands Tachyon 3 up on rough terrain, a 10-degree slope, and the physical robot without fine-tuning.
  • CaR does not remove objective design; it moves the burden from weighted reward functions to inequality constraints, and it leaves pure maximization tasks such as 'walk as fast as possible' outside its scope.
  • The four constraint templates give task designers a reusable vocabulary for composing terminal and throughout-episode requirements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the pseudocode is taken literally, the multiplier update descends rather than ascends, so the implemented objective may be the conventional constrained-RL dual (min over $\lambda$ of the max over $\pi$) rather than the max-over-$\lambda$ objective in Eq. (7); the reported balancing would then be an empirical feature of the alternating scheme, not a consequence of the stated dual problem.
  • A direct test: freeze the multipliers at their final learned values and retrain from scratch. If the final policy is unchanged, the dynamic adjustment is not what produces the behavior; if it degrades, the adaptive weighting is load-bearing.
  • A natural extension is a hybrid objective that adds a small reward for pure maximization while keeping constraints for safety and posture, since the paper's own limitation discussion points toward combining rewards with constraints.
  • The quantile critic's apparent robustness to shifting target distributions could transfer to other reinforcement learning settings with non-stationary objectives, such as curriculum learning or meta-learned rewards.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes 'Constraints as Rewards' (CaR), a method that replaces the reward function in a reinforcement learning problem with a set of constraint functions, and solves the resulting constrained problem with a Lagrangian-based algorithm called QRSAC-Lagrangian. The authors claim that solving the Lagrangian problem with a zero reward function automatically balances the weights of different task objectives, because the Lagrange multipliers act as the weights. The method is evaluated on a standing-up task for a six-wheeled telescopic-legged robot (Tachyon 3) in simulation and on the real robot, where it succeeds from all tested initial poses, whereas several manually designed reward functions fail. The paper also proposes four constraint-function designs and an ablation study of the five constraints used in the task.

Significance. If the theoretical foundation were sound, the contribution would be significant: it addresses reward engineering, a major practical bottleneck in robot learning, and the real-robot demonstration is a genuine asset. The paper ships a concrete algorithmic recipe (QRSAC-Lagrangian), a clear constraint-design taxonomy, and a reproducible-sounding experimental protocol. However, the central theoretical claim of automatic weight tuning rests on an optimization formulation that is ill-posed as written, and the actual algorithm solves a different saddle-point problem. The empirical standing-up result is credible and valuable, but the manuscript must be revised to make the optimization objective correct and to clarify what the Lagrange multipliers are doing. The current text does not support the strong statement that the weights are automatically balanced in the sense of Eq. (7).

major comments (4)
  1. [Section IV-A, Eq. (5) and Eq. (7)] The optimization problem in Eq. (7) is not well-posed. For any policy π with E_π[Σ γ^t g_m(s_t,a_t)] > 0, the objective Σ_m λ_m E_π[Σ γ^t g_m] grows without bound as λ_m → ∞, so no finite maximizer over λ exists; for E_π[Σ γ^t g_m] < 0, the supremum is approached only as λ_m → 0, which is excluded by the strict constraint λ_m > 0. Thus the 'max over λ' in Eqs. (5) and (7) cannot be the basis for the claimed automatic weight tuning.
  2. [Algorithm 1, line 9] Algorithm 1 updates λ by gradient descent (λ_m ← λ_m − Adam(α_λ, ∇_{λ_m} L(π, λ))) and clamps to nonnegative values. This is the standard saddle-point update for min_{λ≥0} max_π L(π,λ), not the max over λ of Eq. (7). The manuscript therefore describes and analyzes one objective while implementing another. The theoretical justification for automatic balancing collapses unless the text is rewritten to state the actual saddle-point problem and to explain how the zero-reward limit fits within it.
  3. [Section IV-A, paragraph after Eq. (7)] The claim that 'Lagrange multipliers, which serves as weights among constraints, can be tuned automatically' is only true in a limited sense. In the standard constrained RL interpretation, λ_m increases for violated constraints and decreases toward zero for satisfied constraints. In the zero-reward case, once all constraints are satisfied, the Lagrangian objective becomes identically zero and provides no further learning signal; the policy is then only kept inside the feasible set by the constraints themselves. The paper should state this mechanism explicitly rather than implying that Eq. (7) yields a well-posed weighted-sum maximization.
  4. [Section VI-B, Fig. 4a and Table III] The learning curve shows the average of 5 runs but no error bars or variance shading, and Table III reports averages of 10 runs without variance. The claims that QRSAC-Lagrangian converges 'faster and more robustly' than SAC-Lagrangian, and that each ablation constraint improves its corresponding metric, need uncertainty quantification; as presented, the differences in Table III (e.g., Pose 0.925 vs 0.920 vs 0.901) may be within run-to-run noise.
minor comments (6)
  1. [Section IV-A, Eq. (5)] The text calls Eq. (5) the 'Lagrange dual function' and its maximization the 'Lagrange dual'; in standard constrained optimization, the dual problem for constraints E[Σ γ^t g_m] ≥ 0 is min_{λ≥0} max_π L(π,λ), not max_{λ>0} max_π L(π,λ). Please correct the terminology.
  2. [Algorithm 1 and Section IV-A] Algorithm 1 initializes λ = 0 and clamps to max(λ_m, 0), which is inconsistent with the strict positivity constraint λ_m > 0 in Eq. (5). The paper should use λ ≥ 0 consistently.
  3. [Section VI-B, caption of Fig. 4] The caption says 'The weight of the pose parameter shown on the right is the weight of the left-front hip joint', but the figure shows the weight of a pose constraint; please clarify the wording.
  4. [Appendix I-D] Typo: 'sustituting' should be 'substituting'.
  5. [Section VI-B] Typo: 'distrubution' should be 'distribution'.
  6. [Table VI] Typo: 'accelaration' should be 'acceleration'.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: CaR's objective is a substitution, lambda-as-weights is definitional, and the empirical comparisons are external benchmarks.

full rationale

No circular step is exhibited. The central transformation is a substitution: Eq. (7) follows from Eq. (6) by setting r(st, at) = 0, and the Lagrange multipliers are, by construction, the coefficients multiplying each constraint return in the Lagrangian. The 'automatic tuning' claim is attached to the lambda-update mechanism in Algorithm 1 rather than to an independently predicted quantity, so it is a method construction, not a fitted parameter renamed as a prediction. The constraint-function designs in Appendix I are algebraic identities (E[sum gamma^t g] >= 0 iff p_eps >= P(...) or eps >= E[g_hat]), with derivations supplied in the paper; they do not import the target result. The constraint thresholds (10^-3, 10^-2, p_eps, eps) are hand-specified task definitions, not fitted values later called predictions. The experimental claims are checked against external baselines (SAC-Lagrangian, PPO-Lagrangian, CaT), five manually designed reward functions, and a Gym inverted-pendulum benchmark, so the empirical content is not forced by the method's definition. The only overlapping-author citation is [10], used for Tachyon 3 hardware details; it is an implementation platform reference, externally falsifiable, and not load-bearing for the method's validity. I separately note a non-circularity correctness concern: Eq. (5)/(7) maximize over lambda, which is not a well-posed dual objective when constraint slack is positive (the objective can become unbounded), while Algorithm 1 line 9 actually descends on lambda. This is a mathematical-consistency issue, not a reduction of a prediction to its inputs, so it does not raise the circularity score.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The method's task specification rests on hand-chosen constraint thresholds (pose tolerance, safety limits) and on unproven convergence of primal-dual RL. No new physical entities are introduced.

free parameters (6)
  • Final pose threshold ε_pose = 1e-3 rad, 1e-3 m (12 constraints)
    Hand-chosen tolerance for hip joint and knee position at final timestep; defines task success. Location: Sec V-A Eq (12).
  • Inclination threshold ε_incl = 1e-2
    Hand-chosen tolerance on body alignment with z-axis. Location: Sec V-A Eq (16).
  • Swing velocity limit ω_max = 2.0 rad/s
    Hand-chosen safety limit on hip joint angular velocity. Location: Sec V-A Eq (15).
  • Fall-down probability threshold p_ϵ = 0
    Hand-chosen; constrains fall-down probability to zero. Location: Sec V-A Eq (13).
  • Body contact probability threshold p_ϵ = 0
    Hand-chosen; constrains body-floor contact probability to zero. Location: Sec V-A Eq (14).
  • Curriculum schedule = not a scalar
    Starting pose distribution near upright and gradually increasing distance; hand-designed. Location: Sec VI-A.
assumptions (3)
  • domain assumption The Lagrangian dual formulation of Eq (4) is a valid reformulation with no duality gap, and the alternating primal-dual updates converge to a feasible policy.
    Assumed throughout Section IV-A and Algorithm 1; not proven for the non-convex RL objective. The sign error in Eq (5) weakens this assumption.
  • domain assumption The quantile function in QRSAC estimates the distribution of Q values sufficiently well under changing Lagrange multipliers.
    Section IV-C invokes this to justify QRSAC-Lagrangian; no theoretical guarantee.
  • domain assumption The MuJoCo simulation is a faithful model of the real Tachyon 3 robot so that policies transfer without fine-tuning.
    Relies on the robot model described in [10]; no explicit validation of dynamics fidelity beyond the qualitative success.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Constraints as Rewards: Reinforcement Learning for Robots without Reward Functions." pith.science (2026). https://pith.science/paper/VRWAL5XZ

@misc{pith2026250104228,
  author       = {Pith},
  title        = {Pith review of: Constraints as Rewards: Reinforcement Learning for Robots without Reward Functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VRWAL5XZ}},
  note         = {Machine review of arXiv:2501.04228}
}
read the original abstract

Reinforcement learning has become an essential algorithm for generating complex robotic behaviors. However, to learn such behaviors, it is necessary to design a reward function that describes the task, which often consists of multiple objectives that needs to be balanced. This tuning process is known as reward engineering and typically involves extensive trial-and-error. In this paper, to avoid this trial-and-error process, we propose the concept of Constraints as Rewards (CaR). CaR formulates the task objective using multiple constraint functions instead of a reward function and solves a reinforcement learning problem with constraints using the Lagrangian-method. By adopting this approach, different objectives are automatically balanced, because Lagrange multipliers serves as the weights among the objectives. In addition, we will demonstrate that constraints, expressed as inequalities, provide an intuitive interpretation of the optimization target designed for the task. We apply the proposed method to the standing-up motion generation task of a six-wheeled-telescopic-legged robot and demonstrate that the proposed method successfully acquires the target behavior, even though it is challenging to learn with manually designed reward functions.

Figures

Figures reproduced from arXiv: 2501.04228 by the authors.

Figure 1
Figure 1. Standing-up motion generation task of a six-wheeled [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Relationship between uz, vx and vy. 1) Final pose constraint: g(s, a) = ( 0 (t ̸= T) 10−3 − |starget − sangle/position| (t = T). (12) Existing research [5] uses an episodic reward that penalizes the robot’s pose at every timestep to facil￾itate standing up. However, from the perspective of constraints, the intermediate pose of the robot should be arbitrary. Therefore, we constrain the hip joint angle and knee positi… view at source ↗
Figure 3
Figure 3. Final poses of the robot. Leftmost figure shows the initial pose of each row. The policy trained with manually [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Policy execution results on different types of terrain. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Policy execution result on the real robot. (Top) [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Learning curve of each algorithm trained with eq. (17) [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Gain Tuning Is Not What You Need: Reward Gain Adaptation for Constrained Locomotion Learning

    cs.RO 2025-10 conditional novelty 6.0 of 10

    ROGER's online threshold-normalized gain adaptation yields near-zero constraint violations and higher primary reward than fixed-gain and adaptive baselines in quadruped and MuJoCo locomotion, including one-hour real-w...

Reference graph

Works this paper leans on

22 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Learning bipedal robot locomotion from hu- man movement,

    M. Taylor, S. Bashkirov, J. F. Rico, I. Toriyama, N. Miyada, H. Yanag- isawa, and K. Ishizuka, “Learning bipedal robot locomotion from hu- man movement,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) , 2021, pp. 2797–2803

    work page 2021

  2. [2]

    Champion-level drone racing using deep reinforce- ment learning,

    E. Kaufmann, L. Bauersfeld, A. Loquercio, M. M ¨uller, V . Koltun, and D. Scaramuzza, “Champion-level drone racing using deep reinforce- ment learning,” Nature, vol. 620, no. 7976, pp. 982–987, 2023

    2023

  3. [3]

    Robot parkour learning,

    Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao, “Robot parkour learning,” in Conference on Robot Learning (CoRL), 2023

    work page 2023

  4. [4]

    Cat: Constraints as terminations for legged locomotion reinforcement learning,

    E. Chane-Sane, P.-A. Leziart, T. Flayols, O. Stasse, P. Sou `eres, and N. Mansard, “Cat: Constraints as terminations for legged locomotion reinforcement learning,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2024

    work page 2024

  5. [5]

    Learning agile and dynamic motor skills for legged robots,

    J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,” Science Robotics , vol. 4, no. 26, 2019

    work page 2019

  6. [6]

    Boyd and L

    S. Boyd and L. Vandenberghe, Convex optimization . Cambridge university press, 2004

    2004

  7. [7]

    Outracing champion gran turismo drivers with deep reinforcement learning,

    P. Wurman, S. Barrett, K. Kawamoto, J. MacGlashan, K. Subramanian, T. Walsh, R. Capobianco, A. Devlic, F. Eckert, F. Fuchs, L. Gilpin, P. Khandelwal, V . Kompella, H. Lin, P. MacAlpine, D. Oller, T. Seno, C. Sherstan, M. Thomure, and H. Kitano, “Outracing champion gran turismo drivers with deep reinforcement learning,” Nature, vol. 602, pp. 223–228, 02 2022

    work page 2022

  8. [8]

    Benchmarking safe exploration in deep reinforcement learning,

    J. Achiam and D. Amodei, “Benchmarking safe exploration in deep reinforcement learning,” 2019. [Online]. Available: https: //api.semanticscholar.org/CorpusID:208283920

    work page 2019

Show all 22 references
  1. [9]

    Learning to walk in the real world with minimal human effort,

    S. Ha, P. Xu, Z. Tan, S. Levine, and J. Tan, “Learning to walk in the real world with minimal human effort,” in Proceedings of the 2020 Conference on Robot Learning , ser. Proceedings of Machine Learning Research, J. Kober, F. Ramos, and C. Tomlin, Eds., vol. 155. PMLR, 16–18 ...

    2020

  2. [10]

    Real-time perceptive motion control using control barrier functions with analytical smoothing for six-wheeled- telescopic-legged robot tachyon 3,

    N. Takasugi, M. Kinoshita, Y . Kamikawa, R. Tsuzaki, A. Sakamoto, T. Kai, and Y . Kawanami, “Real-time perceptive motion control using control barrier functions with analytical smoothing for six-wheeled- telescopic-legged robot tachyon 3,” in IROS. IEEE, 2024

    2024

  3. [11]

    A comprehensive survey on safe reinforcement learning,

    J. Garc ´ıa, Fern, and o Fern ´andez, “A comprehensive survey on safe reinforcement learning,” Journal of Machine Learning Research , vol. 16, no. 42, pp. 1437–1480, 2015

    2015

  4. [12]

    Penalized proximal policy optimization for safe reinforcement learning,

    L. Zhang, L. Shen, L. Yang, S. Chen, X. Wang, B. Yuan, and D. Tao, “Penalized proximal policy optimization for safe reinforcement learning,” 07 2022, pp. 3719–3725

    2022

  5. [13]

    Robot reinforcement learning on the constraint manifold,

    P. Liu, D. Tateo, H. B. Ammar, and J. Peters, “Robot reinforcement learning on the constraint manifold,” in Proceedings of the 5th Conference on Robot Learning , ser. Proceedings of Machine Learning Research, A. Faust, D. Hsu, and G. Neumann, Eds., vol. 164, 2022, pp. 1357–1366

    2022

  6. [14]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in Proceedings of the 32nd International Conference on Machine Learning , ser. Proceedings of Machine Learn- ing Research, F. Bach and D. Blei, Eds., vol. 37. Lille, France: PMLR, ...

    2015

  7. [15]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” CoRR, vol. abs/1707.06347, 2017. [Online]. Available: http://arxiv.org/abs/ 1707.06347

    2017 arXiv

  8. [16]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy an...

    2018

  9. [17]

    Soft actor-critic algorithms and applications,

    T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V . Kumar, H. Zhu, A. Gupta, P. Abbeel, and S. Levine, “Soft actor-critic algorithms and applications,” 2019. [Online]. Available: https://arxiv.org/abs/1812.05905

    2019 arXiv

  10. [18]

    Evaluation of constrained reinforcement learning algorithms for legged locomotion,

    J. Lee, L. Schroth, V . Klemm, M. Bjelonic, A. Reske, and M. Hutter, “Evaluation of constrained reinforcement learning algorithms for legged locomotion,” CoRR, vol. abs/2309.15430, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2309.15430

    2023 doi arXiv

  11. [19]

    Learning agile soccer skills for a bipedal robot with deep reinforcement learning,

    T. Haarnoja, B. Moran, G. Lever, S. H. Huang, D. Tirumala, J. Hump- lik, M. Wulfmeier, S. Tunyasuvunakool, N. Y . Siegel, R. Hafner, M. Bloesch, K. Hartikainen, A. Byravan, L. Hasenclever, Y . Tassa, F. Sadeghi, N. Batchelor, F. Casarini, S. Saliceti, C. Game, N. Sreen- dra, K...

    2024

  12. [20]

    R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduc- tion, 2nd ed. The MIT Press, 2018

    2018

  13. [21]

    Mujoco: A physics engine for model-based control

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control.” in IROS. IEEE, 2012

    2012

  14. [22]

    Openai gym,

    G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016. [Online]. Available: arXiv:1606.01540 APPENDIX I CONSTRAINT FUNCTION DERIVATION In the derivation, we assume discrete state and action sets but the results can also ...

    2016 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.