Pith. sign in

REVIEW 4 major objections 5 minor 28 references

Stability Enhancement in Reinforcement Learning via Adaptive Control Lyapunov Function

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SAC-CLF filters a soft actor-critic policy through a quadratic-programming controller built on an LQR-derived control Lyapunov function, and claims this keeps learning safe and stable on nonlinear and satellite attitude tasks.

desk verdict The paper's engineering instinct is fine but the safety guarantee collapses: the QP slack is in the wrong place, the adaptive-stability analysis isn't a proof, and the ablation tables are swapped. read the letter →

arxiv 2504.19473 v1 pith:CL52XPNJ submitted 2025-01-18 cs.LG cs.RO

Donghe Chen , Han Wang , Lin Cheng , Shengping Gong This is my paper · ORCID
classification cs.LGcs.RO
keywords ReinforcementlearningSafeexplorationControlLyapunovfunctionQuadraticprogrammingSoftActor-CriticSatelliteattitudeAdaptiveconstraintsinputsmoothing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a reinforcement learning agent can explore and learn in safety-critical control tasks without leaving a certified stable region. It introduces SAC-CLF, which takes each action proposed by Soft Actor-Critic and projects it through a quadratic program that enforces a control Lyapunov function decrease condition, so the closed-loop state stays inside a safe energy ball. Three mechanisms carry the argument: an LQR-based procedure to design the Lyapunov function for nonlinear systems, an adaptive tightening or relaxing of the stability constraint to handle unmodeled dynamics, and a smoothing term that penalizes abrupt control changes while still prioritizing safety. If this works as claimed, model-free RL could be applied to real physical systems like spacecraft during training rather than only after extensive simulation.

What carries the argument

The central object is the quadratic program in Eq. (30): given the RL action u_RL(t), it solves for the control input u(t) closest to that action while satisfying the CLF condition partial V/partial e [f(x)+g(x)u] <= -eta(t)V(e), actuator limits, and a smoothing penalty against the previous input. The Lyapunov function is built from the positive definite matrix P obtained by solving the algebraic Riccati equation for the linearized nominal system, so V(e)=e^T P e. The adaptive strength eta(t)=eta0(1+k_eta(t)) is updated from the discrepancy between desired and actual Lyapunov derivatives, and the $\beta$-weighting term makes the input a compromise between the RL recommendation and the previously applied control. This QP layer is what converts an unconstrained RL policy into one that, by construction, respects a stability certificate at every step.

What would settle it

Run the NCT or satellite model from an initial error inside the claimed safe ball D with the LQR-derived P and check whether the QP in Eq. (30) ever becomes infeasible or allows V(e) to rise above V0 during an episode; one such trajectory would show the D subset of B assumption, on which the safety guarantee rests, fails for that system.

Watch

Extended reading notes

Core claim

The central claim is that solving the quadratic program in Eq. (30) at each control step, with a Lyapunov function V(e)=e^T P e obtained by linearizing the system and solving the algebraic Riccati equation, keeps the state inside a safe energy ball D={e: V(e)<V0} while the RL policy keeps learning. The QP minimizes the deviation from the RL-suggested action plus a vibration-dampening term, subject to the CLF decrease constraint and actuator limits. An adaptive parameter eta(t) adjusts the constraint's conservatism by comparing the desired and actual Lyapunov derivative, so the method claims robustness to model uncertainty without permanently sacrificing performance. Simulations on the NCT system and satellite attitude control are presented as evidence that the customized LQR-designed CLF outperforms a unit-matrix CLF, that adaptive constraints reduce cost and variance under model bias, and that the smoothing term reduces command oscillation.

Load-bearing premise

The guarantee rests on assuming that the quadratic function V(e)=e^TPe from the linearized LQR problem is a valid control Lyapunov function for the true nonlinear system over an energy ball D that lies entirely inside the set of states where some admissible input makes V decrease, and that training starts inside that ball; the paper states but never verifies this containment for either test system.

Editorial extensions

If this is right

  • If the central claim is correct, RL agents can be trained on physical systems from the first episode while the controlled state remains inside a prescribed safe energy ball.
  • The LQR-based CLF design would replace hand-crafted Lyapunov functions for nonlinear regulator problems, giving a systematic recipe that starts from a local linear model and yields a global safety filter.
  • Adaptive constraint strength would let the method tighten safety margins when unmodeled dynamics push the Lyapunov derivative off target and relax them when the system is behaving well, so robustness does not demand permanently conservative control.
  • The vibration-dampening term would reduce actuator wear and command chatter without resorting to naive low-pass filtering that ignores the safety constraint.
  • On the demonstrated tasks, the method claims lower cost and lower variance than fixed-constraint baselines, meaning safer exploration also leads to better final performance rather than just constrained behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the same QP safety filter could be attached to other actor-critic algorithms beyond SAC, since the filter only needs a proposed action and a Lyapunov certificate to project onto.
  • The paper's proof of the safe energy ball assumes D is contained in the safe state set B, but it never verifies this containment computationally; a natural testable extension is to estimate the largest V0 for which D subset of B holds, which would turn the guarantee from formal to practically certified.
  • The adaptive eta(t) rule is a heuristic driven by the derivative discrepancy; one could instead estimate the disturbance bound d(x) online and set eta(t) from that bound, making the robustness margin explicit rather than emergent.
  • The smoothing rule is equivalent to a constrained low-pass filter on the RL action; its real payoff would be on hardware with actuator limits and wear, so an experiment comparing actuator usage and tracking error on a physical testbed would sharpen the claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SAC-CLF, an integration of Soft Actor-Critic with a quadratic Control Lyapunov Function designed through linearization and LQR. The method solves a QP at each control step to project the RL-recommended action onto a set defined by a CLF decrease condition, with an adaptively tuned decay rate eta(t) and a smoothing term that penalizes changes from the previous control input. The authors claim three contributions: a task-specific CLF design, adaptive constraint strength for robustness to unmodeled dynamics, and safety-prioritized input smoothing. The paper includes a formal forward-invariance theorem, an adaptive-gain analysis based on a transfer function, Algorithm 1, and simulations on a nonlinear control benchmark and a satellite attitude control problem.

Significance. If the safety guarantee and robustness results were established, the paper would be a useful practical contribution: the LQR-based construction gives a concrete, non-circular way to obtain a CLF candidate for nonlinear systems, and the integration with SAC addresses a real deployment concern in safe reinforcement learning. The experiments cover two relevant aerospace-style benchmarks, and the idea of adapting the CLF constraint severity is interesting. However, the central theoretical claims are not supported: the implemented QP does not contain the claimed slack relaxation, the forward-invariance proof is circular, and the adaptive analysis is not a valid stability argument. The paper also does not provide code or a full hyperparameter set, which limits reproducibility. The significance of the contribution therefore cannot be assessed until these load-bearing issues are resolved.

major comments (4)
  1. [III-D, Eq. (30)] In Eq. (30), the slack variable ε appears only in the objective term Kεε and not in the CLF inequality, so the stated relaxation is not implemented: any feasible point is solved with ε=0 and the QP is hard-constrained. The text on page 6 says ε allows "a small, controlled violation", but the constraint in Eq. (30) is the hard condition ∂V/∂e [f(x)+g(x)u] ≤ −η(t)V(e). Consequently, whenever the CLF decrease condition cannot be met at the current state, the QP is infeasible and Algorithm 1 has no defined control output; this includes states reached under the unmodeled dynamics d(x) that the adaptive constraint is supposed to handle. The claimed safety during learning is therefore not delivered by the implemented mechanism.
  2. [III-A, Theorem 2] The proof of Theorem 2 assumes the property it is supposed to establish: step 4 states that "If the CLF decrease condition is satisfied at e(0), and the system dynamics preserve this property, then it is satisfied for all e(t)", which is exactly forward invariance. The theorem also assumes a control law u=k(e) with V̇≤0 exists along the trajectory, but never shows that the QP in Eq. (30) produces such a control, nor that the LQR-derived V(e)=eᵀ P e satisfies the condition in Definition 2 over the operating region. Definition 3 requires D⊆B, but the paper neither computes V0 nor verifies D⊆B for either the NCT system or the satellite model; without that, Theorem 2 does not apply to the experiments.
  3. [III-B, Eqs. (15)-(27)] The adaptive-gain analysis does not establish robustness. Eq. (19) asks that −∂V/∂e d(x)=η0 kη(t) V(x), which requires exact knowledge of the unknown disturbance d(x) to choose kη(t); Eq. (22) is a first-order heuristic whose convergence to that value is not proven. The transfer function in Eq. (23) treats the time-domain quantity V(e(t)) as a Laplace variable V(E(s)), and Eq. (25) gives a "pole" s2 that depends on s through V(E(s)); the left-half-plane conclusion is therefore not a valid stability statement. These gaps matter because Eq. (30) omits d(x), so the adaptation is the only mechanism claimed to compensate for model bias, but no bound on ∂V/∂e d(x) or feasibility certificate is provided.
  4. [IV-C and IV-D, Tables II and III] The hyperparameter tables appear to invert the experimental conditions. In Table II, the satellite rows labeled "adaptive constraints" have ωη=0.0 while the corresponding "constant constraints" rows have ωη=0.01; in Table III, the rows labeled "with vibration-dampening term" set β=0.0 and the rows labeled "without" set β=1.0, whereas Eq. (30) defines β as the smoothing weight. As written, the figures labeled as showing adaptive or smoothed versions are actually the non-adaptive or unsmoothed configurations, so the empirical support for contributions (2) and (3) is not reproducible from the reported settings.
minor comments (5)
  1. [III-A, Theorem 1] Theorem 1 is a restatement of the standard CLF definition, not a theorem; consider calling it Definition.
  2. [III-B, Eqs. (16)-(17)] The notation switches between V(e(t)) and V(x(t)) in the same derivation; unify the argument of the Lyapunov function to avoid ambiguity.
  3. [IV] The simulations compare only variants of the proposed method; the conclusion that SAC-CLF "outperforms other leading algorithms" is not supported by any baseline such as unconstrained SAC, CPO, or Lagrangian methods.
  4. [IV, Tables I-III] Reproducibility would benefit from reporting the LQR weights Q and R, learning rates, network sizes, and the actual values of Kε (whether 10^8 or 108 in the tables).
  5. [Throughout] Minor typos and notation issues include "adpative" in Table II, and the inconsistent use of V(x(t)) versus V(e(t)) in Eqs. (19)-(22).

Circularity Check

2 steps flagged · score 6.0 of 10

Theorem 2's safety proof assumes the CLF-decrease preservation it is supposed to prove, and the adaptive-constraint robustness guarantee is built into the Eq. (19) matching condition.

  1. self definitional [Section III-A, Theorem 2 (proof step 4), with Definition 3 Eq. (9)]
    "Since V(e(0)) < V0 and Vdot(e) ≤ 0, it follows that V(e(t)) ≤ V(e(0)) < V0 for all t ≥ 0. If the CLF decrease condition is satisfied at e(0), and the system dynamics preserve this property, then it is satisfied for all e(t). Therefore, e(t) ∈ D for all t ≥ 0, ensuring that if the initial error is within the safe energy ball, all subsequent errors will also remain within the safe energy ball."

    The theorem is supposed to prove forward invariance of the safe energy ball D, but the proof's final step assumes exactly the conclusion: 'the system dynamics preserve this property' is the claim that every future state admits a control keeping V non-increasing. Since D is defined (Eq. 9) as a sublevel set contained in B, every state in D has, by definition, some input satisfying the CLF decrease condition; that pointwise existence says nothing about whether the QP-selected input in Eq. (30) keeps the trajectory inside D. The proof provides no independent mechanism and therefore reduces the theorem to its own premise.

  2. self definitional [Section III-B, Eqs. (16)-(22)]
    "To align the actual constraints with the desired constraints, it is necessary to establish a relationship between the error term e(t) and the system’s state, specifically: − ∂V/∂e d(x(t)) = η0kη(t)V(x(t)) ... If a suitable kη(t) can be found such that this condition holds, then the satisfaction of the desired constraints ... automatically ensures the satisfaction of the actual constraints."

    The claimed robustness property that adaptive η(t) makes the actual (disturbed) CLF constraint follow from the desired (nominal) constraint is achieved by construction: Eq. (19) is exactly the equality that, when substituted into Eq. (16), turns the actual constraint into Eq. (21) as an algebraic consequence of Eq. (20). This is a matching condition defining what kη would need to be, not a proof that the update law (22) converges to it or that the QP in Eq. (30), which omits d(x), remains feasible. The compensation is therefore equivalent to the assumed equality rather than an independently derived prediction.

full rationale

The LQR-based CLF construction is anchored in an external standard (the algebraic Riccati equation, Eq. 13), and the experimental comparisons use independent cost measurements, so there is no load-bearing self-citation or renamed-known-result issue. However, the paper's formal safety guarantee is circular. Theorem 2 asserts that initial membership in the safe energy ball D implies all future states stay in D, but the proof's step 4 assumes that the CLF decrease condition is preserved along the trajectory, which is exactly the forward-invariance property to be shown. Definition 3 ensures only that every state in D has some CLF-decreasing control input; it does not show the QP in Eq. (30) selects such an input at every future state, especially under the unmodeled dynamics d(x) that the adaptive mechanism is meant to handle. A second, milder construction-level circularity is the adaptive constraint: Eq. (19) is chosen so that the actual constraint (21) follows from the desired constraint (20) by substitution, so the disturbance compensation is the definition of kη, not a proven consequence; no convergence or QP-feasibility argument closes the gap. These steps make the theoretical safety claim partially circular (score 6), while the empirical demonstrations remain independent evidence that the method works in simulation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on several hand-chosen gains and on the unverified inclusion D subset of B. No new physical entities are introduced; the safe energy ball is a mathematical set, not a new object.

free parameters (5)
  • eta0 (baseline CLF decay rate) = 0.1, 0.01, 0.3, 0.03 (per experiment)
    Hand-chosen per environment and setting; directly sets the conservatism of the safety constraint.
  • omega_eta (adaptation speed) = 0.0 or 0.01
    Hand-chosen; controls how quickly the constraint-strength gain responds to the Lyapunov derivative mismatch.
  • beta (smoothing weight) = 0.0 or 1.0
    Hand-chosen; Table III appears to swap the labels for the with and without vibration-dampening conditions.
  • K_epsilon (slack penalty) = 1e8
    Set large to enforce constraints, but in Eq. (30) the slack variable does not appear in the constraint, so this parameter has no effect in the stated QP.
  • LQR weights Q and R = not fully specified
    Chosen by hand; the CLF matrix P and hence the Lyapunov function V(e)=e^T P e are determined by these weights.
assumptions (4)
  • domain assumption f and g are known exactly
    Stated in Section II-A: 'The functions f and g are assumed to be known.' The QP constraint in Eq. (30) evaluates them explicitly.
  • ad hoc to paper The LQR-derived V(e)=e^T P e is a valid CLF and the safe energy ball D satisfies D subset of B
    Definition 3 (Eq. 9) imposes D subset of B as a condition, but the paper never verifies this inclusion or gives a method to compute V0 for the NCT or satellite models.
  • domain assumption The disturbance d(x) can be estimated well enough through delta(t) and the adaptation loop converges
    Section III-B uses delta as a real-time signal; no measurement model or convergence proof is given.
  • domain assumption Initial state lies inside D
    Theorem 2 starts with e(0) in D; the simulations do not report how initial states were chosen relative to D.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stability Enhancement in Reinforcement Learning via Adaptive Control Lyapunov Function." pith.science (2026). https://pith.science/paper/CL52XPNJ

@misc{pith2026250419473,
  author       = {Pith},
  title        = {Pith review of: Stability Enhancement in Reinforcement Learning via Adaptive Control Lyapunov Function},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CL52XPNJ}},
  note         = {Machine review of arXiv:2504.19473}
}
read the original abstract

Reinforcement Learning (RL) has shown promise in control tasks but faces significant challenges in real-world applications, primarily due to the absence of safety guarantees during the learning process. Existing methods often struggle with ensuring safe exploration, leading to potential system failures and restricting applications primarily to simulated environments. Traditional approaches such as reward shaping and constrained policy optimization can fail to guarantee safety during initial learning stages, while model-based methods using Control Lyapunov Functions (CLFs) or Control Barrier Functions (CBFs) may hinder efficient exploration and performance. To address these limitations, this paper introduces Soft Actor-Critic with Control Lyapunov Function (SAC-CLF), a framework that enhances stability and safety through three key innovations: (1) a task-specific CLF design method for safe and optimal performance; (2) dynamic adjustment of constraints to maintain robustness under unmodeled dynamics; and (3) improved control input smoothness while ensuring safety. Experimental results on a classical nonlinear system and satellite attitude control demonstrate the effectiveness of SAC-CLF in overcoming the shortcomings of existing methods.

Figures

Figures reproduced from arXiv: 2504.19473 by the authors.

Figure 1
Figure 1. Framework of SAC-CLF The SAC-CLF framework, illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The hierarchical relationships: the state space contains [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The CLF from an LQR-based control policy ensures [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Optimal control input may not satisfy CLF constraints, [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Safety-Prioritized Control Input Smoothing: Com [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The customized CLF demonstrates superior conver [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 8
Figure 8. Figure 8: Effectiveness of the vibration-dampening term in con [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 18 canonical work pages

  1. [1]

    On-board modeling of gravity fields of elongated asteroids using hopfield neural networks,

    Y . Zhao, H. Yang, S. Li, and Y . Zhou, “On-board modeling of gravity fields of elongated asteroids using hopfield neural networks,” Astrodynamics, vol. 7, no. 1, pp. 101–114, Mar. 2023. [Online]. Available: https://doi.org/10.1007/s42064-022-0151-3

    work page doi:10.1007/s42064-022-0151-3 2023

  2. [2]

    Closed-loop deep neural network optimal control algorithm and error analysis for powered landing under uncertainties,

    W. Li, Y . Song, L. Cheng, and S. Gong, “Closed-loop deep neural network optimal control algorithm and error analysis for powered landing under uncertainties,” Astrodynamics, vol. 7, no. 2, pp. 211–228, Jun. 2023. [Online]. Available: https://doi.org/10.1007/ s42064-022-0153-1

    work page 2023

  3. [3]

    Low-thrust trajectory optimization with averaged dynamics using analytical switching detection,

    D. Wu, L. Cheng, S. Gong, and H. Baoyin, “Low-thrust trajectory optimization with averaged dynamics using analytical switching detection,” Journal Of Guidance, Control, And Dynamics , vol. 47, 0 100 200 300 400 500 Step 0.0 0.2 0.4 0.6Control Input Magnitude without vibration-dampening term with vibration-dampening term (a) NCT System 0 100 200 300 400 50...

    2024

  4. [4]

    Spacecraft relative motion control near an asteroid with uncertainties: a lyapunov redesign approach,

    W. Wang, G. Mengali, A. A. Quarta, and H. Baoyin, “Spacecraft relative motion control near an asteroid with uncertainties: a lyapunov redesign approach,” IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 4, pp. 4507–4517, Aug. 2024, conference Name: IEEE Transactions on Aerospace and Electronic Systems. [Online]. Available: https://ieeex...

    work page arXiv 2024

  5. [5]

    Robust controller design: recent emerging concepts for control of mechatronic systems,

    C. M. Ionescu, E. H. Dulf, M. Ghita, and C. I. Muresan, “Robust controller design: recent emerging concepts for control of mechatronic systems,” Journal of the Franklin Institute , vol. 357, no. 12, pp. 7818–7844, Aug. 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0016003220303999

    work page 2020

  6. [6]

    Dynamic-matching adaptive sliding mode control for hypersonic vehicles,

    C. Qu, L. Cheng, S. Gong, and X. Huang, “Dynamic-matching adaptive sliding mode control for hypersonic vehicles,” Aerospace Science and Technology, vol. 149, p. 109159, Jun. 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S127096382400292X

    work page 2024

  7. [7]

    Adaptive non-affine control for the short-period model of a generic hypersonic flight vehicle,

    Y . Wang and Q. Wu, “Adaptive non-affine control for the short-period model of a generic hypersonic flight vehicle,” Aerospace Science and Technology, vol. 66, pp. 193–202, Jul. 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1270963816307933

    work page 2017

  8. [8]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” Jul. 2019, arXiv:1509.02971. [Online]. Available: http://arxiv.org/abs/1509.02971

    arXiv 2019

Show all 28 references
  1. [9]

    Trust Region Policy Optimization,

    J. Schulman, S. Levine, P. Moritz, M. I. Jordan, and P. Abbeel, “Trust Region Policy Optimization,” Apr. 2017, arXiv:1502.05477 [cs]. [Online]. Available: http://arxiv.org/abs/1502.05477

    2017 arXiv

  2. [10]

    Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey,

    W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey,” in 2020 IEEE Symposium Series on Computational Intelligence (SSCI) , Dec. 2020, pp. 737–744, arXiv:2009.13303 [cs]. [Online]. Available: http://arxiv.org/a...

    2020 arXiv

  3. [11]

    Safety- aware adaptive reinforcement learning with applications to brushbot JOURNAL OF IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS 10 navigation,

    M. Ohnishi, L. Wang, G. Notomista, and M. Egerstedt, “Safety- aware adaptive reinforcement learning with applications to brushbot JOURNAL OF IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS 10 navigation,” Jan. 2018, arXiv:1801.09627 version: 1. [Online]. Available: http:...

    2018 arXiv

  4. [12]

    AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles,

    S. Shah, D. Dey, C. Lovett, and A. Kapoor, “AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles,” Jul. 2017, arXiv:1705.05065 [cs]. [Online]. Available: http://arxiv.org/abs/1705. 05065

    2017 arXiv

  5. [13]

    Control Barrier Function Based Quadratic Programs for Safety Critical Systems,

    A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control Barrier Function Based Quadratic Programs for Safety Critical Systems,” IEEE Transactions on Automatic Control , vol. 62, no. 8, pp. 3861– 3876, Aug. 2017, arXiv:1609.06408 [cs, math]. [Online]. Available: http://arxiv...

    2017 arXiv

  6. [14]

    Safe Model-based Reinforcement Learning with Stability Guarantees,

    F. Berkenkamp, M. Turchetta, A. Schoellig, and A. Krause, “Safe Model-based Reinforcement Learning with Stability Guarantees,” in Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/pape...

    2017

  7. [15]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep...

    2015

  8. [16]

    Playing atari with deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” Dec. 2013, arXiv:1312.5602. [Online]. Available: http: //arxiv.org/abs/1312.5602

    2013 arXiv

  9. [17]

    Mastering the game of Go without human knowledge,

    D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y . Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis, “Mastering the game of Go without human knowledge,” Nature, vol. 550...

    2017

  10. [18]

    Local gaussian process regression for real time online model learning,

    D. Nguyen-tuong, J. Peters, and M. Seeger, “Local gaussian process regression for real time online model learning,” in Advances in Neural Information Processing Systems , vol. 21. Curran Associates, Inc., 2008. [Online]. Available: https://proceedings.neurips.cc/paper/ 2008/ha...

    2008

  11. [19]

    Safe reinforcement learning for legged locomotion,

    T.-Y . Yang, T. Zhang, L. Luu, S. Ha, J. Tan, and W. Yu, “Safe reinforcement learning for legged locomotion,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , Oct. 2022, pp. 2454–2461, iSSN: 2153-0866. [Online]. Available: https://ieeexplore...

    2022

  12. [20]

    Safe exploration techniques for rein- forcement learning – an overview,

    M. Pecka and T. Svoboda, “Safe exploration techniques for rein- forcement learning – an overview,” in Modelling and Simulation for Autonomous Systems , J. Hodicky, Ed. Cham: Springer International Publishing, 2014, pp. 357–375

    2014

  13. [21]

    A Review of Safe Reinforcement Learning: Methods, Theory and Applications,

    S. Gu, L. Yang, Y . Du, G. Chen, F. Walter, J. Wang, and A. Knoll, “A Review of Safe Reinforcement Learning: Methods, Theory and Applications,” May 2024, arXiv:2205.10330 [cs]. [Online]. Available: http://arxiv.org/abs/2205.10330

    2024 arXiv

  14. [22]

    DeepSafeMPC: deep learning-based model predictive control for safe multi-agent reinforcement learning,

    X. Wang, H. Pu, H. J. Kim, and H. Li, “DeepSafeMPC: deep learning-based model predictive control for safe multi-agent reinforcement learning,” Mar. 2024, arXiv:2403.06397. [Online]. Available: http://arxiv.org/abs/2403.06397

    2024 arXiv

  15. [23]

    A General Safety Framework for Learning- Based Control in Uncertain Robotic Systems,

    J. F. Fisac, A. K. Akametalu, M. N. Zeilinger, S. Kaynama, J. Gillula, and C. J. Tomlin, “A General Safety Framework for Learning- Based Control in Uncertain Robotic Systems,” IEEE Transactions on Automatic Control, vol. 64, no. 7, pp. 2737–2752, Jul. 2019, conference Name: IE...

    2019

  16. [24]

    Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor,” Aug. 2018, arXiv:1801.01290. [Online]. Available: http://arxiv.org/abs/1801.01290

    2018 arXiv

  17. [25]

    Torque saturation in bipedal robotic walking through control lyapunov function-based quadratic programs,

    K. Galloway, K. Sreenath, A. D. Ames, and J. W. Grizzle, “Torque saturation in bipedal robotic walking through control lyapunov function-based quadratic programs,” IEEE Access , vol. 3, pp. 323– 332, 2015, conference Name: IEEE Access. [Online]. Available: https://ieeexplore.i...

    2015

  18. [26]

    Adaptive cruise control: experimental validation of advanced controllers on scale-model cars,

    A. Mehra, W.-L. Ma, F. Berg, P. Tabuada, J. W. Grizzle, and A. D. Ames, “Adaptive cruise control: experimental validation of advanced controllers on scale-model cars,” 2015 American Control Conference (ACC) , pp. 1411–1418, Jul. 2015, conference Name: 2015 American Control Con...

    2015

  19. [27]

    Rapidly exponentially stabilizing control lyapunov functions and hybrid zero dynamics,

    A. D. Ames, K. Galloway, K. Sreenath, and J. W. Grizzle, “Rapidly exponentially stabilizing control lyapunov functions and hybrid zero dynamics,” IEEE Transactions on Automatic Control , vol. 59, no. 4, pp. 876–891, Apr. 2014, conference Name: IEEE Transactions on Automatic Co...

    2014

  20. [28]

    Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem,

    K. G. Vamvoudakis and F. L. Lewis, “Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem,” Automatica, vol. 46, no. 5, pp. 878–888, May 2010. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S0005109810000907 Dong...

    2010

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.