Pith. sign in

REVIEW 3 major objections 6 minor 56 references

Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that training a physics-regulated RL agent from initial states on the boundary of its Lyapunov safety envelope learns safe policies with fewer episodes and far lower failure rates than uniform random sampling.

desk verdict The reader's baseline concern is valid and undercuts the headline comparison, but the boundary-sampling idea is clean and deserves a revision, not a desk reject. read the letter →

arxiv 2412.13224 v1 pith:2MIR4SYJ submitted 2024-12-17 cs.RO cs.AIcs.LG

Hongpeng Cao , Yanbing Mao , Lui Sha , Marco Caccamo This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords safereinforcementlearningworst-casesamplingLyapunovsafetyenvelopecornercasesphysics-regulateddeepsafety-criticalcyber-physicalsystemsinitial-conditionrobustcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deep reinforcement-learning policies are usually trained from one fixed initial condition or from uniformly random states, so the rare corner cases where real systems fail are rarely seen. This paper proposes a concrete definition of those corner cases: any state on the boundary of the safety envelope, the Lyapunov ellipsoid $\{s : s^\top P s = 1\}$ guaranteed by the physics model. It gives a parameterization that generates such boundary states explicitly and a curriculum that visits them periodically and sparsely during training. Integrated into the physics-regulated RL framework Phy-DRL, this worst-case sampling cuts cart-pole training-episode failures to 0.59% versus 90.6% under random sampling, and one boundary sample suffices to train a quadruped policy that transfers to a real robot. The paper's point is that concentrating training where safety is actually decided is cheaper and safer than hoping random exploration finds those states.

What carries the argument

The load-bearing object is the boundary of the safety envelope, the ellipsoid surface $\{s : s^\top P s = 1\}$, with $P$ the positive-definite Lyapunov matrix computed from the linearized dynamics. Definition 3.1 names every boundary point a worst-case condition; Lemma 3.2 parameterizes these points by rotating a unit sphere with the eigenvectors of $P$ and scaling each axis by $1/\sqrt{\lambda_i(P)}$. This converts the continuous boundary into the finite grid of Algorithm 1, which samples the angles $\theta_r$ sparsely on $[0,2\pi)$ and uses the resulting states as initial conditions for a few periodic episodes. The mechanism's work is to focus all exploration on the surface where safety is about to be lost, while leaving the model-based feedback in Phy-DRL to guide the agent back toward the equilibrium.

What would settle it

After training the cart-pole Phy-DRL agent with Algorithm 1 under the reported settings, initialize it at several thousand states uniformly sampled from the admissible region outside the safety envelope, including the corners where $|x|\approx 0.9$ and $|\theta|\approx 0.8$. If a substantial fraction of those runs violates safety, then boundary-only training does not generalize to all dangerous initial states, and part of the reported gap between worst-case and random sampling is an artifact of where the test states are drawn from.

Watch

Extended reading notes

Core claim

The central claim is that 'worst case' for a deep RL policy in a safety-critical system can be identified exactly with the boundary of the Lyapunov safety envelope, and that training on that boundary renders the envelope invariant. For a positive-definite matrix $P$ defining the envelope $\Omega=\{s:s^\top P s\le 1\}$, Definition 3.1 declares every state with $s^\top P s = 1$ a worst-case condition. Lemma 3.2 then provides the explicit spherical parameterization $s = Q(P)y$, with coordinates built from angles $\theta_1,\dots,\theta_{n-1}$ and eigenvalues $\lambda_i(P)$, so boundary states can be generated without solving an optimization problem. Algorithm 1 turns those states into a periodic, sparse training curriculum. On this basis the paper reports that Phy-DRL, the named framework in which a model-based linear-feedback action is added to the data-driven action, achieves near-zero training failures in the cart-pole experiments, larger verified-safe regions in the 2D quadrotor, and a quadruped policy generalizing to road conditions never seen in training.

Load-bearing premise

The load-bearing premise is that states on the boundary of the Lyapunov safety envelope are the true worst-case conditions for the DRL policy, so training only on those boundary states transfers to every dangerous state the system might reach.

Editorial extensions

If this is right

  • With the (2-5) setting in the cart-pole study, worst-case sampling fails in 1 of 170 training episodes (0.59%) while random sampling fails in 154 of 170 (90.6%).
  • Sparse curricula work: failure rates of 10.0%, 3.75%, and 0.59% are achieved with 30, 80, and 170 episodes respectively, supporting the paper's claim of data-efficient training.
  • The advantage holds without an early-termination bias: worst-case-w.t. fails 0 of 170 episodes versus 85 of 170 for random-w.t.
  • On the 2D quadrotor, Phy-DRL trained with worst-case sampling attains far more internal-envelope test samples than Phy-DRL trained with random sampling, while a pure data-driven DRL with a CLF-style reward reaches zero internal-envelope samples under the same worst-case curriculum.
  • On the quadruped, a policy trained from a single boundary condition (forward 1 m/s on snow) transfers to the real robot and outperforms PD and pure DRL on untrained backward-velocity and wet-road conditions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the boundary parameterization in Lemma 3.2 is not tied to Phy-DRL; any safe-RL scheme carrying a quadratic Lyapunov certificate could adopt the same worst-case curriculum, making the sampling lemma a transferable ingredient.
  • Editorial inference: because the reported failure rates count episodes that violate safety during training, a separate evaluation that starts the trained policy from many states inside the admissible set but outside the envelope would be needed to show the boundary curriculum alone, rather than the model-based stabilizer, is responsible for the safety gap.
  • Editorial inference: a natural testable extension is to blend boundary samples with a small number of deliberately exterior states near the corners of the admissible set, probing whether the envelope-boundary definition under-approximates the states a real disturbance could drive the system into.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a physics-model-guided worst-case sampling strategy for safe reinforcement learning. Worst-case conditions are defined as states on the boundary of a Lyapunov safety envelope Omega = {s : s^T P s <= 1} (Definition 3.1), and Algorithm 1 generates such states via a spherical-coordinate parameterization (Lemma 3.2) and trains the Phy-DRL agent periodically on them. The approach is evaluated on a cart-pole system, a 2D quadrotor, and a simulated and real quadruped robot, showing lower episode failure rates and larger invariant-envelope areas than a uniform random-sampling baseline, with substantially fewer training episodes.

Significance. If the central claim holds, the paper offers a practical way to focus deep RL training on safety-critical states using an available physics model, which could improve data efficiency and safety of learned controllers. The paper's formal lemma (Lemma 3.2) is correct, and the experiments cover multiple platforms including a real quadruped, which is a genuine strength. However, the main empirical comparison is confounded by the choice of the random baseline's sampling support, and the term 'worst-case' is not formally justified. The approach is plausible, but the current evidence does not fully separate the effect of the proposed sampling strategy from the effect of sampling an easier subset of the state space.

major comments (3)
  1. [Section 4.2, Table 2] The headline comparison between worst-case and random sampling is confounded by the support of the initial-state distributions. The random baseline draws uniformly from the axis-aligned box (for cart-pole, intervals [-0.9,0.9], [-3,3], [-0.8,0.8], [-4.5,4.5]), which is the safety set X and a strict superset of the ellipsoidal safety envelope Omega used to define worst-case samples. Consequently, a large fraction of random initial states lie outside Omega and closer to the true safety boundary, making them harder to control. The paper itself states that worst-case sampling 'automatically avoids many samples that are physically infeasible to control' (Section 4.2), which concedes that the two training distributions differ in difficulty. To support the central claim that boundary sampling is superior, the authors must add a matched baseline that samples uniformly from Omega (or from the boundary of Omega) with the same number of episodes; without this, the failure-rate gap in Table 2 (e.g., 0.59% vs 90.6% for setting (2-5)) may be an artifact of task difficulty rather than a benefit of the proposed sampling strategy. The same issue applies to the quadrotor experiment in Section 4.3.
  2. [Definition 3.1 and Section 3] The paper labels states on the boundary of the Lyapunov envelope as 'worst-case conditions' without a formal argument or empirical evidence that these are the most safety-critical states for the learned DRL policy. The safety-critical corners of the constraint set X generally lie outside Omega, since Omega is a subset of X; therefore the name 'worst-case' is not justified by the safety constraints themselves. The only justification offered is the intuitive statement that 'the plant is more likely to violate the safety constraint when its state is near the envelope boundary' (Section 3). If the intended claim is that training on the boundary of Omega renders Omega invariant (Definition 2.1), the paper should either provide a formal argument (e.g., prove that a policy that decreases the Lyapunov function on the boundary suffices for invariance under the given disturbance model) or soften the terminology and clearly state that the method is a heuristic for focusing training on the envelope boundary. This point is load-bearing because the entire method and evaluation are centered on the envelope boundary.
  3. [Section 4.2, IE/EE metrics] The empirical claim that the learned policy 'successfully renders the safety envelope invariant' is not fully supported by the reported numbers. Table 2 shows that the worst-case (2-5) setting still has one failed episode (0.59%) during training, and no statistical variability (e.g., mean and standard deviation over multiple random seeds) is reported for the failed-episode counts or for the IE/EE areas shown in Figures 3, 4, 8, and 9. Since the central contribution is about 'sampling efficiency' and 'robustness', the authors should report repeated-seed statistics and specify how many test samples were used to compute the IE/EE areas. This would allow the reader to judge whether the observed advantages are consistent or within noise.
minor comments (6)
  1. [Section 3, Eq. (12)] In the 'Episode Complexity' paragraph, the sentence 'the number of θ1 is q_n' should refer to q_1, not q_n; the notation is inconsistent with the algorithm input.
  2. [Lemma 3.2 and Algorithm 1] The spherical parameterization in Lemma 3.2 uses angles θ_i in [0,2π) for all i, whereas standard spherical coordinates require θ_i in [0,π] for i≥2 to avoid double-covering the ellipsoid. As written, the grid in Algorithm 1 may contain duplicate points, and the reported episode count in Eq. (12) may overcount distinct sampled states. Please clarify the intended angle ranges or adjust the counting.
  3. [Section 4, Eq. (14)] The formal definition of an EE sample says there exists k such that s(k) is in X\Omega, while the intuitive description says the system 'always stays in the safety set'. These are different statements; please clarify the quantifiers intended in the definition and in the description of the green areas in the figures.
  4. [Section 4.3] For the quadrotor, the random sampling intervals for v_z and v_θ are [-10,10] and [-45,45]; if these are not part of the safety constraints in Eq. (15), the paper should specify how they were chosen and whether they correspond to the state bounds used in the worst-case envelope generation.
  5. [Figure 5(d)] The reward curves for Phy-DRLwc versus Phy-DRLran are shown for five random seeds, but without shaded error bands; adding mean±std bands would make the claimed 'large variance' for random sampling more convincing.
  6. [Conclusion] The phrase 'spare worst-case sampling' should be 'sparse worst-case sampling'.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular reduction found; the worst-case sampling distribution and the IE safety metric share the same Lyapunov envelope by design, but the learned policy is not forced to satisfy the metric, so the evaluation is not vacuous.

full rationale

The paper's new element is Definition 3.1, which labels the boundary of the Lyapunov envelope Ω as the worst-case condition, and Algorithm 1 trains Phy-DRL from sparse boundary samples. The IE safety metric then asks whether trajectories starting in Ω remain in Ω, so the training distribution and the evaluation predicate are aligned by construction. This alignment, however, is not a circular reduction: the policy is a neural network trained with a one-step Lyapunov-shaped reward, and nothing forces the learned policy to render Ω invariant; the worst-case training itself still produced failures in the (2-3) setting (10.0%), so the outcome is not automatic. P and H come from LMI design in the authors' prior Phy-DRL work, not from fitting to the safety outcomes, so no fitted parameter is renamed as a prediction. The main threat to the headline comparison is a confounded baseline: the random sampler draws from the axis-aligned box [−0.9,0.9]×[−3,3]×[−0.8,0.8]×[−4.5,4.5], which strictly contains the ellipsoid Ω, while worst-case samples lie exactly on ∂Ω; the paper itself concedes the gap by saying the worst-case construction 'automatically avoids many samples that are physically infeasible to control' (Section 4.2). That is a mismatch in training-distribution difficulty, not a circular derivation. Self-citations to [9,10] supply the Phy-DRL backbone and the LMI-derived P, but they are prior peer-reviewed results and are not invoked as an unverified uniqueness theorem to forbid alternatives. Hence no circularity; score 1 reflects only the evaluation-alignment and baseline-support concerns.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central heuristic, that the safety envelope boundary constitutes worst-case conditions, is a domain assumption introduced by the paper without independent justification. The free parameters are hyperparameters of the sampling curriculum, not fitted to outcomes. No new physical entities are introduced.

free parameters (2)
  • alpha (Lyapunov decrease rate)
    Pre-defined in Equation (6) to bound the decrease of the Lyapunov value; it shapes the LMI solution P and therefore the safety envelope used for sampling.
  • Sampling counts q_r and period p
    User-chosen parameters in Algorithm 1 that determine the number of worst-case samples and training curriculum length (Equation 12); the paper uses (2-3), (2-4), (2-5) settings.
assumptions (5)
  • domain assumption The real plant is a known linear part plus unknown mismatch f(s,a), and (A,B) suffice to compute a stabilizing F and P via LMI.
    Invoked in Section 2.3 and used to build the safety envelope; the mismatch f is not accounted for in the safety guarantee.
  • ad hoc to paper States on the safety envelope boundary s^T P s = 1 are the worst-case conditions for DRL training.
    Definition 3.1 states this directly; the paper provides no proof that the envelope boundary captures the most safety-critical corner cases.
  • domain assumption Uniform sampling from the safety set X is the appropriate baseline for initial-condition sampling.
    Section 4 uses uniform sampling over X as 'random'; this includes states outside Ω and may disadvantage the baseline.
  • domain assumption DDPG with the Phy-DRL safety-embedded reward can learn to render Ω invariant.
    The experiments rely on this empirical capability of the RL algorithm; results are shown without formal guarantee.
  • standard math Lyapunov stability theorem and positive definiteness of P (Lemma A.1).
    Used in Definition 2.1 and in the proof of Lemma 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning." pith.science (2026). https://pith.science/paper/2MIR4SYJ

@misc{pith2026241213224,
  author       = {Pith},
  title        = {Pith review of: Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2MIR4SYJ}},
  note         = {Machine review of arXiv:2412.13224}
}
read the original abstract

Real-world accidents in learning-enabled CPS frequently occur in challenging corner cases. During the training of deep reinforcement learning (DRL) policy, the standard setup for training conditions is either fixed at a single initial condition or uniformly sampled from the admissible state space. This setup often overlooks the challenging but safety-critical corner cases. To bridge this gap, this paper proposes a physics-model-guided worst-case sampling strategy for training safe policies that can handle safety-critical cases toward guaranteed safety. Furthermore, we integrate the proposed worst-case sampling strategy into the physics-regulated deep reinforcement learning (Phy-DRL) framework to build a more data-efficient and safe learning algorithm for safety-critical CPS. We validate the proposed training strategy with Phy-DRL through extensive experiments on a simulated cart-pole system, a 2D quadrotor, a simulated and a real quadruped robot, showing remarkably improved sampling efficiency to learn more robust safe policies.

Figures

Figures reproduced from arXiv: 2412.13224 by the authors.

Figure 1
Figure 1. Phy-DRL training powered by periodic and sparse [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Worst-case condition generation in for a three [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Worst-case Sampling v.s. Random Sampling, with termination condition. Blue: area of IE samples (14). Green: area of EE samples (15). Ellipse area: safety envelope. The (a) and (b) are the testing result visualized on 𝑥 and 𝜃 dimensions, where (c) and (d) are the results visualized on 𝑣 and 𝑤 dimensions. The size of colored area indicates the safety and robustness of the learned policy, the larger the better. 1.0 0.5… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Worst-case Sampling v.s. Random Sampling, without using termination condition in training. 1.81.92.02.1 x 2.2 3.50 3.75 4.00 4.25 4.50 y 0.4 0.2 0.0 0.2 0.4 (a) Phy-DRLwc 1.8 1.9 2.0 2.1 x 2.2 3.50 3.75 4.00 4.25 4.50 y 0.4 0.2 0.0 0.2 0.4 (b) Phy-DRLran 0.00.2 0.4 0.6…
Figure 5
Figure 5. Figure 5: (a)-(c): The number and locations of IE samples [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Yaw and velocity trajectories under velocity commands [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Yaw and velocity trajectories under velocity commands [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 3
Figure 3. Figure 3: The sample areas of policies trained without the episode [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 8
Figure 8. Figure 8: (2-3) Worst-case Sampling v.s. Random Sampling, with and without termination condition. Blue: area of IE samples. Green: area of EE samples. Ellipse area: safety envelope. 1.0 0.5 0.0 0.5 1.0 x 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 (a) worst-case 1.0 0.5 0.0 0.5…
Figure 9
Figure 9. Figure 9: (2-4) Worst-case Sampling v.s. Random Sampling, with and without termination condition. Blue: area of IE samples. Green: area of EE samples. Ellipse area: safety envelope [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 51 canonical work pages

  1. [1]

    [n. d.]. AI INCIDENT DATABASE. https://incidentdatabase.ai/entities/

    work page

  2. [2]

    Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. 2017. Constrained policy optimization. In International conference on machine learning . PMLR, 22– 31

    work page 2017

  3. [3]

    Felix Berkenkamp, Matteo Turchetta, Angela Schoellig, and Andreas Krause. 2017. Safe model-based reinforcement learning with stability guarantees. Advances in Neural Information Processing Systems 30 (2017)

    work page 2017

  4. [4]

    Rajendra Bhatia. 2009. Positive definite matrices. Princeton university press

    work page 2009

  5. [5]

    Daniel Bogdoll, Jasmin Breitenstein, Florian Heidecker, Maarten Bieshaar, Bern- hard Sick, Tim Fingscheidt, and Marius Zöllner. 2021. Description of corner cases in automated driving: Goals and challenges. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 1023–1028

    work page 2021

  6. [6]

    Joshua Bote. 2023. Waymo driverless car brings San Francisco traffic to a halt during rush hour. https://www.sfgate.com/bayarea/article/waymo-rush-hour- traffic-standstill-17739556.php. (2023)

    work page 2023

  7. [7]

    Stephen Boyd, Laurent El Ghaoui, Eric Feron, and Venkataramanan Balakrishnan

    work page

  8. [8]

    Yihao Cai, Hongpeng Cao, Yanbing Mao, Lui Sha, and Marco Caccamo

    work page

Show all 56 references
  1. [9]

    Hongpeng Cao, Yanbing Mao, Lui Sha, and Marco Caccamo. 2023. Physics-Model- Regulated Deep Reinforcement Learning towards Safety & Stability Guarantees. In 62nd IEEE Conference on Decision and Control . 8300–8305

    2023

  2. [10]

    Hongpeng Cao, Yanbing Mao, Lui Sha, and Marco Caccamo. 2024. Physics- Regulated Deep Reinforcement Learning: Invariant Embeddings. In The Twelfth International Conference on Learning Representations . https://openreview.net/ forum?id=5Dwqu5urzs

    2024

  3. [11]

    Ya-Chien Chang and Sicun Gao. 2021. Stabilizing neural control using self-learned almost Lyapunov critics. In 2021 IEEE International Conference on Robotics and Automation. IEEE, 1803–1809

    2021

  4. [12]

    Richard Cheng, Gábor Orosz, Richard M Murray, and Joel W Burdick. 2019. End- to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 3387–3395

    2019

  5. [13]

    Richard Cheng, Abhinav Verma, Gabor Orosz, Swarat Chaudhuri, Yisong Yue, and Joel Burdick. 2019. Control regularization for reduced variance reinforcement learning. In International Conference on Machine Learning . 1141–1150

    2019

  6. [14]

    Arthur Clavière, Eric Asselin, Christophe Garion, and Claire Pagetti. 2021. Safety Verification of Neural Network Controlled Systems. InProceedings of 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops. 47–54

    2021

  7. [15]

    Xingye Da, Zhaoming Xie, David Hoeller, Byron Boots, Anima Anandkumar, Yuke Zhu, Buck Babich, and Animesh Garg. 2021. Learning a contact-adaptive ICCPS ’25, June 03–05, 2025, Woodstock, NY xxx et al. controller for robust, efficient legged locomotion. InConference on Robot Lea...

    2021

  8. [16]

    Scott Fujimoto, Herke Hoof, and David Meger. 2018. Addressing function ap- proximation error in actor-critic methods. In International conference on machine learning. PMLR, 1587–1596

    2018

  9. [17]

    Pascal Gahinet, Arkadii Nemirovskii, Alan J Laub, and Mahmoud Chilali. 1994. The LMI control toolbox. In Proceedings of 1994 33rd IEEE conference on decision and control, Vol. 3. IEEE, 2038–2041

    1994

  10. [18]

    Siddhant Gangapurwala, Alexander Mitchell, and Ioannis Havoutis. 2020. Guided constrained policy optimization for dynamic quadrupedal robot locomotion.IEEE Robotics and Automation Letters 5, 2 (2020), 3642–3649

    2020

  11. [19]

    Zhenglei He, Kim-Phuc Tran, Sebastien Thomassey, Xianyi Zeng, Jie Xu, and Changhai Yi. 2021. A deep reinforcement learning based multi-criteria decision support system for optimizing textile chemical process. Computers in Industry 125 (2021), 103373

    2021

  12. [20]

    Huang, Nicolas Papernot, Ian J

    Sandy H. Huang, Nicolas Papernot, Ian J. Goodfellow, Yan Duan, and Pieter Abbeel. 2017. Adversarial Attacks on Neural Network Policies. In5th International Conference on Learning Representations, ICLR 2017, Workshop Track Proceedings . https://openreview.net/forum?id=ryvlRyBKl

    2017

  13. [21]

    Laura Humphrey, Bettina Könighofer, Robert Könighofer, and Ufuk Topcu. 2016. Synthesis of Admissible Shields. In Hardware and Software: Verification and Testing. Lecture Notes in Computer Science . Springer, 134–151. https://doi.org/10. 1007/978-3-319-49052-6_9 arXiv:1904.0693...

    2016 arXiv

  14. [22]

    Julian Ibarz, Jie Tan, Chelsea Finn, Mrinal Kalakrishnan, Peter Pastor, and Sergey Levine. 2021. How to train your robot with deep reinforcement learning: lessons we have learned. The International Journal of Robotics Research 40, 4-5 (2021), 698–721

    2021

  15. [23]

    Tobias Johannink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, and Sergey Levine. 2019. Residual reinforcement learning for robot control. In2019 International Conference on Robotics and Automation (ICRA) . IEEE, ...

    2019

  16. [24]

    Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer

  17. [25]

    Alex Kendall, Jeffrey Hawke, David Janz, Przemyslaw Mazur, Daniele Reda, John- Mark Allen, Vinh-Dieu Lam, Alex Bewley, and Amar Shah. 2019. Learning to drive in a day. In 2019 International Conference on Robotics and Automation . IEEE, 8248–8254

    2019

  18. [26]

    B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A Al Sallab, Senthil Yogamani, and Patrick Pérez. 2021. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems 23, 6 (2021), 4909–4926

    2021

  19. [27]

    Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rocktäschel. 2023. A survey of zero-shot generalisation in deep reinforcement learning. Journal of Artificial Intelligence Research 76 (2023), 201–264

    2023

  20. [28]

    Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. 2016. End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research 17, 1 (2016), 1334–1373

    2016

  21. [29]

    Jingqi Li, David Fridovich-Keil, Somayeh Sojoudi, and Claire J Tomlin. 2021. Augmented lagrangian method for instantaneously constrained reinforcement learning problems. In 2021 60th IEEE Conference on Decision and Control (CDC) . IEEE, 2982–2989

    2021

  22. [30]

    Tongxin Li, Ruixiao Yang, Guannan Qu, Yiheng Lin, Steven Low, and Adam Wierman. [n. d.]. Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Control. arXiv preprint https://arxiv.org/pdf/2206.01341.pdf

    arXiv

  23. [31]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2016. Continuous control with deep reinforcement learning. In 4th International Conference on Learning Representations, ICLR

    2016

  24. [32]

    Minghuan Liu, Menghui Zhu, and Weinan Zhang. 2022. Goal-conditioned re- inforcement learning: Problems and solutions. arXiv preprint arXiv:2201.08299 (2022)

    2022 arXiv

  25. [33]

    Bernard Marr. 2021. How Tesla Is Using Artificial Intelligence to Create The Autonomous Cars Of The Future. Bernard Marr & Co. https://bernardmarr.com/how-tesla-is-using-artificial-intelligence-to-create- the-autonomous-cars-of-the-future/

    2021

  26. [34]

    Bhairav Mehta, Manfred Diaz, Florian Golemo, Christopher J Pal, and Liam Paull

  27. [35]

    Fabio Muratore, Fabio Ramos, Greg Turk, Wenhao Yu, Michael Gienger, and Jan Peters. 2022. Robot learning from randomized simulations: A review. Frontiers in Robotics and AI 9 (2022), 799893

    2022

  28. [36]

    NHTSA. [n. d.]. Summary Report: Standing General Order on Crash Reporting for Level 2 Advanced Driver Assistance Systems. National Highway Traffic Safety Administration ([n. d.]). https://www.nhtsa.gov/sites/nhtsa.gov/files/2022- 06/ADAS-L2-SGO-Report-June-2022.pdf

    2022

  29. [37]

    Theodore J Perkins and Andrew G Barto. 2002. Lyapunov design for safe rein- forcement learning. Journal of Machine Learning Research 3, Dec (2002), 803–832

    2002

  30. [38]

    Krishan Rana, Vibhavari Dasagi, Jesse Haviland, Ben Talbot, Michael Milford, and Niko Sünderhauf. [n. d.]. Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics. arXiv preprint https://arxiv. org/pdf/2107.09822.pdf

    arXiv

  31. [39]

    Remo Sasso, Michelangelo Conserva, and Paulo Rauber. 2023. Posterior Sam- pling for Deep Reinforcement Learning. The Fortieth International Conference on Machine Learning (2023), 1–19

    2023

  32. [40]

    Thomas Savage, Dongda Zhang, Max Mowbray, and Ehecatl Antonio Del Río Chanona. 2021. Model-free safe reinforcement learning for chemical processes using Gaussian processes. IFAC-PapersOnLine 54, 3 (2021), 504–509

    2021

  33. [41]

    Lui Sha et al. 2001. Using simplicity to control complexity. IEEE Software 18, 4 (2001), 20–28

    2001

  34. [42]

    Tian Tolentino. 2019. Autonomous aircraft market worth USD 23.7bn by 2030. https://www.traveldailymedia.com/autonomous-aircraft-marke

    2019

  35. [43]

    Akifumi Wachi and Yanan Sui. 2020. Safe reinforcement learning in constrained Markov decision processes. In International Conference on Machine Learning . 9797–9806

    2020

  36. [44]

    Tyler Westenbroek, Fernando Castaneda, Ayush Agrawal, Shankar Sastry, and Koushil Sreenath. 2022. Lyapunov Design for Robust and Efficient Robotic Reinforcement Learning. arXiv:2208.06721 (2022). https://arxiv.org/pdf/2208. 06721.pdf

    2022 arXiv

  37. [45]

    Weiming Xiang, Patrick Musau, Ayana A Wild, Diego Manzanas Lopez, Nathaniel Hamilton, Xiaodong Yang, Joel Rosenfeld, and Taylor T Johnson. 2018. Verifica- tion for machine learning, autonomy, and neural networks survey.arXiv preprint arXiv:1810.01989 (2018)

    2018 arXiv

  38. [46]

    Ruihan Yang, Minghao Zhang, Nicklas Hansen, Huazhe Xu, and Xiaolong Wang

  39. [47]

    Zhaocong Yuan, Adam W Hall, Siqi Zhou, Lukas Brunke, Melissa Greeff, Jacopo Panerati, and Angela P Schoellig. 2022. Safe-control-gym: A unified benchmark suite for safe learning-based control and reinforcement learning in robotics.IEEE Robotics and Automation Letters 7, 4 (202...

    2022

  40. [48]

    Arnold Zachary and Toner Helen. 2021. AI Accidents: An Emerging Threat.Center for Security and Emerging Technology (2021). https://doi.org/10.51593/20200072

    2021 doi

  41. [49]

    Liqun Zhao, Konstantinos Gatsis, and Antonis Papachristodoulou. 2023. Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approach. In 62nd IEEE Conference on Decision and Control . IEEE, 1320–1325

    2023

  42. [50]

    Bingzhuo Zhong, Hongpeng Cao, Majid Zamani, and Marco Caccamo. 2023. Towards safe ai: Sandboxing dnns-based controllers in stochastic games. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 15340–15349

    2023

  43. [51]

    Chris Ziegler. 2016. A Google self-driving car caused a crash for the first time. https://www.theverge.com/2016/2/29/11134344/google-self-driving-car- crash-report Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning ICCPS ’25, June 03–05, 2025, Woodstock, ...

    2016

  44. [1994]

    Linear matrix inequalities in system and control theory . SIAM

  45. [2017]

    In Computer Aided Verification: 29th International Conference, CA V 2017

    Reluplex: An efficient SMT solver for verifying deep neural networks. In Computer Aided Verification: 29th International Conference, CA V 2017. Springer, 97–117

    2017

  46. [2020]

    In Conference on Robot Learning

    Active domain randomization. In Conference on Robot Learning . PMLR, 1162–1176

  47. [2022]

    2022 International Conference on Learning Representations (2022)

    Learning vision-guided quadrupedal locomotion end-to-end with cross- modal transformers. 2022 International Conference on Learning Representations (2022)

    2022

  48. [2024]

    arXiv preprint arXiv:2409.05898 (2024)

    Simplex-enabled Safe Continual Learning Machine. arXiv preprint arXiv:2409.05898 (2024)

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.