Pith. sign in

REVIEW 3 major objections 6 minor 78 references

Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-level MARL architecture with CBF safety filters can reach near-perfect safety and success in dense multi-agent driving.

desk verdict Solid empirical template for hierarchical MARL with CBF filters, but the headline safety guarantee rests on a flawed application of single-agent forward invariance to coupled multi-agent dynamics. read the letter →

arxiv 2507.14850 v2 pith:SX36ZMBF submitted 2025-07-20 cs.LG cs.AIcs.RO

H. M. Sabbir Ahmad , Ehsan Sabouni , Alexander Wasilkoff , Param Budhraja , Zijian Guo , Songyuan Zhang , Chuchu Fan , Christos Cassandras
show 1 more author
Wenchao Li
This is my paper · ORCID
classification cs.LGcs.AIcs.RO
keywords hierarchicalmulti-agentreinforcementlearningcontrolbarrierfunctionssafety-criticalautonomousdrivingskill-basedCBF-QPsafetyfilterpointwise-in-timepartiallyobservableMARL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a hierarchical multi-agent reinforcement learning method in which a high-level policy chooses cooperative skills for all agents, and each agent's low-level policy executes its skill through a control-barrier-function safety filter. The authors claim this decomposition turns pointwise-in-time safety into a per-agent, local property that holds during both training and deployment, unlike constrained-MDP approaches that only guarantee safety on average over trajectories. They report near-perfect success and safety rates, around 99 percent, across dense traffic scenarios including merging, intersections, roundabouts, bottlenecks, and tollgates, while also improving success-weighted time and energy efficiency over the baselines. If correct, the method shows that hard safety constraints and cooperative task learning can coexist in multi-agent systems without sacrificing sample efficiency.

What carries the argument

The load-bearing object is the hierarchical policy composition $\pi_H \circ \pi_z$: a high-level policy $\pi_H$ selects a joint skill $z$ for all agents, and each agent's low-level policy executes its skill through a parametric quadratic program (QP). The QP uses High-Order Control Barrier Functions (HOCBFs) to enforce inter-agent and road-boundary constraints $b_j^i(s^i, s^j) \ge 0$ pointwise in time, along with Control Lyapunov Functions (CLFs) that steer the agent toward the skill's termination condition. A CBF is a function whose nonnegative level set is forward invariant, so enforcing the CBF inequality keeps the agent inside the safe set for all future times. The QP parameters are learned by backpropagating through the KKT conditions of the QP, so the RL objective shapes the controller parameters while the safety filter remains a hard layer.

What would settle it

Record the CBF constraint values $b_j^i(s^i_t, s^j_t)$ throughout an episode under the trained policies; if any of them ever goes negative during evaluation in the simulator, Theorem 5.1's promise is violated in the tested setting. Alternatively, perturb the vehicle model parameters in the simulator and measure whether the near-perfect success rate degrades materially.

Watch

Extended reading notes

Core claim

The paper's central claim is that splitting the multi-agent control problem into a high-level skill-selection policy and a low-level CBF-based skill-execution policy makes safety a forward-invariant property of each skill, and therefore of the whole hierarchical policy. Theorem 5.1 states that satisfying the CBF constraints in the low-level quadratic program guarantees safe execution of the skills, and hence guarantees the safety of the overall approach. Empirically, the method achieves roughly 98 to 99 percent success rates in five traffic environments and in lidar-based target and spread scenarios, with lower success-weighted time and energy than the baselines. The experiments use a kinematic bicycle model for the CBF constraints while the simulator itself runs on a physics engine, which is why the measured success is near-perfect rather than exactly 100 percent.

Load-bearing premise

The safety guarantee is certified only for the kinematic bicycle model used inside the CBF constraints, while the actual training and evaluation run on a physics-engine simulator, so the guarantee's transfer to the evaluated system is an unproven assumption.

Editorial extensions

If this is right

  • If Theorem 5.1 holds for the modeled dynamics, agents cannot violate the modeled safety constraints during training, so exploration in hierarchical RL never carries the system into unsafe states.
  • Because the safety filter acts per agent from local observations, the approach supports decentralized execution after centralized training, with both the high-level and low-level policies conditioned only on the agent's own observation and history.
  • The near-perfect success rates across conflicting traffic topologies imply that cooperative driving tasks such as merging, roundabout crossing, and bottleneck coordination can be handled mostly through skill selection, without unsafe low-level maneuvers.
  • The measured convergence within roughly 300k iterations, versus 1M for baseline methods, indicates that the hierarchy reduces sample complexity compared with flat safe MARL baselines.
  • Skills trained in one environment transfer to another with only a small drop in success, suggesting that the CBF safety layer makes low-level behaviors reusable across tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Since the safety guarantee is proved only for the kinematic bicycle model used inside the CBF constraints, while the simulator runs a different physics engine, the honest interpretation is that the empirical 99 percent success is a robustness observation rather than a certified transfer; verifying the guarantee under model mismatch, for instance with robust CBFs, is the natural next step.
  • The safety layer is per-agent and independent of the high-level task reward, so one could decouple task-level exploration from safety enforcement, allowing aggressive coordination exploration without needing to penalize unsafe low-level actions.
  • Restricting safety to individual-skill execution may limit cooperative maneuvers that require jointly planned space-time corridors; extending CBF constraints to joint skill pairs would test whether the hierarchy can handle tighter coordination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes HMARL-CBF, a hierarchical multi-agent reinforcement learning method in which a high-level policy selects among interpretable skills and a low-level CBF-QP controller executes the skill subject to pointwise safety constraints. The central theoretical claim is Theorem 5.1: satisfying the CBF constraints in (19) guarantees safe execution of the skills and hence safety of the entire approach. The method is validated in five MetaDrive traffic environments and a lidar-based suite, reporting near-perfect success rates and improved time/energy efficiency over several baselines, together with ablation, generalization, and skill-transfer studies.

Significance. If the safety guarantee were established, the hierarchical skill-based decomposition with pointwise CBF filters would be a valuable contribution to safe MARL, as it promises safety during training and deployment while avoiding the trajectory-level constraints of CMDP formulations. The experimental effort is substantial: five seeds, multiple baselines, ablations, generalization tests, and skill-transfer experiments. However, the paper's central theoretical claim is not supported by the proof given, and the empirical evaluation does not compensate for the missing robustness argument under model mismatch. With the guarantee removed, the contribution reduces to an empirical study of a plausible heuristic, which materially weakens its significance.

major comments (3)
  1. [§5.2, Theorem 5.1 and Eq. (36)] The proof of Theorem 5.1 applies the single-agent forward-invariance result Theorem 3.4 to inter-agent constraints b_i^j(s_i,s_j), but the time derivative of b_i^j contains \dot{s}_j, which is driven by agent j's control and is not a decision variable in agent i's QP (19). Equation (36) explicitly treats \dot{p}_j and \dot{d}_j as measured quantities, with only a_i appearing in the constraint. Without a bound on other agents' future controls or a joint CBF / potential-function argument, each agent can satisfy its individual QP constraint at every step and the pair can still collide if the other agent's subsequent control differs from the value assumed when the constraint was imposed. Thus the central claim that satisfying (19) 'guarantees' safety is not established. The authors should either prove a multi-agent forward-invariance result under explicit assumptions on other-agent behavior, or use robust/joint CBFs that account for the coupled dynamics.
  2. [§6.6.1 and Eq. (34)] The CBF constraints are computed from the kinematic bicycle model (34), while the simulator uses the PyBullet physics engine, as acknowledged in §6.6.1. Theorem 3.4's forward-invariance guarantee applies only to the exact model (3). Model mismatch means the guaranteed invariant set for the bicycle model need not be invariant for the simulated vehicle; the paper neither quantifies this mismatch nor implements robust CBFs (despite a remark in §5.2 that they could be used in the presence of model uncertainty). Consequently, the claim that safety is guaranteed 'during both the training phase and real-world deployment' (Section 1, contribution 2) does not transfer to the evaluated system. The measured 99% success rate is an empirical observation, not a certificate.
  3. [§6.4, Table 1 and Figures 4–6] The paper claims 'almost 100% success rate ... from the beginning of training,' but the training curves in Figures 4 and 6 do not display the first iterations in sufficient detail to support that phrase. More importantly, the baseline comparison is not apples-to-apples: the baselines 'use 2000+ hours of individual driving experience in addition to the training iterations' while the proposed method trains only on the joint data. The authors should either run all methods under identical sample budgets or clearly separate pre-training/experience requirements, since the large success-rate gap may partly reflect this asymmetry.
minor comments (6)
  1. [§3.2, Eq. (8)] The CLF definition uses \eta(s) without defining it; please specify that it is a class-K function or otherwise clarify its role.
  2. [§6.6.1, Eq. (36)] The displayed equation for the inter-agent CBF constraint is corrupted and unreadable in the manuscript; please fix the typesetting.
  3. [§6.3] The sentence 'Further details about the implementation can be found in 6.2' appears to refer to the wrong section; it should point to Section 6.6.
  4. [§6.6.2, Target Bicycle dynamics] In the bicycle model for the Target Bicycle environment, the derivatives of cos \theta and sin \theta omit the wheelbase L; as written the equations are only valid for L=1.
  5. [§5.1, Eq. (14)] The indicator function I(x) is defined as '1 if x<0 for some i, 0 otherwise,' which is ambiguous; clarify the index and the intended condition.
  6. [§2, Related Work] Reference [44] is from the same group and appears to anticipate parts of the method; please clarify the novel contribution with respect to [44] in the related work.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the safety theorem is a standard CBF application (with an unverified multi-agent premise), and the empirical results are measured, not fitted.

full rationale

The paper's derivation chain is not circular. The low-level QP policy (19) enforces CBF/CLF constraints, and its parameter gradients (21)-(24) are obtained from the KKT conditions of that QP, not from the reported success/safety metrics. The near-100% success rates are measured in MetaDrive's PyBullet-based simulation, so they are independent of the kinematic bicycle model used to construct the CBF constraints; the paper itself states this limitation: 'we only use a dynamic bicycle model for the CBF constraints, the environment uses pybullet physics engine for simulation.' The main caveat is Theorem 5.1: its proof invokes the single-agent CBF forward-invariance Theorem 3.4, but the inter-agent constraint derivative in (36) treats other agents' velocities (dot-p_j, dot-d_j) as measured quantities rather than as part of agent i's controlled dynamics, so the forward-invariance premise is not established for the coupled multi-agent system. This is a soundness/correctness gap, not a circular reduction, because no parameter is fitted to the claimed guarantee and the theorem's conclusion is not used as an input. Self-citations such as [44], [51], and [56] appear in related work or as baselines and are not load-bearing for the central derivation. Overall, the central claim retains independent content, and the circularity score is correspondingly low.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central safety claim rests on standard CBF forward invariance, exact dynamics, initial safety, and feasibility of simultaneous QPs, plus a large set of hand-picked reward and skill parameters. The paper does not verify the CBF condition over the continuous state space or prove QP feasibility under multi-agent coupling, and it uses an approximate model for the safety filter. The empirical contribution therefore carries most of the weight.

free parameters (5)
  • CBF/CLF QP parameters phi (class-K coefficients, safety circle radius, CLF convergence rates) = not reported
    Learned by policy gradient in the low-level skill policies (Eq. 19); the claim that they remain valid CBF parameters is not verified beyond box constraints (Section 6.6.1).
  • Safety penalty weights p_j^i in extrinsic reward (Eq. 14) = not reported
    Hand-chosen positive weights that convert safety violations into reward terms; they affect cooperation and the measured success rate.
  • lambda in revised low-level reward (Eq. 28) = not reported
    Balances high-level advantage and intrinsic skill reward; set by hand and not specified numerically.
  • Intrinsic reward coefficients c1, c2, c3, c4 (Eq. 39) = not reported
    Shape smoothness, speed and heading tracking, and lane centering; values are not provided.
  • Skill termination increments dv, Delta v, Delta theta = not reported
    Define speed-up, slow-down, and turn targets for each skill; chosen per environment and not reported.
assumptions (5)
  • standard math CBF forward-invariance theorem (Theorem 3.4 from [54]) guarantees safety of the set C when inequalities (7) hold for all t.
    Invoked in Theorem 5.1 without proof; standard control theory result.
  • domain assumption Agent dynamics are control-affine and exactly known, as in Eq. (3).
    The CBF constraints in Eq. (19) are derived from this model, but the simulator uses PyBullet physics (Section 6.6.1).
  • domain assumption Candidate functions b_j^i are valid CBFs/HOCBFs for the system.
    Learned parameters phi are only constrained to be nonnegative; no offline verification over the state space is performed.
  • ad hoc to paper The pointwise CBF QPs for all agents remain feasible simultaneously under decentralized execution.
    Theorem 5.1 treats other agents' states as exogenous inputs and does not prove feasibility of the coupled multi-agent QP; no argument addresses conflicting constraints.
  • domain assumption Safety constraints are initially satisfied.
    Stated in the Problem Formulation: 'We assume that the constraints are initially satisfied upon their introduction.'

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems." pith.science (2026). https://pith.science/paper/SX36ZMBF

@misc{pith2026250714850,
  author       = {Pith},
  title        = {Pith review of: Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SX36ZMBF}},
  note         = {Machine review of arXiv:2507.14850}
}
read the original abstract

We address the problem of safe policy learning in multi-agent safety-critical autonomous systems. In such systems, it is necessary for each agent to meet the safety requirements at all times while also cooperating with other agents to accomplish the task. Toward this end, we propose a safe Hierarchical Multi-Agent Reinforcement Learning (HMARL) approach based on Control Barrier Functions (CBFs). Our proposed hierarchical approach decomposes the overall reinforcement learning problem into two levels learning joint cooperative behavior at the higher level and learning safe individual behavior at the lower or agent level conditioned on the high-level policy. Specifically, we propose a skill-based HMARL-CBF algorithm in which the higher level problem involves learning a joint policy over the skills for all the agents and the lower-level problem involves learning policies to execute the skills safely with CBFs. We validate our approach on challenging environment scenarios whereby a large number of agents have to safely navigate through conflicting road networks. Compared with existing state of the art methods, our approach significantly improves the safety achieving near perfect (within 5%) success/safety rate while also improving performance across all the environments.

Figures

Figures reproduced from arXiv: 2507.14850 by the authors.

Figure 1
Figure 1. At the higher level, agents choose skills in a decentralized manner based on [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the various METADRIVE environments. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 4
Figure 4. From left to right: success rate in merging, tollgate, and bottleneck environments. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: From left to right: average episode length in merging, tollgate, and bottleneck environments. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Plot of the success rate of the agents with our algorithm vs the baselines in merging, [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Success/safety rate vs reward for Target and Spread environment. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Success/safety rate vs reward for Bicycle Target environment. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: From left to right: illustration of i. inter agent safety constraints on another lane, ii. inter [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Visualization of agents’ trajectories under the trained model: for each vehicle, its ten most [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Visualization of skills under the trained model. From left to right: (a) acceleration [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 74 canonical work pages

  1. [1]

    Lowe, R., Y . Wu, A. Tamar, et al. Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems (NeurIPS). 2017

    work page 2017

  2. [2]

    Samvelyan, C

    Rashid, T., M. Samvelyan, C. Schroeder. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In International Conference on Machine Learning, pages 4295–4304. 2018

    work page 2018

  3. [3]

    Lever, A

    Sunehag, P., G. Lever, A. Gruslys, et al. Value-decomposition networks for cooperative multi- agent learning. In Advances in Neural Information Processing Systems (NeurIPS). 2017

    work page 2017

  4. [4]

    Zhao, Y ., Y . Yang, Z. Lu, et al. Multi-agent first order constrained optimization in policy space. Advances in Neural Information Processing Systems, 36, 2024

    work page 2024

  5. [5]

    Constrained Markov decision processes

    Altman, E. Constrained Markov decision processes. Routledge, 2021

    work page 2021

  6. [6]

    Grudzien Kuba, Y

    Gu, S., J. Grudzien Kuba, Y . Chen, et al. Safe multi-agent reinforcement learning for multi-robot control. Artificial Intelligence, 319:103905, 2023

    work page 2023

  7. [7]

    Tang, H., J. Hao, T. Lv, et al. Hierarchical deep multiagent reinforcement learning with temporal abstraction, 2019

    work page 2019

  8. [8]

    Wang, R., K. Wang, F. Xu, et al. Hierarchical cooperative multi-agent reinforcement learning with dual coordination mechanism. In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial 22 Intelligence and Thirteenth Symposium on Educational Advances in Artificial In...

    work page 2023

Show all 78 references
  1. [9]

    Mahadevan, R

    Ghavamzadeh, M., S. Mahadevan, R. Makar. Hierarchical multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems, 13:197–229, 2006

    2006

  2. [10]

    Wu, C., A. R. Kreidieh, K. Parvate, et al. Flow: A modular learning framework for mixed autonomy traffic. IEEE Transactions on Robotics, 38(2):1270–1286, 2022

    2022

  3. [11]

    Son, K., D. Kim, W. J. Kang, et al. Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In International Conference on Machine Learning, pages 5887–5896. 2019

    2019

  4. [12]

    Farquhar, T

    Foerster, J., G. Farquhar, T. Afouras, et al. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 2018

    2018

  5. [13]

    Pertsch, K., Y . Lee, J. J. Lim. Accelerating reinforcement learning with learned skill priors. In Conference on Robot Learning (CoRL). 2020

    2020

  6. [14]

    Parr, R., S. Russell. Reinforcement learning with hierarchies of machines. In Advances in Neural Information Processing Systems, pages 1043–1049. 1997

    1997

  7. [15]

    Dietterich, T. G. Hierarchical reinforcement learning with the maxq value function decomposi- tion. Journal of Artificial Intelligence Research, 13:227–303, 2000

    2000

  8. [16]

    Sutton, R. S., D. Precup, S. Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1-2):181–211, 1999

    1999

  9. [17]

    Precup, D., R. S. Sutton. Temporal abstraction in reinforcement learning.PhD thesis, University of Massachusetts Amherst, 2000

    2000

  10. [18]

    Dayan, P., G. E. Hinton. Feudal reinforcement learning. Advances in Neural Information Processing Systems, pages 271–278, 1992

    1992

  11. [19]

    Vezhnevets, A. S., S. Osindero, T. Schaul, et al. Feudal networks for hierarchical reinforcement learning. In International Conference on Machine Learning, pages 3540–3549. 2017

    2017

  12. [20]

    Nachum, O., S. Gu, H. Lee, et al. Data-efficient hierarchical reinforcement learning. In Advances in Neural Information Processing Systems, pages 3303–3313. 2018

    2018

  13. [21]

    Bacon, P.-L., M. Harb, D. Precup. The option-critic architecture. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31. 2017

    2017

  14. [22]

    Bacon, M

    Harb, M., P.-L. Bacon, M. Klissarov, et al. Waiting for the right time: When to commit to action in hierarchical reinforcement learning. International Conference on Learning Representations (ICLR), 2018

    2018

  15. [23]

    Li, S., R. Wang, M. Tang, et al. Hierarchical reinforcement learning with advantage-based auxiliary rewards. Advances in Neural Information Processing Systems, 32, 2019

    2019

  16. [24]

    Synnaeve, A

    Gehring, J., G. Synnaeve, A. Krause, et al. Hierarchical skills for efficient exploration.Advances in Neural Information Processing Systems, 34:11553–11564, 2021

    2021

  17. [25]

    Ahilan, S., P. Dayan. Feudal multi-agent hierarchies for cooperative reinforcement learning, 2019

    2019

  18. [26]

    Givony, T

    Tessler, C., S. Givony, T. Zahavy, et al. A deep hierarchical approach to lifelong learning in minecraft. In AAAI Conference on Artificial Intelligence, pages 1553–1561. 2017

    2017

  19. [27]

    Gregor, K., D. J. Rezende, D. Wierstra. Variational intrinsic control. arXiv preprint arXiv:1611.07507, 2016

    2016 arXiv

  20. [28]

    Mankowitz, D. J., T. A. Mann, S. Mannor. Adaptive skills, adaptive partitions (asap). In Advances in Neural Information Processing Systems (NeurIPS). 2016

    2016

  21. [29]

    Yang, M., Y . Yang, Z. Lu, et al. Hierarchical multi-agent skill discovery. InNeurIPS. 2023

    2023

  22. [30]

    Bacon, M

    Harb, J., P.-L. Bacon, M. Klissarov, et al. When waiting is not an option: Learning options with a deliberation cost. In AAAI Conference on Artificial Intelligence. 2018

    2018

  23. [31]

    Gupta, S

    Sharma, A., A. Gupta, S. Levine. Skill-based meta-reinforcement learning. In International Conference on Machine Learning (ICML). 2020

    2020

  24. [32]

    Weaver, C

    Hao, C., C. Weaver, C. Tang, et al. Skill-critic: Refining learned skills for reinforcement learning. arXiv preprint arXiv:2306.08388, 2023. 23

    2023 arXiv

  25. [33]

    Zhao, Y ., Y . Yang, Z. Lu, et al. Multi-agent first order constrained optimization in policy space. Advances in Neural Information Processing Systems, 36:39189–39211, 2023

    2023

  26. [34]

    Nachum, E

    Chow, Y ., O. Nachum, E. Duenez-Guzman, et al. A lyapunov-based approach to safe reinforce- ment learning. Advances in neural information processing systems, 31, 2018

    2018

  27. [35]

    Nachum, A

    Chow, Y ., O. Nachum, A. Faust, et al. Lyapunov-based safe policy optimization for continuous control. arXiv preprint arXiv:1901.10031, 2019

    1901 arXiv

  28. [36]

    Liu, Y ., J. Ding, X. Liu. Ipo: Interior-point policy optimization under constraints. Proceedings of the AAAI Conference on Artificial Intelligence, 34:4940–4947, 2020

    2020

  29. [37]

    Liu, M., T. Yang, M. Wang. A natural policy gradient primal-dual method for constrained markov decision processes. In Advances in Neural Information Processing Systems (NeurIPS). 2020

    2020

  30. [38]

    Ding, Z., K. Wei, H. Lu. Upper confidence primal-dual reinforcement learning for cmdps. In Advances in Neural Information Processing Systems (NeurIPS). 2020

    2020

  31. [39]

    Huang, W., J. Ji, B. Zhang, et al. Safedreamer: Safe reinforcement learning with world models. In The Twelfth International Conference on Learning Representations. 2024

    2024

  32. [40]

    Wang, Y ., S. S. Zhan, R. Jiao, et al. Enforcing hard constraints with soft barriers: Safe reinforcement learning in unknown stochastic environments. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., Proceedings of the 40th International Conferen...

    2023

  33. [41]

    Gu, S., J. G. Kuba, Y . Chen, et al. Safe multi-agent reinforcement learning for multi-robot control. Artificial Intelligence, page 103905, 2023

    2023

  34. [42]

    So, O., C. Ge, C. Fan. Solving minimum-cost reach avoid using reinforcement learning. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, C. Zhang, eds.,Advances in Neural Information Processing Systems, vol. 37, pages 30951–30984. Curran Associates, Inc., 2024

    2024

  35. [43]

    Yu, D., H. Ma, S. Li, et al. Reachability constrained reinforcement learning. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, S. Sabato, eds.,Proceedings of the 39th International Conference on Machine Learning, vol. 162 of Proceedings of Machine Learning Research...

    2022

  36. [44]

    Sabbir Ahmad, V

    Sabouni, E., H. Sabbir Ahmad, V . Giammarino, et al. Reinforcement learning-based receding horizon control using adaptive control barrier functions for safety-critical systems*. In 2024 IEEE 63rd Conference on Decision and Control (CDC), pages 401–406. 2024

    2024

  37. [45]

    Berducci, L., S. Yang, R. Mangharam, et al. Learning adaptive safety for multi-agent systems. 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 2859–2865, 2023

    2024

  38. [46]

    Cheng, Y ., P. Zhao, N. Hovakimyan. Safe and efficient reinforcement learning using disturbance- observer-based control barrier functions. In Learning for Dynamics and Control Conference, pages 104–115. PMLR, 2023

    2023

  39. [47]

    Notomista, P

    Emam, Y ., G. Notomista, P. Glotfelter, et al. Safe reinforcement learning using robust control barrier functions. IEEE Robotics and Automation Letters, 10:2886–2893, 2021

    2021

  40. [48]

    Pereira, M. A., Z. Wang, I. Exarchos, et al. Safe optimal control using stochastic barrier functions and deep forward-backward sdes, 2020

    2020

  41. [49]

    Gao, Z., G. Yang, A. Prorok. Online control barrier functions for decentralized multi-agent navigation. In 2023 International Symposium on Multi-Robot and Multi-Agent Systems (MRS), pages 107–113. IEEE, 2023

    2023

  42. [50]

    Orosz, R

    Cheng, R., G. Orosz, R. M. Murray, et al. End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Ar...

    2019

  43. [51]

    Zhang, S., O. So, M. Black, et al. Discrete GCBF proximal policy optimization for multi-agent safe optimal control. In The Thirteenth International Conference on Learning Representations. 2025. 24

    2025

  44. [52]

    Mahadevan, M

    Makar, R., S. Mahadevan, M. Ghavamzadeh. Hierarchical multi-agent reinforcement learning. In Proceedings of the fifth international conference on Autonomous agents , pages 246–253. 2001

    2001

  45. [53]

    Mahadevan

    Rohanimanesh, K., S. Mahadevan. Learning to take concurrent actions. In S. Becker, S. Thrun, K. Obermayer, eds., Advances in Neural Information Processing Systems, vol. 15. MIT Press, 2002

    2002

  46. [54]

    Ames, A. D., K. Galloway, J. W. Grizzle. Control lyapunov functions and hybrid zero dynamics. In 2012 IEEE 51st IEEE Conference on Decision and Control (CDC), pages 6837–6842. 2012

    2012

  47. [55]

    Xiao, W., C. Belta. High-order control barrier functions. IEEE Transactions on Automatic Control, 67(7):3655–3662, 2022

    2022

  48. [56]

    Xiao, W., C. G. Cassandras, C. Belta. Hierarchical Optimal Control with Barrier Functions, pages 109–120. Springer International Publishing, Cham, 2023

    2023

  49. [57]

    Sreenath

    Nguyen, Q., K. Sreenath. Exponential control barrier functions for enforcing high relative- degree safety-critical constraints. In 2016 American Control Conference (ACC), pages 322–328. IEEE, 2016

    2016

  50. [58]

    Sutton, R. S., D. Precup, S. Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1):181–211, 1999

    1999

  51. [59]

    Black, A

    Chuck, C., K. Black, A. Arjun, et al. Granger causal interaction skill chains. Transactions on Machine Learning Research, 2024

    2024

  52. [60]

    Iqbal, S., F. Sha. Actor-attention-critic for multi-agent reinforcement learning. In International conference on machine learning, pages 2961–2970. PMLR, 2019

    2019

  53. [61]

    Yu, C., A. Velu, E. Vinitsky, et al. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35:24611–24624, 2022

    2022

  54. [62]

    Dukkipati

    Gupta, S., A. Dukkipati. Probabilistic view of multi-agent reinforcement learning: A unified approach. 2019

    2019

  55. [63]

    Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229–256, 1992

    1992

  56. [64]

    Samvelyan, C

    Rashid, T., M. Samvelyan, C. S. De Witt, et al. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21(178):1–51, 2020

    2020

  57. [65]

    Hong, Y ., Y . Jin, Y . Tang. Rethinking individual global max in cooperative multi-agent reinforcement learning. Advances in neural information processing systems, 35:32438–32449, 2022

    2022

  58. [66]

    Quiroz, R

    Xu, M., M. Quiroz, R. Kohn, et al. Variance reduction properties of the reparameterization trick. In The 22nd international conference on artificial intelligence and statistics, pages 2711–2720. PMLR, 2019

    2019

  59. [67]

    Choi, J. J., D. Lee, K. Sreenath, et al. Robust control barrier–value functions for safety-critical control. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 6814–6821. IEEE, 2021

    2021

  60. [68]

    Amos, B., J. Z. Kolter. Optnet: Differentiable optimization as a layer in neural networks. In International conference on machine learning, pages 136–145. PMLR, 2017

    2017

  61. [69]

    Li, Q., Z. Peng, L. Feng, et al. Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning, 2022

    2022

  62. [70]

    Wolski, P

    Schulman, J., F. Wolski, P. Dhariwal, et al. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  63. [71]

    Haarnoja, T., A. Zhou, P. Abbeel, et al. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. PMLR, 2018

    2018

  64. [72]

    Kulkarni, T. D., K. Narasimhan, A. Saeedi, et al. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. Advances in neural information processing systems, 29, 2016. 25

    2016

  65. [73]

    Peng, Z., Q. Li, K. M. Hui, et al. Learning to simulate self-driven particles system with coordinated policy optimization.Advances in Neural Information Processing Systems, 34:10784– 10797, 2021

    2021

  66. [74]

    De Witt, C. S., T. Gupta, D. Makoviichuk, et al. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020

    2011 arXiv

  67. [75]

    Yang, Y ., R. Luo, M. Li, et al. Mean field multi-agent reinforcement learning. InInternational conference on machine learning, pages 5571–5580. PMLR, 2018

    2018

  68. [76]

    Narvekar, S., B. Peng, M. Leonetti, et al. Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research, 21(181):1–50, 2020

    2020

  69. [77]

    Nayak, S., K. Choi, W. Ding, et al. Scalable multi-agent reinforcement learning through intelligent information aggregation, 2023

    2023

  70. [78]

    Gu, S., J. G. Kuba, M. Wen, et al. Multi-agent constrained policy optimisation, 2022. 26

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.