Red-Team Multi-Agent Reinforcement Learning for Emergency Braking Scenario

Kaifeng Wang; Xiaoqiang Meng; Xin Gao; Xueyuan Li; Yinsong Chen; Zirui Li

REVIEW 5 major objections 5 minor 32 references

Red-Team Multi-Agent Reinforcement Learning for Emergency Braking Scenario

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that training nearby vehicles as red-team agents raises an autonomous vehicle's emergency-braking collision rate from 5% to 85% in simulation, exposing corner cases that static scenario modeling misses.

desk verdict The framework is worth a look, but the headline 5%-to-85% collision-rate claim is a cross-condition comparison that Table II does not support. read the letter →

arxiv 2507.15587 v1 pith:RXSD7YA4 submitted 2025-07-21 cs.LG cs.AI

Yinsong Chen , Kaifeng Wang , Xiaoqiang Meng , Xueyuan Li , Zirui Li , Xin Gao This is my paper · ORCID

classification cs.LGcs.AI

keywords red-teamreinforcementlearningsafety-criticalscenariogenerationautonomousvehicledecision-makingemergencybrakingmulti-agentcornercasediscoveryconstrainedMarkovdecisionprocess

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an autonomous vehicle's emergency-braking policy can be stress-tested by re-casting nearby vehicles as adversarial 'red-team' agents trained with multi-agent reinforcement learning. The authors report that this red-team interference raises the AV's collision rate from 5% to 85% in simulation, while the adversarial vehicles remain bound by traffic rules. If correct, the framework offers a way to actively discover corner cases that are rare in natural driving data, instead of waiting for them to appear. That matters because sparse real-world data makes such failure scenarios hard to find before deployment.

What carries the argument

The load-bearing machinery is threefold: the CGMDP, a Markov decision process augmented with a weighted interaction graph $G(V,E)$ and a constraint set $\Omega$ that forbids illegal or non-human maneuvers; the DC-GPPO algorithm, which extends PPO with a Lagrangian cost term and a hard action filter to keep red-team behavior within traffic rules while maximizing disruption; and the PTZ model, which computes a threat zone from time-to-collision and the AV's acceleration and heading to reward red-team actions that intensify danger. Together they convert 'find corner cases' into an RL objective with safety constraints, so the red-team learns to provoke failures without itself driving illegally.

What would settle it

Take the trained red-team policies and run them against several independently trained AV emergency-braking policies, such as different RL algorithms or a production rule-based controller, in the same SUMO scenario. If the collision-rate increase drops sharply or disappears against any of those policies, the claim that red-team interference exposes general corner cases in AV decision-making would be undermined; if the increase persists across diverse AV policies, the claim is strongly supported.

Watch

Extended reading notes

Core claim

The central claim is that safety-critical driving scenarios, specifically lead-vehicle emergency braking, can be made far more dangerous to an AV by training background vehicles as coordinated red-team agents. The paper models the red-team decision process as a Constraint Graph Representation Markov Decision Process (CGMDP), which encodes vehicles and their interactions as a weighted graph and applies a hard action-space constraint plus a soft behavioral cost, and couples this with a Policy Threat Zone (PTZ) model that quantifies the threat red-team vehicles pose to the AV. In simulation, the resulting DC-GPPO algorithm raised the AV collision rate from a 5% baseline to 75% with one interfering vehicle and 85% with multiple interfering vehicles, and produced a variety of collision trajectories the authors identify as corner cases. The paper argues this demonstrates that active adversarial exploration by red-team vehicles exposes decision-making weaknesses that static scenario modeling and data-driven generation miss.

Load-bearing premise

The evaluation assumes that the single AV decision-making model from the authors' earlier work is a representative competent baseline, since every reported collision-rate increase is measured against that one fixed policy.

Editorial extensions

If this is right

If the 5%-to-85% result holds, red-team RL can serve as an automated adversarial test that ranks AV policies by how easily they are broken, not just by average performance.
The framework can generate labeled corner-case trajectories, including collision locations and AV and red-team traces, that could be replayed as training data for more robust AV decision-making.
Because the red-team vehicles are constrained to obey traffic rules, the discovered failures are more likely to correspond to plausible real-world incidents than unconstrained adversarial perturbations.
The same CGMDP-plus-PTZ recipe should transfer to other safety-critical scenarios, such as cut-ins, lane changes, and intersections, though the paper only demonstrates emergency braking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A surprising result of this kind deserves cross-policy validation: the 85% collision rate was measured against one fixed AV policy from the authors' earlier work, so a natural extension is to test the same red-team policies against several independently trained AV policies; if the collision-rate jump shrinks, the corner cases may be specific to that one policy.
The reward design rewards collisions directly, so the red-team may be finding the easiest collision rather than the most diverse or most safety-relevant failure; an extension would reward coverage of distinct failure modes.
The PTZ model relies on the AV's own acceleration and heading as signals of avoidance intent; in a real deployment the AV would likely obscure or randomize such signals, so the red-team's information advantage could shrink in practice.
The 5% baseline comes from the same simulation environment and the authors' DRS-PPO policy, so an independent replication in a different simulator or with a production AV stack would settle how much of the effect is framework-specific versus environment-specific.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, quick take on arXiv:2507.15587.

The paper is a decent engineering demonstration of using multi-agent RL as a red-team to stress-test an AV emergency-braking policy in SUMO. It is not a new idea—Feng, Ding, Chen et al. have all done adversarial scenario generation—but the specific combination of a graph-based MDP with hard action-space constraints, Lagrangian soft constraints, and a policy threat zone is a reasonable, clearly described extension. The training curves with 5 seeds suggest the method learns something, and the trajectory plots give a concrete sense of the generated corner cases.

The soft spots are real. The headline result, "red-team vehicles increased the collision rate from 5% to 85%," is not supported by Table II. The 5% is the SVI baseline, the 85% is the MVI DC-GPPO result, and the MVI baseline cell is blank. Without measuring the multi-vehicle no-interference baseline, you cannot attribute the jump to the red-team—the added vehicles alone, even without RL, might raise collision rates substantially. Second, the primary metric is collision rate, which is exactly what the red-team reward optimizes, so reporting a rise is a statement about training success, not an emergent property. Third, the evaluation lacks error bars on test results, no random or rule-based red-team baseline, and uses only the authors' own prior AV policy as the victim. Fourth, several key components are underspecified: the functions f1–f4, Phi1–Phi2, the constraint set Omega, and the constants C0, C1 are all undefined in the math. That makes reproduction impossible without the code, which is not provided.

None of these are fatal, but they are load-bearing. The paper would survive peer review if the authors fix the comparison, add the missing baselines and specifications, and ideally release the simulator code. The core direction—using RL adversaries to find failures in AV policies—remains valuable, and this paper's framework is a plausible step toward that, just not a validated one yet.

Who should read it? People working on AV safety validation and adversarial scenario generation. It is a reasonable cite for the method, but not for the quantitative claim. I would give it a serious referee, but with the expectation of major revisions.

My recommendation: send it out, but insist on the baseline fix and the missing details.

Referee Report

5 major / 5 minor

Summary. The paper proposes a Red-Team Multi-Agent Reinforcement Learning (RMARL) framework for generating adversarial emergency-braking scenarios. Background vehicles are treated as red-team agents trained with a proposed DC-GPPO algorithm operating on a Constraint Graph Representation MDP (CGMDP), with a Policy Threat Zone (PTZ) model intended to amplify threat. Experiments in SUMO compare four red-team training algorithms against an AV decision-making model from the authors' prior work [14]. The paper reports that red-team interference raises AV collision rates from 5% to 85% and claims the framework generates diverse corner cases.

Significance. The constrained-adversary formulation and the graphical state representation are reasonable ideas, and the use of SUMO with multiple RL baselines and five training seeds is a credible start. If the reported result were supported, the work would provide a useful adversarial scenario-generation method for safety validation. However, the central quantitative claim is currently unsupported because it compares a single-vehicle baseline with a multi-vehicle treatment condition, and the evaluation lacks the controls and statistical detail needed to substantiate 'significantly impacts.' The PTZ model is also left unconnected to the optimization objective. With targeted experiments and clearer formalization, the contribution could be made solid.

major comments (5)

[V.C, Table II] The abstract and Section V.C claim that red-team vehicles increased the AV collision rate from 5% to 85%, but the 5% value is the BL row under SVI and the 85% value is DC-GPPO under MVI, where the MVI BL entry is '/'. The comparison therefore conflates the number of interfering vehicles with the effect of red-team training. A multi-vehicle baseline with no red-team interference must be measured and reported; without it, the causal contribution of the red-team is unidentified.
[V.B, V.C, Table II] Table II reports point estimates only. Fig. 4 shows standard deviations for training rewards across 5 seeds, but no variance, confidence intervals, or test-episode counts are given for collision rate, travel time, lateral acceleration, or average speed. As a result, differences such as GPPO 52% vs GD3QN 50% in SVI and the claimed superiority of DC-GPPO are not statistically assessable. The paper should report means and standard deviations over seeds for all test metrics, and it should also compare against a random or rule-based red-team policy to show that the learned adversarial behavior, rather than merely adding interfering vehicles, is responsible for the collision-rate increase.
[III.C.2, IV.C, Eq. (6)] The PTZ model is a stated contribution, but Eq. (6) defines DPTZ as a set and is never used in the reward Eq. (13), the composite advantage Eq. (12), or the policy update. The text says red-team vehicles 'leverage' the PTZ to identify risks, but no mechanism is specified. Please state exactly how DPTZ or the threat functions Phi1 and Phi2 enter the reward, cost, or action selection; otherwise the PTZ contribution is unfalsifiable.
[IV.C, Eq. (14)] The cost function Cint = C1 for 'inertial interference' is never formally defined. The only example is a red-team vehicle failing to intervene promptly during an AV lane change; without a quantitative condition for when C1 is incurred, the dual-constraint mechanism is not reproducible. Please define inertial interference in terms of observable state variables, for instance relative position and velocity thresholds together with the timing of the red-team action relative to the AV's maneuver.
[V.A] All reported collision-rate increases are measured against a single AV policy from the authors' prior work [14]. To support the claim that the framework generates generally safety-relevant corner cases, the experiments should include at least one independently trained or rule-based AV baseline, or show that discovered scenarios transfer to other AV policies. Otherwise the red-team's success may reflect exploitation of one brittle policy rather than discovery of genuinely general corner cases.

minor comments (5)

[Throughout] The manuscript contains many typographic inconsistencies, such as 'A Vs' with a space, the repeated reference to 'IEEEexample', and 'Fig. 4 present' instead of 'Fig. 4 presents'; a careful proofreading pass is needed.
[References] References [16] and [19] are the same paper by Kipf and Welling; please cite it once and remove the duplicate.
[III.C.3] The action-space description claims a 23-dimensional space from 11 levels of longitudinal acceleration and 13 levels of steering angle with steering restricted to zero longitudinal acceleration. The arithmetic behind 23 is not shown and should be derived explicitly.
[Eq. (6)] The set-builder notation with a vertical bar inside braces and the division sign between Phi1 and Phi2 is nonstandard; please rewrite the definition of DPTZ with explicit logical conditions or separate clauses.
[Fig. 5] The figure caption states that markers indicate collision locations, but in the printed grayscale version the markers are difficult to distinguish from lane and trajectory lines; please use larger or colored symbols and provide a zoomed inset for each collision.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline 5%-to-85% collision-rate result restates the red-team's own reward objective, and the 5% baseline is imported from the authors' prior work [14] with no MVI control in Table II.

fitted input called prediction [Section IV.C Eq. (13); Section V.C 'Test Results' and Table II]
"Rcol = C0, if collision ... In [14], without interference, the A V’s collision rate was only 5%, but under red-team interference it rose to 85%, indicating a substantial degradation of A Vs decision-making safety."

The red-team policy is trained to maximize R_col, a constant reward on AV collision, and the paper's headline metric CR is the frequency of exactly that collision event. The 'from 5% to 85%' result therefore measures how well the reward was optimized, not an independent emergent property of the framework. Reporting a metric that is the training objective as evidence that the framework 'significantly impacts AVs decision-making safety' is a fitted input being presented as an outcome by construction.
self citation load bearing [Section V.A 'Simulation Setup'; Section V.C 'Test Results'; Table II]
"A Vs use the decision-making model from our previous work [14] ... In [14], without interference, the A V’s collision rate was only 5%, but under red-team interference it rose to 85% ... [Table II:] MVI CR / 67.00 55.00 70.00 85.00"

The 5% baseline and the fixed AV policy under attack are taken from [14], whose author list overlaps substantially with the present paper (Wang, Chen, Li, Gao). The baseline is neither re-derived nor re-measured here, and for the MVI condition in which the 85% collision rate was obtained, Table II leaves the baseline cell blank ('/'). Thus the '5% to 85%' comparison is an imported self-cited number compared against a new condition with no measured control, so the causal contribution of the red-team framework is not established independently of the authors' own prior work.

full rationale

Score 6 reflects that the paper's headline quantitative result is partially circular: collision rate is a direct term in the red-team reward (Eq. 13), so a large part of the reported 5%-to-85% increase is a measure of reward optimization, not an independent emergent validation. The evaluation is also anchored to the authors' own prior AV policy [14]; the 5% baseline is imported from that self-citation, and Table II has no MVI baseline ('/'), so the claimed comparison is uncontrolled. The framework does contain independent algorithmic content (CGMDP constraints, PTZ model, DC-GPPO), and the comparison against PPO/GD3QN/GPPO is a legitimate internal baseline, which prevents a higher score. The missing MVI control is primarily a correctness/statistical concern; I do not treat it as an additional circular step beyond the self-citation dependence. The PTZ/TTC modeling is not circular because TTC is a standard external safety metric used to shape rewards, not defined by the target claim.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central result rests on a small set of domain assumptions about the simulator, the fixed AV baseline, and the information available to red-team agents, plus a collection of unspecified reward, cost, threshold, and constraint parameters that are effectively free design choices. No new physical entities are introduced.

free parameters (6)

Collision reward constant C0
Reward in Eq. (13) is central to training red-team agents, but C0 is never given a numeric value; it defines the strength of the collision objective.
Inertial interference cost constant C1
Cost in Eq. (14) shapes the soft constraint against non-disruptive behavior; value not reported.
Reward functions f1(d), f2(delta_v), f3(a_lat, a_lon)
Functional forms are not specified, so their shapes and weights are free design choices that directly affect learned red-team behavior.
Cost function f4(TTC)
TTC-based cost term in Eq. (14) is not defined; its parameters determine how risk is penalized.
PTZ threat functions Phi1 and Phi2
Eq. (6) defines the Policy Threat Zone as a set comprehension, but the actual TTC threshold and danger-level formulas are not given.
Constraint set Omega
Hard constraints on actions are described qualitatively; the specific rules are not enumerated, making Omega a hand-set design choice.

assumptions (5)

domain assumption The AV policy from the authors' prior work [14] is a competent, fixed baseline for safety-critical decision-making.
Used in Section V.C; all reported collision-rate increases are measured against this single AV policy.
domain assumption SUMO's default driver model adequately simulates realistic human-driven background traffic.
Used in Section V.A; background vehicles rely on SUMO's default model, and no validation against real driving data is provided.
domain assumption V2X communication assumptions (all red-team vehicles can communicate; human-driven vehicles cannot) hold in the scenario.
Stated in Section III.C.1 for the adjacency matrix; these assumptions simplify information sharing and may not reflect real deployments.
domain assumption TTC below two seconds is the appropriate definition of a safety-critical emergency braking scenario.
Section III.A uses TTC below 2 s from [32] to define the scenario; the paper's scope is limited by this choice.
domain assumption Graph features from six nearest neighbors and the node, adjacency, and mask matrices capture all interaction information needed for red-team policies.
State representation in Eqs. (2)-(5) restricts information; the paper provides no ablation showing this is sufficient.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Red-Team Multi-Agent Reinforcement Learning for Emergency Braking Scenario." pith.science (2026). https://pith.science/paper/RXSD7YA4

@misc{pith2026250715587,
  author       = {Pith},
  title        = {Pith review of: Red-Team Multi-Agent Reinforcement Learning for Emergency Braking Scenario},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RXSD7YA4}},
  note         = {Machine review of arXiv:2507.15587}
}

read the original abstract

Current research on decision-making in safety-critical scenarios often relies on inefficient data-driven scenario generation or specific modeling approaches, which fail to capture corner cases in real-world contexts. To address this issue, we propose a Red-Team Multi-Agent Reinforcement Learning framework, where background vehicles with interference capabilities are treated as red-team agents. Through active interference and exploration, red-team vehicles can uncover corner cases outside the data distribution. The framework uses a Constraint Graph Representation Markov Decision Process, ensuring that red-team vehicles comply with safety rules while continuously disrupting the autonomous vehicles (AVs). A policy threat zone model is constructed to quantify the threat posed by red-team vehicles to AVs, inducing more extreme actions to increase the danger level of the scenario. Experimental results show that the proposed framework significantly impacts AVs decision-making safety and generates various corner cases. This method also offers a novel direction for research in safety-critical scenarios.

Figures

Figures reproduced from arXiv: 2507.15587 by the authors.

**Figure 3.** Constraint Graph Representation Markov Decision Process. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

**Figure 2.** Illustration of scenarios.(a) is the lead vehicle emergency [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Illustration of reward curves.(a) is the single-vehicle inter [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Trajectories of red-team vehicles and AVs. AV0 and AV1 represent the trajectories of AVs, while RV0 and RV1 represent the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 25 canonical work pages

[14]

Dynamic Residual Safe Reinforcement Learning for Multi-Agent Safety-Critical Scenarios Decision-Making

K. Wang, Y . Chen, Q. Liu, X. Li, and X. Gao, “Dynamic residual safe reinforcement learning for multi-agent safety-critical scenarios decision- making,” arXiv preprint arXiv:2504.06670 , 2025

work page Pith review arXiv 2025
[1]

Edge computing for autonomous driving: Opportunities and challenges,

S. Liu, L. Liu, J. Tang, B. Yu, Y . Wang, and W. Shi, “Edge computing for autonomous driving: Opportunities and challenges,” Proceedings of the IEEE, vol. 107, no. 8, pp. 1697–1716, 2019

2019
[2]

A survey on safety-critical driving scenario generation—a methodological perspec- tive,

W. Ding, C. Xu, M. Arief, H. Lin, B. Li, and D. Zhao, “A survey on safety-critical driving scenario generation—a methodological perspec- tive,” IEEE Transactions on Intelligent Transportation Systems , vol. 24, no. 7, pp. 6971–6988, 2023

2023
[3]

Intelligent driving in- telligence test for autonomous vehicles with naturalistic and adversarial environment,

S. Feng, X. Yan, H. Sun, Y . Feng, and H. X. Liu, “Intelligent driving in- telligence test for autonomous vehicles with naturalistic and adversarial environment,” Nature communications, vol. 12, no. 1, p. 748, 2021

work page 2021
[4]

Cmts: A conditional multiple trajectory synthesizer for generating safety-critical driving scenarios,

W. Ding, M. Xu, and D. Zhao, “Cmts: A conditional multiple trajectory synthesizer for generating safety-critical driving scenarios,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) , 2020, pp. 4314–4321

work page 2020
[5]

Generating critical test scenarios for auto- mated vehicles with evolutionary algorithms,

M. Klischat and M. Althoff, “Generating critical test scenarios for auto- mated vehicles with evolutionary algorithms,” in 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2019, pp. 2352–2358

work page 2019
[6]

Diversifying latent flows for safety-critical scenarios generation,

D. Gao, Y . Qin, X. Tao, and J. LU, “Diversifying latent flows for safety-critical scenarios generation,” in 2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC) , 2024, pp. 2665–2670

work page 2024
[7]

Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method

W. Ding, B. Chen, M. Xu, and D. Zhao, “Learning to collide: An adaptive safety-critical scenarios generating method,” 2020. [Online]. Available: https://arxiv.org/abs/2003.01197

work page Pith review arXiv 2020

Show all 32 references

[8]

Risk-aware attention-based td3 approach for vehicle decision-making in dynamic traffic,

Y . Niu, C. Hu, and Z. Wang, “Risk-aware attention-based td3 approach for vehicle decision-making in dynamic traffic,” in 2024 IEEE Interna- tional Conference on Unmanned Systems (ICUS) , 2024, pp. 1351–1356

2024
[9]

Risk- aware vehicle trajectory prediction under safety-critical scenarios,

Q. Wang, D. Xu, G. Kuang, C. Lv, S. E. Li, and B. Nie, “Risk- aware vehicle trajectory prediction under safety-critical scenarios,” IEEE Transactions on Intelligent Transportation Systems , pp. 1–16, 2025. (a) (b) (c) (d) (e) (f) (g) (h) Figure 5: Trajectories of red-team vehi...

2025
[10]

Safety risk evaluation based autonomous vehicle decision-making approach for cut- in emergency scenario,

Y . Wang, Y . Cao, B. Sun, T. Gong, J. Xu, J. Lu, and S. Yang, “Safety risk evaluation based autonomous vehicle decision-making approach for cut- in emergency scenario,” in 2024 IEEE 22nd International Conference on Industrial Informatics (INDIN) , 2024, pp. 1–8

2024
[11]

A decision- making strategy for vehicle autonomous braking in emergency via deep reinforcement learning,

Y . Fu, C. Li, F. R. Yu, T. H. Luan, and Y . Zhang, “A decision- making strategy for vehicle autonomous braking in emergency via deep reinforcement learning,” IEEE Transactions on Vehicular Technology , vol. 69, no. 6, pp. 5876–5888, 2020

2020
[12]

Human decision-making in high-risk driving scenarios: A cognitive modeling perspective,

Z. Li, H. Huang, H. Cheng, J. Jiang, X. Li, and A. Zgonnikov, “Human decision-making in high-risk driving scenarios: A cognitive modeling perspective,” in 2024 IEEE International Automated Vehicle Validation Conference (IAVVC), 2024, pp. 1–8

2024
[13]

Decision-making for complex scenario using safe reinforcement learning,

J. Xu, X. Pei, and K. Lv, “Decision-making for complex scenario using safe reinforcement learning,” in 2020 4th CAA International Conference on Vehicular Control and Intelligence (CVCI) , 2020, pp. 1–6

2020
[15]

Markov decision processes,

F. Garcia and E. Rachelson, “Markov decision processes,” Markov Decision Processes in Artificial Intelligence , pp. 1–38, 2013

2013
[17]

Prox- imal policy optimization algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

2017 arXiv
[18]

High- dimensional continuous control using generalized advantage estimation,

J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estimation,” arXiv preprint arXiv:1506.02438 , 2015

2015 arXiv
[19]

Semi-supervised classification with graph convolutional networks,

T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” 2017. [Online]. Available: https: //arxiv.org/abs/1609.02907

2017 arXiv
[20]

Benchmarking safe exploration in deep reinforcement learning,

A. Ray, J. Achiam, and D. Amodei, “Benchmarking safe exploration in deep reinforcement learning,” arXiv preprint arXiv:1910.01708 , vol. 7, no. 1, p. 2, 2019

1910 arXiv
[21]

Microscopic traffic simulation using sumo,

P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y .-P. Fl ¨otter¨od, R. Hilbrich, L. L ¨ucken, J. Rummel, P. Wagner, and E. Wießner, “Microscopic traffic simulation using sumo,” in 2018 21st international conference on intelligent transportation systems (ITSC) . Ieee, 20...

2018
[22]

Dueling network architectures for deep reinforcement learning,

Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in International conference on machine learning. PMLR, 2016, pp. 1995– 2003

2016
[23]

Deep sparse rectifier neural networks,

X. Glorot, A. Bordes, and Y . Bengio, “Deep sparse rectifier neural networks,” Journal of Machine Learning Research , vol. 15, pp. 315– 323, 2011

2011
[24]

Mv-stghat: Multi-view spatial-temporal graph hybrid attention network for decision- making of connected and autonomous vehicles,

Q. Liu, Y . Tang, X. Li, F. Yang, K. Wang, and Z. Li, “Mv-stghat: Multi-view spatial-temporal graph hybrid attention network for decision- making of connected and autonomous vehicles,” IEEE Transactions on Vehicular Technology, pp. 1–16, 2024

2024
[25]

Multi-vehicles decision-making in interactive highway exit: A graph reinforcement learning approach,

X. Gao, T. Luan, X. Li, Q. Liu, Z. Li, and F. Yang, “Multi-vehicles decision-making in interactive highway exit: A graph reinforcement learning approach,” in 2022 IEEE 17th Conference on Industrial Elec- tronics and Applications (ICIEA) . IEEE, 2022, pp. 534–539

2022
[26]

Rate gqn: A deviations-reduced decision-making strategy for connected and auto- mated vehicles in mixed autonomy,

X. Gao, X. Li, Q. Liu, Z. Ma, T. Luan, F. Yang, and Z. Li, “Rate gqn: A deviations-reduced decision-making strategy for connected and auto- mated vehicles in mixed autonomy,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 1, pp. 613–625, 2023

2023
[27]

Ethical alignment decision making for connected autonomous vehicle in traffic dilemmas via reinforcement learning from human feedback,

X. Gao, T. Luan, X. Li, Q. Liu, Z. Ma, X. Meng, and Z. Li, “Ethical alignment decision making for connected autonomous vehicle in traffic dilemmas via reinforcement learning from human feedback,” IEEE Internet of Things Journal , vol. 11, no. 23, pp. 38 585–38 600, 2024

2024
[28]

A human feedback- driven decision-making method based on multi-modal deep reinforce- ment learning in ethical dilemma traffic scenarios,

X. Gao, T. Luan, X. Li, Q. Liu, X. Meng, and Z. Li, “A human feedback- driven decision-making method based on multi-modal deep reinforce- ment learning in ethical dilemma traffic scenarios,” in 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC...

2023
[29]

Adversarial evaluation of autonomous vehicles in lane-change scenarios,

B. Chen, X. Chen, Q. Wu, and L. Li, “Adversarial evaluation of autonomous vehicles in lane-change scenarios,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 8, pp. 10 333–10 342, 2022

2022
[30]

Multilevel graph reinforce- ment learning for consistent cognitive decision-making in heterogeneous mixed autonomy,

X. Gao, Z. Ma, X. Li, X. Meng, and Z. Li, “Multilevel graph reinforce- ment learning for consistent cognitive decision-making in heterogeneous mixed autonomy,” arXiv preprint arXiv:2408.08516 , 2024

2024 arXiv
[31]

Efficient genera- tion of safety-critical scenarios combining dynamic and static scenario parameters,

Z. Wang, X. Li, D. Wei, L. Wang, and Y . Huang, “Efficient genera- tion of safety-critical scenarios combining dynamic and static scenario parameters,” IEEE Transactions on Intelligent Vehicles, pp. 1–16, 2024

2024
[32]

Defining time-to-collision thresholds by the type of lead vehicle in non-lane-based traffic environments,

S. Das and A. K. Maurya, “Defining time-to-collision thresholds by the type of lead vehicle in non-lane-based traffic environments,” IEEE Transactions on Intelligent Transportation Systems , vol. 21, no. 12, pp. 4972–4982, 2020

2020
[33]

Characterizing warfare in red teaming,

A. Yang, H. Abbass, and R. Sarker, “Characterizing warfare in red teaming,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 36, no. 2, pp. 268–285, 2006

2006

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [14]

Dynamic Residual Safe Reinforcement Learning for Multi-Agent Safety-Critical Scenarios Decision-Making

K. Wang, Y . Chen, Q. Liu, X. Li, and X. Gao, “Dynamic residual safe reinforcement learning for multi-agent safety-critical scenarios decision- making,” arXiv preprint arXiv:2504.06670 , 2025

work page Pith review arXiv 2025

[2] [1]

Edge computing for autonomous driving: Opportunities and challenges,

S. Liu, L. Liu, J. Tang, B. Yu, Y . Wang, and W. Shi, “Edge computing for autonomous driving: Opportunities and challenges,” Proceedings of the IEEE, vol. 107, no. 8, pp. 1697–1716, 2019

2019

[3] [2]

A survey on safety-critical driving scenario generation—a methodological perspec- tive,

W. Ding, C. Xu, M. Arief, H. Lin, B. Li, and D. Zhao, “A survey on safety-critical driving scenario generation—a methodological perspec- tive,” IEEE Transactions on Intelligent Transportation Systems , vol. 24, no. 7, pp. 6971–6988, 2023

2023

[4] [3]

Intelligent driving in- telligence test for autonomous vehicles with naturalistic and adversarial environment,

S. Feng, X. Yan, H. Sun, Y . Feng, and H. X. Liu, “Intelligent driving in- telligence test for autonomous vehicles with naturalistic and adversarial environment,” Nature communications, vol. 12, no. 1, p. 748, 2021

work page 2021

[5] [4]

Cmts: A conditional multiple trajectory synthesizer for generating safety-critical driving scenarios,

W. Ding, M. Xu, and D. Zhao, “Cmts: A conditional multiple trajectory synthesizer for generating safety-critical driving scenarios,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) , 2020, pp. 4314–4321

work page 2020

[6] [5]

Generating critical test scenarios for auto- mated vehicles with evolutionary algorithms,

M. Klischat and M. Althoff, “Generating critical test scenarios for auto- mated vehicles with evolutionary algorithms,” in 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2019, pp. 2352–2358

work page 2019

[7] [6]

Diversifying latent flows for safety-critical scenarios generation,

D. Gao, Y . Qin, X. Tao, and J. LU, “Diversifying latent flows for safety-critical scenarios generation,” in 2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC) , 2024, pp. 2665–2670

work page 2024

[8] [7]

Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method

W. Ding, B. Chen, M. Xu, and D. Zhao, “Learning to collide: An adaptive safety-critical scenarios generating method,” 2020. [Online]. Available: https://arxiv.org/abs/2003.01197

work page Pith review arXiv 2020