Pith. sign in

REVIEW 5 major objections 7 minor 31 references

DRARL: Disengagement-Reason-Augmented Reinforcement Learning for Efficient Improvement of Autonomous Driving Policy

T0 review · 5 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DRARL claims that a self-driving policy can learn from a single driver-disengagement event by finding the out-of-distribution behavior that caused it and retraining in a reason-preserving simulation.

desk verdict A plausible compositional method for learning from disengagements, with an unvalidated reason-detection claim and evaluation gaps that need fixing before the central claim is accepted. read the letter →

arxiv 2506.16720 v1 pith:5TYTWKJV submitted 2025-06-20 cs.RO cs.LG

Weitao Zhou , Bo Zhang , Zhong Cao , Xiang Li , Qian Cheng , Chunyang Liu , Yaqin Zhang , Diange Yang This is my paper · ORCID
classification cs.ROcs.LG
keywords disengagementreasoningout-of-distributiondetectionautonomousdrivingpolicyreinforcementlearningimaginationenvironmentconditionalvariationalautoencoderkerneldensityestimationrobotaxicases
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DRARL claims that autonomous driving policies can improve themselves from rare driver-disengagement events without expert annotation: the system finds when and why the policy failed by detecting out-of-distribution behavior of surrounding objects, then retrains the policy in a reason-augmented imagination environment that replays the recorded scenario while letting the relevant object react. The point is to turn single disengagement records, which are too scarce to train on directly and are sometimes not policy failures at all, into an efficient training signal that generalizes to similar situations. The paper reports that the method identifies casual and non-policy-failure interventions correctly most of the time, and that after training on a small subset of cases the improved policy reaches 0% collisions and an 82% pass rate in one scenario, versus 45% collisions for direct log-replay training.

What carries the argument

The load-bearing object is the disengagement reason $R=\{q_i^{b:d}\}$: the kinematic state sequence of object $i$ from the first out-of-distribution frame $b$ to the disengagement frame $d$. A conditional variational autoencoder trained on the agent's replay buffer generates the distribution of expected future locations, kernel density estimation measures how unlikely the recorded arrival is, and the first frame below the threshold $e_o$ starts the reason. The reason then controls the imagination environment's transition model $V_i$: before $b$ the object follows the recorded log, and after $b$ it follows an interactive model with randomness, so each training episode either reproduces the reason or tests the policy with varied reactions after it.

What would settle it

Collect a labeled set of real disengagement events where the true cause is independently known, run the OOD reason detector, and check whether the detected window contains the true cause; if many true causes are not visible as out-of-distribution dynamic-object trajectories, the training signal is systematically mis-targeted.

Watch

Extended reading notes

Core claim

The central discovery is that the reason for a disengagement can be represented compactly as a set of surrounding-object state sequences that were out-of-distribution relative to the RL agent's historical experience, with the start of the reason set at the first out-of-distribution state and the end set at the moment of driver takeover. Once that reason is known, training in an environment whose transition model switches from log-replay to an interactive driving model at the reason boundary lets the policy learn to handle the failure mode and semantically similar variants, while avoiding both overfitting to one trajectory and becoming overly cautious. The paper's results show that a random choice of reason boundary degrades performance, manually fixed boundaries are sensitive in their effect, and the automatically detected boundary outperforms both.

Load-bearing premise

The method assumes every real policy-failure disengagement has a cause visible as one or more surrounding objects briefly behaving in ways the RL agent's historical data would predict as out-of-distribution; if the cause is a perception failure, a map error, a planning error, or behavior that began before the recorded window, the detected reason is mis-targeted.

Editorial extensions

If this is right

  • If the reason detector is correct, a single disengagement record can support productive policy training, reducing reliance on large accident datasets.
  • Policies improved this way generalize to semantically similar cases rather than only the exact replay, because the interactive post-reason model generates variants.
  • Because casual and non-policy-failure disengagements are filtered out before training, the policy avoids unnecessary, overly conservative updates from irrelevant interventions.
  • The driving side of the method is largely policy-agnostic apart from needing an accessible replay buffer, so it could attach to different reinforcement-learning planners.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same reason-as-out-of-distribution-window idea could apply to any human-supervised autonomous system with a logged replay buffer and takeover timestamps, such as teleoperated robots.
  • A testable extension is to ablate the proximity filter that raised casual-disengagement detection from 92.5% to 99.7%; if accuracy drops without it, much of the filtering is carried by the spatial prior rather than the OOD model.
  • Another extension is to validate detected reason windows against ground-truth causes in synthetic disengagements with known injected faults, which would isolate how precisely the CVAE-based detector localizes the true cause.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes DRARL, a framework for improving an RL driving policy from real-world disengagement events. Disengagement cases are first screened by an out-of-distribution (OOD) state detector: a CVAE-based motion predictor trained on the agent's replay buffer is used to detect dynamic objects whose trajectories deviate from historical distributions, and the resulting object set R and start time b define the disengagement reason. Cases without such a reason are deemed casual or non-policy-failure and discarded. For policy-relevant cases, a reason-augmented imagination environment switches surrounding-object transition models from log replay before b to interactive models after b, and the policy is retrained. Evaluation uses reconstructed real-world disengagement scenarios (pedestrian/cyclist crossing, cut-in, casual, non-policy-failure) in CARLA. The paper reports 92.5%/99.7% casual detection and 94.7% non-policy-failure detection, and improved policy performance over SAC log-replay, random-reason, and fixed-reason baselines.

Significance. If the support were adequate, DRARL would be practically significant: it is a scalable, annotation-free way to turn rare disengagement logs into targeted training environments, and the reason-augmented environment design is a sensible response to the scarcity and single-instance nature of disengagement data. The paper also has methodological strengths: the use of a nondeterministic CVAE predictor to define OOD states, the explicit separation of log-replay and interactive phases at the reason boundary, and the inclusion of random-reason and fixed-reason controls that isolate the value of b. However, the current evidence does not establish the central reason-identification claim, and the policy evaluation has a train/test contamination issue. These are fixable with additional experiments, but they are load-bearing for the paper's conclusions.

major comments (5)
  1. [Section VI.C, Table I; Section IV.A.1] The identification metric in Table I does not validate the central claim that DRARL 'accurately identifies policy-related disengagement reasons.' For casual disengagement, the label is assigned by the method itself: a case is casual exactly when no OOD state is found (Sec. IV.A.1), so the 92.5% rate measures agreement with the detector's own definition, not agreement with any ground-truth cause. For non-policy-failure cases, the garbage-obstacle scenario tests only one type of non-policy cause (perception/planner obstacle avoidance), so the 94.7% rate cannot support the general claim. No ground-truth labels for the responsible object set R or the start timestamp b are used to measure precision/recall or timestamp error, although R and b completely determine the training environment in Eq. (7).
  2. [Section VI.C, 20 m filter] The 'reasonable assumption—that only nearby objects within 20 meters can cause disengagement' is applied post hoc and lifts the casual-detection rate from 92.5% to 99.7%. This threshold is a free parameter; the paper provides no independent evidence that disengagement reasons cannot involve objects farther than 20 m, and using a filter tuned to improve this self-defined metric risks circularity. The comparison should report results with and without the filter and justify the distance threshold from data or an external criterion.
  3. [Section VI.A.2 and VI.D] The evaluation of policy improvement is contaminated: the paper states that less than 5% of cases are randomly selected for training and 'the trained policy will be tested in all cases.' If the test set includes the training cases, Table II and Fig. 6 overstate generalization because the policy may have memorized the replayed trajectories of those exact cases. The evaluation should use a strictly held-out split of the original cases, or should explicitly report training-case-excluded metrics. The text should also specify whether the test cases are original recordings, synthetic variants, or both.
  4. [Section VI.B.2 and Introduction] The natural comparator for DRARL is the authors' prior one-shot disengagement continual-learning method [5], which is described as the direct predecessor in the introduction but is never evaluated as a baseline. Without this comparison, the experiments cannot distinguish the benefit of explicit reason identification from the benefit of any data-augmented retraining scheme, and the claim that DRARL is 'better' than [5] is unsubstantiated.
  5. [Section IV.A.1 and IV.B.1] The entire pipeline assumes that every policy-failure disengagement is caused by a period of OOD behavior of one or more dynamic surrounding objects, as formalized in Eq. (1). The paper acknowledges other causes (perception failures, map errors, planning failures) in the introduction and in the non-policy-failure experiment, but it never tests how the OOD detector behaves when the true cause is not a dynamic-object anomaly, e.g., an ego-vehicle planning error or an issue beginning before the recorded window. A validation set with ground-truth causes is needed to bound the method's applicability.
minor comments (7)
  1. [Throughout] The term 'casual disengagement' is unusual and potentially confusing; consider 'non-policy-triggered disengagement' or 'driver-initiated disengagement without policy failure' to avoid ambiguity.
  2. [Section VI.A.2] The phrase 'less than 5%' should be replaced with the exact number of training cases and the selection procedure (random seed, filtering criteria) for reproducibility.
  3. [Section VI.B.2] The collision rate metric is introduced by saying it is calculated 'in a similar way' to the passing rate, but the formula should be stated explicitly.
  4. [Section VI.D] There is a typo: 'the reason of disengagement's impart' should be 'the reason of disengagement's impact.'
  5. [Abstract and Section I] Minor grammar: 'a out-of-distribution' should be 'an out-of-distribution'; similar grammatical slips appear elsewhere in Sections I and IV.
  6. [Reference [5]] The bibliographic entry for [5] is incomplete; it should include volume, article number, or page range as done for other references.
  7. [Eq. (5)] There is a typo in the definition of q_i: the last component should be \dot{y}_i, matching the definition of q_e.

Circularity Check

2 steps flagged · score 6.0 of 10

Reason-identification validation is self-definitional: the casual/non-policy-failure identification rates reduce to the detector's own OOD definition, while the policy-improvement comparison retains independent content.

  1. self definitional [Section IV.A.2, Section IV.C, Section VI.A.1 (Casual Disengagement Cases), Table I]
    "Based on these observations, this work defines the occurrence of an OOD state as the start of a disengagement reason. ... If the method doesn't find any out-of-distribution state, this case is classified as the casual disengagement. ... this work samples 2400 frames from normal driving data and assumes that every frame is a casual disengagement case."

    The negative class is defined by the detector's own output: a case is 'casual' exactly when no OOD state is found. The evaluation then assumes normal driving frames are casual, and because those frames come from the same historical distribution used to train the CVAE predictor, they are in-distribution by construction. The reported 92.5% 'identifying rate' therefore only checks that the detector labels in-distribution inputs as in-distribution; it does not test whether real-world casual disengagements lack policy-failure causes. The non-policy-failure experiment is structurally identical: a static garbage obstacle produces no OOD surrounding-object state, so it is 'without reason pertaining to the policy' by definition.

  2. fitted input called prediction [Section VI.C, Table I]
    "By introducing a reasonable assumption—that only nearby objects within 20 meters can cause disengagement—the number of mis-recognized disengagement frames reduces to 7 increasing ε to 99.7%."

    The 20 m cutoff is introduced after the initial 92.5% result and applied to the same 2400 normal-driving frames used to define the casual-disengagement metric. The improved 99.7% is then reported as a 'successful identification rate,' but it is a threshold-tuned number rather than a prediction on held-out ground truth. Since the label 'casual' is itself defined as 'no OOD state found,' the cutoff only makes the detector agree better with its own definition; it supplies no evidence that the identified reasons correspond to true causes of real disengagements.

full rationale

The policy-improvement part of DRARL is not circular: the reason-augmented imagination environment (Eq. 7), the SAC/Randomly-Reason/Fixed-Reason baselines, and the reported collision/pass rates constitute an independent empirical comparison, and the self-citation to the authors' prior one-shot continual-learning work [5] is descriptive rather than load-bearing evidence. However, the paper's headline reason-identification claim is partially circular. A 'casual disengagement' is defined as a case with no OOD state, and the evaluation assumes normal driving frames are casual, so Table I's 92.5%/99.7% rates reduce to the detector's own definition plus a fitted 20 m threshold. The qualitative positive examples in Fig. 5 do not provide precision/recall for the responsible-object set R or the start timestamp b. Thus the central reason-identification validation is self-definitional, while the downstream policy gains remain independent; overall, this is partial circularity rather than a fully forced derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on two linked assumptions: (1) the cause of a disengagement is always detectable as out-of-distribution behavior of surrounding dynamic objects using a CVAE trained on the agent's replay buffer, and (2) training in a hybrid replay/interactive simulation built from the detected reason transfers to real driving. Both are domain assumptions that the evaluation does not independently validate: policy-failure reason detection is shown only qualitatively, and casual/non-policy-failure rates are measured against the method's own definition plus a post hoc 20 m filter. The reproduced evaluation in CARLA uses the same interactive behavior model as original training, so the strongest absolute numbers (near-zero collision rates) are best interpreted as relative comparisons against the paper's baselines.

free parameters (4)
  • OOD probability threshold e_o = 0.01%
    States with estimated probability below e_o are labeled out-of-distribution; no sensitivity analysis is provided, and the value directly determines which objects are flagged as the reason. Section IV.B.2 and VI.C.
  • prediction horizon gamma = 2 frames (1.0 s at 0.5 s/frame)
    Chosen to simulate human reaction time (Section VI.C); manually set and affects when OOD states are detected.
  • nearby-object relevance radius = 20 m
    Introduced post hoc after the initial casual-detection experiment, raising the identification rate from 92.5% to 99.7%; not part of the method in Section IV and fitted to the evaluation set (Section VI.C).
  • training-case selection share = <5%
    Randomly selecting under 5% of cases for training; exact share not fixed, and because testing uses all cases, training cases are included in the test set (Section VI.A.2 and VI.D).
assumptions (5)
  • domain assumption Disengagement reasons are always caused by out-of-distribution behavior of one or more dynamic surrounding objects (definition R={q^{b:d}_i}, Section IV.A.1).
    This converts the problem of finding a disengagement cause into finding OOD object trajectories, excluding causes such as perception failure, map error, ego-vehicle planning errors, or road geometry issues.
  • domain assumption The start of a reason is the first OOD state and the end is the driver disengagement time (Section IV.A.2-3).
    The method traces back from disengagement time to find the first OOD state for each object; if the actual failure began earlier outside the recorded window, or the disengagement time is not the end of the relevant cause, the extracted interval is wrong.
  • domain assumption The CVAE trajectory predictor trained on the agent's historical replay buffer D accurately estimates the distribution of expected behavior at run time (Section IV.B.1, Eq. 2).
    OOD detection relies entirely on this model: if the historical data already contains aggressive/abnormal behaviors, true failure-causing behaviors will not be flagged; if the historical data lacks diversity, normal driving will be flagged spuriously.
  • ad hoc to paper Only surrounding objects within 20 meters can cause disengagement (Section VI.C).
    Added after the fact to reduce false positives in casual-disengagement identification; this filter is not derived from a safety principle and is not used in the main method description.
  • domain assumption The interactive transition model f used in the imagination environment captures realistic reactions of surrounding objects (Section V.B, Eq. 9).
    Policy improvement is only as useful as this model; if f differs from real behaviors, the trained policy may not transfer. The paper reuses the models from the original policy training, which risks training the policy to a self-consistent simulation rather than to real-world reasoning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DRARL: Disengagement-Reason-Augmented Reinforcement Learning for Efficient Improvement of Autonomous Driving Policy." pith.science (2026). https://pith.science/paper/5TYTWKJV

@misc{pith2026250616720,
  author       = {Pith},
  title        = {Pith review of: DRARL: Disengagement-Reason-Augmented Reinforcement Learning for Efficient Improvement of Autonomous Driving Policy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5TYTWKJV}},
  note         = {Machine review of arXiv:2506.16720}
}
read the original abstract

With the increasing presence of automated vehicles on open roads under driver supervision, disengagement cases are becoming more prevalent. While some data-driven planning systems attempt to directly utilize these disengagement cases for policy improvement, the inherent scarcity of disengagement data (often occurring as a single instances) restricts training effectiveness. Furthermore, some disengagement data should be excluded since the disengagement may not always come from the failure of driving policies, e.g. the driver may casually intervene for a while. To this end, this work proposes disengagement-reason-augmented reinforcement learning (DRARL), which enhances driving policy improvement process according to the reason of disengagement cases. Specifically, the reason of disengagement is identified by a out-of-distribution (OOD) state estimation model. When the reason doesn't exist, the case will be identified as a casual disengagement case, which doesn't require additional policy adjustment. Otherwise, the policy can be updated under a reason-augmented imagination environment, improving the policy performance of disengagement cases with similar reasons. The method is evaluated using real-world disengagement cases collected by autonomous driving robotaxi. Experimental results demonstrate that the method accurately identifies policy-related disengagement reasons, allowing the agent to handle both original and semantically similar cases through reason-augmented training. Furthermore, the approach prevents the agent from becoming overly conservative after policy adjustments. Overall, this work provides an efficient way to improve driving policy performance with disengagement cases.

Figures

Figures reproduced from arXiv: 2506.16720 by the authors.

Figure 1
Figure 1. The autonomous vehicle is intervened by driver be [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework of disengagement reasoning for policy [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The disengagement case reasoning method. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Collected Disengagement Cases for Evaluation. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Examples of Disengagement Reason Detection. The [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The average reward and pass rate of the fixed reason [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 17 canonical work pages

  1. [5]

    Autonomous driving policy continual learning with one-shot disen- gagement case,

    Z. Cao, X. Li, K. Jiang, W. Zhou, X. Liu, N. Deng, and D. Yang, “Autonomous driving policy continual learning with one-shot disen- gagement case,”IEEE Transactions on Intelligent Vehicles, 2022

    work page 2022

  2. [1]

    A crash injury model involving autonomous vehicle: Investigating of crash and disengagement reports,

    A. Sinha, V . Vu, S. Chand, K. Wijayaratna, and V . Dixit, “A crash injury model involving autonomous vehicle: Investigating of crash and disengagement reports,”Sustainability, vol. 13, no. 14, p. 7938, 2021

    work page 2021

  3. [2]

    Anti-jerk on-ramp merging using deep reinforcement learning,

    Y . Lin, J. McPhee, and N. L. Azad, “Anti-jerk on-ramp merging using deep reinforcement learning,” in2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020, pp. 7–14

    work page 2020

  4. [3]

    Identify, estimate and bound the uncertainty of reinforcement learning for autonomous driving,

    W. Zhou, Z. Cao, N. Deng, K. Jiang, and D. Yang, “Identify, estimate and bound the uncertainty of reinforcement learning for autonomous driving,”IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 8, pp. 7932–7942, 2023

    work page 2023

  5. [4]

    Generalizing from a few examples: A survey on few-shot learning,

    Y . Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a few examples: A survey on few-shot learning,”ACM computing surveys (csur), vol. 53, no. 3, pp. 1–34, 2020

    2020

  6. [6]

    Who make drivers stop? towards driver-centric risk assessment: Risk object identification via causal inference,

    C. Li, S. H. Chan, and Y .-T. Chen, “Who make drivers stop? towards driver-centric risk assessment: Risk object identification via causal inference,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 10 711–10 718

    2020

  7. [7]

    Goal-oriented object importance estimation in on-road driving videos,

    M. Gao, A. Tawari, and S. Martin, “Goal-oriented object importance estimation in on-road driving videos,” in2019 International Confer- ence on Robotics and Automation (ICRA). IEEE, 2019, pp. 5509– 5515

    work page 2019

  8. [8]

    Agent-centric risk assessment: Accident anticipation and risky region localization,

    K.-H. Zeng, S.-H. Chou, F.-H. Chan, J. Carlos Niebles, and M. Sun, “Agent-centric risk assessment: Accident anticipation and risky region localization,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2222–2230

    2017

Show all 31 references
  1. [9]

    Uncertainty- aware reinforcement learning for collision avoidance,

    G. Kahn, A. Villaflor, V . Pong, P. Abbeel, and S. Levine, “Uncertainty- aware reinforcement learning for collision avoidance,”arXiv preprint arXiv:1702.01182, 2017

    2017 arXiv

  2. [10]

    Confidence-aware reinforcement learning for self-driving cars,

    Z. Cao, S. Xu, H. Peng, D. Yang, and R. Zidek, “Confidence-aware reinforcement learning for self-driving cars,”IEEE Transactions on Intelligent Transportation Systems, 2021

    2021

  3. [11]

    Uncertainty-aware model-based re- inforcement learning: Methodology and application in autonomous driving,

    J. Wu, Z. Huang, and C. Lv, “Uncertainty-aware model-based re- inforcement learning: Methodology and application in autonomous driving,”IEEE Transactions on Intelligent Vehicles, 2022

    2022

  4. [12]

    Learning task-relevant representations for generalization via characteristic func- tions of reward sequence distributions,

    R. Yang, J. Wang, Z. Geng, M. Ye, S. Ji, B. Li, and F. Wu, “Learning task-relevant representations for generalization via characteristic func- tions of reward sequence distributions,” inThe 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 2022, pp. 2242–2252

    2022

  5. [13]

    Uncertainty-based offline variational bayesian reinforcement learning for robustness under di- verse data corruptions,

    R. Yang, J. Wang, G. Wu, and B. Li, “Uncertainty-based offline variational bayesian reinforcement learning for robustness under di- verse data corruptions,” inAdvances in Neural Information Processing Systems 38, 2024

    2024

  6. [14]

    Analysis of accident data for test scenario definition in the assess project,

    H. Fagerlind, I. Heinig, M. Vistr ¨om, M. Wisch, L. Sulzberger, M. McCarthy, W. Hulshof, M. Roynard, and S. Schaub, “Analysis of accident data for test scenario definition in the assess project,”Berichte Der Bundesanstalt Fuer Strassenwesen. Unterreihe Fahrzeugtechnik, no. 77, 2010

    2010

  7. [15]

    A comprehensive self-driving car test,

    V . G. Cerf, “A comprehensive self-driving car test,”Communications of the ACM, vol. 61, no. 2, pp. 7–7, 2018

    2018

  8. [16]

    Congested traffic states in empirical observations and microscopic simulations,

    M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,”Physical review E, vol. 62, no. 2, p. 1805, 2000

    2000

  9. [17]

    General lane-changing model mobil for car-following models,

    A. Kesting, M. Treiber, and D. Helbing, “General lane-changing model mobil for car-following models,”Transportation Research Record, vol. 1999, no. 1, pp. 86–94, 2007

    1999

  10. [18]

    Trafficsim: Learning to simulate realistic multi-agent behaviors,

    S. Suo, S. Regalado, S. Casas, and R. Urtasun, “Trafficsim: Learning to simulate realistic multi-agent behaviors,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 10 400–10 409

    2021

  11. [19]

    Simnet: Learning reactive self-driving simulations from real-world observations,

    L. Bergamini, Y . Ye, O. Scheel, L. Chen, C. Hu, L. Del Pero, B. Osi´nski, H. Grimmett, and P. Ondruska, “Simnet: Learning reactive self-driving simulations from real-world observations,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 5119–5125

    2021

  12. [20]

    Overview on deepmind and its alphago zero ai,

    S. D. Holcomb, W. K. Porter, S. V . Ault, G. Mao, and J. Wang, “Overview on deepmind and its alphago zero ai,” inProceedings of the 2018 international conference on big data and education, 2018, pp. 67–71

    2018

  13. [21]

    Generalization in visual reinforcement learning with the reward sequence distribution,

    J. Wang, R. Yang, Z. Geng, Z. Shi, M. Ye, Q. Zhou, S. Ji, B. Li, Y . Zhang, and F. Wu, “Generalization in visual reinforcement learning with the reward sequence distribution,”CoRR, vol. abs/2302.09601, 2023

    2023 arXiv

  14. [22]

    Quan- tifying generalization in reinforcement learning,

    K. Cobbe, O. Klimov, C. Hesse, T. Kim, and J. Schulman, “Quan- tifying generalization in reinforcement learning,” inInternational Conference on Machine Learning. PMLR, 2019, pp. 1282–1289

    2019

  15. [23]

    Tra- jectron++: Dynamically-feasible trajectory forecasting with heteroge- neous data,

    T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone, “Tra- jectron++: Dynamically-feasible trajectory forecasting with heteroge- neous data,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 683–700

    2020

  16. [24]

    On estimation of a probability density function and mode,

    E. Parzen, “On estimation of a probability density function and mode,” The annals of mathematical statistics, vol. 33, no. 3, pp. 1065–1076, 1962

    1962

  17. [25]

    M. J. Kochenderfer,Decision making under uncertainty: theory and application. MIT press, 2015

    2015

  18. [26]

    Lane change and merge maneuvers for connected and automated vehicles: A survey,

    D. Bevly, X. Cao, M. Gordon, G. Ozbilgin, D. Kari, B. Nelson, J. Woodruff, M. Barth, C. Murray, A. Kurt,et al., “Lane change and merge maneuvers for connected and automated vehicles: A survey,” IEEE Transactions on Intelligent Vehicles, vol. 1, no. 1, pp. 105–120, 2016

    2016

  19. [27]

    A comprehensive review of the development of adaptive cruise control systems,

    L. Xiao and F. Gao, “A comprehensive review of the development of adaptive cruise control systems,”Vehicle system dynamics, vol. 48, no. 10, pp. 1167–1192, 2010

    2010

  20. [28]

    Application of microscopic pedestrian simulation model,

    K. Teknomo, “Application of microscopic pedestrian simulation model,”Transportation Research Part F: Traffic Psychology and Behaviour, vol. 9, no. 1, pp. 15–27, 2006

    2006

  21. [29]

    Layered costmaps for context-sensitive navigation,

    D. V . Lu, D. Hershberger, and W. D. Smart, “Layered costmaps for context-sensitive navigation,” in2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2014, pp. 709– 715

    2014

  22. [30]

    Carla: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” inConference on robot learning. PMLR, 2017, pp. 1–16

    2017

  23. [31]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational conference on machine learning. PMLR, 2018, pp. 1861–1870

    2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.