Pith. sign in

REVIEW 4 major objections 49 references

Causal training teaches RL driving policies to cooperate with rule-based recovery so they stop spending most of an episode immobilized.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 04:19 UTC pith:4B23YPNT

load-bearing objection Solid hybrid-recovery engineering paper with a clean ablation idea, but the central causal claim is confounded by ordinary recovery-aware fine-tuning. the 4 major comments →

arxiv 2607.03177 v1 pith:4B23YPNT submitted 2026-07-03 cs.SE cs.AIcs.LG

CRRL: A Causality-Based Reinforcement Learning Framework for Autonomous System Recovery

Safia Fatima , Kai Olav Ellefsen , Leon Moonen This is my paper
classification cs.SE cs.AIcs.LG
keywords causal-guided RLrule-based recoveryautonomous drivingself-healing systemsPPOMAPE-KBayesian networkreward shaping
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard reinforcement-learning drivers in complex traffic often freeze for long stretches of an episode; bolting a reverse-recovery rule onto a pretrained policy can make rewards worse because the policy was never taught to work with unplanned interventions. CRRL fixes the coordination problem, not by inventing a new recovery gadget, but by training the policy under a causal signal. A Bayesian network built from multi-vehicle driving logs supplies predicted collision risk that reshapes the reward whenever recovery is active, so the agent learns to anticipate stalls and resume cleanly after a reverse maneuver. A four-condition ablation across straight road, roundabout, and T-junction shows that the causal training itself—not the recovery module alone—is what lifts reward, distance, and speed, including in many episodes that never trigger recovery. The result matters for any autonomous system that must keep a learned controller and a safety rulebook working as a team rather than fighting each other.

Core claim

The issue is not missing recovery mechanisms but a lack of policies trained to collaborate with them. When the recovery module is held fixed, a PPO policy trained with causal reward shaping from a Bayesian network of driving logs significantly improves reward, distance, and velocity and reduces stalled time relative to the same recovery paired with a pretrained policy; causal-guided training is the primary source of the gain.

What carries the argument

CRRL: a hybrid MAPE-K pipeline in which a Bayesian-network causal model, fit on discretized multi-vehicle driving logs, shapes the PPO reward during recovery (and optionally ranks reverse actions by predicted collision risk), teaching the learned forward policy to cooperate with a rule-based stall-recovery module.

Load-bearing premise

The claim depends on an expert-structured Bayesian network whose probabilities, fit on discretized simulator logs, being accurate enough to guide both training rewards and recovery choices.

What would settle it

Re-run the key C-versus-B ablation after replacing the causal bonus with a random or constant bonus of the same magnitude; if the gaps in reward, distance, and stalled time vanish, the causal structure is not doing the work claimed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Policies trained under causal reward shaping can finish many complex episodes with zero recovery interventions while still outperforming vanilla PPO.
  • Adding heuristic recovery to a policy never exposed to it can degrade reward; cooperation must be learned during training.
  • Once the policy is causally trained, runtime causal risk ranking yields further efficiency: fewer recovery attempts and shorter recovery durations.
  • Gains appear across straight, roundabout, and T-junction geometries, with the largest effects in the hardest layout.
  • Hybrid learned-plus-rule recovery becomes useful only when the learned component is trained under the recovery signal.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same training-time causal shaping pattern could apply to other managed/managing pairs beyond driving, such as microservice repair or robot-arm recovery.
  • Learning or updating the causal graph online, rather than fitting it once offline, might track distribution shift without fixed expert edges.
  • Collision inflation under recovery points toward treating causal failure probability as a hard constraint budget rather than only a soft reward bonus.
  • Zero-recovery competence in roundabouts suggests causal shaping may improve ordinary lane-keeping, not only stall escape.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper proposes CRRL, a hybrid MAPE-K-style framework that couples a Bayesian-network causal model (fit on discretized multi-vehicle CARLA logs) with PPO. Causal estimates of P(collision|action,state) shape the reward during recovery and, in one condition, rank reverse actions. A four-condition ablation (pretrained vs causal-trained policy × none/heuristic/causal recovery) across three Town07 scenarios (straight, roundabout, T-junction; n=20 episodes each) is used to argue that causal-guided training, not the recovery module alone, is the primary source of gains in reward, distance, velocity, and stalled time. Zero-recovery roundabout episodes and D-vs-B comparisons are offered as supporting evidence that the policy learns cooperative and proactive behaviour.

Significance. If the isolation claim holds, the work supplies a concrete, reproducible recipe for training learned controllers to cooperate with engineered safety modules—an important and under-addressed problem in self-adaptive and autonomous systems. The four-condition design, non-parametric tests with Cohen’s d, zero-recovery episode analysis, and promised replication package are genuine strengths. The honest limitations section and the observation that recovery alone can degrade a frozen policy are also valuable. The contribution is therefore of clear interest to the TAAS / self-adaptive and safe-RL communities, provided the causal component can be cleanly separated from ordinary recovery-aware fine-tuning.

major comments (4)
  1. Section 4.1 asserts that C vs B isolates causal training because both share identical heuristic recovery, so differences “can only arise from differences in learned policy weights.” This is incorrect. Condition C fine-tunes PPO for up to 2000 episodes under active recovery, action blending (α=0.8), and causally shaped rewards (Alg. 1, §§4.5.2, 4.6); Condition B freezes the pretrained weights and never experiences recovery. The large T-junction effects (d=0.97–1.86, p<0.001) are therefore confounded with ordinary adaptation to recovery interventions. The paper itself flags the threat in §6 (“Conditions C and D may reflect adaptation to recovery rather than superior driving”) yet still attributes the gains to the Bayesian P(collision|·) (Eqs. 5, 10). A non-causal recovery-aware fine-tuning control (identical recovery exposure and blending, but environmental reward only) is required before
  2. The causal model itself is largely expert-specified (edge constraints in §4.4) and evaluated only by downstream RL metrics; no held-out calibration, interventional check, or comparison against a non-causal risk estimator is reported. If the discretized BN is misspecified, both the dense shaping signal and the risk-ranked selection lose their claimed causal content. At minimum the manuscript should quantify predictive accuracy of P(collision=1|evidence) on held-out logs and show that a non-causal baseline (e.g., frequency tables or a small MLP) does not produce comparable C-vs-B gains.
  3. Table 12 reports 60 Mann–Whitney tests on n=20 episodes per cell with no multiple-comparison correction and high episode variance (Figs. 4–7). The authors note the issue in §6 but still treat the single-star results as supportive. Either apply a family-wise or FDR correction and re-state which claims survive, or pre-register the primary contrast (C vs B on the T-junction) and treat the remainder as exploratory. With the present sample size the mixed/negative straight-road results already weaken the generality claim.
  4. RQ5 and Table 11 show recovery raises collisions by 155–607 % (C vs A). Even the more favourable D-vs-B comparison leaves absolute collision counts of several events per episode—far above any deployable threshold. The paper correctly notes this limitation, yet the abstract and conclusions still speak of “effective RL policies that cooperate with rule-based safety components.” Either constrain recovery inside a CMDP / safety layer that demonstrably bounds collisions, or qualify the contribution strictly as a simulation study of cooperation under high collision cost.

Circularity Check

0 steps flagged

No significant circularity: causal reward shaping and ablation metrics are external to the fitted Bayesian network; C-vs-B is a validity confound, not a by-construction reduction.

full rationale

The paper's central claim (causal-guided PPO training produces policies that cooperate with rule-based recovery, isolated by C vs B) is supported by an empirical four-condition ablation on external performance metrics (episode reward, distance traveled, average velocity, stalled-time fraction, off-road fraction) measured in CARLA evaluation episodes. These metrics are not algebraically or statistically forced by the CausalNex Bayesian network whose CPDs are fit on discretized multi-vehicle logs (Section 4.4) and then used only for dense reward shaping (Eqs. 5, 10) and optional risk-ranked selection. The shaped reward R_shaped = R_base + (1 - P(failure|a,s)) * 0.5 is an input to PPO updates; the reported gains are observed outcomes of the resulting policy, not a re-expression of the fitted P(collision). No uniqueness theorem, self-citation chain, or ansatz is load-bearing for the numerical results. The acknowledged internal-validity threat (Section 6: C/D may reflect adaptation to recovery exposure rather than causal accuracy) is a confounding concern, not circularity: it does not make any claimed prediction equal its inputs by construction. Self-citations are limited to standard MAPE-K, PPO, and CARLA background. Score 1 reflects only the minor, non-load-bearing nature of any residual self-reference; the derivation chain is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 1 invented entities

The central claim rests on a handful of modeling and experimental choices rather than free mathematical parameters. The causal DAG is largely expert-specified, continuous signals are discretized by hand-chosen bins, recovery is triggered by fixed velocity/time thresholds, and action blending uses a fixed 0.8/0.2 ratio. These are domain assumptions and free design parameters that, if altered, could change the measured cooperation benefit.

free parameters (5)
  • action blend ratio α = 0.8
    Fixed at 0.8 causal / 0.2 PPO during recovery (Eq. 4); chosen by authors, not learned.
  • causal reward coefficient = 0.5
    Multiplier 0.5 on (1−P(failure)) bonus (Eq. 5 / Table 2); hand-set.
  • stalled-trigger thresholds = 0.5 km/h / 50 steps / 5 km/h / 30 steps
    v < 0.5 km/h for >50 steps triggers recovery; success when v > 5 km/h; max 30 recovery steps. All fixed design choices that define the failure states the policy is trained to anticipate.
  • velocity-dependent candidate pools
    Hand-crafted reverse/forward action sets for three speed bins (Section 3.4); not learned.
  • discretization bins for CausalNex nodes
    Continuous telemetry (speed, throttle, distance, TTC, …) binned into categorical levels before structure/parameter learning; bin edges are domain-chosen.
axioms (4)
  • domain assumption A Bayesian network whose edges are largely expert-specified and whose CPDs are estimated from discretized multi-vehicle CARLA logs yields a usable approximation to interventional collision risk P(collision|do(action),state).
    Invoked throughout Sections 4.4–4.5 for both reward shaping and risk-ranked selection; no interventional validation of the DAG is reported.
  • domain assumption Rule-based reverse maneuvers with fixed throttle −0.3 are preferable to learned reverse actions because reverse states are rare and safety-critical.
    Stated in Section 3.3 as the motivation for the hybrid split; underpins the entire architecture.
  • domain assumption CARLA Town07 with 30 NPCs and 10 pedestrians is a sufficiently realistic proxy for evaluating recovery cooperation.
    All training and evaluation occur in this single map under fixed weather/spawn conditions (Section 4.3).
  • ad hoc to paper Mann-Whitney U tests on n=20 independent episodes per cell, without multiple-comparison correction, adequately support claims of statistical significance.
    Section 5.6 reports 60 tests; authors themselves note the lack of family-wise correction as a threat.
invented entities (1)
  • CRRL hybrid architecture (causal-shaped PPO + rule-based recovery + dual-use Bayesian network) no independent evidence
    purpose: To train a policy that anticipates and cooperates with external recovery interventions rather than treating them as unanticipated disturbances.
    The composite system is the paper's main technical proposal; its value is demonstrated only inside the reported CARLA ablations.

pith-pipeline@v1.1.0-grok45 · 32139 in / 3462 out tokens · 37192 ms · 2026-07-12T04:19:09.228429+00:00 · methodology

0 comments
read the original abstract

Traditional reinforcement learning (RL) for recovery in autonomous systems lacks causal understanding and generalizes poorly to novel failure scenarios. RL policies often stall in failure states, spending up to 70% of an episode immobilized. Rule-based recovery alone is inadequate, and adding heuristic recovery to a pretrained PPO policy worsens rewards because policies cannot coordinate well with unanticipated interventions. The issue is not missing recovery mechanisms but a lack of policies trained to collaborate with them. We introduce CRRL, a causal-guided RL framework that trains policies to work effectively with rule-based recovery. The recovery detects stalled states and assists the agent. Causal relations from driving logs shape the training signal, teaching the policy to anticipate stalls and adjust actions in recovery contexts. The framework follows MAPE-K, with sensor collection, causal model construction, and hybrid RL policy training corresponding to Monitor, Analyze, and Plan/Execute, respectively. We evaluate CRRL through a four-condition ablation study across three driving scenarios, with 20 episodes per condition. We find that causal training significantly improves reward, distance, and velocity. Moreover, 9 of 20 roundabout episodes required zero recovery intervention, confirming navigation competence. These results show that causal-guided training produces effective RL policies that cooperate with rule-based safety components.

Figures

Figures reproduced from arXiv: 2607.03177 by Kai Olav Ellefsen, Leon Moonen, Safia Fatima.

Figure 1
Figure 1. Figure 1: Overview of the proposed causal-guided hybrid RL framework, structured as a [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The three evaluation scenarios. (1) Straight road: two vehicles 41 m apart on opposing lanes. (2) Roundabout: vehicles [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Multi-Agent Causal Graph for Two-Car Interaction Scenarios. Node colours indicate category: action (blue), state (green), [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Training progress over 2000 episodes (50-episode moving [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Reward distributions across all four ablation conditions described in Table [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparative performance across four ablation conditions. Each bar represents the mean value over 20 episodes, with error [PITH_FULL_IMAGE:figures/full_fig_p022_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: C vs B: Effect of causal training (key comparison). Both conditions use identical heuristic recovery modules, differences [PITH_FULL_IMAGE:figures/full_fig_p024_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: D vs C: Effect of causal inference at runtime. Both conditions use causal-trained policy weights, D additionally uses the causal [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: T-junction: all four conditions (strongest scenario). Per-episode boxplots show progressive improvement from A to D. [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 18 canonical work pages

  1. [1]

    FORMS: Unifying Reference Model for Formal Specification of Distributed Self-Adaptive Systems

    D. Weyns, S. Malek, and J. Andersson. “FORMS: Unifying Reference Model for Formal Specification of Distributed Self-Adaptive Systems. ” In: ACM Trans. Auton. Adapt. Syst.7.1 (2012), 8:1–8:61.doi: 10.1145/2168260.2168268

    work page doi:10.1145/2168260.2168268 2012

  2. [2]

    A Survey on Self-Healing Systems: Approaches and Systems

    H. Psaier and S. Dustdar. “A Survey on Self-Healing Systems: Approaches and Systems. ” In:Computing91.1 (2011), pp. 43–73.doi: 10.1007/s00607- 010-0107-y

    work page doi:10.1007/s00607- 2011

  3. [3]

    Multi-Objective Reinforcement Learning for AUV Thruster Failure Recovery

    S. R. Ahmadzadeh, P. Kormushev, and D. G. Caldwell. “Multi-Objective Reinforcement Learning for AUV Thruster Failure Recovery. ” In:IEEE Symp. Adaptive Dynamic Programming and Reinforcement Learning (ADPRL). IEEE, 2014, pp. 1–8.doi: 10.1109/adprl.2014.7010621

    work page doi:10.1109/adprl.2014.7010621 2014

  4. [4]

    Recovery-by-Learning: Restoring Autonomous Cyber-physical Systems from Sensor Attacks

    F. Akowuah et al. “Recovery-by-Learning: Restoring Autonomous Cyber-physical Systems from Sensor Attacks. ” In:IEEE 27th Int’l Conf. Embedded and Real-Time Computing Systems and Applications (RTCSA). 2021, pp. 61–66.doi: 10.1109/rtcsa52859.2021.00015

    work page doi:10.1109/rtcsa52859.2021.00015 2021

  5. [5]

    Causal Inference Q-Network: Toward Resilient Reinforcement Learning

    C.-H. H. Yang et al. “Causal Inference Q-Network: Toward Resilient Reinforcement Learning. ” In:Self-Supervision for Reinforcement Learning Workshop - ICLR 2021. 2021

    2021

  6. [6]

    Gasse et al.Causal Reinforcement Learning Using Observational and Interventional Data

    M. Gasse et al.Causal Reinforcement Learning Using Observational and Interventional Data. 2021. arXiv: 2106.14421

    Pith/arXiv arXiv 2021

  7. [7]

    M. R. Samsami et al.Causal Imitative Model for Autonomous Driving. 2021. arXiv: 2112.03908

    Pith/arXiv arXiv 2021

  8. [8]

    Howard and L

    R. Howard and L. Kunze.Extending Structural Causal Models for Autonomous Vehicles to Simplify Temporal System Construction & Enable Dynamic Interactions Between Agents. 2025. arXiv: 2406.01384

    Pith/arXiv arXiv 2025

  9. [9]

    Fault Prediction and Recovery Using Machine Learning Techniques and the HTM Algorithm in Vehicular Network Environment

    S. Zidi et al. “Fault Prediction and Recovery Using Machine Learning Techniques and the HTM Algorithm in Vehicular Network Environment. ” In: IEEE Open J. Intelligent Transportation Systems5 (2024), pp. 132–145.doi: 10.1109/OJITS.2023.3347484

    work page doi:10.1109/ojits.2023.3347484 2024

  10. [10]

    Self-Healing IoT Sensor Networks with Isolation Forest Algorithm for Autonomous Fault Detection and Recovery

    S. K. Devi, R. Thenmozhi, and D. S. Kumar. “Self-Healing IoT Sensor Networks with Isolation Forest Algorithm for Autonomous Fault Detection and Recovery. ” In:Int’l Conf. Automation and Computation (AUTOCOM). IEEE, 2024, pp. 451–456.doi: 10.1109/AUTOCOM60220.2024.10486184

    work page doi:10.1109/autocom60220.2024.10486184 2024

  11. [11]

    Self-Adaptive Software: Landscape and Research Challenges

    M. Salehie and L. Tahvildari. “Self-Adaptive Software: Landscape and Research Challenges. ” In:ACM Trans. Autonomous and Adaptive Systems (TAAS)4.2 (2009), 14:1–14:42.doi: 10.1145/1516533.1516538

    doi:10.1145/1516533.1516538 2009

  12. [12]

    Recovery RL: Safe Reinforcement Learning With Learned Recovery Zones

    B. Thananjeyan et al. “Recovery RL: Safe Reinforcement Learning With Learned Recovery Zones. ” In:IEEE Robotics and Automation Letters6.3 (2021), pp. 4915–4922.doi: 10.1109/LRA.2021.3070252

    doi:10.1109/lra.2021.3070252 2021

  13. [13]

    Uncertainty in Self-adaptive Systems: A Research Community Perspective

    S. M. Hezavehi et al. “Uncertainty in Self-adaptive Systems: A Research Community Perspective. ” In:ACM Trans. Auton. Adapt. Syst.15.4 (2021), 10:1–10:36.doi: 10.1145/3487921

    work page doi:10.1145/3487921 2021

  14. [14]

    Reinforcement Learning: A Survey

    L. P. Kaelbling, M. L. Littman, and A. W. Moore. “Reinforcement Learning: A Survey. ” In:J. Artificial Intelligence Research4 (1996), pp. 237–285. doi: 10.1613/jair.301

    doi:10.1613/jair.301 1996

  15. [15]

    Causal Inference

    J. Pearl. “Causal Inference. ” In:Proceedings of Ws. Causality: Objectives and Assessment at NIPS 2008. PMLR, 2010, pp. 39–58

    2008

  16. [16]

    Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University Press. 384 Pp., 2000, ISBN 0521773628

    R. P. McDonald. “Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University Press. 384 Pp., 2000, ISBN 0521773628.” In:Psychometrika67.2 (2002), pp. 321–322.doi: 10.1007/BF02294851

    work page doi:10.1007/bf02294851 2000

  17. [17]

    Peters, D

    J. Peters, D. Janzing, and B. Scholkopf.Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press, 2017

    2017

  18. [18]

    Research on Application of Deep Reinforcement Learning in Traffic Signal Control

    S. Zhancheng. “Research on Application of Deep Reinforcement Learning in Traffic Signal Control. ” In:Int’l Conf. Frontiers of Signal Processing (ICFSP). 2021, pp. 17–21.doi: 10.1109/ICFSP53514.2021.9646417. Manuscript submitted to ACM 30 Safia Fatima, Kai Olav Ellefsen, and Leon Moonen

    work page doi:10.1109/icfsp53514.2021.9646417 2021

  19. [19]

    DeepDRAMA: Deep Reinforcement Learning-based Disaster Recovery with Mitigation Awareness in EONs

    R. Zou et al. “DeepDRAMA: Deep Reinforcement Learning-based Disaster Recovery with Mitigation Awareness in EONs. ” In:IEEE Global Communications Conf. (GLOBECOM). 2021, pp. 1–6.doi: 10.1109/globecom46510.2021.9685680

    doi:10.1109/globecom46510.2021.9685680 2021

  20. [20]

    A Reinforcement Learning Approach to Automatic Error Recovery

    Q. Zhu and C. Yuan. “A Reinforcement Learning Approach to Automatic Error Recovery. ” In:Annual IEEE/IFIP Int’l Conf. Dependable Systems and Networks (DSN’07). 2007, pp. 729–738.doi: 10.1109/dsn.2007.11

    work page doi:10.1109/dsn.2007.11 2007

  21. [21]

    Multi-Agent Architecture for Fault Recovery in Self-Healing Systems

    P. K. Rajput and G. Sikka. “Multi-Agent Architecture for Fault Recovery in Self-Healing Systems. ” In:J. Ambient Intelligence and Humanized Computing12.2 (2021), pp. 2849–2866.doi: 10.1007/s12652-020-02443-8

    work page doi:10.1007/s12652-020-02443-8 2021

  22. [22]

    MicroRAS: Automatic Recovery in the Absence of Historical Failure Data for Microservice Systems

    L. Wu et al. “MicroRAS: Automatic Recovery in the Absence of Historical Failure Data for Microservice Systems. ” In:IEEE/ACM 13th Int’l Conf. Utility and Cloud Computing (UCC). 2020, pp. 227–236.doi: 10.1109/ucc48980.2020.00041

    doi:10.1109/ucc48980.2020.00041 2020

  23. [23]

    A Comprehensive Survey on Safe Reinforcement Learning

    J. Garcıa and F. Fernández. “A Comprehensive Survey on Safe Reinforcement Learning. ” In:J. Machine Learning Research16.1 (2015), pp. 1437–1480

    2015

  24. [24]

    Improving Scalability and Reward of Utility-Driven Self-Healing for Large Dynamic Architectures

    S. Ghahremani, H. Giese, and T. Vogel. “Improving Scalability and Reward of Utility-Driven Self-Healing for Large Dynamic Architectures. ” In: ACM Trans. Autonomous and Adaptive Systems (TAAS)14.3 (2020), 12:1–12:41.doi: 10.1145/3380965

    work page doi:10.1145/3380965 2020

  25. [25]

    Deep Learning for Effective and Efficient Reduction of Large Adaptation Spaces in Self-adaptive Systems

    D. Weyns et al. “Deep Learning for Effective and Efficient Reduction of Large Adaptation Spaces in Self-adaptive Systems. ” In:ACM Trans. Autonomous and Adaptive Systems17.1-2 (2022), 1:1–1:42.doi: 10.1145/3530192

    work page doi:10.1145/3530192 2022

  26. [26]

    S. Wang, D. Jia, and X. Weng.Deep Reinforcement Learning for Autonomous Driving. 2019. arXiv: 1811.11329

    Pith/arXiv arXiv 2019

  27. [27]

    Autonomous Driving Decision Control Based on Improved Proximal Policy Optimization Algorithm

    Q. Song et al. “Autonomous Driving Decision Control Based on Improved Proximal Policy Optimization Algorithm. ” In:Applied Sciences13.11 (2023), p. 6400.doi: 10.3390/app13116400

    work page doi:10.3390/app13116400 2023

  28. [28]

    Deep Reinforcement Learning for Autonomous Driving: A Survey

    B. R. Kiran et al. “Deep Reinforcement Learning for Autonomous Driving: A Survey. ” In:IEEE Trans. Intelligent Transportation Systems23.6 (2022), pp. 4909–4926.doi: 10.1109/TITS.2021.3054625

    doi:10.1109/tits.2021.3054625 2022

  29. [29]

    Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic

    R. Valiente et al. “Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic. ” In:IEEE Open J. Intelligent Transportation Systems3 (2022), pp. 397–410.doi: 10.1109/OJITS.2022.3172981

    work page doi:10.1109/ojits.2022.3172981 2022

  30. [30]

    Filos et al.Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?2020

    A. Filos et al.Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?2020. arXiv: 2006.14911

    Pith/arXiv arXiv 2020

  31. [31]

    S. J. Grimbly, J. Shock, and A. Pretorius.Causal Multi-Agent Reinforcement Learning: Review and Open Problems. 2021. arXiv: 2111.06721

    Pith/arXiv arXiv 2021

  32. [32]

    Howard and L

    R. Howard and L. Kunze.Evaluating Temporal Observation-Based Causal Discovery Techniques Applied to Road Driver Behaviour. 2023. arXiv: 2302.00064

    Pith/arXiv arXiv 2023

  33. [33]

    Simulation-Based Counterfactual Causal Discovery on Real World Driver Behaviour

    R. Howard and L. Kunze. “Simulation-Based Counterfactual Causal Discovery on Real World Driver Behaviour. ” In:IEEE Intelligent Vehicles Symp. (IV). 2023, pp. 1–8.doi: 10.1109/IV55152.2023.10186705

    doi:10.1109/iv55152.2023.10186705 2023

  34. [34]

    Xu et al.TraffNet: Learning Causality of Traffic Generation for What-if Prediction

    M. Xu et al.TraffNet: Learning Causality of Traffic Generation for What-if Prediction. 2024. arXiv: 2303.15954

    Pith/arXiv arXiv 2024

  35. [35]

    CityLifeSim: A High-Fidelity Pedestrian and Vehicle Simulation with Complex Behaviors

    C. Y. Wang et al. “CityLifeSim: A High-Fidelity Pedestrian and Vehicle Simulation with Complex Behaviors. ” In:IEEE 2nd Int’l Conf. Intelligent Reality (ICIR). 2022, pp. 11–16.doi: 10.1109/ICIR55739.2022.00018

    work page doi:10.1109/icir55739.2022.00018 2022

  36. [36]

    CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning

    D. McDuff et al. “CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning. ” In:First Conf. Causal Learning and Reasoning. PMLR, 2022, pp. 559–575

    2022

  37. [37]

    Interventional Root Cause Analysis of Failures in Multi-Sensor Fusion Perception Systems

    S. Wang et al. “Interventional Root Cause Analysis of Failures in Multi-Sensor Fusion Perception Systems. ” In:Proceedings 2025 Network and Distributed System Security Symp.Internet Society, 2025.doi: 10.14722/ndss.2025.240036

    work page doi:10.14722/ndss.2025.240036 2025

  38. [38]

    Spirtes, C

    P. Spirtes, C. Glymour, and R. Scheines.Causation, Prediction, and Search. Ed. by J. Berger et al. Vol. 81. Springer New York, 1993.doi: 10.1007/978- 1-4612-2748-9

    doi:10.1007/978- 1993

  39. [39]

    Optimal Structure Identification with Greedy Search

    D. M. Chickering. “Optimal Structure Identification with Greedy Search. ” In:J. machine learning research3.Nov (2002), pp. 507–554

    2002

  40. [40]

    Investigating Causal Relations by Econometric Models and Cross-spectral Methods

    C. W. J. Granger. “Investigating Causal Relations by Econometric Models and Cross-spectral Methods. ” In:Econometrica37.3 (1969), pp. 424–438. doi: 10.2307/1912791. JSTOR: 1912791

    doi:10.2307/1912791 1969

  41. [41]

    Detecting and Quantifying Causal Associations in Large Nonlinear Time Series Datasets

    J. Runge et al. “Detecting and Quantifying Causal Associations in Large Nonlinear Time Series Datasets. ” In:Science Advances5.11 (2019), eaau4996. doi: 10.1126/sciadv.aau4996. [42]Welcome to CausalNex’s API Docs and Tutorials! — Causalnex 0.12.1 Documentation. https://causalnex.readthedocs.io/en/latest/

    doi:10.1126/sciadv.aau4996 2019

  42. [42]

    Model-Based Reinforcement Learning: A Survey

    T. M. Moerland et al. “Model-Based Reinforcement Learning: A Survey. ” In:Foundations and Trends in Machine Learning16.1 (2023), pp. 1–118. doi: 10.1561/2200000086

    doi:10.1561/2200000086 2023

  43. [43]

    MAPE-K Formal Templates to Rigorously Design Behaviors for Self-Adaptive Systems

    D. G. D. L. Iglesia and D. Weyns. “MAPE-K Formal Templates to Rigorously Design Behaviors for Self-Adaptive Systems. ” In:ACM Trans. Autonomous and Adaptive Systems (TAAS)10.3 (2015), pp. 1–31.doi: 10.1145/2724719

    work page doi:10.1145/2724719 2015

  44. [44]

    CARLA: An Open Urban Driving Simulator

    A. Dosovitskiy et al. “CARLA: An Open Urban Driving Simulator. ” In:Conf. Robot Learning. PMLR, 2017, pp. 1–16

    2017

  45. [45]

    The Cityscapes Dataset for Semantic Urban Scene Understanding

    M. Cordts et al. “The Cityscapes Dataset for Semantic Urban Scene Understanding. ” In:IEEE Conf. Computer Vision and Pattern Recognition (CVPR). IEEE, 2016, pp. 3213–3223.doi: 10.1109/CVPR.2016.350

    doi:10.1109/cvpr.2016.350 2016

  46. [46]

    D. P. Kingma and M. Welling.Auto-Encoding Variational Bayes. 2022. arXiv: 1312.6114

    Pith/arXiv arXiv 2022

  47. [47]

    Schulman et al.Proximal Policy Optimization Algorithms

    J. Schulman et al.Proximal Policy Optimization Algorithms. 2017. arXiv: 1707.06347

    Pith/arXiv arXiv 2017

  48. [48]

    Razak.Autonomous Driving in Carla Using Deep Reinforcement Learning

    I. Razak.Autonomous Driving in Carla Using Deep Reinforcement Learning. https://github.com/idreesshaikh/Autonomous-Driving-in-Carla-using- Deep-Reinforcement-Learning. 2023

    2023

  49. [49]

    Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping

    A. Y. Ng, D. Harada, and S. Russell. “Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping. ” In:Icml. Vol. 99. Citeseer, 1999, pp. 278–287. Manuscript submitted to ACM

    1999