REVIEW 3 major objections 7 minor 25 references
Enhancing Uncertainty Quantification for Runtime Safety Assurance Using Causal Risk Analysis and Operational Design Domain
T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A Bayesian network turns live weather, road, and speed data into a runtime safety confidence score.
desk verdict Plausible integration of HARA/FTA with Bayesian confidence scoring for runtime safety cases, but the unified BN promised in the abstract is never assembled and the evaluation reports no numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is the Bayesian Network assembled from fault-tree structure and safety-argument patterns. Fault-tree events become BN nodes, AND/OR gates become conditional probability tables, and ODD attributes become evidence variables; Monte Carlo Dropout provides the model-uncertainty node and a scenario-coverage metric feeds the data-completeness node. At runtime the network is instantiated with live environmental observations, and posterior inference over the objective node produces the confidence score. The DL ontology is supporting formalization rather than the inference engine.
What would settle it
In the paper's own driving-simulation setup, systematically vary rain, fog, illumination, road surface, and ego speed; at each condition, record the BN posterior confidence and the empirically observed detection error rate over many runs. If the confidence score does not track the measured failure probability (for instance, high confidence while misses are frequent), the central claim that the BN quantifies runtime safety confidence is falsified.
Extended reading notes
Core claim
The central claim is that integrating model, data, and environmental uncertainties in one Bayesian Network, instantiated at runtime with current ODD observations, produces a dynamic and context-aware measure of safety confidence. The authors define ODD attributes through a risk-based causal analysis (HARA plus fault trees), formalize them in a Description Logic ontology aligned with OpenODD, and connect them to assurance goals for data appropriateness, model robustness, and testing adequacy. The BN's posterior distribution, computed with Bayes' theorem over real-time evidence, gives the likelihood of correct behavior and its associated variance. The case study is the Object Detection component of an Automated Valet Parking system.
Load-bearing premise
The live confidence score is only as trustworthy as the conditional probability tables, which the paper says are typically filled in by domain experts, with no procedure or evidence given to show those probabilities match how the object detector actually fails.
Editorial extensions
If this is right
- Confidence scores can be recomputed continuously as weather, lighting, road surface, or ego speed change, so safety assurance reflects the current situation.
- The BN separates confidence into data, model, and testing dimensions, making the weakest assurance component visible.
- ODD boundaries can be refined from simulation traces (for example, by decision trees), allowing the confidence model to tighten as new edge cases are found.
- The ontology-based formalization allows automated consistency checking and querying of ODD, safety cases, and confidence models in one place.
Reading between the lines
- A testable extension the authors do not run: in the driving simulator, record empirical detection failure rates per environmental condition and compare them with BN posterior confidence; strong correlation would warrant trust in the score, poor correlation would show the conditional probability tables need calibration.
- If the conditional probability tables were learned online from operational data rather than fixed by experts, the framework could self-calibrate and the confidence scores would carry empirical meaning.
- The same artifact-to-BN pipeline could be applied to planning and control components, reusing the HARA/FTA step to give a whole-vehicle runtime safety case.
- One implicit consequence is that an ODD exit can itself be treated as a safety event: when the posterior confidence drops below a threshold, the system could trigger a handover or minimal-risk maneuver.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a methodology for runtime safety assurance of DNN-based components by combining Hazard Analysis and Risk Assessment (HARA), fault tree analysis (FTA), Operational Design Domain (ODD) specification, and Bayesian Networks (BNs). At design time, ODD classes and attributes are derived from ability graphs and risk-based causal analysis, then formalized in Description Logic aligned with ASAM OpenODD. Safety argument patterns from the literature are formalized, and BN-based 'confidence models' are constructed for data appropriateness, model uncertainty, and testing adequacy. At runtime, the paper claims to instantiate these BNs with real-time ODD observations (e.g., weather, lighting, ego speed) and to compute posterior probabilities that yield a dynamic, context-aware confidence score. The approach is demonstrated in a proof-of-concept tool (Open-DSC) and a CARLA simulation of an Automated Valet Parking (AVP) object detection component.
Significance. The idea of linking ODD conditions to safety-confidence estimates at runtime is timely and relevant, as static safety cases are widely recognized as insufficient for learning-enabled autonomous systems. The paper's formal ontology and traceability contribution, if made concrete, could support tool-supported safety assurance. However, the current manuscript does not demonstrate the central claim: no unified BN is presented, the CPTs are expert-set without calibration or validation, and the case study reports no quantitative evidence that the computed confidence tracks actual detector performance. The approach is defensible in principle, but the load-bearing pieces—unified model, parameter estimation, and empirical validation—are missing.
major comments (3)
- [Section VI and Section VII] The paper's abstract and Section VII claim that model, data, and environmental uncertainties are 'integrated into a unified Bayesian Network' and that runtime instantiation yields a single safety confidence. This is not demonstrated. Section VI presents three separate BNs (Figs. 9-11) for data appropriateness, model uncertainty, and testing adequacy, plus a separate ODD-sufficiency BN in Fig. 12. No DAG containing all of these together is defined, and no principled aggregation of their posteriors into a single safety-confidence value is specified. Section VII states that 'we instantiate the Bayesian models defined in Fig. 9, 10, and 11' and compute confidence over 'each assurance dimension,' but no combined objective node or joint query is defined. Without a unified model (or a defined aggregation rule), the central claimed contribution—a dynamic and context-aware measure of safety confidence—is not operationalized. The authors must either connect the subnetworks into one BN with ODD attributes as parent variables and a top-level safety objective, or explicitly define how the per-dimension posteriors are combined.
- [Section VI.1, 'Constructing the Initial BN'] The CPTs for all nodes in the confidence BNs are 'typically defined by domain experts' (Section VI.1), with no elicitation procedure, no data-driven estimation, and no calibration step. Since the runtime confidence is computed as a posterior over these CPTs, any error in the expert-set values propagates directly into the reported confidence. The paper provides no argument or experiment showing that these posteriors correlate with actual detector failures or with any ground-truth safety metric. This is a load-bearing gap because the numerical confidence values are otherwise formal but meaningless. The manuscript should provide at least (a) a sensitivity analysis over CPTs, (b) calibration against simulation or field data (e.g., comparing predicted confidence with empirically observed detection error rates), or (c) a concrete procedure for updating CPTs from runtime data with a demonstration.
- [Section VIII.D, 'Simulating Runtime Behavior in CARLA'] The CARLA demonstration does not report any numerical evaluation. Fig. 21 shows a single confidence value but no comparison with ground-truth object detection success/failure, no correlation analysis, and no baseline (e.g., plain MC-dropout uncertainty without ODD conditioning). Consequently, the claim that the BN provides 'a dynamic and context-aware measure of uncertainty' is not supported by evidence. The case study should include quantitative results: e.g., confidence values under varying ODD conditions plotted against actual detection error rates, or a comparison between the BN-based confidence and a baseline uncertainty estimator.
minor comments (7)
- [Abstract and introduction] There is a typo in the abstract: 'A VP' should be 'AVP'. Also, the phrase 'dynamic and context-aware measure of uncertainty' is used in the abstract and later in Section III, but the term 'confidence' is used interchangeably with 'uncertainty'; please clarify the relationship between the two concepts.
- [Section IV.A.3 and Fig. 7] The decision-tree-based boundary refinement is described only briefly, and Fig. 7 presents rules without any information about how they were trained, on how much data, or with what accuracy. At minimum, please clarify the source and size of the simulation traces and the accuracy of the learned boundaries.
- [Section IV.B, Axioms (A15), (A17), (A19)] The axioms use 'T' instead of the top symbol '⊤' inconsistently with the surrounding axioms (e.g., A1, A3). Please correct the notation for uniformity.
- [Section V, Formalization of Assurance Goals] Several axioms, such as (A29) 'supportedBy ⊑ supportedBy', are trivially true and do not add information; please remove or replace them with meaningful constraints. Also, the text refers to 'Fig. 14 and Fig.' with a missing figure number.
- [Section VIII.A] The taxonomy figures appear to be duplicated: the text says 'illustrated in Fig. 14 and Fig.' and Fig. 13 and Fig. 14 both show the same taxonomy. Please consolidate or clearly differentiate the two figures.
- [References and links] Reference [2] is incomplete (no title, year, or publisher). Some URLs in the text (e.g., git.fortiss.org links) are placeholders and should either be made accessible or removed. Please also check the spelling of 'casual' (instead of 'causal') in Fig. 5's caption.
- [Section VI.1, Eq. (3)] The scenario coverage metric m is used only for the DataMetric node's CPT, but the paper does not explain how m is converted into a conditional probability, nor how 'relevant scenarios' are identified for the AVP case. Please clarify this mapping.
Circularity Check
No significant circularity: the runtime confidence is an explicit model evaluation, not a fitted prediction, and the causal structure is an input assumption rather than a hidden equivalence.
full rationale
The derivation chain in the paper is: HARA/FTA identifies hazards and ODD attributes; these are formalized and used to build Bayesian networks with expert-defined CPTs; at runtime, ODD observations are entered as evidence to compute a posterior over hazard-related nodes. This is a deductive evaluation of an explicit probabilistic model, not an empirical prediction derived from fitted data. The fact that ODD attributes are selected from the same hazard analysis is the paper's stated modeling methodology, not a hidden circular reduction: the paper does not claim to have independently discovered that rain or fog degrade detection, nor does it calibrate the CPTs to failure data and then report those same data as a prediction. The self-citations ([22] for the GSN safety case pattern, [10] for ACPs) are used as prior templates and inputs, not as load-bearing evidence that the confidence values are correct or as uniqueness arguments forbidding alternatives. The main weaknesses are validation and completeness gaps: no single unified BN connecting Figs. 9-12 is shown, no aggregation of per-dimension confidences is defined, the CPTs are expert-set without calibration, and the CARLA demonstration does not correlate computed confidence with actual object-detection failures. These are important correctness concerns but are not circularity, because the runtime output is not asserted to be equivalent to its inputs by construction, nor is any fitted parameter renamed as a prediction.
Assumptions & free parameters
free parameters (2)
- CPT entries for BN nodes (DataMetric, OddFC, OddSuff, DataComp, BnModelUnc, ModelUnc, TestDist, TestUcn, ODD_Suff) =
Not specified; expert-defined
- Decision tree thresholds for ODD boundary refinement =
e.g., Vehicle_lighting <= 60.48, Fog <= 4828.34, Sun_Angle <= 0.19, Vehicle_speed <= 43.79, Rain <= 1.84
assumptions (4)
- domain assumption The FTA structure, with AND/OR gates, faithfully represents the causal dependencies that determine object detection safety.
- domain assumption Expert-provided CPTs are accurate enough for safety decisions.
- domain assumption The ODD classes and attributes derived from ability graphs and HARA cover all safety-relevant conditions.
- domain assumption DL axioms A1-A48 correctly formalize the semantics of ODD, safety argument, and BN.
Cite this review
Pith. "Pith review of Enhancing Uncertainty Quantification for Runtime Safety Assurance Using Causal Risk Analysis and Operational Design Domain." pith.science (2026). https://pith.science/paper/TFSYLF2X
@misc{pith2026250703515,
author = {Pith},
title = {Pith review of: Enhancing Uncertainty Quantification for Runtime Safety Assurance Using Causal Risk Analysis and Operational Design Domain},
year = {2026},
howpublished = {\url{https://pith.science/paper/TFSYLF2X}},
note = {Machine review of arXiv:2507.03515}
}
read the original abstract
Ensuring the runtime safety of autonomous systems remains challenging due to deep learning components' inherent uncertainty and their sensitivity to environmental changes. In this paper, we propose an enhancement of traditional uncertainty quantification by explicitly incorporating environmental conditions using risk-based causal analysis. We leverage Hazard Analysis and Risk Assessment (HARA) and fault tree modeling to identify critical operational conditions affecting system functionality. These conditions, together with uncertainties from the data and model, are integrated into a unified Bayesian Network (BN). At runtime, this BN is instantiated using real-time environmental observations to infer a probabilistic distribution over the safety estimation. This distribution enables the computation of both expected performance and its associated variance, providing a dynamic and context-aware measure of uncertainty. We demonstrate our approach through a case study of the Object Detection (OD) component in an Automated Valet Parking (AVP).
Figures
Figures from the paper (16 more)
Reference graph
Works this paper leans on
-
[1]
ASAM. OpenODD: Concept Paper. https://www.asam. net/standards/detail/openodd, October 2021. [Online]
work page 2021
- [2]
-
[3]
Quantitative projection coverage for testing ml- enabled autonomous systems
Chih-Hong Cheng, Chung-Hao Huang, and Hirotoshi Yasuoka. Quantitative projection coverage for testing ml- enabled autonomous systems. In Automated Technology for Verification and Analysis: 16th International Sympo- sium, ATVA 2018, Los Angeles, CA, USA, October 7-10, 2018, Proceedings 16 , pages 126–142. Springer, 2018
work page 2018
-
[4]
Operational design domain (odd) framework for driver-automation systems
HongSeok Cho and R John Hansman. Operational design domain (odd) framework for driver-automation systems. 2020
work page 2020
-
[5]
Avsc best practice for describing an operational design domain: Conceptual framework and lexicon
Automated Vehicle Safety Consortium et al. Avsc best practice for describing an operational design domain: Conceptual framework and lexicon. SAE Industry Tech- nologies Consortia, 2020
work page 2020
-
[6]
Dy- namic safety cases for through-life safety assurance
Ewen Denney, Ganesh Pai, and Ibrahim Habli. Dy- namic safety cases for through-life safety assurance. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, volume 2, pages 587–590. IEEE, 2015
work page 2015
-
[7]
Simulation- based safety assurance for an avp system incorporating learning-enabled components, 2023
Hasan Esen and Brian Hsuan-Cheng Liao. Simulation- based safety assurance for an avp system incorporating learning-enabled components, 2023
work page 2023
-
[8]
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059. PMLR, 2016
2016
Show all 25 references
-
[9]
Towards an operational design domain that supports the safety argumentation of an automated driving system
Magnus Gyllenhammar, Rolf Johansson, Fredrik Warg, DeJiu Chen, Hans-Martin Heyn, Martin Sanfridson, Jan S¨oderberg, Anders Thors ´en, and Stig Ursing. Towards an operational design domain that supports the safety argumentation of an automated driving system. In 10th European C...
2020
-
[10]
A new approach to creating clear safety arguments
Richard Hawkins, Tim Kelly, John Knight, and Patrick Graydon. A new approach to creating clear safety arguments. In Advances in Systems Safety: Proceedings of the Nineteenth Safety-Critical Systems Symposium, Southampton, UK, 8-10th February 2011 , pages 3–23. Springer, 2011
2011
-
[11]
Guidance on the assurance of machine learning in autonomous systems (amlas)
Richard Hawkins, Colin Paterson, Chiara Picardi, Yan Jia, Radu Calinescu, and Ibrahim Habli. Guidance on the assurance of machine learning in autonomous systems (amlas). arXiv preprint arXiv:2102.01564 , 2021
2021 arXiv
-
[12]
ISO 26262: Road vehicles – Functional safety
International Organization for Standardization. ISO 26262: Road vehicles – Functional safety. Standard,
-
[13]
ISO/PAS 21448: Road vehicles – Safety of the intended function- ality (SOTIF)
International Organization for Standardization. ISO/PAS 21448: Road vehicles – Safety of the intended function- ality (SOTIF). Publicly Available Specification, 2019. ISO/PAS 21448:2019
2019
-
[14]
A two-level abstraction odd definition language: Part i
Patrick Irvine, Xizhe Zhang, Siddartha Khastgir, Edward Schwalb, and Paul Jennings. A two-level abstraction odd definition language: Part i. In 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC) , pages 2614–2621. IEEE, 2021
2021
-
[15]
Redefining safety for autonomous vehicles
Philip Koopman and William Widen. Redefining safety for autonomous vehicles. In International Conference on Computer Safety, Reliability, and Security , pages 300–
-
[16]
Identifying the oper- ational design domain for an automated driving system through assessed risk
Chung Won Lee, Nasif Nayeer, Danson Evan Garcia, Ankur Agrawal, and Bingbing Liu. Identifying the oper- ational design domain for an automated driving system through assessed risk. In 2020 IEEE Intelligent Vehicles Symposium (IV), pages 1317–1322. IEEE, 2020
2020
-
[17]
New definition and spec- ification of operational design condition for autonomous railway system
Rim Louhichi and Insaf Sassi. New definition and spec- ification of operational design condition for autonomous railway system. In 33rd European Safety and Reliabil- ity Conference , pages 1918–1925. Research Publishing Services, 2023
1918
-
[18]
Sahara: a security- aware hazard and risk analysis method
Georg Macher, Harald Sporer, Reinhard Berlach, Eric Armengaud, and Christian Kreiner. Sahara: a security- aware hazard and risk analysis method. In 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 621–624. IEEE, 2015
2015
-
[19]
Ability and skill graphs for system modeling, online monitoring, and decision support for vehicle guidance systems
Andreas Reschka, Gerrit Bagschik, Simon Ulbrich, Mar- cus Nolte, and Markus Maurer. Ability and skill graphs for system modeling, online monitoring, and decision support for vehicle guidance systems. In 2015 Ieee in- telligent vehicles symposium (Iv) , pages 933–939. IEEE, 2015
2015
-
[20]
Operational design domains in automated vehicles: A review of state-of-the-art standards, chal- lenges, and proposed solution
Ali Shakeri. Operational design domains in automated vehicles: A review of state-of-the-art standards, chal- lenges, and proposed solution. 2024
2024
-
[21]
Formal verification of bayesian network classifiers
Andy Shih, Arthur Choi, and Adnan Darwiche. Formal verification of bayesian network classifiers. In Inter- national Conference on Probabilistic Graphical Models , pages 427–438. PMLR, 2018
2018
-
[22]
Towards continuous assurance case creation for ads with the evidential tool bus
Lev Sorokin, Radouane Bouchekir, Tewodros A Beyene, Brian Hsuan-Cheng Liao, and Adam Molin. Towards continuous assurance case creation for ads with the evidential tool bus. In European Dependable Computing Conference, pages 49–61. Springer, 2024
2024
-
[23]
Acclimatizing the operational design domain for autonomous driving systems
Chen Sun, Zejian Deng, Wenbo Chu, Shen Li, and Dongpu Cao. Acclimatizing the operational design domain for autonomous driving systems. IEEE Intelligent Transportation Systems Magazine, 14(2):10–24, 2021
2021
-
[24]
A safety case pattern for systems with machine learning components
Ernest Wozniak, Carmen C ˆarlan, Esra Acar-Celik, and Henrik J Putzer. A safety case pattern for systems with machine learning components. In Computer Safety, Reliability, and Security. SAFECOMP 2020 Workshops: DECSoS 2020, DepDevOps 2020, USDAI 2020, and WAISE 2020, Lisbon, P...
2020
-
[2018]
ISO 26262:2018, Parts 1–12
2018
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.