Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Function

Francesco Braghin; Mohammed Irshadh Ismaaeel Sathyamangalam Imran; Satyesh Shanker Awasthi; Stefano Arrigoni

REVIEW 4 major objections 6 minor 49 references

Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Function

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bayesian optimization finds all critical driving scenarios with one to two orders of magnitude fewer simulations.

desk verdict The case study is honest and the benchmark is useful, but the 'orders of magnitude' claim rests on an oracle stopping rule and a strawman baseline; it merits peer review with major revision. read the letter →

arxiv 2507.22769 v2 pith:7E7SBSS7 submitted 2025-07-30 cs.RO cs.SYeess.SY

Satyesh Shanker Awasthi , Mohammed Irshadh Ismaaeel Sathyamangalam Imran , Stefano Arrigoni , Francesco Braghin This is my paper · ORCID

classification cs.ROcs.SYeess.SY

keywords BayesianoptimizationautonomousdrivingvalidationcriticalscenariodiscoveryGaussianprocesssurrogateThompsonsamplingmotionplanningsimulation-basedtesting

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that exhaustive simulation is not necessary to locate the dangerous scenarios of an autonomous driving function. It casts critical-scenario discovery as the maximization of a safety-violation metric over continuous scenario parameters and solves that maximization with Bayesian Optimization, letting a Gaussian-process surrogate choose which parameter sets to simulate next. Tested on a model-predictive motion planner in a two-lane highway operational design domain, the framework recovers every critical region that an exhaustive baseline found, including off-road events and solver infeasibility, using one to two orders of magnitude fewer simulations. The point of the work is that validation budgets could be concentrated on the parameter regions most likely to fail, shortening development and homologation loops for autonomous driving functions.

What carries the argument

The load-bearing mechanism is the Bayesian Optimization loop: a Gaussian process with a squared exponential kernel models the map from scenario parameters to a criticality metric; an acquisition function then selects the next parameter set, the co-simulation evaluates the metric, and the surrogate is refit. Thompson Sampling is the acquisition function that naturally balances exploration and exploitation, and it is compared with Probability of Improvement. Critical regions are defined by DBSCAN clustering of the exhaustive baseline results, and the loop stops when at least one simulation has landed in every cluster that the baseline identified, subject to an upper budget one order of magnitude below the baseline count. The two metrics used are the maximum lateral displacement of the ego vehicle (continuous) and the MPC solver's infeasibility status (discrete), which correspond to the two failure modes: off-road excursions and solver failure.

What would settle it

Construct a parameter space where a genuine critical region lies between the grid lines of the baseline discretization, so the baseline and its DBSCAN clusters do not contain it; run the BO framework with the paper's stopping rule. If the loop terminates having hit every baseline-identified cluster but never the hidden region, the completeness claim fails.

Watch

Extended reading notes

Core claim

The paper claims that critical-scenario discovery for an autonomous driving function can be treated as a black-box optimization problem: find the scenario parameters $x$ that maximize a criticality metric $f(x)$ computed by one simulation run. A Bayesian Optimization loop with a Gaussian process surrogate and an acquisition function recovers all critical regions found by exhaustive combinatorial testing while using a fraction of the simulation budget. In the 3-parameter case, 15 seed simulations plus acquisition iterations found both critical clusters in under 100 simulations on average, against a 1,331-run full-factorial baseline; in the 6-parameter case, 30 seed simulations plus iterations matched clusters found in a 19,800-run baseline. The paper further claims that the workflow is scenario-agnostic, so any logical scenario with continuous parameters inside an operational design domain can be plugged into the same loop, and that the hybrid strategy of alternating the criticality metric used to train the surrogate is what makes the higher-dimensional search reliable.

Load-bearing premise

The evaluation assumes that the DBSCAN clusters found in the exhaustive baseline are the complete and correct set of critical regions, and that finding one simulation in each cluster is enough to prove all critical scenarios were discovered.

Editorial extensions

If this is right

A validation campaign can substitute a BO-guided search for a full-factorial sweep and still claim to have seen the same critical regions, at a fraction of the simulation cost.
Continuous criticality metrics, like maximum lateral displacement, should guide the surrogate when possible; discrete metrics alone rely on exploration and can miss clusters under exploitative acquisition functions.
Acquisition functions that balance exploration and exploitation, such as Thompson Sampling, are more reliable for multi-cluster discovery than a greedy Probability of Improvement.
In higher-dimensional parameter spaces, alternating the metric used to train the surrogate is the paper's recommended fix for reliably finding small, hard-to-locate failure clusters.
The framework's scenario-agnostic formulation means the same loop can be re-applied to other logical scenarios and other driving functions without changing the optimization machinery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the authors leave implicit is that the stopping rule presupposes the baseline-identified clusters are the complete set of critical regions; in a real development setting with no exhaustive baseline, the loop would need an alternative stopping rule such as a budget cap or a surrogate-convergence criterion.
A natural stress test would be to run the same BO loop on the same motion planner with a critical region deliberately placed between the grid lines of the baseline discretization; the reported speed-up is defined relative to the grid, not to an oracle.
The hybrid metric strategy suggests a general recipe: when failure modes are heterogeneous, no single scalar objective captures them all, so alternating or multi-objective acquisition may be necessary.
A direct extension is to multi-agent urban scenarios by treating all agents' initial poses and behaviors as the parameter vector and using a collision or near-miss metric as the objective; the machinery would carry over intact.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a legitimate case study, but the headline 'orders of magnitude fewer simulations' is not backed by the evidence as written. The two soft spots are the oracle stopping rule and the lack of a random/LHS baseline. Both are fixable.

What's actually new: most of the machinery is textbook BO, as the authors acknowledge. The value is in the benchmark: two acquisition functions (Thompson Sampling vs Probability of Improvement) crossed with two criticality metrics (continuous lateral displacement vs discrete solver status), including a hybrid strategy that alternates metrics. That comparison is useful and not something I've seen in the cited BO-for-AV papers. The 3D/6D scalability study on their own MPC planner is also concrete, and the co-simulation setup (Simulink + acados + CarMaker) is non-trivial. They report failure cases honestly, e.g., PI fails to find both clusters in 5 of 10 runs, and they note the single-metric strategy in 6D had an outlier run of 472 simulations.

Where it gets soft. The stopping criterion in Section IV terminates when one instance in each of the baseline-identified DBSCAN clusters is found. That means the reported simulation counts are hitting times for known targets, not discovery costs for unknown critical regions. In a real validation campaign, you don't know these clusters without running the exhaustive DoE you're trying to avoid. The abstract's claim is therefore stronger than what is demonstrated. This is not a fatal flaw for a case study, but it needs to be said plainly.

The bigger practical problem is the baseline. The paper only compares against full-factorial DoE. In the 6D case, the critical clusters cover about 35% of the space (6,174 off-road + 720 solver-failure out of 19,800), and the smaller solver-failure cluster is about 3.6% of the space. Random or LHS sampling at the same budget would likely find both clusters with high probability within a few hundred runs. Without that comparison, the speed-up over 'brute-force' is a low bar. The authors should add a random/LHS baseline at equal total budget.

I also wish the DBSCAN parameters and the alternation schedule for the hybrid strategy were specified in more detail, but those are minor.

Bottom line: the paper deserves a serious referee. It's a well-written, honest engineering study with a useful empirical comparison. But I'd send it back for major revision: add a random baseline, replace or supplement the oracle stopping rule with a practical one, and temper the abstract. If the authors do that, the result will be a solid contribution to scenario-based validation.

Referee Report

4 major / 6 minor

Summary. This paper introduces a Bayesian optimization (BO) framework for discovering safety-critical scenarios during virtual validation of an autonomous driving function. The logical scenario parameters (first three, then six) are searched by a Gaussian-process surrogate and an acquisition function (Thompson Sampling or probability of improvement) against a criticality metric: maximum lateral displacement or MPC solver status. The case study is an MPC-based motion planner in a two-lane highway with a preceding vehicle; the baseline is a full-factorial discretization (1,331 runs in 3D and 19,800 runs in 6D), and DBSCAN clustering of the baseline results defines the critical clusters. BO runs are stopped when at least one point in each baseline-defined critical cluster has been found, and the paper reports mean/median counts below 100 in 3D and around 100-200 in 6D, claiming one to two orders of magnitude fewer simulations than brute-force DoE.

Significance. Accelerated critical-scenario search is an important problem, and the paper offers a transparent case study with two failure modes, repeated random-seed experiments, and a clearly described co-simulation setup. The central quantitative claim, however, is not established by the current evaluation: the stopping rule uses the baseline cluster labels as targets, no comparison to random sampling or Latin hypercube sampling at equal budget is provided, and the success criterion of one point per cluster is weak. If the authors reframe the claim as targeted re-identification of known critical clusters and add appropriate comparator methods, the work would be a useful empirical contribution. The paper does not provide code or data, which limits reproducibility.

major comments (4)

[Section IV, stopping criterion] The paper states: 'The iteration of the Bayesian Optimization framework simulations terminate when at least one instance in each of the critical clusters identified in the baseline simulations are identified.' Because the target clusters are produced by the exhaustive baseline, the simulation counts in Figures 6 and 8 are hitting times for a known target set, not the cost of discovering unknown critical regions. In a real validation campaign the clusters are unknown without first running the baseline that the method is meant to replace. The abstract's claim that the framework 'identifies hazardous situations ... using orders of magnitude fewer simulations' is therefore not demonstrated as a de novo discovery cost. Please either add a deployable stopping rule and evaluate de novo discovery, or explicitly reframe the results as targeted search for known clusters and adjust the title and abstract claims accordingly.
[Section V, Figures 6-8] No comparison is made against random sampling, Latin hypercube sampling, or any other model-free search at the same simulation budget. This matters because the off-road cluster occupies 56 of 1,331 points (~4%) in 3D, and in 6D the two critical clusters together occupy 6,894 of 19,800 points (~35%). At budgets of 50-200 simulations, random or LHS sampling would be expected to hit these clusters by chance a substantial fraction of the time. The reported one-to-two-order speed-up relative to the full-factorial grid does not establish that BO itself is responsible for the efficiency. Same-budget random/LHS detection rates should be reported, and the BO advantage over these baselines should be quantified.
[Section V-A, DBSCAN clustering and success criterion] The success criterion is defined as finding at least one instance in each baseline-derived critical cluster. This is weak because it does not measure coverage of the critical regions, and the clusters themselves depend on the arbitrarily chosen baseline discretization and on the DBSCAN hyperparameters (eps, min_samples), which are not reported. A single point in a cluster could be an artifact of the grid or of the clustering parameters, and critical regions missed by the baseline grid are invisible to the evaluation. Please report the DBSCAN hyperparameters, provide a sensitivity analysis of the clusters, and use a stronger discovery metric such as the volume fraction of each cluster that is covered, or require multiple points per cluster.
[Section V-B and Figure 8] The statistical reporting is not complete enough to support the scalability claims. The 'hybrid strategy' of Set F, said to alternate between the two criticality metrics, is never defined (no schedule, no equations, no pseudocode). Figure 8's caption labels the categories as A and B although the text discusses Sets E and F, and the text states that a 472-simulation outlier in Set E is omitted from the figure while the reported mean is affected by it. Furthermore, in the 3D case, sets C and D that failed to find both clusters in some runs have their means excluded (Section V-A), which biases the acquisition-function comparison. Please report success rates, include all runs (including failures and outliers), and give an exact definition of the hybrid metric alternation.

minor comments (6)

[Abstract and I-D.2] The claim that this is the first work applying such a framework to a developed ADF is not supported by the cited related work, since references [34]-[36] already apply BO or scenario-exploration methods to automated-driving test-case search; please temper the novelty claim or explain the precise distinction.
[Section IV, paragraph after Table I] The sentence 'Different combinations of the acquisition functions and Hence this paper also benchmarks for the stopping criteria...' is ungrammatical and should be revised.
[Table I and Table II] Table I has an extra bracket in the x_0,ego range ('[-0.5,0.5]]'), and Table II's caption reverses the mapping of Clusters 1 and 3 to off-road events and a priori solver failure relative to the description in Section V-A.
[Figure 8 caption] The caption uses 'A' and 'B' for the two simulation sets, while the text refers to Sets E and F; the caption should be corrected for consistency.
[Equation (7)] The probability-of-improvement formula does not specify the behavior when the posterior variance sigma(x) is zero; standard implementations add a small jitter, which should be stated.
[Section III-B and Sections V-A/V-B] The GP surrogate is trained on the discrete solver-status metric (values such as 0 and 4) using a Gaussian likelihood and a squared-exponential kernel; the authors should justify this treatment of a discrete response or use a surrogate model better suited to classification outputs.

Circularity Check

1 steps flagged · score 4.0 of 10

Efficiency claim is measured with an oracle stopping rule: the BO run stops once it hits the baseline's DBSCAN clusters, so 'orders of magnitude fewer simulations' is a hitting-time to known targets rather than a de novo discovery cost.

other [Section IV (Framework Validation Setup), stopping-criterion paragraph; Section V-A, baseline ground-truth paragraph]
"The iteration of the Bayesian Optimization framework simulations terminate when at least one instance in each of the critical clusters identified in the baseline simulations are identified. ... This comprehensive dataset serves as the ground truth for validating the sample efficiency of the Bayesian optimization approach, which seeks to discover these same critical regions using significantly fewer simulations."

The reported speed-up in Figures 6 and 8 is the number of simulations needed to hit at least one point in each precomputed baseline cluster, because the stopping rule defines success using the exhaustive baseline's DBSCAN clusters. The abstract's central claim--that the framework 'identifies hazardous situations ... using orders of magnitude fewer simulations'--is therefore demonstrated only as the cost of hitting a target set that is known in advance from the very brute-force DoE the method is meant to replace. The BO surrogate does not fit those cluster labels, so this is evaluation circularity rather than fitting-to-a-parameter circularity; the method still has independent content, but the headline efficiency number is not a de novo discovery cost.

full rationale

The BO loop itself is not derivationally circular: the Gaussian Process surrogate is trained only on the criticality metric outputs of the simulations, and the acquisition function selects new points without seeing the baseline DBSCAN cluster labels. No load-bearing self-citation chain is present; references to the authors' earlier motion-planner work are used only to describe the case-study ADF, not to justify the BO result. However, the evaluation protocol makes the central efficiency claim self-referential in an important way. The stopping criterion in Section IV uses 'at least one instance in each of the critical clusters identified in the baseline simulations' as the success condition, and Section V-A calls the baseline dataset the 'ground truth' for 'discover[ing] these same critical regions.' Thus the mean/median counts in Figures 6 and 8 are hitting times to a set of targets that is unknown in a real validation campaign without first running the exhaustive DoE. The claim of 'one to two orders of magnitude fewer simulations' is therefore conditional on oracle knowledge of the very clusters the method is supposed to discover. This is partial evaluation circularity, not a complete reduction of the derivation to its inputs, so a score of 4 is appropriate; concerns about whether DBSCAN clusters are complete or correct are correctness risks, not circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests mainly on the completeness of the baseline clusters as ground truth and on the fidelity of the co-simulation oracle. No new entities are postulated. The GP, acquisition functions, and thresholds introduce several fitted or hand-chosen settings.

free parameters (6)

GP hyperparameters (length scales, signal variance, noise) = estimated by maximizing log marginal likelihood on each dataset
These are standard BO model parameters fitted to simulation data, not to the target criticality claim, so they do not by themselves invalidate the results.
PI margin parameter xi = not specified
Equation (7) defines Probability of Improvement with a margin xi >= 0, but the value used in the case study is not reported; this affects exploration in Sets C and D.
Seed sample size factor = 5x parameter dimension (15 for 3D, 30 for 6D)
Chosen from a rule of thumb in [40], not derived; it directly affects total simulation counts.
Off-road failure threshold = 3.5 m lateral displacement
Set to match lane width; this converts the continuous metric into a binary failure label used for cluster interpretation.
Baseline discretization steps = e.g., 2 m/s speed steps, 5 m distance steps, giving 1331 and 19800 grid points
The speed-up ratio is computed against this arbitrary grid, so the grid resolution is a free choice that determines the headline factor.
DBSCAN hyperparameters (eps, min_samples) = not reported
The number and shape of critical clusters, which define the success criterion, depend on these unreported parameters (Section V).

assumptions (6)

standard math Standard GP regression and BO convergence assumptions (zero-mean GP prior, SE kernel, marginal likelihood hyperparameter fitting).
Invoked in Section III.B to model f(x) and select x_next; these are textbook results and not the paper's contribution.
domain assumption The IPG CarMaker plus MPC co-simulation is a faithful oracle for the ADF's real-world behavior.
All criticality labels come from this closed-loop simulation; no real-world or independent validation is provided (Section IV).
domain assumption Control Barrier Functions prevent collisions, so off-road events and solver infeasibility are the only failure modes.
Stated in Section IV: 'no collisions occur because of the inherent modeling of the MPC where CBF is a hard constraint'.
ad hoc to paper The DBSCAN clusters from the exhaustive baseline are the complete ground-truth set of critical regions.
The BO stopping criterion and success metric are defined as finding at least one instance in these clusters (Section IV, Section V), making the baseline's clustering output load-bearing.
domain assumption Wind and road friction are not modeled in the planner and act as unmodeled disturbances.
Section IV says the wind and road parameters 'are not explicitly considered in the modeling and closed-loop control, therefore they should act as disturbance factors'.
ad hoc to paper The discretization of the full-factorial baseline is representative of exhaustive testing cost.
The reported order-of-magnitude speed-up is a ratio to this specific grid, so the grid resolution is an assumption about what brute-force testing costs (Section V-A).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Function." pith.science (2026). https://pith.science/paper/7E7SBSS7

@misc{pith2026250722769,
  author       = {Pith},
  title        = {Pith review of: Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Function},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7E7SBSS7}},
  note         = {Machine review of arXiv:2507.22769}
}

read the original abstract

Rigorous Verification and Validation (V&V) of Autonomous Driving Functions (ADFs) is paramount for ensuring the safety and public acceptance of Autonomous Vehicles (AVs). Current validation relies heavily on simulation to achieve sufficient test coverage within the Operational Design Domain (ODD) of a vehicle, but exhaustively exploring the vast parameter space of possible scenarios is computationally expensive and time-consuming. This work introduces a framework based on Bayesian Optimization (BO) to accelerate the discovery of critical scenarios. We demonstrate the effectiveness of the framework on an Model Predictive Controller (MPC)-based motion planner, showing that it identifies hazardous situations, such as off-road events, using orders of magnitude fewer simulations than brute-force Design of Experiments (DoE) methods. Furthermore, this study investigates the scalability of the framework in higher-dimensional parameter spaces and its ability to identify multiple, distinct critical regions within the ODD of the motion planner used as the case study .

Figures

Figures reproduced from arXiv: 2507.22769 by the authors.

**Figure 1.** Layout of feedback loop established between the developed motion planner and the high-fidelity simulation software [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

**Figure 2.** A scene from the logical scenario being tested and three variable parameters of the logical scenario [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Baseline simulation results of the 3-DOF simulations. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: The two principal clusters identified by the two [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: One complete run each of the 3 simulation categories with the same seed sample. The seed runs and the acquisition runs [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: 4 categories of 10 sets of simulations. A: Criticality [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 8.** Figure 8: For both sets, the initial seed sample size was set [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

**Figure 7.** Figure 7: Simulations performed for wind gust velocity = 15 m/s. each subplot represents the 3-D equivalent simulations for each [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: 2 categories of 10 sets of simulations. A: Criticality [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 43 canonical work pages

[1]

Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?

N. Kalra and S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” Transportation Research Part A: Policy and Practice , vol. 94, pp. 182–193, 2016. [Online]. Available: https://www.sciencedirect.com/scie nce/article/pii/S0965856416302129

work page 2016
[2]

A system- atic approach towards the definition of the terms edge case and corner case for automated driving,

L. Vater, M. Sonntag, J. Hiller, P. Schaudt, and L. Eckstein, “A system- atic approach towards the definition of the terms edge case and corner case for automated driving,” in 2023 3rd International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), 2023, pp. 1–6

work page 2023
[3]

Finding critical scenarios for automated driving systems: A systematic mapping study,

X. Zhang, J. Tao, K. Tan, M. T ¨orngren, J. M. G. S ´anchez, M. R. Ramli, X. Tao, M. Gyllenhammar, F. Wotawa, N. Mohan, M. Nica, and H. Felbinger, “Finding critical scenarios for automated driving systems: A systematic mapping study,” IEEE Trans. Softw. Eng., vol. 49, no. 3, p. 991–1026, 2023. [Online]. Available: https://doi.org/10.1109/TSE.2022.3170122

work page arXiv 2023
[4]

Challenges in autonomous vehicle testing and validation,

P. Koopman and M. Wagner, “Challenges in autonomous vehicle testing and validation,” SAE Int. J. Trans. Safety , 2016

work page 2016
[5]

Virtual testing of automated driving systems. a survey on validation methods,

R. Don `a and B. Ciuffo, “Virtual testing of automated driving systems. a survey on validation methods,” IEEE Access, vol. 10, pp. 24 349–24 367, 2022

work page 2022
[6]

On the Need for a Statistical Foundation in Scenario-Based Testing of Autonomous Vehicles

X. Zhao, R. Aghazadeh-Chakherlou, C.-H. Cheng, P. Popov, and L. Strigini, “On the need for a statistical foundation in scenario- based testing of autonomous vehicles,” 2025. [Online]. Available: https://arxiv.org/abs/2505.02274

work page Pith review arXiv 2025
[7]

Addendum 156 – un regulation no. 157 uniform provisions concerning the approval of vehicles with regards to automated lane keeping system,

United Nations Economic Commission for Europe, “Addendum 156 – un regulation no. 157 uniform provisions concerning the approval of vehicles with regards to automated lane keeping system,” 2021, accessed: 2024-06-10. [Online]. Available: https: //unece.org/transport/documents/2021/03/standards/un-regulation-no-1 57-automated-lane-keeping-systems-alks

work page 2021
[8]

for Standardization of Automation and M

A. for Standardization of Automation and M. Systems, Evolving Land- scapes Of Collaborative Testing For Adas & Ad . ASAM e.V ., 2022

work page 2022

Show all 49 references

[9]

Validation of x-in-the-loop approaches for virtual homologation of automated driving functions,

S. Riedmaier, J. Nesensohn, C. Gutenkunst, B. Schick, T. D ¨user, and H. Abdellatif, “Validation of x-in-the-loop approaches for virtual homologation of automated driving functions,” in 11. Grazer Symposium Virtual Vehicle(GSVF) Graz, 15./16.05. 2018 , 2018

2018
[10]

Vehicle-in-virtual-environment (vve) method for autonomous driving system development, evaluation and demonstration,

X. Cao, H. Chen, S. Y . Gelbal, B. Aksun-Guvenc, and L. Guvenc, “Vehicle-in-virtual-environment (vve) method for autonomous driving system development, evaluation and demonstration,” Sensors, vol. 23, no. 11, 2023. [Online]. Available: https://www.mdpi.com/1424-8220/23 /11/5088

2023
[11]

Correlation of software- in-the-loop simulation with physical testing for autonomous driving,

Z. Fei, M. Andersson, and A. Tingberg, “Correlation of software- in-the-loop simulation with physical testing for autonomous driving,”
[12]

A survey on scenario-based testing for automated driving systems in high-fidelity simulation,

Z. Zhong, Y . Tang, Y . Zhou, V . de Oliveira Neves, Y . Liu, and B. Ray, “A survey on scenario-based testing for automated driving systems in high-fidelity simulation,” 2021. [Online]. Available: https://arxiv.org/abs/2112.00964

2021 arXiv
[13]

Gas: Generating fast & accurate surrogate models for simulations of autonomous vehicle systems,

K. Joshi, C. Hsieh, S. Mitra, and S. Misailovic, “Gas: Generating fast & accurate surrogate models for simulations of autonomous vehicle systems,” in 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE) , 2024, pp. 260–271

2024
[14]

Autonovi-sim: Autonomous vehicle simulation platform with weather, sensing, and traffic control,

A. Best, S. Narang, L. Pasqualin, D. Barber, and D. Manocha, “Autonovi-sim: Autonomous vehicle simulation platform with weather, sensing, and traffic control,” in 2018 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition Workshops (CVPRW) , 2018, pp. 1161–11 618

2018
[15]

Waymax: An accelerated, data-driven simulator for large-scale autonomous driving research,

C. Gulino, J. Fu, W. Luo, G. Tucker, E. Bronstein, Y . Lu, J. Harb, X. Pan, Y . Wang, X. Chen, J. D. Co-Reyes, R. Agarwal, R. Roelofs, Y . Lu, N. Montali, P. Mougin, Z. Yang, B. White, A. Faust, R. McAllister, D. Anguelov, and B. Sapp, “Waymax: An accelerated, data-driven simu...
[16]

A scenario-based platform for testing autonomous vehicle behavior prediction models in simulation,

F. Indaheng, E. Kim, K. Viswanadha, J. Shenoy, J. Kim, D. J. Fremont, and S. A. Seshia, “A scenario-based platform for testing autonomous vehicle behavior prediction models in simulation,” 2021. [Online]. Available: https://arxiv.org/abs/2110.14870

2021 arXiv
[17]

A rule-defined adaptive mpc based motion planner for autonomous driving applications,

M. I. I. Sathyamangalam Imran, S. S. Awasthi, M. Khayyat, S. Arrigoni, and F. Braghin, “A rule-defined adaptive mpc based motion planner for autonomous driving applications,” in 16th International Symposium on Advanced Vehicle Control . Cham: Springer Nature Switzerland, 2024,...

2024
[18]

Data basis for scenario-based validation of had on high- ways,

J. Bock, R. Krajewski, L. Eckstein, J. Klimke, J. Sauerbier, and A. Zlocki, “Data basis for scenario-based validation of had on high- ways,” in 27th Aachen colloquium automobile and engine technology , 2018, pp. 8–10

2018
[19]

A framework for definition of logical scenarios for safety assurance of automated driving,

H. Weber, J. Bock, J. Klimke, C. Roesener, J. Hiller, R. Krajewski, A. Zlocki, and L. E. and, “A framework for definition of logical scenarios for safety assurance of automated driving,” Traffic Injury Prevention , vol. 20, no. sup1, pp. S65–S70, 2019

2019
[20]

Pegasus—first steps for the safe introduction of automated driving,

PEGASUS Consortium, “Pegasus—first steps for the safe introduction of automated driving,” in Verification and Validation of Automated Driving Functions, 2019, project funded by the German Federal Ministry for Economic Affairs and Energy (BMWi); defines scenario-based testing m...

2019
[21]

Verification & Validation Methods (VVM) for Automated Vehicles,

Fraunhofer IESE et al., “Verification & Validation Methods (VVM) for Automated Vehicles,” Project page, Fraunhofer IESE / ASAM, 2019–2023, german Federal Ministry-funded extension of PEGASUS scenario-based validation to SAE L4/5 urban use cases. [Online]. Available: https://ww...

2019
[22]

SET Level: Safety for the autonomous mobility of the future,

Fraunhofer LBF and partners, “SET Level: Safety for the autonomous mobility of the future,” Fraunhofer LBF project page, 2022, develops flexible simulation platforms for VDA L4/L5 automated driving validation. [Online]. Available: https://www.lbf.fraunhofer.de/en/project s/set...

2022
[23]

Webots.hpc: A parallel simulation pipeline for autonomous vehicles,

M. Franchi, R. Kahn, M. Chowdhury, S. Khan, K. Kennedy, L. Ngo, and A. Apon, “Webots.hpc: A parallel simulation pipeline for autonomous vehicles,” in Practice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You , ser. PEARC ’22. New Y...

2022
[24]

NVIDIA Introduces DRIVE Constellation Simulation System to Safely Drive Autonomous Vehicles Billions of Miles in Virtual Reality,

H. Marinez, “NVIDIA Introduces DRIVE Constellation Simulation System to Safely Drive Autonomous Vehicles Billions of Miles in Virtual Reality,” NVIDIA Newsroom, Mar 2018, [Online]. Available: https://nvidianews.nvidia.com/news/nvidia-introduces-drive-constellati on-simulation-...

2018
[25]

Ansys A Vxcelerate Autonomy: End-to-end simulation platform for autonomous vehicle validation,

Ansys, Inc., “Ansys A Vxcelerate Autonomy: End-to-end simulation platform for autonomous vehicle validation,” Product overview / press release, December 19, 2023, accessed: 2025-06-25. [Online]. Available: https://www.ansys.com/products/av-simulation/ansys-avxcelerate-auton omy

2023
[26]

Multi-objective reinforcement learning for critical scenario generation of autonomous vehicles,

J. Wu, C. Lu, A. Arrieta, and S. Ali, “Multi-objective reinforcement learning for critical scenario generation of autonomous vehicles,” 2025. [Online]. Available: https://arxiv.org/abs/2502.15792

2025 arXiv
[27]

Dense reinforcement learning for safety validation of autonomous vehicles,

S. Feng, H. Sun, X. Yan, H. Zhu, Z. Zou, S. Shen, and H. X. Liu, “Dense reinforcement learning for safety validation of autonomous vehicles,” Nature, vol. 615, no. 7953, pp. 620–627, 2023

2023
[28]

Safety-critical scenario generation via reinforcement learning based editing,

H. Liu, L. Zhang, S. K. Sastry Hari, and J. Zhao, “Safety-critical scenario generation via reinforcement learning based editing,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) , 2024, pp. 14 405–14 412

2024
[29]

Dynnpc: Finding more violations induced by ads in simulation testing through dynamic npc behavior generation,

Y . Lu, Y . Tian, D. Wang, B. Chen, and X. Peng, “Dynnpc: Finding more violations induced by ads in simulation testing through dynamic npc behavior generation,” 2025. [Online]. Available: https://arxiv.org/abs/2411.19567

2025
[30]

Testing of autonomous vehicles using surrogate models and stochastic optimization,

H. Beglerovic, M. Stolz, and M. Horn, “Testing of autonomous vehicles using surrogate models and stochastic optimization,” in 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) , 2017, pp. 1–6

2017
[31]

Variable-fidelity model selection for stochastic simulation,

J. Mullins and S. Mahadevan, “Variable-fidelity model selection for stochastic simulation,” Reliability Engineering & System Safety , vol. 131, pp. 40–52, 2014. [Online]. Available: https://www.sciencedirect. com/science/article/pii/S0951832014001331

2014
[32]

Genetic algorithm-based test parameter optimization for adas system testing,

F. Kl ¨uck, M. Zimmermann, F. Wotawa, and M. Nica, “Genetic algorithm-based test parameter optimization for adas system testing,” in 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS) , 2019, pp. 418–425

2019
[33]

Testing vision-based control systems using learnable evolutionary algorithms,

R. Ben Abdessalem, S. Nejati, L. C. Briand, and T. Stifter, “Testing vision-based control systems using learnable evolutionary algorithms,” in 2018 IEEE/ACM 40th International Conference on Software Engi- neering (ICSE), 2018, pp. 1016–1026

2018
[34]

Identification of test cases for automated driving systems using bayesian optimization,

B. Gangopadhyay, S. Khastgir, S. Dey, P. Dasgupta, G. Montana, and P. Jennings, “Identification of test cases for automated driving systems using bayesian optimization,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC) , 2019, pp. 1961–1967

2019
[35]

An application of scenario exploration to find new scenarios for the devel- opment and testing of automated driving systems in urban scenarios,

B. Sch ¨utt, M. Heinrich, S. Marahrens, J. M. Z ¨ollner, and E. Sax, “An application of scenario exploration to find new scenarios for the devel- opment and testing of automated driving systems in urban scenarios,” in Proceedings of the 8th International Conference on Vehicle ...

2022
[36]

A bayesian optimization method for finding the worst-case scenarios of autonomous vehicles,

Y . Huang, J. Sun, and Y . Tian, “A bayesian optimization method for finding the worst-case scenarios of autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems , vol. 26, no. 1, pp. 529–543, 2025

2025
[37]

A framework for definition of logical scenarios for safety assurance of automated driving,

H. Weber, J. Bock, J. Klimke, C. Roesener, J. Hiller, R. Krajewski, A. Zlocki, and L. Eckstein, “A framework for definition of logical scenarios for safety assurance of automated driving,” Traffic injury prevention, vol. 20, no. sup1, pp. S65–S70, 2019

2019
[38]

Criticality metrics for automated driving: A review and suitability analysis of the state of the art,

L. Westhofen, C. Neurohr, T. Koopmann, M. Butz, B. Sch ¨utt, F. Utesch, B. Neurohr, C. Gutenkunst, and E. B ¨ode, “Criticality metrics for automated driving: A review and suitability analysis of the state of the art,” Archives of Computational Methods in Engineering , vol. 30,...

2023
[39]

R. L. Iman, Latin Hypercube Sampling . John Wiley & Sons, Ltd,
[40]

A tutorial on bayesian optimization,

P. I. Frazier, “A tutorial on bayesian optimization,” 2018. [Online]. Available: https://arxiv.org/abs/1807.02811

2018 arXiv
[41]

Matlab version: 9.8.0 (r2020b),

T. M. Inc., “Matlab version: 9.8.0 (r2020b),” Natick, Massachusetts, United States, 2020. [Online]. Available: https://www.mathworks.com

2020
[42]

Carmaker,

IPG Automotive GmbH, “Carmaker,” Karlsruhe, Germany, 2020. [Online]. Available: https://ipg-automotive.com/products-services/simu lation-software/carmaker/

2020
[43]

Fast integrators with sensitivity propagation for use in CasADi,

J. Frey, J. De Schutter, and M. Diehl, “Fast integrators with sensitivity propagation for use in CasADi,” 2023

2023
[44]

acados – a modular open-source framework for fast embedded optimal control,

R. Verschueren, G. Frison, D. Kouzoupis, J. Frey, N. van Duijkeren, A. Zanelli, B. Novoselnik, T. Albin, R. Quirynen, and M. Diehl, “acados – a modular open-source framework for fast embedded optimal control,” Mathematical Programming Computation , 2021

2021
[45]

Virtual validation and uncertainty quantification of an adaptive model predictive controller-based motion planner for autonomous driving systems,

M. I. I. Sathyamangalam Imran, S. S. Awasthi, M. Khayyat, S. Arrigoni, and F. Braghin, “Virtual validation and uncertainty quantification of an adaptive model predictive controller-based motion planner for autonomous driving systems,” Future Transportation , vol. 4, no. 4, pp....

2024
[46]

A density-based al- gorithm for discovering clusters in large spatial databases with noise,

M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-based al- gorithm for discovering clusters in large spatial databases with noise,” in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining , ser. KDD’96. AAAI Press, 1996, p. 226–231

1996
[2008]

Available: https://onlinelibrary.wiley.com/doi/abs/10.1 002/9780470061596.risk0299

[Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1 002/9780470061596.risk0299
[2023]

Available: https://arxiv.org/abs/2310.08710

[Online]. Available: https://arxiv.org/abs/2310.08710

arXiv
[2024]

Available: https://arxiv.org/abs/2406.03040

[Online]. Available: https://arxiv.org/abs/2406.03040

arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?

N. Kalra and S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” Transportation Research Part A: Policy and Practice , vol. 94, pp. 182–193, 2016. [Online]. Available: https://www.sciencedirect.com/scie nce/article/pii/S0965856416302129

work page 2016

[2] [2]

A system- atic approach towards the definition of the terms edge case and corner case for automated driving,

L. Vater, M. Sonntag, J. Hiller, P. Schaudt, and L. Eckstein, “A system- atic approach towards the definition of the terms edge case and corner case for automated driving,” in 2023 3rd International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), 2023, pp. 1–6

work page 2023

[3] [3]

Finding critical scenarios for automated driving systems: A systematic mapping study,

X. Zhang, J. Tao, K. Tan, M. T ¨orngren, J. M. G. S ´anchez, M. R. Ramli, X. Tao, M. Gyllenhammar, F. Wotawa, N. Mohan, M. Nica, and H. Felbinger, “Finding critical scenarios for automated driving systems: A systematic mapping study,” IEEE Trans. Softw. Eng., vol. 49, no. 3, p. 991–1026, 2023. [Online]. Available: https://doi.org/10.1109/TSE.2022.3170122

work page arXiv 2023

[4] [4]

Challenges in autonomous vehicle testing and validation,

P. Koopman and M. Wagner, “Challenges in autonomous vehicle testing and validation,” SAE Int. J. Trans. Safety , 2016

work page 2016

[5] [5]

Virtual testing of automated driving systems. a survey on validation methods,

R. Don `a and B. Ciuffo, “Virtual testing of automated driving systems. a survey on validation methods,” IEEE Access, vol. 10, pp. 24 349–24 367, 2022

work page 2022

[6] [6]

On the Need for a Statistical Foundation in Scenario-Based Testing of Autonomous Vehicles

X. Zhao, R. Aghazadeh-Chakherlou, C.-H. Cheng, P. Popov, and L. Strigini, “On the need for a statistical foundation in scenario- based testing of autonomous vehicles,” 2025. [Online]. Available: https://arxiv.org/abs/2505.02274

work page Pith review arXiv 2025

[7] [7]

Addendum 156 – un regulation no. 157 uniform provisions concerning the approval of vehicles with regards to automated lane keeping system,

United Nations Economic Commission for Europe, “Addendum 156 – un regulation no. 157 uniform provisions concerning the approval of vehicles with regards to automated lane keeping system,” 2021, accessed: 2024-06-10. [Online]. Available: https: //unece.org/transport/documents/2021/03/standards/un-regulation-no-1 57-automated-lane-keeping-systems-alks

work page 2021

[8] [8]

for Standardization of Automation and M

A. for Standardization of Automation and M. Systems, Evolving Land- scapes Of Collaborative Testing For Adas & Ad . ASAM e.V ., 2022

work page 2022