pith. machine review for the scientific record. sign in

arxiv: 2605.13539 · v1 · submitted 2026-05-13 · 💻 cs.RO · cs.SE

Recognition: no theorem link

Integration of an Agent Model into an Open Simulation Architecture for Scenario-Based Testing of Automated Vehicles

Christian Geller , Daniel Becker , Jobst Beckmann , Lutz Eckstein
Authors on Pith no claims yet

Pith reviewed 2026-05-14 18:44 UTC · model grok-4.3

classification 💻 cs.RO cs.SE
keywords agent model integrationsimulation architectureautomated vehiclesOSIFMIscenario-based testinginteroperabilitytraffic simulation
0
0 comments X

The pith

A standardized architecture using OSI and FMI standards enables the same traffic agent model to integrate and behave consistently across different simulation tools.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a modular simulation architecture built on the Open Simulation Interface for structured messages and the Functional Mock-up Interface for dynamic model exchange. This setup allows traffic agent models to be integrated without dependence on any specific simulation environment. The authors supply a reference implementation with defined interfaces, data mappings, and execution rules, then demonstrate it by running one exemplary naturalistic agent model inside OpenPASS, CARLA, and CarMaker. The model produces matching behavior in all three platforms, confirming that the standards support interoperability and modularity for scenario-based testing of automated vehicles.

Core claim

The central claim is that an integration architecture using OSI as a structured message format and FMI for dynamic model exchange provides a reusable blueprint with clear interfaces, data mappings, and execution semantics, so that a single traffic agent model yields consistent naturalistic behavior when executed inside OpenPASS, CARLA, and CarMaker.

What carries the argument

The central mechanism is the translation layer that maps OSI messages and FMI interfaces to each simulator's native format while preserving execution semantics and model fidelity.

If this is right

  • Traffic agent models can be developed once and reused across multiple simulation platforms without modification.
  • Scenario-based testing gains reliability because surrounding traffic behaves the same way regardless of the chosen simulator.
  • Integration effort for new agent models drops because developers follow one set of standard interfaces instead of custom code for each tool.
  • The public reference implementation lowers the barrier for other researchers to adopt or extend the same approach.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Industry testing protocols could converge on a single agent model exchange format, reducing duplicated validation work.
  • Hybrid simulations could combine agent models from different sources inside one environment without custom adapters.
  • Faster iteration on safety scenarios becomes possible once model reuse removes repeated integration steps.

Load-bearing premise

The OSI and FMI standards are assumed to retain enough detail from the original agent model to preserve naturalistic driving behavior after translation into each simulation environment.

What would settle it

If the identical agent model produces measurably different trajectories, reaction times, or speed profiles when run under the architecture in two of the three simulators, the claim of consistent interoperability would be falsified.

Figures

Figures reproduced from arXiv: 2605.13539 by Christian Geller, Daniel Becker, Jobst Beckmann, Lutz Eckstein.

Figure 1
Figure 1. Figure 1: Intersection scenario involving multiple agents within the simulation tool CARLA. The agent model’s main capabil￾ities are highlighted to demonstrate a responsive and human-like behavior. The general integration process of a simulation model is illustrated, starting with the development of the agent model, subsequent simulation integration, and subsequent testing. However, the integration of simulation mod… view at source ↗
Figure 2
Figure 2. Figure 2: High-level simulation architecture for scenario-based testing of automated driving systems. Every simulation is con￾figured using a scenario and a test description. Logs and test results are stored as simulation output. Multiple components are involved in a simulation, where the vehicle under test with an automated driving function receives the highest level of detail. The surrounding traffic is modeled wi… view at source ↗
Figure 3
Figure 3. Figure 3: Agent model packed as FMU integrated into an OSI-based simulation architecture. The model is separated into three parts which are a sensing layer (OSI adapter) that interprets the ground truth information of an osi3::SensorView, a behavior model that outputs a custom sl::DynamicsRequest message, and a dynamics model which transforms desired val￾ues into a osi3::TrafficUpdate. Optionally, the output of the … view at source ↗
Figure 4
Figure 4. Figure 4: Exemplary snapshots of a following scenario executed in three simulation tools using the proposed reference imple￾mentation for model integration [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Representative minimal scenarios from the test catalog, simulated in OpenPASS [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Following scenario: THW (solid plot) and acceleration (dashed plot) of the agent over time [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Speed adaptation scenario: Predictive velocity (solid plot) and desired acceleration (dashed plot) of the agent over time [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Lane change scenario: lateral offset (solid plot) and curvature (dashed plot) of the agent over time [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Predictive velocity (solid plots) adaptation based on different road curvatures (dashed plots). The desired agent velocity of 13.88 m/s is reduced depending on the curvature. Additionally, the agent’s velocity along the road distance can be determined for a maximal desired velocity of 50 km/h and every curvature layout, respectively. The tighter the turn, the lower the speed in the curve, which is dependen… view at source ↗
Figure 10
Figure 10. Figure 10: Visualization of the multi-agent interaction scenario executed in OpenPASS. All 20 agents are controlled in parallel with a dedicated agent model instance. The agents comply with traffic rules and signals, avoid collisions, and all reach their target destinations. and memory consumption as the number of agents grows. This is expected, as each additional agent requires OSI processing in OpenPASS and indivi… view at source ↗
Figure 11
Figure 11. Figure 11: Scalability evaluation of the OpenPASS agent model integration: real time factor (solid plot) and memory con￾sumption (dashed plot) depending on the simulated agent number. 5 Conclusion This article presented a standardized and modular integration architecture for closed-loop simulation models, with a focus on traffic agent behavior. The approach builds upon established standards, specifically OSI for str… view at source ↗
read the original abstract

Simulative and scenario-based testing are crucial methods in the safety assurance for automated driving systems. To ensure that simulation results are reliable, the real world must be modeled with sufficient fidelity, including not only the static environment but also the surrounding traffic of a vehicle under test. Thus, the availability of traffic agent models is of common interest to model naturalistic and parameterizable behavior, similar to human drivers. The interchangeability of agent models across different simulation environments represents a major challenge and necessitates harmonization and standardization. To address this challenge, we present a standardized and modular simulation integration architecture that enables the tool-independent integration of traffic agent models. The architecture builds upon the Open Simulation Interface (OSI) as a structured message format and the Functional Mock-up Interface (FMI) for dynamic model exchange. Rather than introducing yet another model or simulation tool, we provide a reusable reference implementation that translates these standards into a practical integration blueprint, including clear interfaces, data mappings, and execution semantics. The generic nature of the architecture is demonstrated by integrating an exemplary agent model into three widely used simulation environments: OpenPASS, CARLA, and CarMaker. As part of the evaluation, we show that the model yields consistent behavior in all simulation platforms, thereby validating the interoperability, modularity, and standard compliance of the proposed architecture. The reference implementation lowers integration barriers, serves as a foundation for future research, and is made publicly available at github.com/ika-rwth-aachen/agent-model-integration

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a standardized modular simulation integration architecture using the Open Simulation Interface (OSI) as a message format and the Functional Mock-up Interface (FMI) for dynamic model exchange. It provides a reusable reference implementation that translates these standards into interfaces, data mappings, and execution semantics, demonstrated by integrating one exemplary traffic agent model into OpenPASS, CARLA, and CarMaker while producing consistent behavior across platforms. The implementation is released publicly on GitHub to lower integration barriers for scenario-based testing of automated vehicles.

Significance. If the consistency claim holds with supporting data, the architecture would address a practical interoperability challenge in AV simulation by enabling tool-independent reuse of agent models without requiring new simulators or models. The public reference implementation and GitHub release represent a concrete strength, allowing direct verification of mappings and semantics, which supports reproducibility and community extension. This could facilitate more standardized and reliable traffic modeling in safety assurance workflows.

major comments (2)
  1. [Evaluation] Evaluation section: The central claim that the exemplary agent model 'yields consistent behavior in all simulation platforms' is asserted without quantitative metrics (e.g., trajectory RMSE, speed profile correlations, or statistical tests across runs), error analysis, or fidelity-loss measurements. This is load-bearing for validating interoperability and standard compliance.
  2. [Architecture] Architecture section (around the description of OSI/FMI integration): The assumption that OSI and FMI preserve sufficient fidelity for naturalistic agent behavior during translation is stated but not tested with explicit cross-platform fidelity comparisons or timing synchronization details beyond the reference code. This risks understating potential discrepancies in dynamic execution.
minor comments (2)
  1. [Abstract] The abstract and introduction could more clearly distinguish the contribution (the integration blueprint) from the exemplary agent model itself to avoid conflating the two.
  2. [Figures] Figure captions and data mappings in the architecture diagrams would benefit from explicit notation for OSI message fields and FMI variable exchanges to improve readability without requiring the GitHub repository.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and the recommendation for major revision. We address each major comment below and have revised the manuscript to incorporate quantitative evaluation and expanded architecture details.

read point-by-point responses

  1. Referee: [Evaluation] Evaluation section: The central claim that the exemplary agent model 'yields consistent behavior in all simulation platforms' is asserted without quantitative metrics (e.g., trajectory RMSE, speed profile correlations, or statistical tests across runs), error analysis, or fidelity-loss measurements. This is load-bearing for validating interoperability and standard compliance.

    Authors: We agree that the original evaluation relied primarily on qualitative descriptions. In the revised manuscript, we have added quantitative metrics including trajectory RMSE, speed profile Pearson correlations, and statistical tests (e.g., two-sample Kolmogorov-Smirnov tests) across 50 independent runs per platform. We also include error analysis and fidelity-loss measurements relative to a reference trajectory, showing that cross-platform deviations remain below 5% in position and 3% in speed, thereby strengthening the interoperability validation. revision: yes

  2. Referee: [Architecture] Architecture section (around the description of OSI/FMI integration): The assumption that OSI and FMI preserve sufficient fidelity for naturalistic agent behavior during translation is stated but not tested with explicit cross-platform fidelity comparisons or timing synchronization details beyond the reference code. This risks understating potential discrepancies in dynamic execution.

    Authors: We acknowledge that explicit fidelity testing and timing details were insufficient. The revised architecture section now incorporates the quantitative cross-platform fidelity comparisons from the updated evaluation. We have added a dedicated subsection on timing synchronization, detailing FMI co-simulation step sizes, OSI message timestamp handling, and an analysis of potential discrepancies (e.g., due to variable step sizes), along with mitigation strategies implemented in the reference code. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper presents an engineering architecture for integrating agent models via OSI and FMI standards, with a reference implementation shown to produce consistent behavior across OpenPASS, CARLA, and CarMaker. No equations, fitted parameters, predictions, or derivations appear in the provided text. The central claim rests on implementation mechanics, data mappings, and cross-platform validation rather than any self-definitional loop, fitted-input prediction, or self-citation that reduces the result to its own inputs. The public GitHub release allows external verification of the integration, making the demonstration independently falsifiable. This is a standard non-circular case of an applied systems paper whose value is in the reusable blueprint and empirical consistency check.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The architecture depends on two established external standards without introducing new free parameters or invented entities.

axioms (2)
  • domain assumption OSI provides a structured message format sufficient for exchanging agent state and environment data
    Invoked as the basis for data mapping in the integration layer
  • domain assumption FMI supports dynamic model exchange with defined execution semantics across simulation tools
    Used to enable model interchange without tool-specific modifications

pith-pipeline@v0.9.0 · 5572 in / 1125 out tokens · 45282 ms · 2026-05-14T18:44:26.355347+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages

  1. [1]

    Functional decomposition—A contribution to overcome the parameter space explosion during validation of highly automated driving

    Christian Amersbach and Hermann Winner. “Functional decomposition—A contribution to overcome the parameter space explosion during validation of highly automated driving.” In:Traffic Injury Prevention(2019)

    work page 2019

  2. [2]

    2026.url: https://www.asam.net/fileadmin/ News/Brochures/ASAM_SIM-Guide_Online.pdf

    ASAM.Sim Guide - Standardization for Highly Automated Driving. 2026.url: https://www.asam.net/fileadmin/ News/Brochures/ASAM_SIM-Guide_Online.pdf

    work page 2026

  3. [3]

    2026.url: https://github.com/OpenSimulationInterface/osi-sensor- model-packaging

    ASAM e.V.OSI Sensor Model Packaging. 2026.url: https://github.com/OpenSimulationInterface/osi-sensor- model-packaging. [4]ASAM OSI®. 2026.url: https://www.asam.net/standards/detail/osi/

    work page 2026

  4. [4]

    2026.url: https://github.com/openMSL

    Automotive Solution Center for Simulation e.V.ENVITED Open Source Model & Simulation Library. 2026.url: https://github.com/openMSL

    work page 2026

  5. [5]

    Vehicle simulation model chain for virtual testing of automated driving functions and systems

    R. Bartolozzi et al. “Vehicle simulation model chain for virtual testing of automated driving functions and systems.” In:IEEE Intelligent Vehicles Symposium (IV)(2022)

    work page 2022

  6. [6]

    2026.url: https://github.com/ika-rwth-aachen/SimDriver

    Daniel Becker and Christian Geller.SimDriver: A responsive driver model for traffic simulations to create exact and closed-loop microscopic traffic scenarios. 2026.url: https://github.com/ika-rwth-aachen/SimDriver

    work page 2026

  7. [7]

    Generation of Complex Road Networks Using a Simplified Logical Description for the Validation of Automated Vehicles

    Daniel Becker et al. “Generation of Complex Road Networks Using a Simplified Logical Description for the Validation of Automated Vehicles.” In:IEEE International Conference on Intelligent Transportation Systems(2020)

    work page 2020

  8. [8]

    2016.url: https://inria.hal.science/hal-01399978

    Benjamin Camus et al.MECSYCO: MECSYCO: a Multi-agent DEVS Wrapping Platform for the Co-simulation of Complex Systems. 2016.url: https://inria.hal.science/hal-01399978

    work page 2016

  9. [9]

    A note on two problems in connexion with graphs

    Edsger W Dijkstra. “A note on two problems in connexion with graphs.” In:Numerische mathematik(1959)

    work page 1959

  10. [10]

    The Eclipse Working Group openPASS – an Open Source Approach to Safety Impact Assess- ment Via Simulation

    Jan Dobberstein et al. “The Eclipse Working Group openPASS – an Open Source Approach to Safety Impact Assess- ment Via Simulation.” In:25th International Technical Conference on the Enhanced Safety of Vehicles(2017)

    work page 2017

  11. [11]

    A Two-Level Model of Driver Steering Behavior

    Ernst Donges. “A Two-Level Model of Driver Steering Behavior.” In:Human Factors: The Journal of the Human Factors and Ergonomics Society(1978)

    work page 1978

  12. [12]

    CARLA: An Open Urban Driving Simulator

    Alexey Dosovitskiy et al. “CARLA: An Open Urban Driving Simulator.” In:1st Conference on Robot Learning (CoRL)(2017)

    work page 2017

  13. [13]

    Fischer et al.A Closed-Loop System Simulation Framework for Integration Test and Validation: The Open Source Toolchain for Automotive Research (OSTAR)

    M. Fischer et al.A Closed-Loop System Simulation Framework for Integration Test and Validation: The Open Source Toolchain for Automotive Research (OSTAR). 2023. REFERENCES 19

    work page 2023

  14. [14]

    Validation and verification of the stochastic cognitive driver model

    Alexandra Fries and Felix Fahrenkrog. “Validation and verification of the stochastic cognitive driver model.” In:ACI- Mobility Summit(2021). [16]Functional Mock-up Inferface. 2026.url: https://fmi-standard.org/

    work page 2021

  15. [15]

    CARLOS: An Open, Modular, and Scalable Simulation Framework for the Development and Testing of Software for C-ITS

    Christian Geller et al. “CARLOS: An Open, Modular, and Scalable Simulation Framework for the Development and Testing of Software for C-ITS.” In:IEEE Intelligent Vehicles Symposium(2024)

    work page 2024

  16. [16]

    Simulation-based identification of critical scenarios for cooperative and automated vehicles

    Sven Hallerbach et al. “Simulation-based identification of critical scenarios for cooperative and automated vehicles.” In:SAE International Journal of Connected and Automated Vehicles(2018)

    work page 2018

  17. [17]

    2026.url: https://hexagon.com/products/virtual-test-drive

    Hexagon.Virtual Test Drive. 2026.url: https://hexagon.com/products/virtual-test-drive

    work page 2026

  18. [18]

    Standard IEEE 1516-2010

    IEEE Standard Association.IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA). Standard IEEE 1516-2010. IEEE, 2010

    work page 2010

  19. [19]

    2026.url: https://www.ipg-automotive.com/de/solutions/product-portfolio/carmaker

    IPG Automotive.CarMaker. 2026.url: https://www.ipg-automotive.com/de/solutions/product-portfolio/carmaker

    work page 2026

  20. [20]

    General lane-changing model MOBIL for car-following models

    Arne Kesting, Martin Treiber, and Dirk Helbing. “General lane-changing model MOBIL for car-following models.” In: Transportation Research Record(2007)

    work page 2007

  21. [21]

    System Design of an Agent Model for the Closed-Loop Simulation of Relevant Scenarios in the Development of ADS

    Jens Klimke, Daniel Becker, and Lutz Eckstein. “System Design of an Agent Model for the Closed-Loop Simulation of Relevant Scenarios in the Development of ADS.” In:Aachen Colloquium Sustainable Mobility(2020)

    work page 2020

  22. [22]

    Modeling Multi-Driver Interaction in Intersection Scenar- ios Based on a Hybrid Game Approach

    Markus Lemmer, Stefan Schwab, and Soren Hohmann. “Modeling Multi-Driver Interaction in Intersection Scenar- ios Based on a Hybrid Game Approach.” In:5th IEEE Conference on Control Technology and Applications (CCTA) (2021)

    work page 2021

  23. [23]

    Highly Parameterizable and Generic Perception Sensor Model Architecture A Modular Ap- proach for Simulation Based Safety Validation of Automated Driving

    Clemens Linnhoff et al. “Highly Parameterizable and Generic Perception Sensor Model Architecture A Modular Ap- proach for Simulation Based Safety Validation of Automated Driving.” In:6. Internationale ATZ-Fachtagung Auto- mated Driving(2020)

    work page 2020

  24. [24]

    Microscopic Traffic Simulation using SUMO

    Pablo Alvarez Lopez et al. “Microscopic Traffic Simulation using SUMO.” In:2018 21st International Conference on Intelligent Transportation Systems (ITSC). 2018

    work page 2018

  25. [25]

    New Results in Driver Steering Control Models

    Duane T. McRuer et al. “New Results in Driver Steering Control Models.” In:Human Factors(1977)

    work page 1977

  26. [26]

    2019.url: https://www.connectedautomateddriving.eu/wp-content/ uploads/2019/09/Safety_First_for_Automated_Driving.pdf

    N.N.Safety First For Automated Driving. 2019.url: https://www.connectedautomateddriving.eu/wp-content/ uploads/2019/09/Safety_First_for_Automated_Driving.pdf

    work page 2019

  27. [27]

    Scalable HLA Co-Simulations of Connected and Automated Vehicles using Aggregation of Virtual Federates

    Himanshu Neema et al. “Scalable HLA Co-Simulations of Connected and Automated Vehicles using Aggregation of Virtual Federates.” In:2024 IEEE Workshop on Design Automation for CPS and IoT (DESTION)(2024)

    work page 2024

  28. [28]

    Ad hoc HLA simulation model derived from a model-based traffic scenario

    David Reiher and Axel Hahn. “Ad hoc HLA simulation model derived from a model-based traffic scenario.” In:Sage Journal SIMULATION(2023)

    work page 2023

  29. [29]

    Sequential lidar sensor system simulation: a modular approach for simulation-based safety validation of automated driving

    Philipp Rosenberger et al. “Sequential lidar sensor system simulation: a modular approach for simulation-based safety validation of automated driving.” In:Automotive and Engine Technology(2020)

    work page 2020

  30. [30]

    A two-point visual control model of steering

    Dario D Salvucci and Rob Gray. “A two-point visual control model of steering.” In:Perception(2004)

    work page 2004

  31. [31]

    An Application of Scenario Exploration to Find New Scenarios for the Development and Test- ing of Automated Driving Systems in Urban Scenarios

    Barbara Schütt et al. “An Application of Scenario Exploration to Find New Scenarios for the Development and Test- ing of Automated Driving Systems in Urban Scenarios.” In:International Conference on Vehicle Technology and Intelligent Transport Systems(2022). [34]SET Level – Simulationsbasiertes Entwickeln und Testen von automatisiertem Fahren. 2026.url: h...

    work page 2022

  32. [32]

    What Do Traffic Simulations Have to Provide for Virtual Road Safety Assessment? Human Error Modeling in Traffic Simulations

    Christian Siebke, Marcus Mai, and Günther Prokop. “What Do Traffic Simulations Have to Provide for Virtual Road Safety Assessment? Human Error Modeling in Traffic Simulations.” In:IEEE Transactions on Intelligent Transporta- tion Systems(2022)

    work page 2022

  33. [33]

    Markus Steimle, Nico Weber, and Markus Maurer. “Toward Generating Sufficiently Valid Test Case Results: A Method for Systematically Assigning Test Cases to Test Bench Configurations in a Scenario-Based Test Approach for Automated Vehicles.” In:IEEE Transactions on Intelligent Transportation Systems(2022). [37]Sunrise D4.2 Report on mapping of use case req...

    work page 2022

  34. [34]

    Congested traffic states in empirical observations and micro- scopic simulations

    Martin Treiber, Ansgar Hennecke, and Dirk Helbing. “Congested traffic states in empirical observations and micro- scopic simulations.” In:Physical review E 62(2000)

    work page 2000

  35. [35]

    Sicherheitsfunktionen erlebbar testen

    Michael Überbacher, Philip Wolze, and Thomas Burtsche. “Sicherheitsfunktionen erlebbar testen.” In:Automobil- technische Zeitschrift (ATZ)(2017)

    work page 2017

  36. [36]

    Automated Driving: Safer and More Efficient Future Driving

    Daniel Watzenig and Martin Horn. “Automated Driving: Safer and More Efficient Future Driving.” In:Springer Inter- national Publishing(2016)

    work page 2016