REVIEW 3 major objections 5 minor 29 references
Systems Engineering for Autonomous Vehicles; Supervising AI using Large Language Models (SSuperLLM)
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper claims that LLMs can carry out real systems-engineering work for autonomous vehicles—requirements writing, requirements auditing, and runtime supervisory control—and demonstrates the supervisory piece in a simulation.
desk verdict A transparent, honest proof-of-concept for LLM-based supervisory control of an AV, but the evidence is a single run and the known rule ambiguity is never actually tested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is an LLM-based supervisory monitor layered over a conventional vehicle controller. The architecture has three working parts: an offboard-generated approved behavior database of atomic, self-contained rules; a context-translation layer that converts numerical state (longitudinal distance, lateral offset, speed, braking distances, predicted pedestrian position) into a natural-language prompt; and an LLM API that returns a fixed-format command like 'Req=2, accel=-2, nudge=1' which the simulation parses. The lower-level plant is a 3-DOF planar bicycle model with time-varying LQR lateral control, and the rules encode speed-maintenance, soft/medium/hard/full braking, and nudging behavior for jaywalking pedestrians. The fixed-response format is what makes the loop testable: the LLM's language output is constrained so a program can act on it deterministically.
What would settle it
Run the supplied simulation with scenario parameters that place the expected pedestrian position exactly at the boundary between 'close crossing' and 'in the middle of the road', and tabulate the LLM's returned requirement over many runs; if the wrong braking command appears at an appreciable rate even with the atomic rule set, the claim of reliable supervisory control fails.
Extended reading notes
Core claim
The paper's central claim is that an LLM can serve as a usable supervisory controller for an autonomous vehicle—not by driving directly, but by turning a context translation of the current vehicle and pedestrian state into a natural-language query, matching that query against an approved set of atomic requirements, and returning a fixed-format action command that a lower-level controller executes. At a 0.5 s call period, this worked in simulation. At a 2 s call period, it failed and produced a virtual vehicle-pedestrian overlap. The paper also reports that an earlier, less carefully worded rule set caused the LLM to confuse 'close crossing' with 'in the middle of the road' and issue a hard-brake command where soft braking was intended, which the paper treats as evidence that requirements must be written explicitly and atomically. The stated conclusion is that LLMs are a productivity multiplier for systems engineering, not a replacement for humans.
Load-bearing premise
The scheme works only if the LLM reliably maps a plain-language description of the situation to the correct rule whenever the rules are written atomically and self-containedly; the paper's own example shows that near-threshold wording can break this mapping.
Editorial extensions
If this is right
- At a 0.5 s LLM decision period, the supervisory monitor produced correct braking and nudging commands through the whole simulation and no vehicle-pedestrian overlap occurred.
- At a 2 s LLM decision period, the same supervisor missed the developing situation and a virtual overlap between vehicle and pedestrian occurred, so supervision frequency is a hard constraint.
- Requirements for the LLM must be atomic, self-contained, clear, measurable, unambiguous, and non-overlapping; an earlier rule formulation made the LLM confuse 'close crossing' with 'in the middle of the road' and issue the wrong braking command.
- The same LLM capability can audit human-written requirements against criteria such as specificity and measurability, and can help decompose a parent requirement into children whose satisfaction implies the parent is satisfied.
- The paper's own conclusion is that these uses make LLMs a productivity multiplier for systems engineering, with humans still required to write and approve rules before the system is trusted.
Reading between the lines
- Reader's inference: the runtime LLM in this design is doing rule classification rather than traffic reasoning; once the rules are fixed, the same loop could run with a deterministic parser, so the LLM's distinctive value is probably in generating and auditing the rule set rather than in live decision-making.
- Testable extension: rewrite the state predicates with explicit numeric thresholds and rerun the near-threshold cases from Section 5.2.3; if the errors disappear, the bottleneck is wording, not the LLM's capability.
- Design implication: the 2 s failure means a supervisory monitor needs a guaranteed response-latency budget, so the LLM's variable latency becomes a safety parameter that must be bounded before deployment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes using Large Language Models (LLMs) throughout the systems-engineering lifecycle of autonomous vehicles: requirements development, requirements audit, functional decomposition, and online supervisory control. After advocating the architecture on general grounds, it presents a proof-of-concept in which a Gemini LLM is prompted with seven natural-language rules and, through a context-translation layer, selects acceleration and nudge commands for a simulated bicycle-model vehicle with an LQR lateral controller. The paper reports a successful run with a 0.5 s LLM call period, a failed run with a 2 s period, and a documented earlier failure in which the LLM confused "close crossing" with "in the middle of the road," producing an incorrect braking command. The discussion concludes that the proof-of-concept shows the hypothesis is true, while cautioning that more research is needed.
Significance. If the central claim were established, the paper would make a useful contribution to explainable supervisory control for autonomous vehicles: an external LLM could audit rule-based behavior selections and produce human-readable justifications, and LLM-assisted requirements work would accelerate a bottleneck in systems engineering. The paper also has practical merits: the simulation is transparent, the vehicle dynamics and LQR design are standard, the source of the demonstrated failure is honestly reported, and the Colab software is made openly available. The evidence, however, is not yet commensurate with the conclusion. The successful run is a single trajectory that does not exercise the previously failing semantic distinction, and no quantitative evaluation of rule-selection accuracy or safety margin is provided. The significance of the idea is therefore real but the support is preliminary.
major comments (3)
- [5.2.1 and 5.2.3] The successful 0.5 s run does not exercise the rules that the LLM previously failed. The reported LLM interactions select only Req=2 and Req=7, and no state description in Section 5.2.1 contains the phrase "expected to be in the middle of the road," which would trigger Req=4 or Req=5. Section 5.2.3 documents that in the older rule formulation the LLM mapped a "close crossing" state to "Req.3, accel=-4,nudge=no," i.e., it applied medium braking instead of soft braking with a nudge. The revised rules in Section 5.1 change the wording but provide no evidence that the ambiguity is resolved, because the successful run simply avoids the problematic states. Since the confused state corresponds to a pedestrian partially across the road, the central claim that the LLM reliably supervises AV behavior is not established by the presented data.
- [5.2 and 6] The empirical basis for the proof-of-concept is a single successful trajectory. The paper reports one run at LLMCallPeriod_s=0.5 and one failed run at LLMCallPeriod_s=2, with no statistical replication, no sensitivity analysis over prompts or rule phrasings, no comparison against a non-LLM baseline, and no quantitative safety metric such as minimum separation, time-to-collision, or rule-selection accuracy. The statement in Section 6 that "the proof-of-concept shows that our hypothesis for LLM and SysEng is true" is therefore too strong. The authors should either add a systematic evaluation over multiple runs and states or temper the conclusion to a preliminary demonstration.
- [5.1 and 5.2.2] The comparison between the 0.5 s and 2 s runs conflates the LLM's rule-selection quality with the supervisory control period. The 2 s run shows a "virtual spatial overlap" between vehicle and pedestrian, but this could be due to the low decision rate rather than to a failure of the LLM to select the correct rule. The LLM's mapping from state text to rule should be evaluated independently of the closed-loop control period, for example by presenting the full set of relevant states offline and measuring classification accuracy against a ground-truth labeling.
minor comments (5)
- [5.2.2] The text says "Refer to Fig. 7" for the LLMCallPeriod_s=2 case, but the relevant figure is Fig. 8; this makes the failed run harder to locate.
- [3.2] The phrase "It is truly impressive!" is informal for a journal; a neutral description of the audit output would be more appropriate.
- [4] The list of contributions is introduced as "following steps a) to d)" but the items are numbered 1 to 4; the cross-reference should be harmonized.
- [5.1] The rules use semantically loaded terms such as "close crossing" and "in the middle of the road" without an explicit operational definition; stating the intended geometric or temporal thresholds would help readers assess whether the revised rules are genuinely more precise.
- [7.2] The LQR weights Q and R are given in the footnote to the code snippet, but they are not included in the main text or in a parameter table; moving them into the appendix text would improve reproducibility.
Circularity Check
No circular reduction: the LLM is an external oracle, and the rules and commands are set before the run rather than fitted to the outcome.
full rationale
I found no circular step that fits the required patterns. The proof-of-concept is empirical: the LLM (Gemini) is invoked through an API and its text responses are parsed into acceleration and nudge commands; the seven rules and command strings in Section 5.1 are authored before the simulation and are not fitted to make the run succeed. The vehicle dynamics and LQR in Section 7 are standard and independent of the LLM. Most informative is Section 5.2.3, where the same style of prompt produced a wrong selection (the LLM answered Req.3 where the paper expected the harder-braking rule), showing that the LLM output is not forced by construction. Section 6 further concedes that LLM maturity and atomic, self-contained rules matter and that caution is needed before definitive conclusions. The self-citation to reference [9] for the SysEng V-model and parent-child requirement decomposition is background methodology, not load-bearing evidence for the empirical claim. The skeptic concern that the successful run does not exercise the ambiguous middle-of-the-road states is a validation-gap argument, not a circularity argument, so it does not raise the circularity score.
Assumptions & free parameters
free parameters (2)
- LQR weights Q and R =
Q=diag([0.5, 0.3, 0, 0.3]), R=[5]
- Deceleration levels for braking (soft, medium, hard, very hard) =
-2, -4, -6, -8 m/s^2
assumptions (5)
- domain assumption Bicycle model (3-DOF) with linear tire stiffness is a sufficient representation of the vehicle for this proof-of-concept.
- standard math The LQR with the linearized model (8) provides adequate lateral control for the simulated scenarios.
- domain assumption A well-formed, atomic requirement set is unambiguous to an LLM, so the LLM will select the correct rule.
- domain assumption SysEng decomposition principle: if all child requirements are met, the parent requirement is met.
- domain assumption The simulation scenario (two jaywalking pedestrians, fixed speeds) is representative enough to test the supervisory concept.
Cite this review
Pith. "Pith review of Systems Engineering for Autonomous Vehicles; Supervising AI using Large Language Models (SSuperLLM)." pith.science (2026). https://pith.science/paper/3LVQBYDQ
@misc{pith2026250110839,
author = {Pith},
title = {Pith review of: Systems Engineering for Autonomous Vehicles; Supervising AI using Large Language Models (SSuperLLM)},
year = {2026},
howpublished = {\url{https://pith.science/paper/3LVQBYDQ}},
note = {Machine review of arXiv:2501.10839}
}
read the original abstract
Generative Artificial Intelligence (GAI) and the idea to use hierarchical models has been around for some years now. GAI has proved to be an extremely useful tool for Autonomous Vehicles (AVs). AVs need to perform robustly in their environment. Thus the AV behavior and short-term trajectory planning needs to be: a) designed and architected using safeguarding and supervisory systems and b) verified using proper Systems Engineering (SysEng) Principles. Can AV Systems Engineering also use Large Language Models (LLM) to help Autonomous vehicles (AV) development? This reader-friendly paper advocates the use of LLMs in 1) requirements (Reqs) development and 2) Reqs verification and 3) provides a proof-of-concept of AV supervisory control. The latter uses a simulation environment of a simple planar (bicycle) vehicle dynamics model and a Linear Quadratic Regulator (LQR) control with an LLM Application Interface (API). The Open-Source simulation SW is available from the author accessible to the readers so that they can engage into the AV stack, LLM API and rules, SysEng and Reqs and fundamental vehicle dynamics and control.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Nick Webb, Dan Smith, Christopher Ludwick, Trent Victor, Qi Hommes, Francesca Favaro, George Ivanov, and Tom Daniel. Waymo’s safety methodologies and safety readiness determinations.arXiv preprint arXiv:2011.00054, 2020
arXiv 2011
-
[2]
Framework for a conflict typology including contributing factors for use in ads safety evaluation
Kristofer D Kusano, John M Scanlon, Mattias Brännström, Johan Engström, and Trent Victor. Framework for a conflict typology including contributing factors for use in ads safety evaluation. In 27th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration, number 23-0328, 2023
work page 2023
-
[3]
Hierarchical model-based imitation learning for planning in autonomous driving
Eli Bronstein, Mark Palatucci, Dominik Notz, Brandyn White, Alex Kuefler, Yiren Lu, Supratik Paul, Payam Nikdel, Paul Mougin, Hongge Chen, et al. Hierarchical model-based imitation learning for planning in autonomous driving. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8652–8659. IEEE, 2022
work page 2022
-
[4]
Efficient information fusion and trajectory aggregation for behavior prediction.URL https://arxiv
B Varadarajan, A Hefny, A Srivastava, KS Refaat, N Nayakanti, A Cornman, K Chen, B Douillard, CP Lam, D Anguelov, et al. Efficient information fusion and trajectory aggregation for behavior prediction.URL https://arxiv. org/abs/2111.14973, 2021
arXiv 2021
-
[5]
Probabilistic prediction of vehicle semantic intention and motion
Yeping Hu, Wei Zhan, and Masayoshi Tomizuka. Probabilistic prediction of vehicle semantic intention and motion. In 2018 IEEE Intelligent Vehicles Symposium (IV), pages 307–313. IEEE, 2018
work page 2018
-
[6]
Embedding synthetic off-policy experience for autonomous driving via zero-shot curricula
Eli Bronstein, Sirish Srinivasan, Supratik Paul, Aman Sinha, Matthew O’Kelly, Payam Nikdel, and Shimon Whiteson. Embedding synthetic off-policy experience for autonomous driving via zero-shot curricula. In Conference on Robot Learning, pages 188–198. PMLR, 2023
work page 2023
-
[7]
Systems engineering for its handbook - section 3 what is systems engineering?, 2023
Federal Highway Administration US Department of Transportation. Systems engineering for its handbook - section 3 what is systems engineering?, 2023. 14 SSuperLLM
work page 2023
-
[8]
Safe by design autonomous driving systems
Marius Bozga and Joseph Sifakis. Safe by design autonomous driving systems. arXiv preprint arXiv:2405.11995, 2024
arXiv 2024
Show all 29 references
-
[9]
Vehicle dynamics and suspension design using systems engineering
Georgios Gatos, Spyridon Karakostas, Andreas Agiotis, and Diomidis Katzourakis. Vehicle dynamics and suspension design using systems engineering. In2024 32nd Mediterranean Conference on Control and Automation (MED), pages 328–333. IEEE, 2024
2024
-
[10]
A survey of algorithms for black-box safety validation of cyber-physical systems
Anthony Corso, Robert Moss, Mark Koren, Ritchie Lee, and Mykel Kochenderfer. A survey of algorithms for black-box safety validation of cyber-physical systems. Journal of Artificial Intelligence Research, 72:377–428, 2021
2021
-
[11]
Trustworthy autonomous system development
Joseph Sifakis and David Harel. Trustworthy autonomous system development. ACM Transactions on Embedded Computing Systems, 22(3):1–24, 2023
2023
-
[12]
Driving with llms: Fusing object-level vector modality for explainable autonomous driving
Long Chen, Oleg Sinavski, Jan Hünermann, Alice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton. Driving with llms: Fusing object-level vector modality for explainable autonomous driving. In 2024 IEEE International Conference on Robotics and Automa...
2024
-
[13]
Advancing requirements engineering through generative ai: Assessing the role of llms
Chetan Arora, John Grundy, and Mohamed Abdelrazek. Advancing requirements engineering through generative ai: Assessing the role of llms. In Generative AI for Effective Software Development, pages 129–148. Springer, 2024
2024
-
[14]
Requirements engineering and large language models: Insights from a panel
Markus Borg. Requirements engineering and large language models: Insights from a panel. IEEE Software, 41(2):6–10, 2024
2024
-
[15]
Normative requirements operational- ization with large language models
Nick Feng, Lina Marsso, Sinem Getir Yaman, Isobel Standen, Yesugen Baatartogtokh, Reem Ayad, Victória Oldem- burgo de Mello, Beverley Townsend, Hanne Bartels, Ana Cavalcanti, et al. Normative requirements operational- ization with large language models. In 2024 IEEE 32nd Inter...
2024
-
[16]
Lessons from the use of natural language inference (nli) in requirements engineering tasks
Mohamad Fazelnia, Viktoria Koscinski, Spencer Herzog, and Mehdi Mirakhorli. Lessons from the use of natural language inference (nli) in requirements engineering tasks. arXiv preprint arXiv:2405.05135, 2024
2024 arXiv
-
[17]
Sae levels of driving automation ™ refined for clarity and international audience
I SAE. Sae levels of driving automation ™ refined for clarity and international audience. https://www.sae.org/blog/sae-j3016-update, 2021
2021
-
[18]
Motion planning constraints for autonomous vehicles, January 25 2024
Diomidis Katzourakis, John P Alsterda, Rami Y Hindiyeh, Robert Chen, and Carson Schultz. Motion planning constraints for autonomous vehicles, January 25 2024. US Patent App. 17/894,812
2024
-
[19]
Autonomous systems–an architectural characterization
Joseph Sifakis. Autonomous systems–an architectural characterization. Models, Languages, and Tools for Concurrent and Distributed Programming: Essays Dedicated to Rocco De Nicola on the Occasion of His 65th Birthday, pages 388–410, 2019
2019
-
[20]
A review of motion planning techniques for automated vehicles
David González, Joshué Pérez, Vicente Milanés, and Fawzi Nashashibi. A review of motion planning techniques for automated vehicles. IEEE Transactions on intelligent transportation systems, 17(4):1135–1145, 2015
2015
-
[21]
Active suspension system with energy storage device, October 27 2020
Diomidis Katzourakis, Christopher L Porritt, Johannes A Huennekens, Huibert Mees, and Paul J Keas. Active suspension system with energy storage device, October 27 2020. US Patent 10,814,690
2020
-
[22]
Steer-by-wire system with multiple steering actuators, September 29 2020
Diomidis Katzourakis, Huibert Mees, and Paul W Choin. Steer-by-wire system with multiple steering actuators, September 29 2020. US Patent 10,787,192
2020
-
[23]
Interactive motion planning for autonomous vehicles with joint optimization
Yuxiao Chen, Sushant Veer, Peter Karkus, and Marco Pavone. Interactive motion planning for autonomous vehicles with joint optimization. arXiv preprint arXiv:2310.18301, 2023
2023 arXiv
-
[24]
Council post: Safety of the intended functionality (sotif) for autonomous driv- ing
Shaoshan Liu. Council post: Safety of the intended functionality (sotif) for autonomous driv- ing. https://www.forbes.com/sites/forbestechcouncil/2022/09/23/safety-of-the-intended-functionality-sotif-for- autonomous-driving/, 2022
2022
-
[25]
Iso 26262-1:2018, road vehicles — functional safety, part 1: V ocabulary
ISO. Iso 26262-1:2018, road vehicles — functional safety, part 1: V ocabulary. https://www.iso.org/standard/68383.html, 2018
2018
-
[26]
A survey on the explainability of supervised machine learning
Nadia Burkart and Marco F Huber. A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research, 70:245–317, 2021
2021
-
[27]
Driving simulator parameteriza- tion using double-lane change steering metrics as recorded on five modern cars
Diomidis Katzourakis, Joost CF de Winter, Stefan de Groot, and Riender Happee. Driving simulator parameteriza- tion using double-lane change steering metrics as recorded on five modern cars. Simulation Modelling Practice and Theory, 26:96–112, 2012
2012
-
[28]
Vehicle dynamics and control
Rajesh Rajamani. Vehicle dynamics and control. Springer Science & Business Media, 2011
2011
-
[29]
Dynamic programming and optimal control: Volume I, volume 4
Dimitri Bertsekas. Dynamic programming and optimal control: Volume I, volume 4. Athena scientific, 2012. 15
2012
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.