pith. machine review for the scientific record. sign in

arxiv: 2604.08031 · v1 · submitted 2026-04-09 · 💻 cs.RO · cs.CV

Recognition: unknown

Open-Ended Instruction Realization with LLM-Enabled Multi-Planner Scheduling in Autonomous Vehicles

Jiawei Liu , Xun Gong , Fen Fang , Muli Yang , Bohao Qu , Yunfeng Hu , Hong Chen , Xulei Yang
show 1 more author
Qing Guo
Authors on Pith no claims yet

Pith reviewed 2026-05-10 17:23 UTC · model grok-4.3

classification 💻 cs.RO cs.CV
keywords autonomous vehicleslarge language modelsmotion planninginstruction realizationmodel predictive controlhuman-machine interactionmulti-planner scheduling
0
0 comments X

The pith

LLM-generated scripts schedule multiple MPC planners to convert open-ended passenger instructions into safe autonomous vehicle controls.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a framework in which a large language model interprets natural-language instructions from passengers and produces executable scripts. These scripts dynamically schedule several model predictive control planners according to real-time vehicle feedback, then convert the resulting trajectories into control signals. The design separates slow semantic reasoning from fast vehicle control at different timescales. Experiments in a new closed-loop benchmark show higher task-completion rates than prior instruction-realization methods, fewer LLM queries, safety and compliance levels matching specialized autonomous-driving systems, and tolerance to LLM inference delays.

Core claim

The framework uses an LLM to interpret open-ended instructions and generate executable scheduling scripts that select and sequence multiple MPC-based motion planners on the basis of real-time feedback, thereby producing a transparent, traceable chain from high-level commands to low-level control signals. A closed-loop benchmark is introduced to evaluate this process. Experiments demonstrate improved task completion, reduced LLM query costs, safety and compliance comparable to specialized AD approaches, and substantial tolerance to LLM latency.

What carries the argument

LLM-enabled multi-planner scheduler that produces executable scripts to choose and switch among MPC motion planners based on real-time feedback.

Load-bearing premise

The introduced closed-loop benchmark is a sufficient proxy for the real-world challenges of open-ended instruction realization.

What would settle it

A demonstration, in either a higher-fidelity simulator or a real vehicle, that the framework fails to complete the same instructions at the reported rates or violates safety constraints that the benchmark claims are satisfied.

Figures

Figures reproduced from arXiv: 2604.08031 by Bohao Qu, Fen Fang, Hong Chen, Jiawei Liu, Muli Yang, Qing Guo, Xulei Yang, Xun Gong, Yunfeng Hu.

Figure 1
Figure 1. Figure 1: Human-Like HMI Using Open-Ended Language Instructions. Here, “open-ended” refers to diverse natural-language phrasings. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A Scheduling-Centric Framework Powered by LLM. It leverages an LLM to interpret passenger instruction and generate an [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Categorization of LLM-Driven AD Methods from a Scheduling Perspective. (a) Mode I: LLM sets system parameters at startup, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Instruction Intent Distribution. trained, probabilistic LLM from directly generating numer￾ical, safety-critical control signals [23, 33]. (ii) Decision￾Making Traceability: Human-readable scripts serve as an interface, enabling a transparent mapping from LLM tex￾tual reasoning to executed actions, simplifying inspection, debugging, and validation by developers or external audi￾tors. (iii) Safety Robustnes… view at source ↗
Figure 5
Figure 5. Figure 5: Intent Recognition Score of Various LLMs. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Key Frames in Instruction Execution Process. Animated results are available in the [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Latency Sensitivity Analysis. and Expert Progress, showcasing the potential of combining high-level LLM with low-level planners for driving tasks. Notably, it is non-trivial that our framework can execute risky instructions while matching the safety of specialized AD methods that do not follow instructions. This stems from our decoupled design, in which the fast MPC-based control loop ensures instruction r… view at source ↗
read the original abstract

Most Human-Machine Interaction (HMI) research overlooks the maneuvering needs of passengers in autonomous driving (AD). Natural language offers an intuitive interface, yet translating passenger open-ended instructions into control signals, without sacrificing interpretability and traceability, remains a challenge. This study proposes an instruction-realization framework that leverages a large language model (LLM) to interpret instructions, generates executable scripts that schedule multiple model predictive control (MPC)-based motion planners based on real-time feedback, and converts planned trajectories into control signals. This scheduling-centric design decouples semantic reasoning from vehicle control at different timescales, establishing a transparent, traceable decision-making chain from high-level instructions to low-level actions. Due to the absence of high-fidelity evaluation tools, this study introduces a benchmark for open-ended instruction realization in a closed-loop setting. Comprehensive experiments reveal that the framework significantly improves task-completion rates over instruction-realization baselines, reduces LLM query costs, achieves safety and compliance on par with specialized AD approaches, and exhibits considerable tolerance to LLM inference latency. For more qualitative illustrations and a clearer understanding.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes an instruction-realization framework for autonomous vehicles that uses an LLM to interpret open-ended passenger instructions, generates executable scripts to schedule multiple MPC-based motion planners in response to real-time feedback, and converts the resulting trajectories into control signals. The scheduling-centric architecture is intended to decouple semantic reasoning from low-level vehicle control at different timescales, yielding a transparent decision chain. Because no high-fidelity evaluation tools exist, the authors introduce a new closed-loop benchmark; experiments on this benchmark report higher task-completion rates than instruction-realization baselines, lower LLM query costs, safety and compliance comparable to specialized AD methods, and robustness to LLM inference latency.

Significance. If the benchmark faithfully captures the relevant vehicle dynamics, sensor noise, and instruction distributions, the work would offer a concrete, traceable route for integrating LLMs into safety-critical AD control loops without sacrificing interpretability. The explicit separation of timescales and the multi-planner scheduling mechanism are technically interesting contributions that could influence future HMI designs. The creation of an open benchmark also addresses a genuine evaluation gap, provided its representativeness can be established.

major comments (2)
  1. [§4] §4 (Benchmark and Evaluation Setup): All central performance claims—improved task-completion rates, reduced LLM costs, safety parity with specialized AD, and latency tolerance—are derived exclusively from experiments in the newly introduced closed-loop benchmark. The manuscript acknowledges the absence of high-fidelity tools yet supplies no quantitative validation (e.g., comparison of vehicle model order, tire/road friction, sensor noise statistics, or traffic density against real-world or high-fidelity simulator data) that the proxy reproduces the dynamics that would determine whether the reported gains transfer. This is load-bearing for every experimental conclusion.
  2. [§5.3] §5.3 (Comparative Experiments): The paper states that the framework “significantly improves task-completion rates over instruction-realization baselines,” but the results section does not report statistical significance tests, confidence intervals, or the number of independent runs per condition. Without these, it is impossible to determine whether the observed differences are robust or could be artifacts of the particular benchmark scenarios.
minor comments (2)
  1. [§3] The abstract and §3 would benefit from an explicit statement of the MPC cost functions and constraint sets used by the individual planners; this would clarify how safety and compliance are enforced at the low level.
  2. [Figure 2] Figure 2 (system architecture) caption should indicate the exact interface between the generated script and the real-time feedback loop (e.g., which state variables are passed back to the LLM scheduler).

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the detailed and constructive feedback on our manuscript. We address each major comment point by point below, indicating the revisions we will incorporate where appropriate.

read point-by-point responses

  1. Referee: [§4] §4 (Benchmark and Evaluation Setup): All central performance claims—improved task-completion rates, reduced LLM costs, safety parity with specialized AD, and latency tolerance—are derived exclusively from experiments in the newly introduced closed-loop benchmark. The manuscript acknowledges the absence of high-fidelity tools yet supplies no quantitative validation (e.g., comparison of vehicle model order, tire/road friction, sensor noise statistics, or traffic density against real-world or high-fidelity simulator data) that the proxy reproduces the dynamics that would determine whether the reported gains transfer. This is load-bearing for every experimental conclusion.

    Authors: We acknowledge that the benchmark serves as a proxy and that direct quantitative validation against real-world or high-fidelity data would strengthen transferability claims. As noted in the manuscript, high-fidelity tools are unavailable, which is why this benchmark was introduced. In the revision, we will expand §4 to include explicit parameter values and sources for the vehicle model (nonlinear bicycle model with Pacejka tire parameters from standard literature), sensor noise statistics (Gaussian variances drawn from typical AD sensor specs), and traffic scenario distributions (sampled to match NGSIM-like densities). A new limitations subsection will discuss assumptions and expected generalization conditions. However, performing side-by-side quantitative comparisons to inaccessible high-fidelity simulators or real-vehicle logs is not feasible within current resources. revision: partial

  2. Referee: [§5.3] §5.3 (Comparative Experiments): The paper states that the framework “significantly improves task-completion rates over instruction-realization baselines,” but the results section does not report statistical significance tests, confidence intervals, or the number of independent runs per condition. Without these, it is impossible to determine whether the observed differences are robust or could be artifacts of the particular benchmark scenarios.

    Authors: We agree that statistical rigor is necessary. The revised manuscript will state that all conditions were evaluated over 20 independent runs using distinct random seeds for instruction generation, initial states, and disturbances. We will report 95% confidence intervals for task-completion rates, LLM query costs, and safety metrics, along with p-values from appropriate tests (paired t-tests for normally distributed metrics or Wilcoxon signed-rank tests otherwise). These additions will appear in §5.3 and the corresponding tables. revision: yes

standing simulated objections not resolved
  • Direct quantitative validation of benchmark dynamics (model order, friction, noise, traffic) against real-world or high-fidelity simulator data, as no such accessible tools exist and obtaining them would require resources beyond the scope of this study.

Circularity Check

0 steps flagged

No circularity; experimental claims rest on introduced benchmark without reduction to inputs or self-citations

full rationale

The paper describes an LLM-based multi-planner scheduling framework for instruction realization in autonomous vehicles and reports empirical improvements from experiments in a newly introduced closed-loop benchmark. No equations, derivations, fitted parameters renamed as predictions, or load-bearing self-citations appear in the provided text that would reduce the claimed task-completion rates, cost reductions, or safety parity to the inputs by construction. The benchmark is explicitly motivated by the acknowledged absence of high-fidelity tools rather than being defined in terms of the results it produces. Per the hard rules, concerns about benchmark fidelity fall under external validity rather than circularity, as no self-referential reduction is exhibited.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Based solely on the abstract, no free parameters, axioms, or invented entities are explicitly stated. The framework relies on standard LLM capabilities and MPC planners without introducing new postulated objects.

pith-pipeline@v0.9.0 · 5510 in / 1173 out tokens · 68031 ms · 2026-05-10T17:23:29.011485+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 4 canonical work pages · 1 internal anchor

  1. [1]

    Carplay ultra, the next generation of carplay, begins rolling out today, 2025

    Apple, Inc. Carplay ultra, the next generation of carplay, begins rolling out today, 2025. Accessed: 2025-05-15. 1

    2025

  2. [2]

    Explainable artificial intelligence for autonomous driving: A comprehensive overview and field guide for future research directions.IEEE Access, 2024

    Shahin Atakishiyev, Mohammad Salameh, Hengshuai Yao, and Randy Goebel. Explainable artificial intelligence for autonomous driving: A comprehensive overview and field guide for future research directions.IEEE Access, 2024. 3

    2024

  3. [3]

    An advanced lane-keeping assistance system with switchable assistance modes.IEEE Transactions on Intelligent Transportation Systems, 21(1): 385–396, 2020

    Yougang Bian, Jieyun Ding, Manjiang Hu, Qing Xu, Jian- qiang Wang, and Keqiang Li. An advanced lane-keeping assistance system with switchable assistance modes.IEEE Transactions on Intelligent Transportation Systems, 21(1): 385–396, 2020. 1

    2020

  4. [4]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 2

    2020

  5. [5]

    Argoverse: 3d tracking and forecasting with rich maps

    Ming-Fang Chang, John Lambert, Patsorn Sangkloy, Jag- jeet Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, et al. Argoverse: 3d tracking and forecasting with rich maps. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8748–8757, 2019. 2

    2019

  6. [6]

    Omnire: Omni urban scene reconstruction

    Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Goj- cic, Sanja Fidler, Marco Pavone, et al. Omnire: Omni urban scene reconstruction. InThe Thirteenth International Con- ference on Learning Representations, 2025. 8

    2025

  7. [7]

    Li auto unveils next-gen au- tonomous driving architecture mindvla, 2025

    China Automotive News. Li auto unveils next-gen au- tonomous driving architecture mindvla, 2025. Accessed: 2025-05-18. 1

    2025

  8. [8]

    Personalized autonomous driving with large lan- guage models: Field experiments

    Can Cui, Zichong Yang, Yupeng Zhou, Yunsheng Ma, Juanwu Lu, Lingxi Li, Yaobin Chen, Jitesh Panchal, and Zi- ran Wang. Personalized autonomous driving with large lan- guage models: Field experiments. In2024 IEEE 27th Inter- national Conference on Intelligent Transportation Systems (ITSC), pages 20–27, 2024. 4

    2024

  9. [9]

    Drivemlm: aligning multi-modal large language models with behavioral planning states for autonomous driv- ing.Visual Intelligence, 3(22), 2025

    Erfei Cui, Wenhai Wang, Zhiqi Li, Jiangwei Xie, Haoming Zou, Hanming Deng, Gen Luo, Lewei Lu, Xizhou Zhu, and Jifeng Dai. Drivemlm: aligning multi-modal large language models with behavioral planning states for autonomous driv- ing.Visual Intelligence, 3(22), 2025. 4

    2025

  10. [10]

    Parting with misconceptions about learning- based vehicle motion planning

    Daniel Dauner, Marcel Hallgarten, Andreas Geiger, and Kashyap Chitta. Parting with misconceptions about learning- based vehicle motion planning. InProceedings of the Con- ference on Robot Learning, pages 1268–1281, 2023. 6

    2023

  11. [11]

    Deepseek-v3 technical report, 2025

    DeepSeek-AI and et al. Deepseek-v3 technical report, 2025. 6

    2025

  12. [12]

    Carla: An open urban driving simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. InProceedings of the Conference on Robot Learn- ing, pages 1–16. PMLR, 2017. 2

    2017

  13. [13]

    Cooperative driving us- ing a hierarchy of mixed-integer programming and tracking control

    Jan Eilbrecht and Olaf Stursberg. Cooperative driving us- ing a hierarchy of mixed-integer programming and tracking control. In2017 IEEE Intelligent Vehicles Symposium (IV), pages 673–678, 2017. 4

    2017

  14. [14]

    LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models

    Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, et al. Libero-plus: In-depth robustness analysis of vision- language-action models.arXiv preprint arXiv:2510.13626,

    work page internal anchor Pith review arXiv

  15. [15]

    Tsang, Ming-Ming Cheng, and Qing Guo

    Yuxiang Fu, Jiakun Ding, Renzhi Wang, Qian Fu, Ivor W. Tsang, Ming-Ming Cheng, and Qing Guo. Benchmark- ing drag*for eye direction transformation and beyond.Visual Intelligence, 3(29), 2025. 8

    2025

  16. [16]

    Real-time integrated power and thermal management of connected hevs based on hierarchical model predictive control.IEEE/ASME Transactions on Mechatron- ics, 26(3):1271–1282, 2021

    Xun Gong, Jieyu Wang, Baolin Ma, Liang Lu, Yunfeng Hu, and Hong Chen. Real-time integrated power and thermal management of connected hevs based on hierarchical model predictive control.IEEE/ASME Transactions on Mechatron- ics, 26(3):1271–1282, 2021. 3

    2021

  17. [17]

    Mak- ing large language models better planners with reasoning- decision alignment

    Zhijian Huang, Tao Tang, Shaoxiang Chen, Sihao Lin, Ze- qun Jie, Lin Ma, Guangrun Wang, and Xiaodan Liang. Mak- ing large language models better planners with reasoning- decision alignment. InEuropean Conference on Computer Vision, pages 73–90, 2024. 3

    2024

  18. [18]

    Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning.arXiv preprint arXiv:2503.07608, 2025

    Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, and Xing- gang Wang. Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reason- ing.arXiv preprint arXiv:2503.07608, 2025. 3

    work page arXiv 2025

  19. [19]

    A survey on vision-language- action models for autonomous driving, 2025

    Sicong Jiang, Zilin Huang, Kangan Qian, Ziang Luo, Tianze Zhu, Yang Zhong, Yihong Tang, Menglin Kong, Yunlong Wang, Siwen Jiao, Hao Ye, Zihao Sheng, Xin Zhao, Tuopu Wen, Zheng Fu, Sikai Chen, Kun Jiang, Diange Yang, Seongjin Choi, and Lijun Sun. A survey on vision-language- action models for autonomous driving, 2025. 1, 3

    2025

  20. [20]

    Towards learning- based planning: The nuplan benchmark for real-world au- tonomous driving

    Napat Karnchanachari, Dimitris Geromichalos, Kok Seang Tan, Nanxiang Li, Christopher Eriksen, Shakiba Yaghoubi, Noushin Mehdipour, Gianmarco Bernasconi, Whye Kit Fong, Yiluan Guo, and Holger Caesar. Towards learning- based planning: The nuplan benchmark for real-world au- tonomous driving. In2024 IEEE International Conference on Robotics and Automation (I...

    2024

  21. [21]

    Singular perturbations and order reduction in con- trol theory—an overview.Automatica, 12(2):123–132, 1976

    Petar V Kokotovic, Robert E O’Malley Jr, and Peddapullaiah Sannuti. Singular perturbations and order reduction in con- trol theory—an overview.Automatica, 12(2):123–132, 1976. 3

    1976

  22. [22]

    An environment for autonomous driving decision-making, 2018

    Edouard Leurent. An environment for autonomous driving decision-making, 2018. 2

    2018

  23. [23]

    Harnessing and evaluating the intrin- sic extrapolation ability of large language models for vehicle trajectory prediction

    Jiawei Liu, Yanjiao Liu, Xun Gong, Tingting Wang, Hong Chen, and Yunfeng Hu. Harnessing and evaluating the intrin- sic extrapolation ability of large language models for vehicle trajectory prediction. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (V...

    2025

  24. [24]

    2509.13769 , archivePrefix =

    Yuechen Luo, Fang Li, Shaoqing Xu, Zhiyi Lai, Lei Yang, Qimao Chen, Ziang Luo, Zixun Xie, Shengyin Jiang, Ji- axin Liu, et al. Adathinkdrive: Adaptive thinking via rein- forcement learning for autonomous driving.arXiv preprint arXiv:2509.13769, 2025. 3

    work page arXiv 2025

  25. [25]

    Integrating llms with its: Recent advances, potentials, challenges, and future directions.IEEE Trans- actions on Intelligent Transportation Systems, 26(5):5674– 5709, 2025

    Doaa Mahmud, Hadeel Hajmohamed, Shamma Almentheri, Shamma Alqaydi, Lameya Aldhaheri, Ruhul Amin Khalil, and Nasir Saeed. Integrating llms with its: Recent advances, potentials, challenges, and future directions.IEEE Trans- actions on Intelligent Transportation Systems, 26(5):5674– 5709, 2025. 1, 2, 5

    2025

  26. [26]

    Revisiting coroutines.ACM Transactions on Programming Languages and Systems (TOPLAS), 31(2):1–31, 2009

    Ana L ´ucia De Moura and Roberto Ierusalimschy. Revisiting coroutines.ACM Transactions on Programming Languages and Systems (TOPLAS), 31(2):1–31, 2009. 2

    2009

  27. [27]

    Natural language interac- tions in autonomous vehicles: Intent detection and slot fill- ing from passenger utterances

    Eda Okur, Shachi H Kumar, Saurav Sahay, Asli Ar- slan Esme, and Lama Nachman. Natural language interac- tions in autonomous vehicles: Intent detection and slot fill- ing from passenger utterances. InInternational Conference on Computational Linguistics and Intelligent Text Process- ing, pages 334–350. Springer, 2019. 2

    2019

  28. [28]

    Llm evaluators recognize and favor their own generations.Ad- vances in Neural Information Processing Systems, 37: 68772–68802, 2024

    Arjun Panickssery, Samuel Bowman, and Shi Feng. Llm evaluators recognize and favor their own generations.Ad- vances in Neural Information Processing Systems, 37: 68772–68802, 2024. 6

    2024

  29. [29]

    Conditional driving from natural language instructions

    Junha Roh, Chris Paxton, Andrzej Pronobis, Ali Farhadi, and Dieter Fox. Conditional driving from natural language instructions. InProceedings of the Conference on Robot Learning, pages 540–551, 2020. 2

    2020

  30. [30]

    Languagempc: Large language models as decision makers for autonomous driving, 2025

    Hao Sha, Yao Mu, Yuxuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, and Mingyu Ding. Languagempc: Large language models as decision makers for autonomous driving, 2025. 4

    2025

  31. [31]

    Lmdrive: Closed-loop end-to-end driving with large language models

    Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L Waslander, Yu Liu, and Hongsheng Li. Lmdrive: Closed-loop end-to-end driving with large language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15120–15130, 2024. 3

    2024

  32. [32]

    Event-triggered real-time scheduling of sta- bilizing control tasks.IEEE Transactions on Automatic con- trol, 52(9):1680–1685, 2007

    Paulo Tabuada. Event-triggered real-time scheduling of sta- bilizing control tasks.IEEE Transactions on Automatic con- trol, 52(9):1680–1685, 2007. 3

    2007

  33. [33]

    Are language models actually useful for time series forecasting?Advances in Neural Information Processing Systems, 37:60162–60191, 2024

    Mingtian Tan, Mike Merrill, Vinayak Gupta, Tim Althoff, and Tom Hartvigsen. Are language models actually useful for time series forecasting?Advances in Neural Information Processing Systems, 37:60162–60191, 2024. 5

    2024

  34. [34]

    Spatial routines for a simu- lated speech-controlled vehicle

    Stefanie Tellex and Deb Roy. Spatial routines for a simu- lated speech-controlled vehicle. InProceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interac- tion, pages 156–163, 2006. 2

    2006

  35. [35]

    Robots that use language.Annual Re- view of Control, Robotics, and Autonomous Systems, 3(1): 25–55, 2020

    Stefanie Tellex, Nakul Gopalan, Hadas Kress-Gazit, and Cynthia Matuszek. Robots that use language.Annual Re- view of Control, Robotics, and Autonomous Systems, 3(1): 25–55, 2020. 2

    2020

  36. [36]

    Tesla model 3 owner’s manual, 2025

    Tesla, Inc. Tesla model 3 owner’s manual, 2025. Accessed:

    2025

  37. [37]

    Global technology: China’s robotaxi market - the road to commercialization, 2025

    The Goldman Sachs Group, Inc. Global technology: China’s robotaxi market - the road to commercialization, 2025. Ac- cessed: 2025-05-15. 1

    2025

  38. [38]

    Con- gested traffic states in empirical observations and micro- scopic simulations.Physical review E, 62(2):1805, 2000

    Martin Treiber, Ansgar Hennecke, and Dirk Helbing. Con- gested traffic states in empirical observations and micro- scopic simulations.Physical review E, 62(2):1805, 2000. 6

    2000

  39. [39]

    The effects of lead time of take-over request and nondriving tasks on taking-over con- trol of automated vehicles.IEEE Transactions on Human- Machine Systems, 48(6):582–591, 2018

    Jingyan Wan and Changxu Wu. The effects of lead time of take-over request and nondriving tasks on taking-over con- trol of automated vehicles.IEEE Transactions on Human- Machine Systems, 48(6):582–591, 2018. 1

    2018

  40. [40]

    Chatgpt as your vehicle co-pilot: An initial attempt.IEEE Transactions on Intelligent Vehicles, 8 (12):4706–4721, 2023

    Shiyi Wang, Yuxuan Zhu, Zhiheng Li, Yutong Wang, Li Li, and Zhengbing He. Chatgpt as your vehicle co-pilot: An initial attempt.IEEE Transactions on Intelligent Vehicles, 8 (12):4706–4721, 2023. 4

    2023

  41. [41]

    A learning-based approach for lane departure warning systems with a personalized driver model.IEEE Transactions on Ve- hicular Technology, 67(10):9145–9157, 2018

    Wenshuo Wang, Ding Zhao, Wei Han, and Junqiang Xi. A learning-based approach for lane departure warning systems with a personalized driver model.IEEE Transactions on Ve- hicular Technology, 67(10):9145–9157, 2018. 1

    2018

  42. [42]

    Alpamayo-r1: Bridging reasoning and action prediction for generalizable autonomous driving in the long tail.arXiv preprint arXiv:2511.00088, 2025

    Yan Wang, Wenjie Luo, Junjie Bai, Yulong Cao, Tong Che, Ke Chen, Yuxiao Chen, Jenna Diamond, Yifan Ding, Wen- hao Ding, et al. Alpamayo-r1: Bridging reasoning and action prediction for generalizable autonomous driving in the long tail.arXiv preprint arXiv:2511.00088, 2025. 3

    work page arXiv 2025

  43. [43]

    Emergent analogical reasoning in large language models.Nature Hu- man Behaviour, 7(9):1526–1541, 2023

    Taylor Webb, Keith J Holyoak, and Hongjing Lu. Emergent analogical reasoning in large language models.Nature Hu- man Behaviour, 7(9):1526–1541, 2023. 4

    2023

  44. [44]

    A survey of joint intent detection and slot filling models in natural language understanding.ACM Computing Surveys, 55(8):1–38, 2022

    Henry Weld, Xiaoqi Huang, Siqu Long, Josiah Poon, and Soyeon Caren Han. A survey of joint intent detection and slot filling models in natural language understanding.ACM Computing Surveys, 55(8):1–38, 2022. 3

    2022

  45. [45]

    Dilu: A knowledge-driven approach to autonomous driving with large language models

    Licheng Wen, Daocheng Fu, Xin Li, Xinyu Cai, MA Tao, Pinlong Cai, Min Dou, Botian Shi, Liang He, and Yu Qiao. Dilu: A knowledge-driven approach to autonomous driving with large language models. InThe Twelfth International Conference on Learning Representations, 2024. 3, 4, 6

    2024

  46. [46]

    To- ward human-vehicle collaboration: Review and perspec- tives on human-centered collaborative automated driving

    Yang Xing, Chen Lv, Dongpu Cao, and Peng Hang. To- ward human-vehicle collaboration: Review and perspec- tives on human-centered collaborative automated driving. Transportation research part C: emerging technologies, 128: 103199, 2021. 1, 2

    2021

  47. [47]

    Qwen3 technical report, 2025

    An Yang and et al. Qwen3 technical report, 2025. 6

    2025

  48. [48]

    Diffusion-es: Gradient-free planning with diffusion for autonomous and instruction-guided driving

    Brian Yang, Huangyuan Su, Nikolaos Gkanatsios, Tsung- Wei Ke, Ayush Jain, Jeff Schneider, and Katerina Fragki- adaki. Diffusion-es: Gradient-free planning with diffusion for autonomous and instruction-guided driving. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15342–15353, 2024. 6

    2024

  49. [49]

    Ex- ploring compositional generalization of large language mod- els

    Haoran Yang, Hongyuan Lu, Wai Lam, and Deng Cai. Ex- ploring compositional generalization of large language mod- els. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies (Volume 4: Student Research Workshop), pages 16–24, 2024. 4

    2024

  50. [50]

    Deep open intent classification with adaptive decision boundary

    Hanlei Zhang, Hua Xu, and Ting-En Lin. Deep open intent classification with adaptive decision boundary. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 14374–14382, 2021. 3

    2021

  51. [51]

    Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma

    Zewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. Autovla: A vision- language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning, 2025. 3, 5

    2025