Pith. sign in

REVIEW 5 major objections 5 minor 49 references

SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sharing a visual encoder between VLM and end-to-end planner plus trajectory chain-of-thought yields state-of-the-art open-loop planning on nuScenes.

desk verdict A sensible VLM-E2E architecture whose headline comparison does not survive arithmetic: the claimed 0.03 L2 gain over DriveVLM is actually 0.12 in Table 1, and the rest of the evaluation lacks error bars and latency data. read the letter →

arxiv 2505.16805 v1 pith:RXYW34HC submitted 2025-05-22 cs.CV

Xuesong Chen , Linjiang Huang , Tao Ma , Rongyao Fang , Shaoshuai Shi , Hongsheng Li This is my paper
classification cs.CV
keywords vision-languagemodelend-to-endautonomousdrivingtrajectorychain-of-thoughtsharedvisualencodernuScenesopen-loopplanningQ-Formerbanktemporaldecoupling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SOLVE, a planning framework that couples a vision-language model (VLM) with an end-to-end (E2E) driving planner and lets them share a visual encoder instead of cooperating only after each branch has produced a trajectory. The central claim is that this feature-level synergy, together with a Trajectory Chain-of-Thought (T-CoT) that selects and refines candidate trajectories from a bank, sets a new state-of-the-art for open-loop planning on nuScenes: SOLVE-VLM reports 0.28 m average $L_2$ displacement error and 0.20% average collision rate, lower than OmniDrive and DriveVLM. If the claim holds, VLM common-sense reasoning can improve planning accuracy and collision avoidance without forcing the entire pipeline to run at VLM speed, because VLM trajectories are consumed asynchronously by the real-time E2E planner. The paper also argues the shared encoder is what lets its E2E-only branch match DriveVLM-Dual, since question-answering supervision teaches the encoder which visual cues matter for planning.

What carries the argument

The load-bearing object is the Sequential Q-Former (SQ-Former), a Q-Former-style querying module that compresses multi-view image features into 384 visual queries by sequentially aligning collector queries with whole-image cues, 3D detection queries, and lane queries, using a temporal memory bank. Sharing this module between the VLM and E2E planner is what makes feature-level synergy concrete. The second mechanism is the Trajectory Chain-of-Thought (T-CoT): a 36-entry trajectory bank built by k-means clustering on training trajectories per navigation command, top-k retrieval combined with one MLP ego-status trajectory, trajectory tokens inserted into the prompt, and a two-stage select-then-refine output. The third mechanism is temporal decoupling: the VLM predicts trajectories over a longer horizon at lower frequency, stores them in memory, and the real-time E2E planner later uses the freshest stored trajectory as an extra initialization query.

What would settle it

Run the same SOLVE models in a closed-loop simulator or with reactive agents and compare collision and progress against the open-loop nuScenes numbers; if the $L_2$ and collision gains shrink or invert once the planner's own outputs feed back into the scene, the central state-of-the-art claim is not evidence of safer real-world driving. A cheaper check is to replace the VLM's selection step in T-CoT with a similarity-based rule that picks the bank trajectory closest to the ego history; if the gap to full SOLVE-VLM is within noise, the VLM's scene reasoning is not load-bearing.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the VLM and the E2E planner should be seen as one learning system rather than two modules joined after inference. Their shared Sequential Q-Former (SQ-Former) compresses multi-view images into 384 visual tokens that both branches consume, so the VLM's perception, prediction, and planning question-answering directly shapes the features the E2E planner uses, and the E2E branch's 3D detection and lane tasks feed back into VLM reasoning. Trajectory prediction is then staged: the VLM selects the best candidate from a bank of clustered driving trajectories plus an ego-status MLP guess, and refines the chosen waypoints. The paper reports that this yields the lowest open-loop $L_2$ displacement and collision rate among compared VLM-based planners on nuScenes, and that asynchronous VLM trajectory initialization further improves the E2E planner.

Load-bearing premise

The load-bearing premise is that the nuScenes open-loop scores—average distance between the predicted and expert trajectory, and the simulated collision rate—actually capture planning quality and safety; the paper itself cites AD-MLP and BEV-Planner as baselines showing that ego status alone can reach competitive scores, so those metrics can be satisfied without robust scene understanding.

Editorial extensions

If this is right

  • If the shared encoder is the source of the gain, other VLM-plus-E2E systems could adopt feature-level sharing rather than post-processing and see comparable reductions in $L_2$ error and collision rate.
  • If T-CoT is what makes VLM waypoint generation reliable, then any autoregressive planner that struggles with numeric coordinates can be anchored to a retrieved reference trajectory before refining it.
  • If the asynchronous VLM-to-E2E initialization works at the reported temporal offset, a high-latency VLM can improve a real-time planner without forcing the whole stack to run at VLM speed.
  • If the reported numbers hold under the standard nuScenes open-loop protocol, SOLVE-VLM becomes the strongest compared VLM-based planner on that benchmark and SOLVE-E2E matches the best E2E-only planner in the table.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gap to AD-MLP is narrow (0.28 vs 0.35 m), so a sympathetic reader might still infer that most of the open-loop score is already explained by ego-history and trajectory-prior retrieval; a decisive test would be to run SOLVE-VLM in scenes where ego status is ambiguous, such as occluded intersections or unusual traffic layouts.
  • A testable extension is to reuse the trajectory bank and T-CoT with a smaller or faster VLM backbone and measure how much of the reported accuracy gain survives, which would separate the reasoning contribution from the retrieval prior.
  • The paper's own ablation implies a direct experimental check: retraining the SQ-Former with only detection and lane supervision, without the VLM QA stage, should reproduce the reported 1.5 cm $L_2$ degradation in the E2E branch; if it does not, the feature-sharing mechanism is not the cause.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper proposes SOLVE, a framework that couples a Vision-Language Model (LLaVA-1.5) with an end-to-end planner through a shared Sequential Q-Former (SQ-Former) visual encoder, a trajectory-bank-based Trajectory Chain-of-Thought (T-CoT) for coarse-to-fine VLM trajectory generation, and a temporal-decoupling memory that lets the E2E planner use asynchronously generated VLM trajectories as initialization. The authors evaluate on nuScenes open-loop planning and report lower L2 displacement and collision rates than several baselines, with ablations on query count, SQ-Former task order, T-CoT reference count, and feature sharing.

Significance. If the reported margins are robust, SOLVE is a useful step toward integrating VLM reasoning with E2E planners: the feature-level synergy through SQ-Former and the T-CoT mechanism are well motivated, and the temporal-decoupling strategy is a principled way to address VLM latency. The ablation study is internally consistent, and the trajectory bank is constructed from the training split rather than the evaluation labels, so I do not see a circularity problem in the planning metric. However, the external validity of open-loop nuScenes L2/collision as a safety proxy is questionable, and the paper's own Table 1 shows ego-status-only baselines are competitive, so the significance of the headline SOTA claim is currently limited to the open-loop benchmark.

major comments (5)
  1. [Section 4.2, Table 1: the claimed margins are inconsistent.] The text states that SOLVE-VLM surpasses OmniDrive and DriveVLM by 0.05 and 0.03 in L2 error, but Table 1 lists DriveVLM at 0.40 and SOLVE-VLM at 0.28, a difference of 0.12. The value 0.03 matches the gap to DriveVLM-Dual (0.31). This misidentification of the baseline affects the paper's central 'state-of-the-art' claim and must be corrected.
  2. [Section 4.2, Table 1: no statistical support is provided.] No standard errors, confidence intervals, multiple-seed runs, or paired significance tests are reported. The key margins are small in absolute terms (0.03 m L2, or 0.10 percentage points in collision rate, which corresponds to a few events on roughly 6,000 evaluation samples), so run-to-run variability could change the conclusions. Please provide variance estimates or significance tests for Table 1 and the main ablations.
  3. [Abstract and Section 3.3: the real-time claim is not measured.] The paper repeatedly claims real-time or efficient operation, and the temporal-decoupling strategy is motivated by VLM latency, but no runtime or latency measurements are reported. Please include inference times for SOLVE-VLM and SOLVE-E2E, the asynchronous delay, and the hardware used; otherwise the efficiency claim is unverified.
  4. [Section 4.2, Table 1: the open-loop metric is acknowledged to be a weak proxy.] The paper itself cites AD-MLP and BEV-Planner, which use only ego status, as achieving competitive L2 and collision results. This is an explicit limitation of the open-loop nuScenes benchmark as a proxy for planning quality. Because the conclusion about 'more robust and reliable autonomous driving' rests on these metrics, please add closed-loop evaluation (e.g., nuPlan) or, at minimum, a quantitative analysis of when SOLVE meaningfully differs from ego-status-only baselines.
  5. [Section 4.4, Table 4: the T-CoT ablation does not isolate the retrieval prior.] The with/without-CoT comparison shows a 1.1 cm improvement, but it does not separate the contribution of the trajectory-bank retrieval from the VLM's chain-style reasoning. A control where the best-retrieved trajectory is used directly without VLM refinement, or where the VLM selects among random trajectories, is needed to support the claim that T-CoT itself, rather than the trajectory prior, drives the improvement.
minor comments (5)
  1. [Section 4.1 and Tables 2-5: units are inconsistent.] Table 1 uses meters, while Tables 2-5 use centimeters without clear emphasis. Please state the unit in each table caption or convert to a single consistent unit.
  2. [Section 3.2: the notation for trajectory selection is confusing.] The text uses 'kl' and 'kl+1' for the number of retrieved trajectories, which reads as a product of k and l. Please use a single symbol, e.g., k, and define it clearly.
  3. [Table 1: the ego-status condition should be made explicit in the text.] The comparison text refers to 'BEV-planner' while the table distinguishes BEV-Planner and BEV-Planner++; since the table also mixes methods with and without ego status, the authors should state which rows are being compared and keep all SOTA comparisons within the same ego-status condition.
  4. [Front matter and references: typos and formatting issues should be cleaned.] Examples include 'V ision', 'V oyager Research', 'LLaV A v1.5', and the malformed author list in reference [2]. These do not affect the technical content but should be corrected before publication.
  5. [Figure 2: the asynchronous memory path is hard to distinguish.] The figure is dense, and the VLM-to-E2E trajectory memory and the trajectory-token injection are not visually separated. Adding a legend or labeled callouts would improve readability.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; reported gains are measured against held-out nuScenes ground truth, not against fitted quantities.

full rationale

The paper's derivation chain is self-contained with respect to circularity. The trajectory bank in T-CoT is built by k-means clustering of training-set trajectories (Section 3.2), and candidates are retrieved by similarity to the current ego feature vector, but the final L2 and collision metrics are evaluated on held-out nuScenes data, so the reported predictions are not equivalent by construction to any fitted constant. The E2E planning-query initialization from clustering trajectories (Section 3.3, citing SparseDrive) and the asynchronous VLM-to-E2E memory mechanism use historical VLM outputs as priors, not ground-truth future trajectories, so there is no self-definitional reduction or label leakage. The paper cites prior work such as AD-MLP, BEV-Planner, OmniDrive, and DriveVLM for components and comparisons, but these are external results, not load-bearing same-author citations, and no uniqueness theorem or ansatz is imported from the authors' own prior work. The numerical discrepancy between the text's claim of a 0.03 L2 gain over DriveVLM and Table 1's 0.40 vs 0.28 (a 0.12 difference) is an internal reporting inconsistency, and the absence of variance estimates or significance tests weakens the SOTA claim; however, neither issue is a circularity pattern. The score is set to 1 to reflect the minor reporting concern without alleging that the derivation reduces to its inputs.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central empirical claims rest on standard training procedures plus several domain assumptions about evaluation metrics, temporal priors, and trajectory-bank retrieval. Free parameters are architecture choices tuned on the same benchmark used for the final claim. No new physical or quasi-physical entities are introduced; SQ-Former, trajectory tokens, and the trajectory bank are composed from existing primitives, so the invented-entity ledger is empty. The most fragile assumptions are the validity of open-loop metrics and the safety of using training-set trajectory clusters as priors.

free parameters (5)
  • SQ-Former collector query count = 384
    Chosen based on ablation in Table 2: 256 queries lose features, 512 add computation without gain. Affects the VLM's visual input capacity and final L2.
  • Trajectory bank cluster count = 36
    Number of k-means clusters on training-set future trajectories, per navigation command. The candidate pool size drives the T-CoT selection process.
  • Top-k retrieved trajectories = 5 plus 1 MLP prediction (6 candidates)
    Selected via ablation in Table 4: 4 candidates underperform, 8 add computational cost with no accuracy gain.
  • SQ-Former task order = image -> detection -> lane
    Chosen via ablation in Table 3; reversing detection and lane order loses 0.3 cm L2, and omitting image encoding loses 0.7 cm L2.
  • Ablation training epochs = 6
    Ablations use 6 epochs to save computational resources (Section 4.4); the main model training epoch count is not reported, so final results may use different settings.
assumptions (6)
  • standard math Standard transformer training, LoRA fine-tuning, and k-means clustering behave as expected and generalize to nuScenes
    The paper relies on off-the-shelf components without formal guarantees, which is normal for empirical ML but still an unproved background assumption.
  • domain assumption nuScenes open-loop L2 and collision rate are valid proxies for planning quality and safety
    Used as the sole evaluation in Section 4.2; prior work shows ego-status-only planners score nearly as well, which questions the metric's sensitivity to scene understanding.
  • domain assumption A VLM trajectory from a previous frame remains a useful planning prior for the current frame despite the temporal offset
    Underpins the async memory mechanism in Section 3.3; no sensitivity analysis to delay is provided.
  • ad hoc to paper Training-set trajectory clusters retrieved by ego-history similarity do not leak future ground truth or artificially inflate open-loop accuracy
    The T-CoT pull-back in Section 3.2 depends on this; the paper does not measure cross-split scenario similarity or analyze the prior's contribution.
  • domain assumption OmniDrive QA annotations are sufficient for VLM planning training and transfer to SOLVE's shared encoder
    Training stage one uses OmniDrive QA pairs (Section 3.4); no analysis of annotation quality or coverage is provided.
  • domain assumption The shared SQ-Former can serve both VLM and E2E branches without negative transfer
    Joint training in Section 3.4 assumes one feature extractor is optimal for both tasks; only end-task metrics validate this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving." pith.science (2026). https://pith.science/paper/RXYW34HC

@misc{pith2026250516805,
  author       = {Pith},
  title        = {Pith review of: SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RXYW34HC}},
  note         = {Machine review of arXiv:2505.16805}
}
read the original abstract

The integration of Vision-Language Models (VLMs) into autonomous driving systems has shown promise in addressing key challenges such as learning complexity, interpretability, and common-sense reasoning. However, existing approaches often struggle with efficient integration and realtime decision-making due to computational demands. In this paper, we introduce SOLVE, an innovative framework that synergizes VLMs with end-to-end (E2E) models to enhance autonomous vehicle planning. Our approach emphasizes knowledge sharing at the feature level through a shared visual encoder, enabling comprehensive interaction between VLM and E2E components. We propose a Trajectory Chain-of-Thought (T-CoT) paradigm, which progressively refines trajectory predictions, reducing uncertainty and improving accuracy. By employing a temporal decoupling strategy, SOLVE achieves efficient cooperation by aligning high-quality VLM outputs with E2E real-time performance. Evaluated on the nuScenes dataset, our method demonstrates significant improvements in trajectory prediction accuracy, paving the way for more robust and reliable autonomous driving systems.

Figures

Figures reproduced from arXiv: 2505.16805 by the authors.

Figure 1
Figure 1. Previous methods combine VLM and end-to-end [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall framework of the proposed SOLVE. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The detail of the proposed SQ-Former. We first capture [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The illustration of combination of the proposed trajec [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results of SOLVE, where red lines, blue lines and yellow lines mean the planning results from VLM, E2E-Async and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 26 canonical work pages

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. NeurIPS, 35: 23716–23736, 2022. 1

    2022

  2. [2]

    A versatile vision-language model for understanding, localization, text reading, and beyond

    J Bai, S Bai, S Yang, S Wang, S Tan, P Wang, J Lin, C Zhou, and J Qwen-VL Zhou. A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023. 6

    arXiv 2023

  3. [3]

    nuscenes: A mul- timodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A mul- timodal dataset for autonomous driving. In CVPR, pages 11621–11631, 2020. 5

    2020

  4. [4]

    nuplan: A closed-loop ml-based plan- ning benchmark for autonomous vehicles

    Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based plan- ning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021. 1

    arXiv 2021

  5. [5]

    Hierarchical adaptive path-tracking control for au- tonomous vehicles

    Changfang Chen, Yingmin Jia, Minglei Shu, and Yinglong Wang. Hierarchical adaptive path-tracking control for au- tonomous vehicles. IEEE Transactions on Intelligent Trans- portation Systems, 16(5):2900–2912, 2015. 1

    work page 2015

  6. [6]

    Driving with llms: Fusing object-level vec- tor modality for explainable autonomous driving

    Long Chen, Oleg Sinavski, Jan H ¨unermann, Alice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton. Driving with llms: Fusing object-level vec- tor modality for explainable autonomous driving. In ICRA, pages 14093–14100. IEEE, 2024. 3

    work page 2024

  7. [7]

    End-to-end autonomous driving: Challenges and frontiers

    Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, An- dreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers. IEEE TPAMI, 2024. 1, 2

    work page 2024

  8. [8]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13243,

    arXiv

Show all 49 references
  1. [9]

    Asynchronous large language model en- hanced planner for autonomous driving

    Yuan Chen, Zi-han Ding, Ziqin Wang, Yan Wang, Lijun Zhang, and Si Liu. Asynchronous large language model en- hanced planner for autonomous driving. In ECCV, pages 22–38. Springer, 2025. 2

    2025

  2. [10]

    Causal confusion in imitation learning

    Pim De Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. NeurIPS, 32, 2019. 1, 3

    2019

  3. [11]

    Large scale interactive mo- tion forecasting for autonomous driving: The waymo open motion dataset

    Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp, Charles R Qi, Yin Zhou, et al. Large scale interactive mo- tion forecasting for autonomous driving: The waymo open motion dataset. In ICCV, pages 9710–9719, 2021. 1

    2021

  4. [12]

    Eva-02: A visual representation for neon genesis

    Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xin- long Wang, and Yue Cao. Eva-02: A visual representation for neon genesis. Image and Vision Computing, 149:105171,

  5. [13]

    Drive like a human: Rethinking autonomous driving with large language models

    Daocheng Fu, Xin Li, Licheng Wen, Min Dou, Pinlong Cai, Botian Shi, and Yu Qiao. Drive like a human: Rethinking autonomous driving with large language models. In WACV, pages 910–919, 2024. 1, 3

    2024

  6. [14]

    Densetnt: End-to-end trajectory prediction from dense goal sets

    Junru Gu, Chen Sun, and Hang Zhao. Densetnt: End-to-end trajectory prediction from dense goal sets. In ICCV, pages 15303–15312, 2021. 1

    2021

  7. [15]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 5

    2021 arXiv

  8. [16]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. In CVPR, pages 17853–17862, 2023. 1, 2, 6

    2023

  9. [17]

    Emma: End-to-end multimodal model for autonomous driving

    Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, et al. Emma: End-to-end multimodal model for autonomous driving. arXiv preprint arXiv:2410.23262,

    arXiv

  10. [18]

    Vad: Vectorized scene representation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. In ICCV, pages 8340– 8350, 2023. 2, 6

    2023

  11. [19]

    Senna: Bridging large vision-language mod- els and end-to-end autonomous driving

    Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xing- gang Wang. Senna: Bridging large vision-language mod- els and end-to-end autonomous driving. arXiv preprint arXiv:2410.22313, 2024. 1

    arXiv

  12. [20]

    Inaction: Interpretable action decision making for au- tonomous driving

    Taotao Jing, Haifeng Xia, Renran Tian, Haoran Ding, Xiao Luo, Joshua Domeyer, Rini Sherony, and Zhengming Ding. Inaction: Interpretable action decision making for au- tonomous driving. In ECCV, pages 370–387. Springer, 2022. 1, 3

    2022

  13. [21]

    Au- tonomous driving at ulm university: A modular, robust, and sensor-independent fusion approach

    Felix Kunz, Dominik Nuss, J ¨urgen Wiest, Hendrik Deusch, Stephan Reuter, Franz Gritschneder, Alexander Scheel, Manuel St¨ubler, Martin Bach, Patrick Hatzelmann, et al. Au- tonomous driving at ulm university: A modular, robust, and sensor-independent fusion approach. In IEEE I...

    2015

  14. [22]

    Pointpillars: Fast encoders for object detection from point clouds

    Alex H Lang, Sourabh V ora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In CVPR, pages 12697–12705, 2019. 1

    2019

  15. [23]

    Exploring the causality of end-to-end autonomous driving

    Jiankun Li, Hao Li, Jiangjiang Liu, Zhikang Zou, Xiaoqing Ye, Fan Wang, Jizhou Huang, Hua Wu, and Haifeng Wang. Exploring the causality of end-to-end autonomous driving. arXiv preprint arXiv:2407.06546, 2024. 1, 3

    2024 arXiv

  16. [24]

    Blip: Bootstrapped language-image pre- training

    Junnan Li et al. Blip: Bootstrapped language-image pre- training. arXiv preprint arXiv:2201.12086, 2023. 1

    2023 arXiv

  17. [25]

    Deep learning for lidar point clouds in autonomous driving: A review

    Ying Li, Lingfei Ma, Zilong Zhong, Fei Liu, Michael A Chapman, Dongpu Cao, and Jonathan Li. Deep learning for lidar point clouds in autonomous driving: A review. IEEE Transactions on Neural Networks and Learning Systems, 32 (8):3412–3432, 2020. 1

    2020

  18. [26]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In ECCV, pages 1–

  19. [27]

    Is ego status all you need for open-loop end-to-end autonomous driving? In CVPR, pages 14864–14873, 2024

    Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M Alvarez. Is ego status all you need for open-loop end-to-end autonomous driving? In CVPR, pages 14864–14873, 2024. 2, 6

    2024

  20. [28]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 36, 2024. 1, 6

    2024

  21. [29]

    Multimodal motion prediction with stacked transformers

    Yicheng Liu, Jinghuai Zhang, Liangji Fang, Qinhong Jiang, and Bolei Zhou. Multimodal motion prediction with stacked transformers. In CVPR, pages 7577–7586, 2021. 1

    2021

  22. [30]

    A language agent for autonomous driving

    Jiageng Mao, Junjie Ye, Yuxi Qian, Marco Pavone, and Yue Wang. A language agent for autonomous driving. arXiv preprint arXiv:2311.10813, 2023. 1, 3

    2023 arXiv

  23. [31]

    Deep learning-based vehicle behavior prediction for autonomous driving applica- tions: A review

    Sajjad Mozaffari, Omar Y Al-Jarrah, Mehrdad Dianati, Paul Jennings, and Alexandros Mouzakitis. Deep learning-based vehicle behavior prediction for autonomous driving applica- tions: A review. IEEE Transactions on Intelligent Trans- portation Systems, 23(1):33–47, 2020. 1

    2020

  24. [32]

    Deep learning for safe autonomous driving: Current challenges and future direc- tions

    Khan Muhammad, Amin Ullah, Jaime Lloret, Javier Del Ser, and Victor Hugo C de Albuquerque. Deep learning for safe autonomous driving: Current challenges and future direc- tions. IEEE Transactions on Intelligent Transportation Sys- tems, 22(7):4316–4336, 2020. 1

    2020

  25. [33]

    Decision-making framework for automated driving in highway environments

    Samyeul Noh and Kyounghwan An. Decision-making framework for automated driving in highway environments. IEEE Transactions on Intelligent Transportation Systems, 19 (1):58–71, 2017. 1

    2017

  26. [34]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652–660, 2017. 1

    2017

  27. [35]

    Safety-enhanced autonomous driving using inter- pretable sensor fusion transformer

    Hao Shao, Letian Wang, Ruobing Chen, Hongsheng Li, and Yu Liu. Safety-enhanced autonomous driving using inter- pretable sensor fusion transformer. In Conference on Robot Learning, pages 726–737. PMLR, 2023. 1, 3

    2023

  28. [36]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. arXiv preprint arXiv:2312.14150, 2023. 3, 4

    2023 arXiv

  29. [37]

    Pip: Planning- informed trajectory prediction for autonomous driving

    Haoran Song, Wenchao Ding, Yuxuan Chen, Shaojie Shen, Michael Yu Wang, and Qifeng Chen. Pip: Planning- informed trajectory prediction for autonomous driving. In ECCV, pages 598–614. Springer, 2020. 1

    2020

  30. [38]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, pages 2446–2454, 2020. 1

    2020

  31. [39]

    Sparsedrive: End-to-end au- tonomous driving via sparse scene representation

    Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Hao- ran Wu, and Sifa Zheng. Sparsedrive: End-to-end au- tonomous driving via sparse scene representation. arXiv preprint arXiv:2405.19620, 2024. 5

    2024 arXiv

  32. [40]

    Motion planning for autonomous driv- ing: The state of the art and future perspectives.IEEE Trans- actions on Intelligent Vehicles, 8(6):3692–3711, 2023

    Siyu Teng, Xuemin Hu, Peng Deng, Bai Li, Yuchen Li, Yunfeng Ai, Dongsheng Yang, Lingxi Li, Zhe Xuanyuan, Fenghua Zhu, et al. Motion planning for autonomous driv- ing: The state of the art and future perspectives.IEEE Trans- actions on Intelligent Vehicles, 8(6):3692–3711, 2023. 1

    2023

  33. [41]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289, 2024. 1, 2, 3, 4, 6

    2024 arXiv

  34. [42]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 6

    2023 arXiv

  35. [43]

    Exploring object-centric temporal modeling for efficient multi-view 3d object detection

    Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xi- angyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. In ICCV, pages 3621–3631, 2023. 3, 6

    2023

  36. [44]

    Omnidrive: A holistic llm-agent framework for autonomous driving with 3d perception, reasoning and planning

    Shihao Wang, Zhiding Yu, Xiaohui Jiang, Shiyi Lan, Min Shi, Nadine Chang, Jan Kautz, Ying Li, and Jose M Alvarez. Omnidrive: A holistic llm-agent framework for autonomous driving with 3d perception, reasoning and planning. arXiv preprint arXiv:2405.01533, 2024. 3, 4, 5, 6

    2024 arXiv

  37. [45]

    Drivecot: Integrating chain-of-thought reasoning with end-to-end driving

    Tianqi Wang, Enze Xie, Ruihang Chu, Zhenguo Li, and Ping Luo. Drivecot: Integrating chain-of-thought reasoning with end-to-end driving. arXiv preprint arXiv:2403.16996, 2024. 4

    2024 arXiv

  38. [46]

    Drive anywhere: Generalizable end-to-end au- tonomous driving with multi-modal foundation models

    Tsun-Hsuan Wang, Alaa Maalouf, Wei Xiao, Yutong Ban, Alexander Amini, Guy Rosman, Sertac Karaman, and Daniela Rus. Drive anywhere: Generalizable end-to-end au- tonomous driving with multi-modal foundation models. In ICRA, pages 6687–6694. IEEE, 2024. 3

    2024

  39. [47]

    Para-drive: Parallelized architecture for real- time autonomous driving

    Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Parallelized architecture for real- time autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15449–15458, 2024. 2

    2024

  40. [48]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Let- ters, 2024. 3

    2024

  41. [49]

    Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes

    Jiang-Tian Zhai, Ze Feng, Jinhao Du, Yongqiang Mao, Jiang-Jiang Liu, Zichang Tan, Yifu Zhang, Xiaoqing Ye, and Jingdong Wang. Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes. arXiv preprint arXiv:2305.10430, 2023. 1, 2, 4, 6

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.