Pith. sign in

REVIEW 3 major objections 5 minor 30 references

The paper argues that a dedicated spatial stream bypassing the language bottleneck prevents spatial representation collapse in vision-language-action driving models, yielding state-of-the-art supervised closed-loop performance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 03:17 UTC pith:VMF3EYMG

load-bearing objection Sensible dual-stream VLA design with a credible NAVSIM score, but the ablation doesn't pin down why it works. the 3 major comments →

arxiv 2607.13926 v1 pith:VMF3EYMG submitted 2026-07-15 cs.RO

S-squared-VLA: Decoupling Semantic and Spatial Streams in Vision-Language-Action Models for Autonomous Driving

Jianguo Yu , Rukang Wang , Duanfeng Chu , Chen Wang , Renju Feng , Liping Lu This is my paper
classification cs.RO
keywords autonomous drivingvision-language-action modelstrajectory planningspatial representation collapsedual-stream architecturebird's-eye-view perceptionclosed-loop simulationsupervised fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard vision-language-action (VLA) driving models route perception through an autoregressive language head, which the paper says destroys fine-grained spatial and geometric information—a 'spatial representation collapse.' The paper proposes S2-VLA, which splits processing into a semantic stream for language-based intent reasoning and a separate spatial stream that pulls uncompressed visual features directly from the visual encoder, supervised with bird's-eye-view map and agent prediction tasks. A dual-stream planning adapter fuses the two with cascaded attention, so planning tokens receive both intent and geometric constraints. On the NAVSIM closed-loop benchmark, this reaches a Predictive Driver Model Score (PDMS) of 87.1 with a 98.4 no-collision rate, the best among VLA models under purely supervised fine-tuning. If correct, it shows that keeping spatial geometry out of the token bottleneck is what makes VLA planning safe and boundary-aware.

Core claim

Standard VLA driving models route perception through an autoregressive language head, which the paper calls a 'spatial representation collapse': discrete tokenization and deep abstraction progressively erase fine-grained geometry, so trajectories drift off-lane or collide. S2-VLA counters this with a semantic stream that reads multi-scale features from a vision-language backbone, plus an independent spatial stream that extracts dense visual queries from the vision transformer, bypasses the language model, and is trained with auxiliary bird's-eye-view map and agent-prediction heads to encode metric layout. A Dual-Stream Planning Adapter then lets planning tokens first attend to semantic inten

What carries the argument

The load-bearing mechanism is the spatial stream: a parallel pathway that takes learnable visual queries plus patch tokens from the vision transformer, bypasses the language model's autoregressive decoding, and is trained with auxiliary bird's-eye-view semantic map and agent bounding-box heads so its features encode metric layout. The second piece is the Dual-Stream Planning Adapter, a cascade of cross-attention blocks in which planning tokens first attend to semantic features and ego-state memory, then to dense spatial features, so intent and geometry are fused rather than entangled. Together these prevent the compression and quantization that the paper says makes single-stream VLAs collide

Load-bearing premise

The central claim rests on the assumption that the PDMS gains in the cumulative ablation come from decoupling the streams, since the rows add semantic features, spatial features, and auxiliary supervision without matching model capacity, parameter count, or compute.

What would settle it

The decisive experiment is a parameter- and compute-matched single-stream VLA given the same auxiliary BEV map and agent losses: if it matches 87.1 PDMS and 98.4 NC, then the dual-stream decoupling itself is not the explanation, and the observed gains come from added supervision or capacity rather than from preventing spatial representation collapse.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • S2-VLA reaches PDMS 87.1 with a single front camera, outperforming LiDAR-fused end-to-end planners like ARTEMIS (87.0) and DRAMA (86.9) and trailing DiffusionDrive (88.1) by less than a point without iterative diffusion decoding.
  • No Collision rises to 98.4 and Ego Progress to 81.6, the highest among all evaluated VLM/VLA methods, so the safety gains come from spatial grounding rather than comfort trade-offs.
  • Semantic features alone plateau at PDMS 85.6 and Drivable Area Compliance 94.0; only adding the spatial stream and auxiliary perception raises DAC to 94.9, confirming that language-only features cannot enforce drivable-area compliance.
  • The full model with auxiliary BEV map and agent prediction adds +0.9 PDMS over the spatial-stream-only variant, with the largest gains in NC and Ego Progress, so geometric supervision contributes directly to collision avoidance.
  • A 2-billion-parameter dual-stream VLA beats larger or reasoning-augmented VLA baselines (ReCogDrive 86.5, ImagiDrive 86.4) under pure supervised fine-tuning, indicating the architecture rather than scale drives the result.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • By extension, any VLA that emits continuous control through discrete autoregressive tokens should show the same spatial collapse; the decoupled design is a transferable recipe for robot manipulation and other continuous-control VLA settings.
  • A direct test of the mechanism: feed the spatial stream two or more cameras or a short temporal history and measure PDMS and DAC; if the design's value scales with spatial context, the 'preserve geometry' explanation is strengthened.
  • The adapter's ordering (semantic attention before spatial refinement) implies a decide-then-verify hypothesis; an ablation that reverses or parallelizes the two attention stages would show whether the ordering is load-bearing.
  • Since the BEV map head covers only 32 meters ahead and 64 meters wide, the method's explicit geometric prior is local; the paper leaves open whether longer-horizon plans would need a learned map prior or HD maps to avoid degradation beyond that range.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes S2-VLA, a vision-language-action model for autonomous driving that decouples semantic and spatial processing. The semantic stream uses an InternVL3-2B backbone with multi-scale feature extraction and learnable action queries; the spatial stream bypasses the autoregressive language model by directly leveraging ViT features, adding a tile-level transformer, a BEV map head, and an agent detection head. A dual-stream planning adapter fuses the two streams via cascaded cross-attention and produces waypoints. On the NAVSIM closed-loop benchmark, the authors report PDMS 87.1, the highest among VLM/VLA methods under SFT, with NC 98.4. The central claim is that this design mitigates 'spatial representation collapse' inherent to conventional VLAs.

Significance. If fully validated, the paper would make a useful architectural contribution: it provides a concrete instantiation of a dual-stream VLA and reports strong closed-loop results on a standard benchmark. The decision to evaluate with NAVSIM's external PDMS, rather than an in-house metric, is a strength, as is the use of the same InternVL3-2B backbone for the primary VLM baseline. However, the paper's mechanistic explanation—that decoupling and bypassing the language bottleneck is the cause of the gains—is not supported by the current ablation design, and the reported benchmark differences are presented without variance. The core empirical result (a high PDMS) may be correct, but the causal claim requires further controls. As presented, the contribution is a competitive system with a plausible but unproven design rationale.

major comments (3)
  1. [Table III and §IV-C] The central mechanistic claim—that bypassing the autoregressive language bottleneck via a dedicated spatial stream drives the improvement—is not identifiable from the cumulative ablation. Row 3 (addition of the spatial stream) simultaneously adds 64 visual queries per patch, a tile-level Transformer encoder, an upsampling/convolution stack, and the associated parameters; row 4 adds the map and agent heads plus their auxiliary losses. There is no 'entangled' control in which the same spatially dense visual features are injected into the LLM token sequence and processed through the autoregressive layers, nor a matched-capacity control with additional transformer layers in the semantic path. The +0.6 PDMS (85.6→86.2) and +0.9 PDMS (86.2→87.1) could therefore result from added capacity, added compute, or the auxiliary supervision rather than from decoupling per se. The paper should add at le
  2. [Tables II and III] All results are reported as single numbers with no variance, number of seeds, or statistical significance. The headline PDMS 87.1 differs from ReCogDrive (86.5), ImagiDrive (86.4), and ARTEMIS (87.0) by margins of 0.1–0.7 points. In closed-loop benchmarks, such differences can easily lie within run-to-run noise. This directly affects the paper's SOTA claim and also the ablation increments, some of which are as small as +0.6 PDMS. At minimum, the authors should report standard deviations over at least 3–5 seeds, or otherwise demonstrate that the differences are reproducible.
  3. [Eq. (1) and Table I] Equation (1) defines L_map with class-specific weights ω_c 'to emphasize critical topological features,' but Table I sets all ω_c = 1.0. If all weights are indeed unity, the stated class-imbalance motivation is not implemented in the experiments; if the value in Table I is only a default or the weights differ per class, the table is incomplete. This is a reproducibility issue and should be clarified. It does not by itself invalidate the main result, but it is a specific inconsistency in the method as reported.
minor comments (5)
  1. [§III.B] The first training stage is described as 'SFT on the ReCogDrive VQA dataset' [15]. ReCogDrive is cited as a framework, not a dataset; the reader cannot tell whether this is a named dataset within the ReCogDrive project or a typo. Please clarify the dataset source and license/release status.
  2. [General] The paper alternates between 'NAVSIM' and 'NA VSIM' even within the abstract and Section III. Please standardize the spelling.
  3. [Fig. 4] The red circles mentioned in the text are not clearly visible in the figure as printed; consider enlarging them or adding zoomed insets.
  4. [§II-C, Eq. (6)] The residual connection in Eq. (6) uses P_vis = P_fuse + MHCA3(...) but the subsequent FFN output is P^(l) = P_vis + FFN(P_vis). This is fine, but the notation P_fuse and P^(l-1) could be defined more explicitly in the text for readers who do not infer the block-level recurrence.
  5. [Table III] The row labels in Table III are '√' marks without textual component names; this makes it hard to map rows to the paragraphs in §IV-C. Adding component names in the row header or a legend would improve readability.

Circularity Check

0 steps flagged

No significant circularity: benchmark results and ablations are self-contained, and the causal claims, while confounded by capacity, are not reduced to their inputs by construction.

full rationale

The paper's central quantitative claim (PDMS 87.1 on NAVSIM) is evaluated by an external closed-loop benchmark, and the reported metrics are not inserted back into the loss or architecture in a way that forces the score. The proposed model's components are trained with auxiliary map/agent objectives, but these are additional supervision signals rather than quantities recomputed from the evaluation metric, so there is no fitted-input-called-prediction pattern. The core mechanistic argument relies on an ablation sequence (Table III), which is cumulative and does not control parameter count, compute, or an entangled variant that feeds the same dense spatial features through the LLM; however, this is an internal-validity/confound concern about attribution, not circularity. No quoted step reduces a claimed derivation to its own definition. The paper does cite prior work by overlapping authors (e.g., ARTEMIS in the comparison table), but ARTEMIS is used as a baseline, not as the justification for the central design, and no uniqueness theorem or load-bearing conclusion is imported from a self-citation. The N_act=64 choice follows VLA-Adapter, but that is a hyperparameter design choice, not the predicted outcome. Overall, the derivation chain is self-contained with respect to external benchmarks; any weakness is methodological confound rather than circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The paper's central empirical claims rest on the external NAVSIM benchmark, borrowed baseline numbers, and a cumulative ablation without matched compute. The main upstream assumptions are the validity of PDMS as a safety proxy, comparability of heterogeneous baselines, and the attribution of gains to decoupling rather than extra capacity.

free parameters (6)
  • Loss weights (λ_plan, λ_agent, λ_map, λ_smooth, λ_reg, λ_cls, ω_c) = 1.0 / 0.1 / 0.5 / 0.5 / 1.0 / 1.0 / 1.0
    Hand-chosen in Table I; no sensitivity analysis is reported, and ω_c=1.0 contradicts the claim of class-specific map loss weighting.
  • Semantic layer subset L = {3, 8, 13, 18, 23, 24}
    Sparse multi-scale sampling layers are hand-selected; no ablation of layer choice is provided.
  • Number of action/visual queries N_act / N_vis = 64 / 64
    Query counts set by design; N_act follows VLA-Adapter, N_vis is chosen similarly, with no sweep.
  • Waypoints M and agent queries N_agent = M=8, N_agent=30
    Output dimensions chosen for the NAVSIM trajectory format and DETR-style detection head.
  • LoRA rank/alpha, learning rate, batch size, stage epochs = r=8, α=16, lr=1e-4, batch=16, epochs 3/4/4
    Training hyperparameters from Table I, not justified by experiments.
  • BEV range and grid resolution = X∈[0,32]m, Y∈[-32,32]m; 24x24 grid to 128x256 output
    Chosen for the front-camera setup; not ablated.
axioms (5)
  • domain assumption NAVSIM PDMS is a valid and sufficient proxy for closed-loop driving safety.
    All safety-related conclusions are based on this simulation benchmark; no real-world or additional simulator validation is provided.
  • domain assumption Baseline scores from different papers are directly comparable under the 'SFT-only' comparison.
    Table II mixes scores reported by original papers with the authors' runs, without a shared retraining pipeline, seeds, or hyperparameters.
  • domain assumption Monocular front-view images plus auxiliary GT supervision provide sufficient BEV spatial and geometric priors for planning.
    The map and agent heads are trained on GT BEV maps and boxes; failure modes of this monocular-to-BEV reconstruction are not analyzed.
  • ad hoc to paper The ablation gains are attributable to the stated components rather than to added parameters, compute, or supervision volume.
    Table III adds modules cumulatively without matched-capacity or matched-compute controls, so the causal attribution is assumed.
  • domain assumption InternVL3-2B pretrained features transfer to driving after LoRA fine-tuning.
    The method relies on this backbone's semantic and visual representations; no comparison with other VLM backbones is given.

pith-pipeline@v1.3.0-alltime-deepseek · 11311 in / 12785 out tokens · 121905 ms · 2026-08-02T03:17:12.109178+00:00 · methodology

0 comments
read the original abstract

Vision-Language Models (VLMs) have demonstrated remarkable potential for high-level reasoning in autonomous driving, yet they fundamentally struggle to generate precise, low-level control actions. This limitation is rooted in a semantic-physical gap caused by the inherent mismatch between discrete language tokens and continuous trajectory planning. While Vision-Language-Action (VLA) architectures attempt to bridge this gap by unifying perception and control into a single policy, this entanglement creates a new bottleneck. Standard VLAs experience a severe spatial representation collapse, which irreversibly degrades the fine-grained spatial and geometric priors essential for safe, boundary-aware navigation. To address this limitation, we propose the S-squared-VLA, which explicitly decouples the semantic and spatial streams in Vision-Language-Action models. The semantic stream leverages hierarchical bridging to extract multi-scale VLM features for robust intent reasoning. In parallel, an independent spatial stream bypasses the autoregressive language bottleneck, directly preserving uncompressed spatial features from the visual encoder. By integrating auxiliary perception supervision, this stream explicitly equips the model with rich spatial and geometric priors. Finally, a dual-stream planning adapter fuses high-level semantic intent with precise spatial constraints via cascaded attention mechanisms. Evaluations on the NAVSIM closed-loop benchmark show that S-squared-VLA achieves a Predictive Driver Model Score (PDMS) of 87.1, establishing a new state-of-the-art for VLA models under a purely supervised fine-tuning (SFT) setting. By mitigating the spatial representation collapse of traditional VLMs, our framework significantly outperforms baselines, achieving the highest No Collision (NC) rate of 98.4 among all evaluated methods.

Figures

Figures reproduced from arXiv: 2607.13926 by Chen Wang, Duanfeng Chu, Jianguo Yu, Liping Lu, Renju Feng, Rukang Wang.

Figure 1
Figure 1. Figure 1: Overview of S 2 -VLA compared with existing paradigms. (a) Tradi￾tional E2E models suffer from semantic blindness, lacking the interpretability and high-level cognitive reasoning required for complex traffic scenarios. (b) VLMs produce discrete outputs unsuitable for continuous control. (c) Conventional VLAs suffer from spatial representation collapse, leading to loss of fine-grained spatial and geometric … view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of the S 2 -VLA model. Stream designed to selectively extract and aggregate hierar￾chical features. Multimodal Input and Query Injection. All inputs are tokenized into a unified sequence of text and visual tokens. At the input of the semantic stream, we inject Nact = 64 learnable action queries Qact to explicitly drive the network to aggregate execution-relevant features into action-centric re… view at source ↗
Figure 3
Figure 3. Figure 3: Architecture of the spatial stream. a high-resolution BEV multi-class semantic map Mˆ sem ∈ R C×128×256. Because our architecture relies exclusively on a monocular front-view camera, the Map Head focuses on re￾constructing a local, visible BEV semantic space immediately ahead of the ego-vehicle(specifically spanning X ∈ [0, 32] m and Y ∈ [−32, 32] m). To mitigate the severe class imbalance between backgrou… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative analysis. The green curves represent ground-truth expert trajectories, and the orange curves represent the model’s predicted future trajectories. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 6 linked inside Pith

  1. [1]

    Bevdrive-e2e: Imitation with bird’s eye view perception for interpretable end-to-end autonomous driving,

    J. Du, Y . Song, X. Pan, S. Su, J. Yang, L. Wang, C. Liu, and Q. Chen, “Bevdrive-e2e: Imitation with bird’s eye view perception for interpretable end-to-end autonomous driving,”IEEE Robotics and Automation Letters, vol. 11, no. 4, pp. 4353–4360, 2026

    2026

  2. [2]

    Planning-oriented autonomous driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang,et al., “Planning-oriented autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 17 853–17 862

    2023

  3. [3]

    Vad: Vectorized scene representation for efficient autonomous driving,

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 8340–8350

    2023

  4. [4]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving,

    B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y . Zhang, Q. Zhang,et al., “Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 12 037–12 047

    2025

  5. [5]

    Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving,

    R. Feng, N. Xi, D. Chu, R. Wang, Z. Deng, A. Wang, L. Lu, J. Wang, and Y . Huang, “Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving,”IEEE Robotics and Automation Letters, vol. 11, no. 1, pp. 226–233, 2025

    2025

  6. [6]

    Dilu: A knowledge-driven approach to autonomous driving with large language models,

    L. Wen, D. Fu, X. Li, X. Cai, T. Ma, P. Cai, M. Dou, B. Shi, L. He, and Y . Qiao, “Dilu: A knowledge-driven approach to autonomous driving with large language models,”arXiv preprint arXiv:2309.16292, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Continuously learning, adapting, and improving: a dual- process approach to autonomous driving,

    J. Mei, Y . Ma, X. Yang, L. Wen, X. Cai, X. Li, D. Fu, B. Zhang, P. Cai, M. Dou,et al., “Continuously learning, adapting, and improving: a dual- process approach to autonomous driving,” inProceedings of the 38th International Conference on Neural Information Processing Systems, 2024, pp. 123 261–123 290

    2024

  8. [8]

    Drivelm: Driving with graph visual ques- tion answering,

    C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, J. Beißwenger, P. Luo, A. Geiger, and H. Li, “Drivelm: Driving with graph visual ques- tion answering,” inEuropean conference on computer vision. Springer, 2024, pp. 256–274

    2024

  9. [9]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model,

    Z. Xu, Y . Zhang, E. Xie, Z. Zhao, Y . Guo, K.-Y . K. Wong, Z. Li, and H. Zhao, “Drivegpt4: Interpretable end-to-end autonomous driving via large language model,”IEEE Robotics and Automation Letters, vol. 9, no. 10, pp. 8186–8193, 2024

    2024

  10. [10]

    Reasonplan: Unified scene prediction and decision reasoning for closed-loop autonomous driving,

    X. Liu, Z. Zhong, Q. Zhang, Y . Guo, Y . Zheng, J. Wang, D. Zhao, Y .-F. Liu, Z. Su, Y . Gao,et al., “Reasonplan: Unified scene prediction and decision reasoning for closed-loop autonomous driving,” inConference on Robot Learning. PMLR, 2025, pp. 3051–3068

    2025

  11. [11]

    A survey on vision-language- action models for autonomous driving,

    S. Jiang, Z. Huang, K. Qian, Z. Luo, T. Zhu, Y . Zhong, Y . Tang, M. Kong, Y . Wang, S. Jiao,et al., “A survey on vision-language- action models for autonomous driving,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 4524–4536

    2025

  12. [12]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control,

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid,et al., “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” inConference on Robot Learning. PMLR, 2023, pp. 2165–2183

    2023

  13. [13]

    Vla-adapter: An effective paradigm for tiny-scale vision-language-action model,

    Y . Wang, P. Ding, L. Li, C. Cui, Z. Ge, X. Tong, W. Song, H. Zhao, W. Zhao, P. Hou,et al., “Vla-adapter: An effective paradigm for tiny-scale vision-language-action model,” inProceedings of the AAAI conference on artificial intelligence, vol. 40, no. 22, 2026, pp. 18 638– 18 646

    2026

  14. [14]

    Simlingo: Vision-only closed-loop autonomous driving with language-action alignment,

    K. Renz, L. Chen, E. Arani, and O. Sinavski, “Simlingo: Vision-only closed-loop autonomous driving with language-action alignment,” in Proceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 11 993–12 003

    2025

  15. [15]

    Recogdrive: A reinforced cognitive framework for end- to-end autonomous driving,

    Y . Li, K. Xiong, X. Guo, F. Li, S. Yan, G. Xu, L. Zhou, L. Chen, H. Sun, B. Wang,et al., “Recogdrive: A reinforced cognitive framework for end- to-end autonomous driving,”arXiv preprint arXiv:2506.08052, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning,

    Z. Zhou, T. Cai, S. Z. Zhao, Y . Zhang, Z. Huang, B. Zhou, and J. Ma, “Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning,” in Advances in Neural Information Processing Systems, 2025

    2025

  17. [17]

    Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation,

    H. Fu, D. Zhang, Z. Zhao, J. Cui, D. Liang, C. Zhang, D. Zhang, H. Xie, B. Wang, and X. Bai, “Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 24 823–24 834

    2025

  18. [18]

    Cot-vla: Visual chain-of-thought reasoning for vision-language-action models,

    Q. Zhao, Y . Lu, M. J. Kim, Z. Fu, Z. Zhang, Y . Wu, Z. Li, Q. Ma, S. Han, C. Finn,et al., “Cot-vla: Visual chain-of-thought reasoning for vision-language-action models,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 1702–1713

    2025

  19. [19]

    Diffvla: Vision-language guided diffusion planning for autonomous driving,

    A. Jiang, Y . Gao, Z. Sun, Y . Wang, J. Wang, J. Chai, Q. Cao, Y . Heng, H. Jiang, Y . Dong,et al., “Diffvla: Vision-language guided diffusion planning for autonomous driving,”arXiv preprint arXiv:2505.19381, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,

    J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, H. Tian, Y . Duan, W. Su, J. Shao,et al., “Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,”arXiv preprint arXiv:2504.10479, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Multimodal language models see better when they look shallower,

    H. Chen, J. Lin, X. Chen, Y . Fan, J. Dong, X. Jin, H. Su, J. Fu, and X. Shen, “Multimodal language models see better when they look shallower,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 6688–6706

    2025

  22. [22]

    Beyond the final layer: Attentive multilayer fusion for vision transformers,

    L. Ciernik, M. Morik, L. Thede, L. Eyring, S. Nakajima, Z. Akata, and L. Muttenthaler, “Beyond the final layer: Attentive multilayer fusion for vision transformers,”arXiv preprint arXiv:2601.09322, 2026

    Pith/arXiv arXiv 2026

  23. [23]

    Llava-uhd: an lmm perceiving any aspect ratio and high-resolution images,

    Z. Guo, R. Xu, Y . Yao, J. Cui, Z. Ni, C. Ge, T.-S. Chua, Z. Liu, and G. Huang, “Llava-uhd: an lmm perceiving any aspect ratio and high-resolution images,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 390–406

    2024

  24. [24]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in European conference on computer vision. Springer, 2020, pp. 213– 229

    2020

  25. [25]

    Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking,

    D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone,et al., “Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking,” in Advances in Neural Information Processing Systems, 2024, pp. 28 706– 28 719

    2024

  26. [26]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen,et al., “Lora: Low-rank adaptation of large language models.” Iclr, vol. 1, no. 2, p. 3, 2022

    2022

  27. [27]

    Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,

    K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger, “Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,”IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 11, pp. 12 878–12 895, 2023

    2023

  28. [28]

    Para- drive: Parallelized architecture for real-time autonomous driving,

    X. Weng, B. Ivanovic, Y . Wang, Y . Wang, and M. Pavone, “Para- drive: Parallelized architecture for real-time autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 15 449–15 458

    2024

  29. [29]

    Drama: An efficient end-to-end motion planner for autonomous driving with mamba,

    C. Yuan, Z. Zhang, J. Sun, S. Sun, Z. Huang, C. D. W. Lee, D. Li, Y . Han, A. Wong, K. P. Tee,et al., “Drama: An efficient end-to-end motion planner for autonomous driving with mamba,”arXiv preprint arXiv:2408.03601, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Imagidrive: A unified imagination-and-planning framework for autonomous driving,

    J. Li, B. Zhang, X. Jin, J. Deng, X. Zhu, and L. Zhang, “Imagidrive: A unified imagination-and-planning framework for autonomous driving,” arXiv preprint arXiv:2508.11428, 2025

    arXiv 2025