Pith. sign in

REVIEW 3 major objections 65 references

WCog-VLA turns reactive vision-language driving models into proactive planners by coupling semantic world forecasts with joint multi-agent trajectory generation, reaching 92.9 PDMS on NAVSIM.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 08:28 UTC pith:LQ5FJDKL

load-bearing objection Clean SOTA engineering on NAVSIM (92.9 PDMS) by gluing agent-token VLM forecasting to a fast multi-agent diffusion world model; the Game-CoT half is reverse-engineered from GT and therefore the weakest link in the 'proactive' story. the 3 major comments →

arxiv 2607.08375 v1 pith:LQ5FJDKL submitted 2026-07-09 cs.CV cs.AI

WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving

Xuerun Yan , Zhexi Lian , Nuoheng Zhang , Shiyu Fang , Haoran Wang , Chen Lv , Jia Hu , Binyang Song This is my paper
classification cs.CV cs.AI
keywords end-to-end autonomous drivingVision-Language-Actionworld cognitionGame-CoTAligned Decoupled Diffusion Transformermulti-agent trajectoriesNAVSIMproactive planning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current vision-language-action models for autonomous driving remain reactive: they either lack structured 3D world understanding or only partially forecast the future, so they cannot negotiate with other road users. This paper claims that dual-level world cognition closes that gap. At the semantic level a vision-language backbone ingests multi-view images, injects 3D agent tokens, and runs a four-step Game-theoretic Chain-of-Thought that treats the ego vehicle as leader in a Stackelberg game. At the generative level an Aligned Decoupled Diffusion Transformer, conditioned on those cognitive states and aligned to a latent scene representation, jointly synthesizes physically plausible trajectories for the ego vehicle and surrounding agents in few denoising steps. An 85k-sample Game-CoT dataset supplies the missing strategic supervision. On the NAVSIM closed-loop benchmark the resulting 2B-parameter system reaches a state-of-the-art PDMS of 92.9 while improving safety metrics and accelerating inference. A reader who cares about safer, more human-like driving should care because the work shows a concrete path from language-model scene talk to interactive, proactive motion.

Core claim

WCog-VLA establishes that bridging semantic-level world forecasting (3D agent tokens plus Game-CoT reasoning) with generative-level joint multi-agent trajectory synthesis (via the Aligned Decoupled Diffusion Transformer) converts reactive vision-language-action models into proactive end-to-end planners, evidenced by a state-of-the-art PDMS of 92.9 on NAVSIM.

What carries the argument

The Aligned Decoupled Diffusion Transformer (ADDT): a condition encoder whose intermediate features are cosine-aligned to a pre-trained VAE scene latent, so a separate generation decoder can recover joint multi-agent trajectories from VLM cognitive states in far fewer denoising steps.

Load-bearing premise

The 85 thousand Game-CoT chains auto-generated by a vision-language model, guided by already-known ground-truth actions, are assumed to teach genuine strategic social reasoning rather than merely rationalize the correct answer after the fact.

What would settle it

Retrain an otherwise identical model on Game-CoT labels produced without ground-truth action hints (or with deliberately wrong hints) and measure whether PDMS and safety metrics fall substantially below the reported 92.9; a large drop would show the supervision is post-hoc rationalization.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Unifying semantic agent forecasts with generative joint multi-agent trajectories raises both overall PDMS and safety scores (no-collision, time-to-collision) above prior end-to-end and VLA baselines on NAVSIM.
  • Explicit game-theoretic chain-of-thought data improves planning beyond trajectory-only or generic driving VQA supervision.
  • Scene-representation alignment inside the diffusion encoder lets the planner keep high trajectory quality with only five denoising steps, cutting inference latency relative to standard diffusion transformers.
  • Each of the four training stages (3D perception pre-training, VLM world-cognition fine-tuning, ADDT supervised fine-tuning, and diffusion GRPO) is required for the full reported gain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the dual-level bridge generalizes, the same semantic-to-generative pattern could be reused in multi-agent robotics domains where language models currently plan without joint physical interaction.
  • Once hallucination rates of auto-generated Stackelberg-style chains are measured, the same annotation recipe could supply low-cost social reasoning data for other VLA systems.
  • The authors note that road-geometry and map-topology evolution are still missing; adding dynamic map foresight is a direct next experiment that should further lift long-horizon EPDMS-style metrics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper proposes WCog-VLA, a dual-level Vision-Language-Action framework for end-to-end autonomous driving that couples semantic world forecasting with generative multi-agent trajectory synthesis. At the semantic level, multi-view images, text instructions, and TrackFormer-derived 3D agent tokens are fed to an InternVL3-2B backbone; agent hidden states drive a world head for current 3D perception and future agent trajectories, while vision/text states support a four-step Game-theoretic Chain-of-Thought (Game-CoT). At the generative level, an Aligned Decoupled Diffusion Transformer (ADDT) conditions on VLM hidden states, aligns intermediate encoder features to a pre-trained multi-agent VAE latent, and denoises joint multi-agent trajectories. The authors construct 85k Game-CoT annotations via Qwen3-VL-Plus with ground-truth action hints, train in four stages (perception pre-training, VLM SFT, ADDT SFT, DiffGRPO RFT), and report a SOTA PDMS of 92.9 on NAVSIM v1 (camera-only) plus strong EPDMS on NAVSIM v2, with ablations attributing gains to 3D perception, dual-level cognition, ADDT alignment, VQA mixture, and RFT.

Significance. If the dual-level design is the primary driver of the reported gains, the work is a solid systems contribution to VLA-based driving: it unifies agent-centric 3D tokens, language-level social reasoning, and joint multi-agent diffusion planning in one trainable stack, and it shows that camera-only planning can surpass several lidar-using E2E baselines on NAVSIM. The ADDT decoupling-plus-alignment idea is practically useful (5-step denoising with competitive PDMS and large speedups versus text-token action generation). Ablations in Tables 3–7 are systematic and credit-worthy: each claimed module is removed and produces a consistent PDMS drop. The 85k Game-CoT resource could be valuable to the community if its quality and independence from GT actions are better established. The result is incremental relative to concurrent VLA+world-model lines, but the empirical package (SOTA closed-loop score, efficiency, and component ablations) is of clear interest to the autonomous-driving community.

major comments (3)
  1. §3.3 (Game-CoT annotation): The pipeline explicitly feeds GT actions as “guiding hints” so that Qwen3-VL-Plus reconstructs causal chains to the already-known optimal action under a fixed Stackelberg template. This makes the four-step CoT closer to post-hoc rationalization of logged behavior than independent strategic forecasting. The paper reports no human audit rates, inter-annotator agreement, free-generation (no-GT-hint) baselines, or hallucination/consistency metrics for the 85k set. Table 6’s +0.8 PDMS from CoT alone is therefore ambiguous (ordinary VQA regularization vs. genuine game-theoretic supervision). Because dual-level “strategic social reasoning” is a central claim of the abstract and contributions list, the authors should either (i) quantify annotation fidelity and independence from GT, or (ii) substantially soften claims that Game-CoT supplies proactive world cognition ra
  2. §3.1–3.2 and Table 4: Semantic world-head futures (Eq. 6) and ADDT joint multi-agent trajectories (Eqs. 2–4, 8) are both presented as world cognition, yet the manuscript never measures consistency between them (e.g., ADE/FDE between world-head forecasts and ADDT surrounding-agent outputs, or conflict rate under the same scene). Table 4 shows only modest synergy (semantic Cur+Fut 88.1, generative-only 87.4, both 89.3 after three-stage SFT). Without a consistency or complementarity analysis, the claim that the framework “bridges” semantic forecasting with generative evolution remains architectural rather than empirically demonstrated as a coherent dual-level world model.
  3. §3.4 / Eq. (9) and evaluation protocol: Stage-4 DiffGRPO optimizes a reward that includes the same NAVSIM PDMS later reported as the primary result (ri = r_PDMS − λ_surr L1). While BC regularization and group-relative advantages are standard, the paper should report the three-stage SFT checkpoint’s full metric breakdown on navtest alongside the RFT model (Table 1 is RFT-only; Table 2 is SFT-only on v2) and, ideally, an ablated RFT reward that does not directly optimize PDMS components. Otherwise the 3.6 PDMS jump from Stage 3→4 (Table 3) is hard to interpret as generalization rather than direct metric optimization on the evaluation score.

Circularity Check

2 steps flagged

Mild circularity from GT-hinted reverse-engineering of Game-CoT (post-hoc rationalization of known actions) and direct use of the reported PDMS as RFT reward; architectural ablations remain independent.

specific steps
  1. fitted input called prediction [Section 3.3 (Game-CoT Reasoning Annotation)]
    "To minimize hallucinatory outputs and ensure logical consistency, we incorporate Ground-Truth (GT) actions as guiding hints. This compels the VLM to reconstruct explicit causal chains linking observed scene contexts to final GT actions. Ultimately, we construct a Game-CoT dataset comprising 85k high-quality annotations on the NAVSIM benchmark."

    The teacher is given the target action and forced to produce a four-step Stackelberg CoT that ends at that action. The resulting 85k annotations are therefore reverse-engineered rationalizations of known GT behavior rather than independent forecasts of interactive futures. When the student VLM is trained on these labels and the paper attributes PDMS gains (Table 6) to 'game-theoretic world cognition,' part of the claimed strategic supervision is by construction the recovery of the GT-linked explanations.

  2. fitted input called prediction [Section 3.4 (Reinforcement Fine-Tuning) + Table 1 / abstract]
    "Ego driving quality is evaluated via the NAVSIM Predictive Driving Model Score (PDMS), whereas surrounding agents are optimized for accurate motion forecasting via a negative L1 displacement penalty. The overall reward is formulated as ri = rPDMS − λsurr LL1(τsurr) ... WCog-VLA achieves a State-Of-The-Art (SOTA) PDMS score of 92.9."

    Stage-4 DiffGRPO directly optimizes a reward that includes the identical PDMS metric later reported as the primary result. The final 92.9 is therefore the outcome of policy search against the evaluation score itself (regularized by BC), making the headline number partly the result of fitting the policy to the reported criterion on the training distribution.

full rationale

This is an empirical systems paper whose central SOTA claim (PDMS 92.9) is obtained by training on a held-out NAVSIM split and comparing against external baselines; the dual-level architecture and ADDT are not derived by construction from the final metric. The only mild circularities are (1) the 85k Game-CoT labels, which are generated by a teacher VLM that is explicitly given GT actions as guiding hints so that its four-step Stackelberg reasoning is forced to terminate at the already-known optimal action, and (2) Stage-4 DiffGRPO, which optimizes a reward containing the same PDMS later reported as the headline result. Both practices are common in imitation+RL pipelines and do not make the held-out ablations or external comparisons tautological; they merely weaken the claim that the semantic half of dual-level cognition constitutes independent strategic foresight rather than post-hoc rationalization. No self-definitional equations, uniqueness theorems imported from the authors, or ansatz-smuggling citations appear. Score 3 reflects these two mild, non-load-bearing issues without over-penalizing a standard engineering result.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The central performance claim rests on standard deep-learning practice plus several paper-specific modeling choices and an auto-generated reasoning corpus whose fidelity is assumed rather than independently verified. No new physical constants or particles are introduced; the free parameters are ordinary training hyper-parameters and loss weights.

free parameters (4)
  • λ_align, λ_world, λ_bc, λ_surr, α_ego, α_surr
    Loss-weighting coefficients that balance denoising, alignment, world-head, behavior-cloning and surrounding-agent terms; chosen by the authors and not derived from first principles.
  • number of DiT blocks (N1=N2=8) and alignment layer index (6th block)
    Architectural hyper-parameters selected for the ADDT design; ablated only partially.
  • denoising steps at inference (default 5)
    Chosen after observing that alignment reduces the required steps; affects both speed and final PDMS.
  • GRPO group size G=6 and discount γ
    Reinforcement fine-tuning hyper-parameters that directly influence the final reported score.
axioms (4)
  • domain assumption Joint multi-agent trajectory distributions can be accurately modeled by a diffusion process conditioned on VLM hidden states.
    Underpins the entire generative world-model claim (Section 3.2).
  • ad hoc to paper Auto-generated Game-CoT traces produced by Qwen3-VL-Plus with GT-action hints are valid supervision for strategic social reasoning.
    Section 3.3; no human validation or inter-annotator agreement is reported.
  • domain assumption A pre-trained VAE latent of multi-agent trajectories is a faithful target for representation alignment.
    Used to define L_align (Eq. 3); quality of the VAE is taken from GenAD.
  • domain assumption NAVSIM closed-loop metrics (PDMS/EPDMS) are a sufficient proxy for real-world proactive driving performance.
    Standard in the subfield but still an unproven transfer assumption.
invented entities (3)
  • Aligned Decoupled Diffusion Transformer (ADDT) no independent evidence
    purpose: Decouples semantic conditioning from high-frequency trajectory generation while aligning intermediate features to a VAE scene latent, enabling fewer denoising steps.
    New architectural module introduced in Section 3.2; no independent evidence outside the paper’s ablations.
  • Game-CoT (Game-theoretic Chain-of-Thought) dataset no independent evidence
    purpose: Supplies 85 k structured four-step reasoning traces that teach Stackelberg-style social interaction.
    Constructed for this work (Section 3.3); quality depends on the generator VLM and GT hints.
  • Agent tokens injected into VLM no independent evidence
    purpose: Provide explicit 3D spatial and dynamic representations of surrounding traffic participants for semantic world cognition.
    Extracted via TrackFormer on BEV features; the injection mechanism is paper-specific even if the extractor is off-the-shelf.

pith-pipeline@v1.1.0-grok45 · 25582 in / 3361 out tokens · 28313 ms · 2026-07-10T08:28:17.043541+00:00 · methodology

0 comments
read the original abstract

Vision-Language-Action (VLA) models have advanced end-to-end autonomous driving. However, existing methods either lack comprehensive world cognition or suffer from fragmented world foresight, inherently confining these models to reactive driving. To address this limitation, we propose WCog-VLA, a novel dual-level World-Cognitive VLA framework that successfully bridges semantic world forecasting with generative world evolution to achieve proactive autonomous driving. At the semantic level, WCog-VLA unifies world cognition and reasoning by incorporating 3D spatial perception and injecting agent tokens to capture the world dynamics, while concurrently enabling Game-theoretic Chain-of-Thought (Game-CoT) reasoning. At the generative level, we introduce the Aligned Decoupled Diffusion Transformer (ADDT) as a powerful generative world model that synthesizes physically-plausible joint multi-agent trajectories. Through scene representation alignment, ADDT reduces the number of denoising steps required and thus significantly accelerates inference. To facilitate strategic reasoning, we further construct a large-scale dataset featuring 85k Game-CoT annotations. Extensive experiments on the NAVSIM benchmark demonstrate that WCog-VLA achieves a State-Of-The-Art (SOTA) PDMS score of 92.9.

Figures

Figures reproduced from arXiv: 2607.08375 by Binyang Song, Chen Lv, Haoran Wang, Jia Hu, Nuoheng Zhang, Shiyu Fang, Xuerun Yan, Zhexi Lian.

Figure 5
Figure 5. Figure 5: Comparison with previous SOTA method on Navtest. Ground truth Ego planning trajectory Surrounding predicted trajectory Front concat view Without generative world cognition With generative world cognition [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization of proactive driving via generative-level world cognition. Ground truth Prediction Future prediction Predicted boxes Ground truth boxes [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of our semantic-level world cognition. the baseline lacks interactive foresight of the oncoming vehicle and generates an ego-only trajectory, resulting in passive deceleration to blindly avoid spu￾rious conflicts. Conversely, our model synthesizes joint multi-agent trajectories that explicitly forecast the oncoming vehicle’s straight trajectory. This foresight enables ego vehicle to confident… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 65 canonical work pages · 26 internal anchors

  1. [1]

    GPT-4 Technical Report

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  2. [2]

    Qwen3-VL Technical Report

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    In: 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

    Chen, K., Li, Y., Zhang, W., Liu, Y., Li, P., Gao, R., Hong, L., Tian, M., Zhao, X., Li, Z., et al.: Automated evaluation of large vision-language models on self-driving corner cases. In: 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 7817–7826. IEEE (2025)

    work page 2025

  4. [4]

    IEEE Transactions on Pattern Analysis and Ma- chine Intelligence46(12), 10164–10183 (2024)

    Chen, L., Wu, P., Chitta, K., Jaeger, B., Geiger, A., Li, H.: End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence46(12), 10164–10183 (2024)

    work page 2024

  5. [5]

    VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning

    Chen,S.,Jiang,B.,Gao,H.,Liao,B.,Xu,Q.,Zhang,Q.,Huang,C.,Liu,W.,Wang, X.: Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13243 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Chen, Y., Wang, Y., Zhang, Z.: Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 26890–26900 (2025)

    work page 2025

  7. [7]

    Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models

    Chi, H., Gao, H.a., Liu, Z., Liu, J., Liu, C., Li, J., Yang, K., Yu, Y., Wang, Z., Li, W., et al.: Impromptu vla: Open weights and open data for driving vision- language-action models. arXiv preprint arXiv:2505.23757 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    IEEE trans- actions on pattern analysis and machine intelligence45(11), 12878–12895 (2022)

    Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., Geiger, A.: Transfuser: Imi- tation with transformer-based sensor fusion for autonomous driving. IEEE trans- actions on pattern analysis and machine intelligence45(11), 12878–12895 (2022)

    work page 2022

  9. [9]

    arXiv preprint arXiv:2601.06474 (2026)

    Dang, C., Wang, J., Li, G., Hou, Z., You, Z., Ye, H., Ma, J., Chen, L., Wang, Y.: Sparseoccvla: Bridging occupancy and vision-language models via sparse queries for unified 4d scene understanding and planning. arXiv preprint arXiv:2601.06474 (2026)

    work page arXiv 2026

  10. [10]

    Advances in Neural Information Processing Systems37, 28706–28719 (2024)

    Dauner, D., Hallgarten, M., Li, T., Weng, X., Huang, Z., Yang, Z., Li, H., Gilitschenski, I., Ivanovic, B., Pavone, M., et al.: Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems37, 28706–28719 (2024)

    work page 2024

  11. [11]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Ding,X.,Han,J.,Xu,H.,Liang,X.,Zhang,W.,Li,X.:Holisticautonomousdriving understanding by bird’s-eye-view injected multi-modal large models. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13668–13677 (2024)

    work page 2024

  12. [12]

    IEEE Robotics and Automation Letters11(1), 226–233 (2025)

    Feng, R., Xi, N., Chu, D., Wang, R., Deng, Z., Wang, A., Lu, L., Wang, J., Huang, Y.: Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving. IEEE Robotics and Automation Letters11(1), 226–233 (2025)

    work page 2025

  13. [13]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Fu, H., Zhang, D., Zhao, Z., Cui, J., Liang, D., Zhang, C., Zhang, D., Xie, H., Wang, B., Bai, X.: Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 24823–24834 (2025)

    work page 2025

  14. [14]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025) WCog-VLA 17

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    iPad: Iterative Proposal-centric End-to-End Autonomous Driving

    Guo, K., Liu, H., Wu, X., Pan, J., Lv, C.: ipad: Iterative proposal-centric end-to- end autonomous driving. arXiv preprint arXiv:2505.15111 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [16]

    arXiv preprint arXiv:2511.19221 (2025)

    Han, J., Tian, M., Zhu, J., He, F., Zhang, H., Guo, S., Zhu, D., Tang, H., Xu, P., Guo, Y., et al.: Percept-wam: Perception-enhanced world-awareness-action model for robust end-to-end autonomous driving. arXiv preprint arXiv:2511.19221 (2025)

    work page arXiv 2025

  17. [17]

    IEEE Transactions on Intelligent Transportation Systems22(4), 2076–2087 (2020)

    Hang, P., Lv, C., Xing, Y., Huang, C., Hu, Z.: Human-like decision making for au- tonomous driving: A noncooperative game theoretic approach. IEEE Transactions on Intelligent Transportation Systems22(4), 2076–2087 (2020)

    work page 2076

  18. [18]

    Advances in neural information processing systems33, 6840–6851 (2020)

    Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems33, 6840–6851 (2020)

    work page 2020

  19. [19]

    Hu, S., Chen, L., Wu, P., Li, H., Yan, J., Tao, D.: St-p3: End-to-end vision-based autonomousdrivingviaspatial-temporalfeaturelearning.In:EuropeanConference on Computer Vision. pp. 533–549. Springer (2022)

    work page 2022

  20. [20]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Hu, Y., Yang, J., Chen, L., Li, K., Sima, C., Zhu, X., Chai, S., Du, S., Lin, T., Wang, W., et al.: Planning-oriented autonomous driving. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 17853– 17862 (2023)

    work page 2023

  21. [21]

    Trans- portation Research Part C: Emerging Technologies180, 105321 (2025)

    Huang, Z., Sheng, Z., Qu, Y., You, J., Chen, S.: Vlm-rl: A unified vision language models and reinforcement learning framework for safe autonomous driving. Trans- portation Research Part C: Emerging Technologies180, 105321 (2025)

    work page 2025

  22. [22]

    EMMA: End-to-End Multimodal Model for Autonomous Driving

    Hwang, J.J., Xu, R., Lin, H., Hung, W.C., Ji, J., Choi, K., Huang, D., He, T., Cov- ington, P., Sapp, B., et al.: Emma: End-to-end multimodal model for autonomous driving. arXiv preprint arXiv:2410.23262 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  23. [23]

    DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Driving

    Jiang, A., Gao, Y., Sun, Z., Wang, Y., Wang, J., Chai, J., Cao, Q., Heng, Y., Jiang, H., Dong, Y., et al.: Diffvla: Vision-language guided diffusion planning for autonomous driving. arXiv preprint arXiv:2505.19381 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    IRL-VLA: Training an Vision-Language-Action Policy via Reward World Model

    Jiang, A., Gao, Y., Wang, Y., Sun, Z., Wang, S., Heng, Y., Sun, H., Tang, S., Zhu, L., Chai, J., et al.: Irl-vla: Training an vision-language-action policy via reward world model. arXiv preprint arXiv:2508.06571 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  25. [25]

    Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving

    Jiang, B., Chen, S., Liao, B., Zhang, X., Yin, W., Zhang, Q., Huang, C., Liu, W., Wang, X.: Senna: Bridging large vision-language models and end-to-end au- tonomous driving. arXiv preprint arXiv:2410.22313 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  26. [26]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Jiang, B., Chen, S., Xu, Q., Liao, B., Chen, J., Zhou, H., Zhang, Q., Liu, W., Huang,C.,Wang,X.:Vad:Vectorizedscenerepresentationforefficientautonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8340–8350 (2023)

    work page 2023

  27. [27]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Jiang, S., Huang, Z., Qian, K., Luo, Z., Zhu, T., Zhong, Y., Tang, Y., Kong, M., Wang, Y., Jiao, S., et al.: A survey on vision-language-action models for au- tonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4524–4536 (2025)

    work page 2025

  28. [28]

    arXiv preprint arXiv:2601.05640 (2026)

    Li, J., Wu, J., Hu, D., Huang, X., Sun, B., Hao, Z., Lang, X., Zhu, X., Zhang, L.: Sgdrive: Scene-to-goal hierarchical world cognition for autonomous driving. arXiv preprint arXiv:2601.05640 (2026)

    work page arXiv 2026

  29. [29]

    arXiv preprint arXiv:2509.20109 (2025)

    Li, P., Zheng, Y., Wang, Y., Wang, H., Zhao, H., Liu, J., Zhan, X., Zhan, K., Lang, X.: Discrete diffusion for reflective vision-language-action models in autonomous driving. arXiv preprint arXiv:2509.20109 (2025)

    work page arXiv 2025

  30. [30]

    DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving

    Li, Y., Shang, S., Liu, W., Zhan, B., Wang, H., Wang, Y., Chen, Y., Wang, X., An, Y., Tang, C., et al.: Drivevla-w0: World models amplify data scaling law in autonomous driving. arXiv preprint arXiv:2510.12796 (2025) 18 X. Yan et al

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  31. [31]

    Li, Y., Wang, Y., Liu, Y., He, J., Fan, L., Zhang, Z.: End-to-end driving with onlinetrajectoryevaluationviabevworldmodel.In:ProceedingsoftheIEEE/CVF International Conference on Computer Vision. pp. 27137–27146 (2025)

    work page 2025

  32. [32]

    ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving

    Li, Y., Xiong, K., Guo, X., Li, F., Yan, S., Xu, G., Zhou, L., Chen, L., Sun, H., Wang, B., et al.: Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving. arXiv preprint arXiv:2506.08052 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation

    Li, Z., Li, K., Wang, S., Lan, S., Yu, Z., Ji, Y., Li, Z., Zhu, Z., Kautz, J., Wu, Z., et al.: Hydra-mdp: End-to-end multimodal planning with multi-target hydra- distillation. arXiv preprint arXiv:2406.06978 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [34]

    IEEE Transactions on Pattern Analysis and Machine Intelligence47(3), 2020–2036 (2024)

    Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., Dai, J.: Bevformer: learningbird’s-eye-viewrepresentationfromlidar-cameraviaspatiotemporaltrans- formers. IEEE Transactions on Pattern Analysis and Machine Intelligence47(3), 2020–2036 (2024)

    work page 2020

  35. [35]

    Liao, B., Chen, S., Yin, H., Jiang, B., Wang, C., Yan, S., Zhang, X., Li, X., Zhang, Y., Zhang, Q., et al.: Diffusiondrive: Truncated diffusion model for end-to-end au- tonomousdriving.In:ProceedingsoftheComputerVisionandPatternRecognition Conference. pp. 12037–12047 (2025)

    work page 2025

  36. [36]

    arXiv preprint arXiv:2602.06521 (2026)

    Liu, L., Song, Z., Jia, C., Ye, H., Hao, X., Chen, L., et al.: Driveworld-vla: Unified latent-space world modeling with vision-language-action for autonomous driving. arXiv preprint arXiv:2602.06521 (2026)

    work page arXiv 2026

  37. [37]

    CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving

    Liu, P., Ning, Q., Lu, X., Liu, H., Ma, W., She, D., Jia, P., Lang, X., Ma, J.: Omnireason: A temporal-guided vision-language-action framework for autonomous driving. arXiv preprint arXiv:2509.00789 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  38. [38]

    arXiv preprint arXiv:2512.12799 (2025)

    Liu, Z., Huang, R., Yang, R., Yan, S., Wang, Z., Hou, L., Lin, D., Bai, X., Zhao, H.: Drivepi: Spatial-aware 4d mllm for unified autonomous driving understanding, perception, prediction and planning. arXiv preprint arXiv:2512.12799 (2025)

    work page arXiv 2025

  39. [39]

    arXiv preprint arXiv:2509.13769 (2025)

    Luo, Y., Li, F., Xu, S., Lai, Z., Yang, L., Chen, Q., Luo, Z., Xie, Z., Jiang, S., Liu, J., et al.: Adathinkdrive: Adaptive thinking via reinforcement learning for autonomous driving. arXiv preprint arXiv:2509.13769 (2025)

    work page arXiv 2025

  40. [40]

    In: European Conference on Computer Vision

    Marcu, A.M., Chen, L., Hünermann, J., Karnsund, A., Hanotte, B., Chidananda, P., Nair, S., Badrinarayanan, V., Kendall, A., Shotton, J., et al.: Lingoqa: Visual question answering for autonomous driving. In: European Conference on Computer Vision. pp. 252–269. Springer (2024)

    work page 2024

  41. [41]

    Peebles,W.,Xie,S.:Scalablediffusionmodelswithtransformers.In:Proceedingsof the IEEE/CVF international conference on computer vision. pp. 4195–4205 (2023)

    work page 2023

  42. [42]

    arXiv preprint arXiv:2505.152981(2), 3 (2025)

    Qian, K., Jiang, S., Zhong, Y., Luo, Z., Huang, Z., Zhu, T., Jiang, K., Yang, M., Fu, Z., Miao, J., et al.: Agentthink: A unified framework for tool-augmented chain- of-thought reasoning in vision-language models for autonomous driving. arXiv preprint arXiv:2505.152981(2), 3 (2025)

    work page arXiv 2025

  43. [43]

    In: Pro- ceedings of the AAAI Conference on Artificial Intelligence

    Qian, T., Chen, J., Zhuo, L., Jiao, Y., Jiang, Y.G.: Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario. In: Pro- ceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 4542–4550 (2024)

    work page 2024

  44. [44]

    Renz, K., Chen, L., Arani, E., Sinavski, O.: Simlingo: Vision-only closed-loop au- tonomousdrivingwithlanguage-actionalignment.In:ProceedingsoftheComputer Vision and Pattern Recognition Conference. pp. 11993–12003 (2025)

    work page 2025

  45. [45]

    In: 2025 IEEE International Conference on Robotics and Automation (ICRA)

    Sun, W., Lin, X., Shi, Y., Zhang, C., Wu, H., Zheng, S.: Sparsedrive: End-to-end autonomous driving via sparse scene representation. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 8795–8801. IEEE (2025) WCog-VLA 19

    work page 2025

  46. [46]

    Advancing Open-source World Models

    Team, R., Gao, Z., Wang, Q., Zeng, Y., Zhu, J., Cheng, K.L., Li, Y., Wang, H., Xu, Y., Ma, S., et al.: Advancing open-source world models. arXiv preprint arXiv:2601.20540 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  47. [47]

    DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models

    Tian, X., Gu, J., Li, B., Liu, Y., Wang, Y., Zhao, Z., Zhan, K., Jia, P., Lang, X., Zhao, H.: Drivevlm: The convergence of autonomous driving and large vision- language models. arXiv preprint arXiv:2402.12289 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  48. [48]

    DDT: Decoupled Diffusion Transformer

    Wang, S., Tian, Z., Huang, W., Wang, L.: Ddt: Decoupled diffusion transformer. arXiv preprint arXiv:2504.05741 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  49. [49]

    Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail

    Wang, Y., Luo, W., Bai, J., Cao, Y., Che, T., Chen, K., Chen, Y., Diamond, J., Ding, Y., Ding, W., et al.: Alpamayo-r1: Bridging reasoning and action pre- diction for generalizable autonomous driving in the long tail. arXiv preprint arXiv:2511.00088 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  50. [50]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Weng, X., Ivanovic, B., Wang, Y., Wang, Y., Pavone, M.: Para-drive: Parallelized architecture for real-time autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15449–15458 (2024)

    work page 2024

  51. [51]

    World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training

    Xiao,J.,Yang,Y.,Chang,X.,Chen,R.,Xiong,F.,Xu,M.,Zheng,W.S.,Zhang,Q.: World-env: Leveraging world model as a virtual environment for vla post-training. arXiv preprint arXiv:2509.24948 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  52. [52]

    arXiv preprint arXiv:2601.05611 (2026)

    Xie, C., Sun, B., Li, T., Wu, J., Hao, Z., Lang, X., Li, H.: Latentvla: Efficient vision-language models for autonomous driving via latent action prediction. arXiv preprint arXiv:2601.05611 (2026)

    work page arXiv 2026

  53. [53]

    In: Proceed- ings of the Winter Conference on Applications of Computer Vision

    Xing, S., Qian, C., Wang, Y., Hua, H., Tian, K., Zhou, Y., Tu, Z.: Openemma: Open-source multimodal model for end-to-end autonomous driving. In: Proceed- ings of the Winter Conference on Applications of Computer Vision. pp. 1001–1009 (2025)

    work page 2025

  54. [54]

    UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving

    Xiong, Z., Ye, X., Yaman, B., Cheng, S., Lu, Y., Luo, J., Jacobs, N., Ren, L.: Unidrive-wm: Unified understanding, planning and generation world model for autonomous driving. arXiv preprint arXiv:2601.04453 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  55. [55]

    IEEE Robotics and Automation Letters9(10), 8186–8193 (2024)

    Xu, Z., Zhang, Y., Xie, E., Zhao, Z., Guo, Y., Wong, K.Y.K., Li, Z., Zhao, H.: Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Letters9(10), 8186–8193 (2024)

    work page 2024

  56. [56]

    DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving

    Yang, Z., Chai, Y., Jia, X., Li, Q., Shao, Y., Zhu, X., Su, H., Yan, J.: Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving. arXiv preprint arXiv:2505.16278 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  57. [57]

    Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think

    Yu, S., Kwak, S., Jang, H., Jeong, J., Huang, J., Shin, J., Xie, S.: Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  58. [58]

    Drones9(4), 281 (2025)

    Yu, Z., Li, J., Wei, Y., Lyu, Y., Tan, X.: Combining camera–lidar fusion and motion planning using bird’s-eye view representation for end-to-end autonomous driving. Drones9(4), 281 (2025)

    work page 2025

  59. [59]

    DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mamba

    Yuan, C., Zhang, Z., Sun, J., Sun, S., Huang, Z., Lee, C.D.W., Li, D., Han, Y., Wong, A., Tee, K.P., et al.: Drama: An efficient end-to-end motion planner for autonomous driving with mamba. arXiv preprint arXiv:2408.03601 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  60. [60]

    arXiv preprint arXiv:2509.19012 (2025)

    Zhang, D., Sun, J., Hu, C., Wu, X., Yuan, Z., Zhou, R., Shen, F., Zhou, Q.: Pure vision language action (vla) models: A comprehensive survey. arXiv preprint arXiv:2509.19012 (2025)

    work page arXiv 2025

  61. [61]

    In: European Conference on Computer Vision

    Zheng, W., Song, R., Guo, X., Zhang, C., Chen, L.: Genad: Generative end-to-end autonomous driving. In: European Conference on Computer Vision. pp. 87–104. Springer (2024) 20 X. Yan et al

    work page 2024

  62. [62]

    Diffusion-Based Planning for Autonomous Driving with Flexible Guidance

    Zheng, Y., Liang, R., Zheng, K., Zheng, J., Mao, L., Li, J., Gu, W., Ai, R., Li, S.E.,Zhan,X.,etal.:Diffusion-basedplanningforautonomousdrivingwithflexible guidance. arXiv preprint arXiv:2501.15564 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  63. [63]

    arXiv preprint arXiv:2503.23463 (2025)

    Zhou, X., Han, X., Yang, F., Ma, Y., Tresp, V., Knoll, A.: Opendrivevla: Towards end-to-end autonomous driving with large vision language action model. arXiv preprint arXiv:2503.23463 (2025)

    work page arXiv 2025

  64. [64]

    AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning

    Zhou, Z., Cai, T., Zhao, S.Z., Zhang, Y., Huang, Z., Zhou, B., Ma, J.: Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. arXiv preprint arXiv:2506.13757 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  65. [65]

    InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

    Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., et al.: Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025