Pith. sign in

REVIEW 3 major objections 5 minor 64 references

A three-stage VLM pipeline internalizes driving logic without tools and cuts inference latency by 88%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 21:08 UTC pith:BZQ5NKQR

load-bearing objection Solid three-stage recipe that actually moves the reliability-efficiency needle on DriveLMM-o1; evaluation stays open-loop VQA, so treat the safety claims as provisional. the 3 major comments →

arxiv 2607.04179 v1 pith:BZQ5NKQR submitted 2026-07-05 cs.CV cs.AI

CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving

Zhaohong Liu , Hao Ye , Xianlin Zhang , Mengshi Qi This is my paper
classification cs.CV cs.AI
keywords Vision-Language ModelsAutonomous DrivingReinforcement LearningKnowledge DistillationChain-of-ThoughtLatent Thought DistillationMulti-turn VerifierDriveLMM-o1
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

End-to-end vision-language models for driving often hallucinate or become overly cautious when fine-tuned only by imitation, while tool-augmented chain-of-thought methods add fragile external APIs and hundreds of tokens of latency. CritiqueDriveVLM claims that a multi-dimensional verifier can guide multi-turn reinforcement learning so the model internalizes perception, logic, and safety checks without any external tools, producing a high-accuracy Teacher. A second step then aligns a Student model’s hidden state at the answer token with the Teacher’s final reasoning state, transferring that logic into a short, CoT-free answer. On the DriveLMM-o1 benchmark the Teacher reaches 76.54% multiple-choice quality and the Student keeps 68.59% while dropping average latency from 3482 ms to 416 ms. If the claim holds, safety-critical driving VLMs can keep deep reasoning without paying the real-time cost of explicit thought or brittle tools.

Core claim

A tool-free Teacher trained by critique-driven multi-turn RL with a multi-dimensional verifier reaches state-of-the-art 76.54% MCQ on DriveLMM-o1, and Latent Thought Distillation that aligns the Student’s hidden answer state with the Teacher’s final think state compresses that capability into a CoT-free Student that still scores 68.59% MCQ while cutting latency 88% to 416 ms.

What carries the argument

Latent Thought Distillation: cosine alignment of the Student’s hidden state at the <answer> token with the Teacher’s hidden state at the final </think> token of its multi-turn trajectory, combined with ordinary cross-entropy on the short answer, so deep logic is stored in latent space rather than emitted as text.

Load-bearing premise

Matching one hidden vector at the answer token is enough to transfer the Teacher’s multi-turn safety and logic checks into a short CoT-free Student for real driving decisions.

What would settle it

On the same DriveLMM-o1 test set, or in closed-loop simulation, a Student trained only with answer cross-entropy (no latent alignment) matches or exceeds the 68.59% MCQ and the safety metrics of the aligned Student; alternatively, a closed-loop driving trial shows the distilled Student fails scenarios the Teacher solves.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. CritiqueDriveVLM proposes a three-stage pipeline for autonomous-driving VLMs that aims to resolve the reliability–latency trade-off of CoT and tool-augmented methods. Stage 1 warms up a Qwen3-VL-8B policy with structured CoT SFT and trains a frozen multi-dimensional verifier (perception/logic/safety scores plus natural-language critique) on ground-truth positives and hard negatives. Stage 2 applies GRPO with composite rewards (format, accuracy, verifier scores) and a step-decay multi-turn penalty (K=2) so that the Teacher internalizes logical refinement without external APIs, reaching 76.54% MCQ and 80.48% Overall Reasoning on DriveLMM-o1. Stage 3 freezes the Teacher, extracts its hidden state at the final </think> token, and aligns the Student’s hidden state at the <answer> token via cosine similarity (Eq. 8) plus cross-entropy, producing a CoT-free Student that retains 68.59% MCQ at 28.83 tokens / 416 ms (88% latency cut versus the Teacher). Ablations (Tables 3–4) and a qualitative pedestrian-crossing case support progressive gains from accuracy reward → verifier → multi-turn interaction and from the latent-alignment term.

Significance. If the reported open-loop gains hold, the work supplies a practical, tool-free route from slow System-2 reasoning to low-latency System-1 execution that is directly relevant to real-time driving VLMs. Strengths include a clean progressive ablation of the multi-turn RL components (Table 4), an explicit isolation of the latent-alignment loss (Table 3), public code, and a concrete latency reduction that is rarely quantified so carefully in this literature. The multi-dimensional verifier and the latent-thought objective are reusable design patterns beyond the present benchmark. The central limitation is evaluation scope: all claims rest on a single open-loop VQA suite (DriveLMM-o1 / nuScenes) without closed-loop control, multi-seed statistics, or real-vehicle validation; that scope is already acknowledged by the authors’ own framing and does not invalidate the reported numbers within their stated setting.

major comments (3)
  1. §4.1–4.3 and Tables 2–3: All quantitative claims (SOTA Teacher MCQ 76.54%, Student 68.59% at 416 ms) rest exclusively on DriveLMM-o1 open-loop VQA. No closed-loop planner metrics, multi-camera temporal evaluation, or real-vehicle transfer is reported. For a safety-critical claim of “highly robust pathway for low-latency autonomous driving,” at least one additional closed-loop or multi-benchmark result (or an explicit, quantified limitation statement) is needed to keep the central reliability–efficiency claim proportionate.
  2. §3.3 Eq. (8) and Table 3: The load-bearing assumption that cosine alignment of h_answer_S with the Teacher’s final h_think_T transfers multi-turn logical depth is supported only by the MCQ lift of the Student over CoT-free SFT (68.59% vs 61.73%). No intermediate diagnostic (e.g., probing of latent risk/perception features, or comparison against token-level CoT distillation or answer-only distillation under matched compute) is provided. A short controlled ablation isolating what is transferred would strengthen the claim that latent thought distillation, rather than simply more answer supervision, is responsible for the retained reasoning depth.
  3. §4.2 Table 2 and §4.1: No error bars, multi-seed runs, or statistical significance tests accompany the 76.54% / 68.59% figures. Given that GRPO and multi-turn sampling are stochastic and that several competing methods lie within a few points, reporting variance (or at least confirming single-run stability) is necessary for the SOTA ranking to be taken as robust.
minor comments (5)
  1. Abstract and throughout: the benchmark is written both as “DriveLMM-01” and “DriveLMM-o1”; standardize on the official spelling used in the cited paper.
  2. §3.2 Eq. (3) and Table 1: the functional form of the step-decay multi-turn penalty P_mt(T) is described only qualitatively (“constant decay value” for K=2). An explicit formula would aid reproducibility.
  3. §3.1: the verifier is trained with Qwen3-VL-235B labels after human filtering, yet the size, architecture, and training recipe of the deployed verifier itself are not stated; a short paragraph would clarify whether it is also an 8B model or a larger frozen judge.
  4. Figure 1 and Figure 3 captions are dense; a one-sentence takeaway under each panel would improve readability.
  5. §4.1 Implementation Details: LoRA rank, target modules, and the precise value of α (verifier weight) are omitted; they belong in Appendix B or the main text for exact reproduction.

Circularity Check

0 steps flagged

No significant circularity: empirical three-stage pipeline with independently defined rewards, verifier, and distillation objective evaluated on held-out DriveLMM-o1 metrics.

full rationale

The paper's central claims are experimental performance numbers (Teacher MCQ 76.54%, Student 68.59% at 28.83 tokens / 416 ms) obtained by training on DriveLMM-o1 and evaluating under the official protocol. Stage-1 verifier is trained on GT positives plus hard negatives scored by an external larger model (Qwen3-VL-235B) followed by human filtering; its scalar outputs and critiques are inputs to Stage-2 GRPO, not derived from the final test metrics. Composite rewards (R_fmt, R_acc against GT, multi-dimensional R_verif, step-decay P_mt) and the GRPO objective (Eqs. 2-7) are defined independently of the reported MCQ/Reasoning scores. Latent Thought Distillation (Eqs. 8-9) aligns Student h_answer_S to Teacher h_think_T by cosine similarity as a design choice; the subsequent MCQ lift over CoT-free SFT baselines is an empirical outcome, not forced by construction. Ablations (Table 4) and efficiency comparisons (Table 3) further isolate each component. Same-dataset reuse across stages is standard practice and does not make the reported gains tautological. No self-definitional equations, fitted-parameter-as-prediction, load-bearing self-citation uniqueness claims, or renamed known results appear. The work is self-contained empirical ML; circularity score is therefore 0.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 3 invented entities

The central performance claims rest on a handful of hand-chosen hyperparameters, the assumption that a frozen multi-dimensional verifier labeled by a larger VLM is a faithful reward model, and the modeling choice that cosine alignment of final hidden states transfers multi-turn reasoning. No free parameters are fitted to the test set itself, but several are chosen without sensitivity analysis.

free parameters (5)
  • λ (alignment loss weight) = 0.5
    Set to 0.5 in Stage 3; balances CE answer loss against latent cosine alignment; no sweep reported.
  • K (max multi-turn limit) = 2
    Fixed to 2; controls how many critique rounds are allowed before the step-decay penalty is applied.
  • α (verifier score weight)
    Scalar multiplier on the sum of perception/logic/safety scores inside R_base; value not numerically stated beyond being a coefficient.
  • G (GRPO group size) = 4
    Number of rollouts per question set to 4; affects advantage normalization variance.
  • learning rates (SFT / GRPO / distillation) = 1e-4 / 2e-6 / 2e-5
    1e-4, 2e-6, 2e-5 respectively; standard but still free choices that affect final numbers.
axioms (4)
  • domain assumption GRPO with format + accuracy + multi-dimensional verifier rewards plus a step-decay multi-turn penalty produces a policy that internalizes logical deduction rather than merely gaming the verifier.
    Invoked throughout Stage 2 (Eqs. 2–7 and Table 1); supported by ablation but not proven.
  • ad hoc to paper Cosine similarity between the Student’s hidden state at the <answer> token and the Teacher’s hidden state at the final </think> token is a sufficient surrogate for transferring multi-turn System-2 reasoning.
    Core of Latent Thought Distillation (Eq. 8); chosen over token-level CoT imitation without theoretical guarantee.
  • domain assumption Discrete scores {0, 0.5, 1.0} on perception, logic and safety produced by Qwen3-VL-235B plus human filtering constitute a reliable multi-dimensional reward model for driving CoT.
    Stage 1 verifier construction; the entire RL signal depends on this labeling pipeline.
  • domain assumption Open-loop MCQ / reasoning scores on DriveLMM-o1 are a meaningful proxy for real-time autonomous-driving safety and latency constraints.
    All quantitative claims rest on this single benchmark; closed-loop evaluation is left to future work.
invented entities (3)
  • Multi-dimensional verifier (perception / logic / safety scores + natural-language critique) no independent evidence
    purpose: Supplies both scalar rewards for GRPO and textual critiques for multi-turn refinement without external perception tools.
    Constructed in Stage 1 from hard negatives and 235B labels; no independent public benchmark of its accuracy is provided.
  • Latent Thought Distillation objective (align h_answer_S with h_think_T) no independent evidence
    purpose: Compresses the Teacher’s fully converged reasoning state into a CoT-free Student.
    Defined by Eq. 8; inspired by prior latent-reasoning work but newly applied here as the sole transfer mechanism.
  • Step-decay multi-turn penalty P_mt(T) no independent evidence
    purpose: Forces first-attempt correctness so the policy does not remain dependent on external critiques.
    Introduced in Stage 2 reward (Eq. 3); value of the decay constant is not numerically specified.

pith-pipeline@v1.1.0-grok45 · 19895 in / 3482 out tokens · 38673 ms · 2026-07-11T21:08:18.783112+00:00 · methodology

0 comments
read the original abstract

End-to-end Vision-Language Models (VLMs) show immense potential in autonomous driving. However, standard Supervised Fine-Tuning (SFT) often suffers from reasoning hallucinations and conservative biases. While traditional tool-augmented frameworks and Chain-of-Thought (CoT) approaches mitigate these issues, they incur exorbitant token consumption and unacceptable latency, rendering real-time deployment impractical. To resolve this reliability-efficiency trade-off, we propose CritiqueDriveVLM, a novel unified three-stage framework internalizing reasoning directly into the VLM. First, we introduce Critique-Driven Multi-Turn Reinforcement Learning (RL) guided by a multi-dimensional verifier. By providing granular scalar feedback and a multi-turn penalty, we force the policy to internalize logical deduction, cultivating a robust System-2 Teacher that achieves high accuracy without fragile external tools. Subsequently, we propose Latent Thought Distillation to overcome the latency bottleneck. By aligning the Student's latent representations with the Teacher's fully converged reasoning states, we compress deep logical capabilities into a fast, CoT-free System-1 Student. Extensive experiments on the widely-used DriveLMM-01 benchmark demonstrate remarkable improvements. Compared to the base model, our tool-free Teacher significantly boosts Multiple Choice Quality (MCQ) from 55.54% to a state-of-the-art 76.54%. Crucially, our distilled Student preserves competitive reasoning depth while drastically minimizing generation length to an average of merely 28 tokens. This slashes inference latency by 88% (from 3482 ms to 416 ms), paving a highly robust pathway for low-latency autonomous driving.Our source code is available at https://github.com/MICLAB-BUPT/CritiqueDriveVLM.

Figures

Figures reproduced from arXiv: 2607.04179 by Hao Ye, Mengshi Qi, Xianlin Zhang, Zhaohong Liu.

Figure 1
Figure 1. Figure 1: Paradigm comparison of VLM-based autonomous driving. (a) Standard SFT is prone to reasoning hallucinations and conservative biases. (b) Tool-Augmented methods suffer from brittle external APIs and high latency. (c) Critique-Driven RL (Teacher) internalizes deep logic without relying on external tools. (d) Latent Thought Distillation (Student) enables instant, CoT-free execution, eliminating the overhead of… view at source ↗
Figure 2
Figure 2. Figure 2: Overall pipeline of the proposed CritiqueDriveVLM framework. (i) Stage 1: Warm-up SFT and Verifier Construction establishing a structural reasoning format and training a multi-dimensional verifier; (ii) Stage 2: Critique-Driven Multi-Turn RL cultivating a highly reliable Teacher via verifier feedback; and (iii) Stage 3: Latent Thought Distillation internalizing the Teacher’s latent representations into a C… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison in a pedestrian crossing scenario. The Baseline hallu￾cinates and misses the pedestrian. Our Stage-2 Teacher corrects this error through verifier-guided critiques, while our Stage-3 Student directly predicts the safe action without explicit CoT overhead. a pedestrian crossing. As illustrated, the standard SFT Baseline suffers from severe visual hallucination; it explicitly reasons th… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

64 extracted references · 18 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2511.21631 (2025)

    Bai, S., Cai, Y., Chen, R., et al.: Qwen3-VL technical report. arXiv preprint arXiv:2511.21631 (2025)

    Pith/arXiv arXiv 2025

  2. [2]

    arXiv preprint arXiv:2502.13923 (2025)

    Bai, S., Chen, K., Liu, X., et al.: Qwen2.5-VL technical report. arXiv preprint arXiv:2502.13923 (2025)

    Pith/arXiv arXiv 2025

  3. [3]

    In: IEEE Conf

    Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuScenes: A multimodal dataset for autonomous driving. In: IEEE Conf. Comput. Vis. Pattern Recog. (2020)

    2020

  4. [4]

    arXiv preprint arXiv:2412.05271 (2025)

    Chen, Z., Wang, W., Cao, Y., et al.: Expanding performance boundaries of open- source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271 (2025)

    Pith/arXiv arXiv 2025

  5. [5]

    Deng, W., Zhang, X., Qi, M.: Active exploring like a pigeon: Reinforcing spatial reasoning via agentic vision-language models. In: Int. Conf. Mach. Learn. (2026)

    2026

  6. [6]

    arXiv preprint arXiv:2311.01460 (2023)

    Deng, Y., Prasad, K., Fernandez, R., et al.: Implicit chain of thought reasoning via knowledge distillation. arXiv preprint arXiv:2311.01460 (2023)

    Pith/arXiv arXiv 2023

  7. [7]

    Ettinger, S., Cheng, S., Caine, B., Liu, C., Zhao, H., Pradhan, S., Chai, Y., Sapp, B., Qi, C.R., Zhou, Y., et al.: Large scale interactive motion forecasting for autonomous driving: The Waymo open motion dataset. In: Int. Conf. Comput. Vis. pp. 9710–9719 (2021)

    2021

  8. [8]

    arXiv preprint arXiv:1807.08048 (2018)

    Fan, H., Zhu, F., Liu, C., Zhang, L., et al.: Baidu Apollo EM motion planner. arXiv preprint arXiv:1807.08048 (2018)

    Pith/arXiv arXiv 2018

  9. [9]

    In: IEEE Conf

    Gao, J., Sun, C., Zhao, H., et al.: VectorNet: Encoding HD maps and agent dynamics from vectorized representation. In: IEEE Conf. Comput. Vis. Pattern Recog. (2020)

    2020

  10. [10]

    Gu, Y., Dong, L., Wei, F., Huang, M.: MiniLLM: Knowledge distillation of large language models. In: Int. Conf. Learn. Represent. (2024)

    2024

  11. [11]

    arXiv preprint arXiv:2501.12948 (2025)

    Guo, D., Yang, D., Zhang, H., et al.: DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

    Pith/arXiv arXiv 2025

  12. [12]

    arXiv preprint arXiv:2412.06769 (2024)

    Hao, S., Sukhbaatar, S., Su, D., et al.: Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769 (2024)

    Pith/arXiv arXiv 2024

  13. [13]

    arXiv preprint arXiv:1503.02531 (2015)

    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

    Pith/arXiv arXiv 2015

  14. [14]

    In: Findings Assoc

    Hsieh, C.Y., Li, C.L., Yeh, C.K., et al.: Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In: Findings Assoc. Comput. Linguist. (ACL) (2023)

    2023

  15. [15]

    Hu, E.J., Shen, Y., Wallis, P., et al.: LoRA: Low-rank adaptation of large language models. In: Int. Conf. Learn. Represent. (2022) 16 Z. Liu et al

    2022

  16. [16]

    In: IEEE Conf

    Hu, Y., Yang, J., Chen, L., et al.: Planning-oriented autonomous driving. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

    2023

  17. [17]

    In: IEEE/RSJ Int

    Ishaq, A., Lahoud, J., More, K., et al.: DriveLMM-o1: A step-by-step reason- ing dataset and large multimodal model for driving scenario understanding. In: IEEE/RSJ Int. Conf. Intell. Robots Syst. (2025)

    2025

  18. [18]

    In: Proc

    Islam, R., Moushi, O.M.: GPT-4o: The cutting-edge advancement in multimodal LLM. In: Proc. Comput. Conf. pp. 47–60. Springer (2025)

    2025

  19. [19]

    Jiang, B., Chen, S., Xu, Q., et al.: VAD: Vectorized scene representation for efficient autonomous driving. In: Int. Conf. Comput. Vis. (2023)

    2023

  20. [20]

    arXiv preprint arXiv:2503.07608 (2025)

    Jiang, B., Chen, S., Zhang, Q., Liu, W., Wang, X.: AlphaDrive: Unleashing the power of VLMs in autonomous driving via reinforcement learning and reasoning. arXiv preprint arXiv:2503.07608 (2025)

    Pith/arXiv arXiv 2025

  21. [21]

    arXiv preprint arXiv:2506.10406 (2025)

    Jiang,Y.,Xiong,Y.,Yuan,Y.,etal.:PAG:Multi-turnreinforcedLLMself-correction with policy as generative verifier. arXiv preprint arXiv:2506.10406 (2025)

    Pith/arXiv arXiv 2025

  22. [22]

    Macmillan (2011)

    Kahneman, D.: Thinking, Fast and Slow. Macmillan (2011)

    2011

  23. [23]

    In: Proc

    Levinson, J., Askeland, J., Becker, J., et al.: Towards fully autonomous driving: Systems and algorithms. In: Proc. IEEE Intell. Veh. Symp. (IV) (2011)

    2011

  24. [24]

    In: Conf

    Li, Y., Du, Y., Zhou, K., et al.: Evaluating object hallucination in large vision- language models. In: Conf. Empir. Methods Nat. Lang. Process. (2023)

    2023

  25. [25]

    Li, Z., Wang, W., Li, H., et al.: BEVFormer: Learning Bird’s-Eye-View represen- tation from multi-camera images via spatiotemporal transformers. In: Eur. Conf. Comput. Vis. (2022)

    2022

  26. [26]

    In: AAAI (2026)

    Liao, D., Qi, M., Liu, L., Ma, H.: Improving batch normalization with test-time adaptation for robust object detection in self-driving. In: AAAI (2026)

    2026

  27. [27]

    Lightman, H., Kosaraju, V., Burda, Y., et al.: Let’s verify step by step. In: Int. Conf. Learn. Represent. (2024)

    2024

  28. [28]

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Adv. Neural Inform. Process. Syst. (2023)

    2023

  29. [29]

    arXiv preprint arXiv:2405.20797 (2024)

    Lu, S., Li, Y., Chen, Q.G., et al.: Ovis: Structural embedding alignment for multi- modal large language model. arXiv preprint arXiv:2405.20797 (2024)

    Pith/arXiv arXiv 2024

  30. [30]

    In: IEEE Conf

    Lv, C., Qi, M., Liu, L., Ma, H.: T2SG: Traffic topology scene graph for topology reasoning in autonomous driving. In: IEEE Conf. Comput. Vis. Pattern Recog. (2025)

    2025

  31. [31]

    Madaan, A., Tandon, N., Gupta, P., et al.: Self-Refine: Iterative refinement with self-feedback. In: Adv. Neural Inform. Process. Syst. (2023)

    2023

  32. [32]

    Nie, M., Peng, R., Wang, C., et al.: Reason2Drive: Towards interpretable and chain-based reasoning for autonomous driving. In: Eur. Conf. Comput. Vis. (2024)

    2024

  33. [33]

    Ouyang, L., Wu, J., Jiang, X., et al.: Training language models to follow instructions with human feedback. In: Adv. Neural Inform. Process. Syst. (2022)

    2022

  34. [34]

    IEEE Trans

    Qi, M., Lv, C., Ma, H.: Robust disentangled counterfactual learning for physical audiovisual commonsense reasoning. IEEE Trans. Pattern Anal. Mach. Intell. (2026)

    2026

  35. [35]

    In: IEEE Conf

    Qi, M., Peng, J., Zhang, X., Ma, H.: Towards balanced multi-modal learning in 3D human pose estimation. In: IEEE Conf. Comput. Vis. Pattern Recog. (2026)

    2026

  36. [36]

    IEEE Trans

    Qi, M., Qin, J., Yang, Y., Wang, Y., Luo, J.: Semantics-aware spatial-temporal binaries for cross-modal video retrieval. IEEE Trans. Image Process. (2021)

    2021

  37. [37]

    In: ACM Int

    Qi, M., Qin, J., Zhen, X., Huang, D., Yang, Y., Luo, J.: Few-shot ensemble learn- ing for video classification with SlowFast memory networks. In: ACM Int. Conf. Multimedia (2020)

    2020

  38. [38]

    IEEE Trans

    Qi, M., Wang, Y., Li, A., Luo, J.: Sports video captioning via attentive motion representation and group relationship modeling. IEEE Trans. Circuit Syst. Video Technol. (2019) CritiqueDriveVLM 17

    2019

  39. [39]

    IEEE Trans

    Qi, M., Wu, Y., Yun, W., Zhang, X., Ma, H.: Explainable action form assessment by exploiting multimodal chain-of-thoughts reasoning. IEEE Trans. Image Process. (2026)

    2026

  40. [40]

    IEEE Trans

    Qi, M., Ye, H., Peng, J., Ma, H.: Action quality assessment via hierarchical pose- guided multi-stage contrastive regression. IEEE Trans. Image Process. (2025)

    2025

  41. [41]

    IEEE Trans

    Qi, M., Zhu, P., Li, X., Bi, X., Qi, L., Ma, H., Yang, M.H.: DC-SAM: In-context segment anything in images and videos via dual consistency. IEEE Trans. Pattern Anal. Mach. Intell. (2026)

    2026

  42. [42]

    In: Findings Assoc

    Qian, K., Jiang, S., Zhong, Y., et al.: AgentThink: A unified framework for tool- augmented chain-of-thought reasoning in vision-language models for autonomous driving. In: Findings Assoc. Comput. Linguist. (EMNLP) (2025)

    2025

  43. [43]

    Radford, A., Kim, J.W., Hallacy, C., et al.: Learning transferable visual models from natural language supervision. In: Int. Conf. Mach. Learn. (2021)

    2021

  44. [44]

    Romero, A., Ballas, N., Kahou, S.E., et al.: FitNets: Hints for thin deep nets. In: Int. Conf. Learn. Represent. (2015)

    2015

  45. [45]

    arXiv preprint arXiv:2506.11234 (2025)

    Rowe, L., de Schaetzen, R., Girgis, R., et al.: Poutine: Vision-language-trajectory pre-training and reinforcement learning post-training enable robust end-to-end autonomous driving. arXiv preprint arXiv:2506.11234 (2025)

    arXiv 2025

  46. [46]

    arXiv preprint arXiv:1707.06347 (2017)

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    Pith/arXiv arXiv 2017

  47. [47]

    In: IEEE Conf

    Shao, H., Hu, Y., Wang, L., et al.: LMDrive: Closed-loop end-to-end driving with large language models. In: IEEE Conf. Comput. Vis. Pattern Recog. (2024)

    2024

  48. [48]

    arXiv preprint arXiv:2511.22570 (2025)

    Shao, Z., Luo, Y., Lu, C., et al.: DeepSeekMath-v2: Towards self-verifiable mathe- matical reasoning. arXiv preprint arXiv:2511.22570 (2025)

    arXiv 2025

  49. [49]

    arXiv preprint arXiv:2402.03300 (2024)

    Shao, Z., Wang, P., Zhu, Q., et al.: DeepSeekMath: Pushing the limits of math- ematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024)

    Pith/arXiv arXiv 2024

  50. [50]

    In: Conf

    Shen, Z., Yan, H., Zhang, L., et al.: CODI: Compressing chain-of-thought into continuous space via self-distillation. In: Conf. Empir. Methods Nat. Lang. Process. (2025)

    2025

  51. [51]

    Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., Yao, S.: Reflexion: Language agents with verbal reinforcement learning. In: Adv. Neural Inform. Process. Syst. (2023)

    2023

  52. [52]

    Sima, C., Renz, K., Chitta, K., et al.: DriveLM: Driving with graph visual question answering. In: Eur. Conf. Comput. Vis. (2024)

    2024

  53. [53]

    In: Findings Assoc

    Thawakar, O., Dissanayake, D., More, K.P., et al.: LlamaV-o1: Rethinking step- by-step visual reasoning in LLMs. In: Findings Assoc. Comput. Linguist. (ACL) (2025)

    2025

  54. [54]

    arXiv preprint arXiv:2402.12289 (2024)

    Tian, X., Gu, J., Li, B., Liu, Y., Zhao, Z., Wang, Y., Zhan, K., Jia, P., Lang, X., Zhao, H.: DriveVLM: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289 (2024)

    Pith/arXiv arXiv 2024

  55. [55]

    arXiv preprint arXiv:2312.09245 (2023)

    Wang, W., Xie, J., Hu, C., et al.: DriveMLM: Aligning multi-modal large language models with behavioral planning states for autonomous driving. arXiv preprint arXiv:2312.09245 (2023)

    arXiv 2023

  56. [56]

    Wang, Y., Kordi, Y., Mishra, S., et al.: Self-Instruct: Aligning language models with self-generated instructions. In: Ann. Meet. Assoc. Comput. Linguist. (2023)

    2023

  57. [57]

    Wei, J., Wang, X., Schuurmans, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. In: Adv. Neural Inform. Process. Syst. (2022)

    2022

  58. [58]

    Xu, G., Jin, P., Wu, Z., et al.: LLaVA-CoT: Let vision language models reason step-by-step. In: Int. Conf. Comput. Vis. (2025) 18 Z. Liu et al

    2025

  59. [59]

    arXiv preprint arXiv:2412.18319 (2024)

    Yao, H., Huang, J., Wu, W., et al.: Mulberry: Empowering MLLM with o1-like reasoning and reflection via collective Monte Carlo tree search. arXiv preprint arXiv:2412.18319 (2024)

    Pith/arXiv arXiv 2024

  60. [60]

    In: ACM Int

    Ye, H., Qi, M., Liu, Z., Liu, L., Ma, H.: SafeDriveRAG: Towards safe autonomous driving with knowledge graph-based retrieval-augmented generation. In: ACM Int. Conf. Multimedia (2025)

    2025

  61. [61]

    arXiv preprint arXiv:2407.06023 (2024)

    Yu, P., Xu, J., Weston, J., Kulikov, I.: Distilling System 2 into System 1. arXiv preprint arXiv:2407.06023 (2024)

    Pith/arXiv arXiv 2024

  62. [62]

    arXiv preprint arXiv:2512.14044 (2025)

    Zhang, Z., Zheng, H., Wang, Y., et al.: OmniDrive-R1: Reinforcement-driven inter- leaved multi-modal chain-of-thought for trustworthy vision-language autonomous driving. arXiv preprint arXiv:2512.14044 (2025)

    Pith/arXiv arXiv 2025

  63. [63]

    arXiv preprint arXiv:2506.13757 (2025)

    Zhou, Z., Cai, T., Zhao, S.Z., et al.: AutoVLA: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. arXiv preprint arXiv:2506.13757 (2025)

    Pith/arXiv arXiv 2025

  64. [64]

    Zhu, P., Qi, M., Li, X., Li, W., Ma, H.: Unsupervised self-driving attention prediction via uncertainty mining and knowledge embedding. In: Int. Conf. Comput. Vis. (2023)

    2023