Pith. sign in

REVIEW 2 major objections 4 minor 2 cited by

Spatial-aware Vision Language Model for Autonomous Driving

T0 review · 2 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LVLDrive claims that injecting LiDAR point-cloud features through a zero-initialized gated Q-Former upgrades a vision-language model with true 3D metric spatial understanding, making it outperform vision-only models in driving planning and

desk verdict LVLDrive is a sensible, well-ablated step for LiDAR-VLM fusion in driving, but its metric-spatial claim rests on a self-built benchmark that mirrors its own training templates, so the headline gap is weaker than it looks. read the letter →

arxiv 2512.24331 v2 pith:IJL6NTNB submitted 2025-12-30 cs.CV

Weijie Wei , Zhipeng Luo , Ling Feng , Venice Erin Liong This is my paper
classification cs.CV
keywords LiDAR-vision-languagemodel3Dmetricspatialreasoningzero-initializedgatedattentionQ-Formerautonomousdrivingplanningquestionansweringopen-loopgroundingevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a pre-trained vision-language model can be upgraded with genuine 3D metric spatial understanding by injecting LiDAR point-cloud features through a zero-initialized gated Q-Former, rather than relying on 2D images alone. On open-loop planning benchmarks and a grounded spatial QA evaluation, the LiDAR-augmented model outperforms its vision-only counterpart, lowering trajectory error, collision rate, and road-boundary intersection rate while sharply raising the quality of parsed 3D object grounding. The reason to care: if correct, explicit 3D metric data—not just image pixels—is necessary to make language-based driving agents trustworthy in safety-critical situations. The paper's core design insight is that unaligned 3D features must be introduced gradually through a learned gate so they do not overwrite the VLM's existing visual-linguistic knowledge.

What carries the argument

The Gradual Fusion Q-Former: a multi-head cross-attention block, based on a prior Q-Former 3D design, in which image features and LiDAR point features are both keys and values, while shared carrier and instance queries are augmented with 3D positional embeddings from reference points. The load-bearing component is the zero-initialized multi-head gate: the point-cloud attention output is multiplied by tanh(g), where g starts at zero, so at the beginning of training the LiDAR stream contributes nothing and its influence ramps up gradually. This preserves the pretrained VLM's stability while allowing the model to learn when to trust 3D cues. The second crucial mechanism is the SA-QA dataset, a

What would settle it

Evaluate LVLDrive zero-shot on an external spatial-QA benchmark built from different scenes, different annotation sources, and different question templates (e.g., a manually constructed test asking for 3D locations of objects not seen in training, with no arrow or mask cue). If mIoU and distance error drop to near chance or to vision-only levels, the paper's claim of learned metric spatial understanding is falsified; if the gap persists, the claim is supported.

Watch

Extended reading notes

Core claim

The central claim is that a VLM's spatial blind spot is not cured by more images or language prompting; it requires a second sensor modality with explicit metric structure. The authors show that by feeding LiDAR point-cloud features into the shared Q-Former as an additional key-value stream, and by gating that stream with a zero-initialized tanh gate per attention head, the model can learn to rely on 3D geometry when needed without catastrophic forgetting of its visual-linguistic priors. They also build a spatial-aware QA dataset derived from ground-truth annotations—including drivable-area checks, 3D attribute descriptions, arrow-guided and masked-box grounding, cross-view distance estimati

Load-bearing premise

The load-bearing premise is that the spatial grounding benchmark measures genuine 3D metric reasoning rather than the model memorizing the answer template from the same ground-truth annotations used to generate its training questions; the training and evaluation question formats are nearly identical, so the reported mIoU may be an in-distribution pattern match.

Editorial extensions

If this is right

  • If LVLDrive's claims hold, VLM-based driving agents can achieve competitive planning metrics while retaining natural-language scene understanding, narrowing the gap with specialized end-to-end planners.
  • A zero-initialized gate is a sufficient mechanism to add a novel modality to a pretrained VLM without destructive interference—a recipe that should transfer to other sensor inputs such as radar or thermal imagery.
  • Explicitly supervising metric outputs (coordinates, sizes, future positions) is what unlocks the model's spatial perception; abstract language supervision alone leaves 3D grounding at zero, so dataset design is the bottleneck, not model capacity.
  • The task designs that mask or arrow-cue objects force the model to bind language to joint 2D-3D features, so the gains should generalize to any scene where calibrated LiDAR and cameras are available.
  • The improvements in collision and road-boundary intersection rates suggest that better metric perception directly translates into safer open-loop planning, a concrete path toward closing the safety gap for LLM-based driving policies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the authors' claims are true but the benchmark is as template-bound as it appears, I would predict that on genuinely out-of-distribution spatial questions—objects seen from novel angles, camera-only occlusions, or question formats never seen in training—the LiDAR advantage will shrink or vanish, because the training distribution already gave away the answer format.
  • A clean testable extension: evaluate the same model zero-shot on an external spatial-QA benchmark constructed from different annotation sources, different scenes, and different question templates; if mIoU collapses, the 'metric understanding' is largely alignment to a template rather than general 3D reasoning.
  • Since the gate starts at zero and ramps up, the gating schedule could be made adaptive per sample—where LiDAR is noisy (fog, darkness, far range) the model should be able to pull the gate back toward zero and explicitly signal when it is ignoring the 3D stream.
  • The dataset is generated automatically from ground-truth annotations at negligible cost and at only 850 training scenes, suggesting this form of cheap synthetic spatial supervision could be applied to any LiDAR-labeled driving corpus, not just the one used here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes LVLDrive, a LiDAR-vision-language framework for autonomous driving. It augments a pre-trained VLM (LLaVA v1.5 with EVA-02 image encoder) with LiDAR point-cloud features through a Gradual Fusion Q-Former that injects 3D features via zero-initialized per-head gates, aiming to avoid catastrophic forgetting while improving metric spatial reasoning. The authors also construct SA-QA, a rule-based spatial QA dataset derived from nuScenes and OpenLane ground truth, and evaluate on nuScenes open-loop planning, DriveLM language metrics, and a self-built BEV 3D-grounding benchmark. They report consistent gains over the OmniDrive-Agent baseline and ablate the fusion configuration, gate design, and SA-QA dataset composition.

Significance. If the reported gains are robust, the paper offers a practical recipe for injecting explicit 3D metric information into VLMs without destroying their pretrained visual-linguistic knowledge, which is a timely and relevant contribution. The zero-initialized gate design is well-motivated, and the ablation study is systematically structured. However, the central evidence for improved 'metric spatial perception' rests on a self-built grounding benchmark that uses the same ground-truth annotations and answer template as the SA-QA training data, so the improvement may partly reflect in-distribution template following rather than a general 3D reasoning capability. The absence of an external spatial QA benchmark (e.g., nuScenes-SpatialQA, which the paper itself cites) and the lack of multi-seed statistics or code/data release further limit the force of the claims.

major comments (2)
  1. [Supplementary Sec. 9; Sec. 4.5; Table 3] The BEV mIoU grounding benchmark is constructed from nuScenes validation ground-truth annotations using the same answer template as SA-QA SP-02 training QAs ('The object is a <category> in the <CAM>, location: (x,y), length: L, ...'). The supplement explicitly states that this template overlap is what makes answer parsing reliable. Consequently, the reported mIoU gains (e.g., 0.18 → 0.21 in Table 3; 0.00 → 0.22 in Table 4) may measure in-distribution output formatting and template adherence rather than a general improvement in 3D metric perception. Since the abstract's central claim is 'metric spatial perception' superiority over vision-only counterparts, this issue is load-bearing. Please evaluate on an external spatial QA benchmark (e.g., nuScenes-SpatialQA [46]) or a held-out set with a different answer template/annotation source, and/or release the benchmark and evaluation code so th
  2. [Table 1 vs. Tables 3/4] There is an internal inconsistency in the reported L2 planning metric. Table 1 reports LVLDrive's average L2 as 0.29 m, while the captions of Tables 3 and 4 state that 'The L2 metric for open-loop planning is consistently around 0.32 meters and is therefore omitted for space.' These two statements cannot both be correct for the same model and evaluation protocol. The discrepancy directly affects the planning-related claims and must be resolved. Additionally, no error bars or multiple-seed results are reported anywhere; given that many of the key differences (e.g., collision rate 0.28% vs. 0.30%, mIoU 0.21 vs. 0.18) are small, the authors should report mean±std over at least three seeds to establish that the differences are not within run-to-run noise.
minor comments (4)
  1. [Abstract; Sec. 1] The abstract contains the typo 'incoperating' (should be 'incorporating'). Section 1 contains a duplicated sentence: 'To explicitly enforce spatial reasoning and encourage the integration of LiDAR features, we further construct SA-QA dataset' appears twice in consecutive paragraphs.
  2. [Sec. 3.1; Sec. 4.2] Minor typos: 'featrues' in Sec. 3.1 should be 'features'; 'Metircs' in Sec. 4.2 should be 'Metrics'.
  3. [Supplementary Sec. 9] The phrase 'nuScenes nuscenes validation set' is redundant; also the choice of the 1.75 m buffer margin for the drivable-area mask and the 15 m distance strata are free parameters whose sensitivity is not examined. Adding a sensitivity analysis would strengthen the benchmark's credibility.
  4. [Sec. 4.3; Table 1] When comparing against end-to-end models (UniAD, VAD, etc.), the values are quoted from their original publications, but the paper adopts a specific 'improved collision rate' definition from BEV-Planner. Please clarify whether all reported numbers use the same evaluation protocol, or at least note which baselines used the stricter definition.

Circularity Check

1 steps flagged · score 6.0 of 10

Metric-spatial claim relies on a grounding benchmark generated with the same answer template as SA-QA training; mIoU partially reduces to template following.

  1. self definitional [Supplementary Sec. 9 'More Details on 3D Grounding Benchmark'; cf. Sec. 4.2 Metrics and Table 5 SP-02]
    "For each selected object, we format its 3D bounding box parameters using the following answer template: The object is a <class> in the <direction>, location: (X, Y), length: L, width: W , height: H, angles in degrees: θ. ... Because similar QA patterns are also used during training, the model typically produces answers that closely follow the desired template. This allows us to reliably parse a BEV bounding box (X, Y, L, W, θ) from the predicted answer and compute the BEV mIoU."

    The 'metric spatial perception' result (Abstract; Sec. 4.2) is measured by parsing model answers into the exact BEV box template used to generate SA-QA training targets (Table 5, SP-02). The benchmark is generated from nuScenes GT with that same template, and the paper explicitly states that template overlap is what makes parsing reliable. Thus the mIoU metric is defined by the training output format; it rewards in-distribution template following rather than independently measuring 3D metric reasoning. Table 4 confirms this: without SA-QA training the model still gives 'seemingly reasonable textual responses' but scores 0.00 because it does not reproduce the trained template. The LiDAR-vs-vision mIoU gap is therefore partly an artifact of the self-built, template-aligned evaluation, not a

full rationale

The fusion mechanism itself is not derivationally circular: the zero-initialized gate is a learned parameter (Eq. 9), the gradual fusion schedule is a training-stability choice, and the open-loop planning numbers in Table 1 are evaluated on the standard external nuScenes benchmark, as are DriveLM language metrics in Table 2. No load-bearing claim is justified only by the authors' own prior work; the base Q-Former 3D block and LLaMA-Adapter gate are external citations. The circularity concern is confined to the 'metric spatial perception' pillar. The grounding benchmark (Supp. Sec. 9) is constructed from nuScenes validation GT with the exact answer template used to generate SA-QA training pairs (Table 5, SP-02), and the paper admits that this template overlap is what permits reliable parsing. Consequently, the reported mIoU partially measures whether the model has memorized the trained output format, not an externally verified 3D metric-reasoning ability. The paper even demonstrates this in Table 4: without SA-QA training, the model produces seemingly reasonable spatial text but cannot be parsed and scores 0.00. Therefore the LiDAR-injection advantage in mIoU is partly an artifact of the self-built, template-aligned evaluation. Separate correctness risk: Table 1 reports average L2 0.29 m for LVLDrive while Tables 3-4 state L2 is 'consistently around 0.32 meters'; this unexplained discrepancy affects the planning claims but is not circularity. Overall, the central planning and language claims have independent support, so this is partial circularity (6), not a fully definitional result (8-10).

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard ML training assumptions plus one load-bearing evaluation assumption: that the self-built grounding benchmark, generated from the same GT and templates as the SA-QA training set, measures genuine 3D metric reasoning rather than in-distribution format recall. No new physical or conceptual entities are introduced; the SA-QA dataset is an artifact of the method, not a postulated entity.

free parameters (2)
  • BEV drivable-area buffer margin = 1.75 m
    Used to label drivable-area QA (SP-01); hand-set, affects generated answers and the binary drivable labels the model learns.
  • Grounding benchmark distance strata = 15 m intervals
    Selected from {15, 20, 25} m to balance object distribution across four ranges; determines which objects are sampled for the grounding metric.
assumptions (5)
  • domain assumption nuScenes 3D annotations and OpenLane lane labels are accurate ground truth for generating QA answers and evaluation targets.
    SA-QA (Sec. 3.4) and the grounding benchmark (Supp. Sec. 9) are generated directly from these labels.
  • domain assumption The Q-Former 3D block (OmniDrive) and zero-init gating (LLaMA-Adapter) preserve pre-trained VLM knowledge while allowing LiDAR injection.
    Sec. 3.3 relies on these prior components without a formal proof that knowledge is preserved.
  • domain assumption Open-loop L2/collision/intersection metrics on nuScenes predict real-world driving safety.
    Table 1 uses these metrics to claim 'reliable driving decision-making'.
  • domain assumption FSDv2 point-cloud features are spatially aligned with the ego coordinate system used for 3D positional embeddings.
    Sec. 3.3 adds 3D positional embeddings from reference points to fuse image and LiDAR features; if the coordinate systems are misaligned, the fusion cannot provide metric reasoning.
  • domain assumption Fine-tuning on template-generated QA transfers to general driving QA and spatial reasoning.
    The model is evaluated on the same template distribution as training; no out-of-distribution spatial benchmark is used to verify transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spatial-aware Vision Language Model for Autonomous Driving." pith.science (2026). https://pith.science/paper/IJL6NTNB

@misc{pith2026251224331,
  author       = {Pith},
  title        = {Pith review of: Spatial-aware Vision Language Model for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IJL6NTNB}},
  note         = {Machine review of arXiv:2512.24331}
}
read the original abstract

While Vision-Language Models (VLMs) show significant promise for end-to-end autonomous driving by leveraging the common sense embedded in language models, their reliance on 2D image cues for complex scene understanding and decision-making presents a critical bottleneck for safety and reliability. Current image-based methods struggle with accurate metric spatial reasoning and geometric inference, leading to unreliable driving policies. To bridge this gap, we propose LVLDrive (LiDAR-Vision-Language), a novel framework specifically designed to upgrade existing VLMs with robust 3D metric spatial understanding for autonomous driving by incoperating LiDAR point cloud as an extra input modality. A key challenge lies in mitigating the catastrophic disturbance introduced by disparate 3D data to the pre-trained VLMs. To this end, we introduce a Gradual Fusion Q-Former that incrementally injects LiDAR features, ensuring the stability and preservation of the VLM's existing knowledge base. Furthermore, we develop a spatial-aware question-answering (SA-QA) dataset to explicitly teach the model advanced 3D perception and reasoning capabilities. Extensive experiments on driving benchmarks demonstrate that LVLDrive achieves superior performance compared to vision-only counterparts across scene understanding, metric spatial perception, and reliable driving decision-making. Our work highlights the necessity of explicit 3D metric data for building trustworthy VLM-based autonomous systems.

Figures

Figures reproduced from arXiv: 2512.24331 by the authors.

Figure 1
Figure 1. (a) Typical image-based VLMs take only images as input and train the LLM through image-conditioned question answering. In contrast, our LVLDrive leverages both image and LiDAR inputs and incorporates carefully designed spatial-aware QAs to encourage effective LiDAR integration and enhance spatial understanding. understanding—precise distances, extents, occlusions, and interactions in 3D. These capabilities are not o… view at source ↗
Figure 2
Figure 2. Overview of LVLDrive. LVLDrive takes text, images, and point clouds as multimodal inputs and employs three pretrained encoders, an LLM, and a Gradual Fusion Q-Former module that bridges visual and linguistic representations to generate task-specific responses. The fire symbol denotes trainable components. with language supervision; and LLaDA [24] adapted poli￾cies to new domains using LLM priors. GPT-style planners … view at source ↗
Figure 3
Figure 3. Gradual Fusion Q-Former. Each block contains two multi-head attention layers. The first layer uses learnable carrier and instance tokens as queries, keys, and values; the keys and values are extended with memory tokens when the memory bank is non￾empty, and both queries and keys are augmented with 3D positional embeddings (3D PE) derived from reference points. The second layer introduces image and point features as … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Examples to illustrate different question-answering pairs in SA-QA. The green dots, box, and line are highlighted only for visualization. The masked region and the red arrows are visible to the vision encoder. are derived from the human-annotated ground-truth labels in…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Steadily moving semi-infinite fracture in plane poroelasticity

    physics.geo-ph 2026-04 unverdicted novelty 7.0 of 10

    XEmbodied achieves SOTA on 18 embodied VQA benchmarks by fusing 3D geometric tokens and distilled physical cues into a 30B VLM with progressive curriculum training.

  2. nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    nuReasoning is a new real-world dataset and benchmark extending nuScenes/nuPlan with 20k clips and multi-type reasoning annotations to evaluate and improve reasoning in long-tail autonomous driving.

Reference graph

Works this paper leans on

88 extracted references · 12 linked inside Pith · cited by 2 Pith papers

  1. [46]

    NuScenes-SpatialQA: A Spa- tial Understanding and Reasoning Benchmark for Vision- Language Models in Autonomous Driving

    Kexin Tian, Jingrui Mao, Yunlong Zhang, Jiwan Jiang, Yang Zhou, and Zhengzhong Tu. NuScenes-SpatialQA: A Spa- tial Understanding and Reasoning Benchmark for Vision- Language Models in Autonomous Driving. InICCVw, 2025. 1, 2, 3

    2025

  2. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 3

    arXiv 2025

  3. [2]

    Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gian- carlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gian- carlo Baldan, and Oscar Beijbom. nuScenes: A multimodal dataset for autonomous driving. InCVPR, 2020. 5, 6, 7, 1, 2

    2020

  4. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. InICCV,

  5. [4]

    SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities. InCVPR, 2024. 3

    2024

  6. [5]

    Persformer: 3d lane detection via perspective transformer and the openlane benchmark

    Li Chen, Chonghao Sima, Yang Li, Zehan Zheng, Jiajie Xu, Xiangwei Geng, Hongyang Li, Conghui He, Jianping Shi, Yu Qiao, and Junchi Yan. Persformer: 3d lane detection via perspective transformer and the openlane benchmark. In European Conference on Computer Vision (ECCV), 2022. 6, 1, 2

    2022

  7. [6]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. InICCV,

  8. [7]

    Spa- tialRGPT: Grounded Spatial Reasoning in Vision Language Models

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Rui- han Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spa- tialRGPT: Grounded Spatial Reasoning in Vision Language Models. InNeurIPS, 2024-10-15. 3

    2024

Show all 88 references
  1. [8]

    Talk2bev: Language-enhanced bird’s-eye view maps for autonomous driving

    Tushar Choudhary, Vikrant Dewangan, Shivam Chandhok, Shubham Priyadarshan, Anushka Jain, Arun K Singh, Sid- dharth Srivastava, Krishna Murthy Jatavallabhula, and K Mad- hava Krishna. Talk2bev: Language-enhanced bird’s-eye view maps for autonomous driving. In2024 IEEE Inter- na...

    2024

  2. [9]

    Talk2car: Taking control of your self-driving car

    Thierry Deruyttere, Simon Vandenhende, Dusan Grujicic, Luc Van Gool, and Marie Francine Moens. Talk2car: Taking control of your self-driving car. InProceedings of the 2019 conference on empirical methods in natural language pro- cessing and the 9th international joint conferen...

    2019

  3. [10]

    Holistic autonomous driving un- derstanding by bird’s-eye-view injected multi-modal large models

    Xinpeng Ding, Jianhua Han, Hang Xu, Xiaodan Liang, Wei Zhang, and Xiaomeng Li. Holistic autonomous driving un- derstanding by bird’s-eye-view injected multi-modal large models. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 13668–13677,

  4. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  5. [12]

    Fsd v2: Improving fully sparse 3d object detection with vir- tual voxels.IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2024

    Lue Fan, Feng Wang, Naiyan Wang, and Zhaoxiang Zhang. Fsd v2: Improving fully sparse 3d object detection with vir- tual voxels.IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2024. 6

    2024

  6. [13]

    Eva-02: A visual representation for neon genesis.Image and Vision Computing, page 105171,

    Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xin- long Wang, and Yue Cao. Eva-02: A visual representation for neon genesis.Image and Vision Computing, page 105171,

  7. [14]

    ORION: A Holistic End-to- End Autonomous Driving Framework by Vision-Language Instructed Action Generation

    Haoyu Fu, Diankun Zhang, Zongchuang Zhao, Jianfeng Cui, Dingkang Liang, Chong Zhang, Dingyuan Zhang, Hongwei Xie, Bing Wang, and Xiang Bai. ORION: A Holistic End-to- End Autonomous Driving Framework by Vision-Language Instructed Action Generation. InICCV, 2025. 7

    2025

  8. [15]

    Multi-Frame, Lightweight & Efficient Vision-Language Mod- els for Question Answering in Autonomous Driving

    Akshay Gopalkrishnan, Ross Greer, and Mohan Trivedi. Multi-Frame, Lightweight & Efficient Vision-Language Mod- els for Question Answering in Autonomous Driving. In CVPRw, 2024. 7

    2024

  9. [16]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. InProceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 3

    2017

  10. [17]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InCVPR, 2021. 3

    2021

  11. [18]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17853–17862, 2023. 7

    2023

  12. [19]

    Drivlme: Enhancing llm-based autonomous driving agents with embodied and social experiences

    Yidong Huang, Jacob Sansom, Ziqiao Ma, Felix Gervits, and Joyce Chai. Drivlme: Enhancing llm-based autonomous driving agents with embodied and social experiences. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3153–3160. IEEE, 2024. 3

    2024

  13. [20]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. 3

    2024 arXiv

  14. [21]

    Omnispa- tial: Towards comprehensive spatial reasoning benchmark for vision language models.arXiv preprint arXiv:2506.03135,

    Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. Omnispa- tial: Towards comprehensive spatial reasoning benchmark for vision language models.arXiv preprint arXiv:2506.03135,

  15. [22]

    Vad: Vectorized scene representation for ef- ficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and 9 Xinggang Wang. Vad: Vectorized scene representation for ef- ficient autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision,...

    2023

  16. [23]

    Textual explanations for self-driving vehicles

    Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, and Zeynep Akata. Textual explanations for self-driving vehicles. InProceedings of the European conference on computer vision (ECCV), pages 563–578, 2018. 2

    2018

  17. [24]

    Driving everywhere with large language model policy adaptation

    Boyi Li, Yue Wang, Jiageng Mao, Boris Ivanovic, Sushant Veer, Karen Leung, and Marco Pavone. Driving everywhere with large language model policy adaptation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14948–14957, 2024. 1, 3

    2024

  18. [25]

    Blip: Bootstrapping language-image pre-training for unified vision- language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision- language understanding and generation. InICML, 2022. 3

    2022

  19. [26]

    Blip- 2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip- 2: bootstrapping language-image pre-training with frozen image encoders and large language models. InICML, 2023. 3

    2023

  20. [27]

    Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M. Alvarez. Is ego status all you need for open- loop end-to-end autonomous driving? InCVPR, 2024. 7

    2024

  21. [28]

    Bevfusion: A simple and robust lidar-camera fusion framework

    Tingting Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia, Zhiwei Lin, Yongtao Wang, Tao Tang, Bing Wang, and Zhi Tang. Bevfusion: A simple and robust lidar-camera fusion framework. InAdvances in Neural Information Processing Systems, 2022. 2

    2022

  22. [29]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InACL Workshop, 2004. 7

    2004

  23. [30]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 3

    2023

  24. [31]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InCVPR, 2024-05-15. 3, 6

    2024

  25. [32]

    Can multi- modal large language models understand spatial relations? arXiv preprint arXiv:2505.19015, 2025

    Jingping Liu, Ziyan Liu, Zhedong Cen, Yan Zhou, Yinan Zou, Weiyan Zhang, Haiyun Jiang, and Tong Ruan. Can multi- modal large language models understand spatial relations? arXiv preprint arXiv:2505.19015, 2025. 3

    2025 arXiv

  26. [33]

    Omnireason: A temporal-guided vision-language- action framework for autonomous driving.arXiv preprint arXiv:2509.00789, 2025

    Pei Liu, Qingtian Ning, Xinyan Lu, Haipeng Liu, Weil- iang Ma, Dangen She, Peng Jia, Xianpeng Lang, and Jun Ma. Omnireason: A temporal-guided vision-language- action framework for autonomous driving.arXiv preprint arXiv:2509.00789, 2025. 7

    2025 arXiv

  27. [34]

    3dsrbench: A comprehensive 3d spatial reasoning benchmark

    Wufei Ma, Haoyu Chen, Guofeng Zhang, Yu-Cheng Chou, Jieneng Chen, Celso de Melo, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 6924–6934, 2025. 3

    2025

  28. [35]

    Gpt-driver: Learning to drive with gpt.arXiv preprint arXiv:2310.01415, 2023

    Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, and Yue Wang. Gpt-driver: Learning to drive with gpt.arXiv preprint arXiv:2310.01415, 2023. 3

    2023 arXiv

  29. [36]

    Lingoqa: Visual question answering for autonomous driving

    Ana-Maria Marcu, Long Chen, Jan H¨unermann, Alice Karn- sund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Visual question answering for autonomous driving. InEuropean Conference on Computer Vision, pages ...

    2024

  30. [37]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744,

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:277...

  31. [38]

    Vlp: Vision language planning for autonomous driving

    Chenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Abhirup Mallik, Alessandro G Allievi, Senem Velipasalar, and Liu Ren. Vlp: Vision language planning for autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14760–14769, 2024. 1, 2

    2024

  32. [39]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InACL, 2002. 7

    2002

  33. [40]

    Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario

    Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang. Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4542–4550, 2024. 2

    2024

  34. [41]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InICML, 2022. 3

    2022

  35. [42]

    Lmdrive: Closed-loop end-to-end driving with large language models

    Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L Waslander, Yu Liu, and Hongsheng Li. Lmdrive: Closed-loop end-to-end driving with large language models. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15120–15130, 2024. 1, 2

    2024

  36. [43]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. InEuropean conference on computer vision, pages 256–274. Springer, 2024. 1, 2, 7

    2024

  37. [44]

    LMAD: Integrated end-to-end vision-language model for Explainable Autonomous Driving.arXiv preprint: 2508.12404, 2025

    Nan Song, Bozhou Zhang, Xiatian Zhu, Jiankang Deng, and Li Zhang. LMAD: Integrated end-to-end vision-language model for Explainable Autonomous Driving.arXiv preprint: 2508.12404, 2025. 7

    2025 arXiv

  38. [45]

    Bev-tsr: Text-scene retrieval in bev space for autonomous driving

    Tao Tang, Dafeng Wei, Zhengyu Jia, Tian Gao, Changwei Cai, Chengkai Hou, Peng Jia, Kun Zhan, Haiyang Sun, Fan JingChen, et al. Bev-tsr: Text-scene retrieval in bev space for autonomous driving. InProceedings of the AAAI Conference on Artificial Intelligence, pages 7275–7283, 2025. 2

    2025

  39. [47]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. InConference on Robot Learning (CoRL), 2024. 3, 7

    2024

  40. [48]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Mar- tinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste 10 Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023. 3

    2023 arXiv

  41. [49]

    Lawrence Zitnick, and Devi Parikh

    Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evalu- ation. InCVPR, 2015. 7

    2015

  42. [50]

    Om- nidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning

    Shihao Wang, Zhiding Yu, Xiaohui Jiang, Shiyi Lan, Min Shi, Nadine Chang, Jan Kautz, Ying Li, and Jose M Alvarez. Om- nidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning. InProceedings of the Computer Vision and Pattern Recognition ...

    2025

  43. [51]

    Mv2dfusion: Leveraging modality-specific object semantics for multi-modal 3d detection

    Zitian Wang, Zehao Huang, Yulu Gao, Naiyan Wang, and Si Liu. Mv2dfusion: Leveraging modality-specific object semantics for multi-modal 3d detection. InIEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. 2

    2025

  44. [52]

    Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understand- ing.arXiv preprint arXiv:2412.10302, 2024

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understand- ing.arXiv preprint arXiv:2412.10302, 2024. 3

    2024 arXiv

  45. [53]

    Segformer: Simple and efficient design for semantic segmentation with transformers

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:12077– 12090, 2021. 3

    2021

  46. [54]

    Chatbev: A visual language model that understands bev maps.arXiv preprint arXiv:2503.13938, 2025

    Qingyao Xu, Siheng Chen, Guang Chen, Yanfeng Wang, and Ya Zhang. Chatbev: A visual language model that understands bev maps.arXiv preprint arXiv:2503.13938, 2025. 2

    2025 arXiv

  47. [55]

    Explainable object-induced action decision for autonomous vehicles

    Yiran Xu, Xiaoyin Yang, Lihang Gong, Hsuan-Chu Lin, Tz- Ying Wu, Yunsheng Li, and Nuno Vasconcelos. Explainable object-induced action decision for autonomous vehicles. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 9523–9532, 2020. 2

    2020

  48. [56]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model.IEEE Robotics and Automation Letters,

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model.IEEE Robotics and Automation Letters,

  49. [57]

    Thinking in space: How multimodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643, 2025. 1, 3

    2025

  50. [58]

    Lidar-llm: Exploring the potential of large language models for 3d lidar understanding

    Senqiao Yang, Jiaming Liu, Renrui Zhang, Mingjie Pan, Ziyu Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Hongsheng Li, Yan- dong Guo, et al. Lidar-llm: Exploring the potential of large language models for 3d lidar understanding. InProceed- ings of the AAAI Conference on Artificial Int...

    2025

  51. [59]

    V2x-vlm: End-to-end v2x cooperative autonomous driv- ing through large vision-language models.arXiv preprint arXiv:2408.09251, 2024

    Junwei You, Haotian Shi, Zhuoyu Jiang, Zilin Huang, Rui Gan, Keshu Wu, Xi Cheng, Xiaopeng Li, and Bin Ran. V2x-vlm: End-to-end v2x cooperative autonomous driv- ing through large vision-language models.arXiv preprint arXiv:2408.09251, 2024. 3

    2024 arXiv

  52. [60]

    Rethinking the open-loop evaluation of end- to-end autonomous driving in nuscenes.arXiv preprint arXiv:2305.10430, 2023

    Jiang-Tian Zhai, Ze Feng, Jinhao Du, Yongqiang Mao, Jiang- Jiang Liu, Zichang Tan, Yifu Zhang, Xiaoqing Ye, and Jing- dong Wang. Rethinking the open-loop evaluation of end- to-end autonomous driving in nuscenes.arXiv preprint arXiv:2305.10430, 2023. 7

    2023 arXiv

  53. [61]

    MiniDrive: More Efficient Vision-Language Models with Multi-Level 2D Features as Text Tokens for Autonomous Driving.arXiv preprint: 2409.07267, 2025-05-

    Enming Zhang, Xingyuan Dai, Min Huang, Yisheng Lv, and Qinghai Miao. MiniDrive: More Efficient Vision-Language Models with Multi-Level 2D Features as Text Tokens for Autonomous Driving.arXiv preprint: 2409.07267, 2025-05-

    2025 arXiv

  54. [62]

    LLaMA-Adapter: Efficient Fine-tuning of language models with Zero-init Attention

    Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao. LLaMA-Adapter: Efficient Fine-tuning of language models with Zero-init Attention. InICLR, 2024. 4

    2024

  55. [63]

    Interndrive: A multimodal large language model for autonomous driving scenario understand- ing

    Ye Zhang and Yiming Nie. Interndrive: A multimodal large language model for autonomous driving scenario understand- ing. InProceedings of the 2024 4th International Conference on Artificial Intelligence, Automation and High Performance Computing, pages 294–305, 2024. 3

    2024

  56. [64]

    MPDrive: Improving spatial understanding with Marker-Based Prompt Learning for Autonomous Driving

    Zhiyuan Zhang, Xiaofan Li, Zhihao Xu, Wenjie Peng, Zi- jian Zhou, Miaojing Shi, and Shuangping Huang. MPDrive: Improving spatial understanding with Marker-Based Prompt Learning for Autonomous Driving. InCVPR, 2025. 7

    2025

  57. [65]

    Opendrivevla: Towards end-to-end au- tonomous driving with large vision language action model

    Xingcheng Zhou, Xuyuan Han, Feng Yang, Yunpu Ma, and Alois C Knoll. Opendrivevla: Towards end-to-end au- tonomous driving with large vision language action model. arXiv preprint arXiv:2503.23463, 2025. 3

    2025

  58. [66]

    Autovla: A vision- language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning

    Zewei Zhou, Tianhui Cai, Seth Z Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. Autovla: A vision- language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. In NeurIPS, 2025. 3 11 Spatial-aware Vision Language Model ...

    2025

  59. [67]

    Further information about our pro- posed SA-QA dataset is presented in Sec

    Abstract This supplementary material provides additional details and analyses of our work. Further information about our pro- posed SA-QA dataset is presented in Sec. 7, and a detailed comparison with NuScenes-SpatialQA is provided in Sec. 8. Additional details on the 3D groun...

  60. [68]

    The specific QA formats and the step-by-step gen- eration procedure are summarized in Tab

    More Details on SA-QA Dataset This dataset is built on top of nuScenes [2] and enriched us- ing the ground-truth annotations provided by both nuScenes and OpenLane [ 5] to generate question-answering (QA) pairs. The specific QA formats and the step-by-step gen- eration procedu...

  61. [69]

    Is object A closer than object B?

    Comparison with NuScenes-SpatialQA NuScenes-SpatialQA [46] is a concurrent study that shares certain similarities with our proposed SA-QA dataset. While both datasets focus on the spatial reasoning of Vision- Language Models (VLMs), SA-QA diverges by prioritiz- ing instruction...

  62. [70]

    For a potential future position at(x, y), is it in a drivable area?

    More Details on 3D Grounding Benchmark Throughout model development, we observed that planning- only evaluation is insufficient and fails to capture a model’s understanding of the spatial distribution of surrounding ob- jects. To better assess this capability, we construct a g...

  63. [71]

    Generate a drivable mask in BEV by buffering lane centerlines with a1.75m margin

  64. [72]

    Yes” if inside; “No

    Check point inclusion against the mask. Binary:“Yes” if inside; “No” otherwise. SP-02 “Identify the object in <CAM, xmin, ymin, xmax, ymax> and describe its 3D information.” None. 1. Project 3D annotations to the 2D planes of all 6 cameras

  65. [73]

    Compute 2D bounding boxes [xmin, ymin, xmax, ymax]clamped to image dims, filtering out candidates that are invisible or too small

  66. [74]

    Sample an object and format the answer

  67. [75]

    The object is a <category>in the<CAM>, location:(x, y), length:<l>, width:<w>, height:<h>, angles in degree:<yaw>

    Format the prompt with the target object’s coordinates. Text:“The object is a <category>in the<CAM>, location:(x, y), length:<l>, width:<w>, height:<h>, angles in degree:<yaw>.” (Rounded to 0.1). SP-03“Identify the object cued by thearrowand describe its 3D information.” Draw ...

  68. [76]

    Draw a visual arrow on the image pointing to the center of the target’s 2D mask

  69. [77]

    Identify the object in the masked regionand describe its 3D information

    Construct the prompt referencing the arrow cue. Text:Same format as SP-02. SP-04“Identify the object in the masked regionand describe its 3D information.” Mask a region of the image. 1–3. Follow steps 1–3 of SP-02 to filter candidates and sample one object

  70. [78]

    Apply a mask to the target’s 2D bounding box region (forcing LiDAR reliance)

  71. [79]

    What objects are on the lane defined by points (x1, y1),(x 2, y2),(x 3, y3)?

    Construct the prompt referencing the masked region. Text:Same format as SP-02. Table 6.Generation logic for spatial reasoning (SR) tasks in the SA-QA dataset. ID Prompt Template Input Data Transformation Generation Logic Answer Generation SR-01“What objects are on the lane def...

  72. [80]

    Aggregate the associated objects located on the selected lane centerline from the OpenLane annotation set [5]

  73. [81]

    The object is a <category>, location

    Format the prompt and answer. List:“The object is a <category>, location...”. SR-02“What is the nearest object in the<DIRECTION> direction?” None. 1. Define 4 spatial sectors (e.g. Front-Left, Back-Left) relative to the current ego-vehicle heading

  74. [82]

    Filter objects located within the target area

  75. [83]

    Calculate Euclidean distances for all candidates and sort in ascending order

  76. [84]

    Please determine the metric distance (in meters) separating the two indicated objects

    Format the prompt and answer. Text:Description of the object with index 0 (minimum distance), using the format from SP-02. SR-03“Please determine the metric distance (in meters) separating the two indicated objects.” Draw arrows on the images

  77. [85]

    Select two distinct visible objects (OA, OB), potentially across different camera views

  78. [86]

    Draw visual arrows pointing toO A andO B in their respective images (following SP-03)

  79. [87]

    D.” (The value is rounded to 0.1 meters). SR-04 “What is the future position of the object at(x, y)afterT second?

    Compute the L2 norm between their 3D centroids: ∥CA −C B∥2. Scalar:“D.” (The value is rounded to 0.1 meters). SR-04 “What is the future position of the object at(x, y)afterT second?” None. 1. Randomly sample one object that possesses a future trajectory within the nuScenes ann...

  80. [88]

    (x f ut, yf ut)

    Randomly select a future position and time interval for the sampled object, and subsequently structure the input prompt and the corresponding target answer. Coordinate:“(x f ut, yf ut).” 2 Table 7.Distribution of object counts across distance ranges in our grounding benchmark....

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.