Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An intention-aware latent world model lets an end-to-end driving planner train without perception annotations and cuts collisions by 46.7 percent versus the previous latent-world baseline.

desk verdict A useful latent-world-model construction for perception-free planning whose headline gains are undermined by a backbone-mismatched LAW baseline. read the letter →

arxiv 2507.00603 v1 pith:JH7R3SZX submitted 2025-07-01 cs.CV

Yupeng Zheng , Pengxuan Yang , Zebin Xing , Qichao Zhang , Yuhang Zheng , Yinfeng Gao , Pengfei Li , Teng Zhang
show 3 more authors
Zhongpu Xia Peng Jia Dongbin Zhao
This is my paper
classification cs.CV
keywords end-to-endautonomousdrivinglatentworldmodelintention-awareplanningperceptionannotation-freeself-supervisedlearningvisionfoundationmodelsnuScenesNavSim
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

World4Drive sets out to show that end-to-end autonomous driving can be trained from raw camera images without perception annotations—no 3D boxes, no HD maps—if the network first builds a latent model of how the physical world evolves under different driving intentions. The proposal is to generate several plausible future trajectories, imagine what the scene will look like under each one, and select the trajectory whose imagined future latent representation best matches the actual future latent extracted from the next frames. This would matter because perception annotation is one of the main bottlenecks to scaling end-to-end planners to new roads and conditions. The paper reports state-of-the-art annotation-free results on the open-loop nuScenes and closed-loop NavSim benchmarks, including an 18.1 percent lower average trajectory displacement error and a 46.7 percent lower collision rate than the previous latent-world baseline, with 3.75 times faster convergence.

What carries the argument

The load-bearing object is the intention-aware latent world model, made of a physical latent encoder, a prediction module the paper calls the world model dreamer, and a world model selector. The dreamer concatenates action tokens $A$ with the current world latent $L_t$ and predicts a future latent $\hat{L}_{t+n}$ for each of $K$ driving intentions through cross-attention. The selector computes the MSE feature distance between each predicted latent and the actual future latent, selects the closest index $j$, trains a ScoreNet with focal loss, and uses the selected distance as the reconstruction loss $\mathcal{L}_{\mathrm{recon}}$ that supplies the self-supervised training signal. The physical latent encoder adds sinusoidal 3D position encodings computed from metric depth and pseudo-semantic masks to the image features, which is the mechanism that injects spatial and semantic information without explicit detection labels.

What would settle it

The decisive experiment is to replace the depth and semantic priors with random or untrained features of the same shape on nuScenes while keeping the intention-aware world model and selector intact: if L2 and collision rate stay near 0.50 m and 0.16%, the priors are not doing the work; if they revert toward LAW's 0.61 m and 0.30%, the spatial-semantic priors carry the claim.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a latent world model can serve as both the generator and the evaluator of multi-modal driving plans. At each step, World4Drive encodes current multi-view images into world latents using spatial priors from metric depth and semantic priors from vision-language segmentation, encodes a fixed vocabulary of trajectories into multi-modal intention queries, and predicts a future latent for each intention. A world model selector then compares each predicted future latent with the latent extracted from the actual future frame, picks the closest one as the winning trajectory, and uses that same distance as a reconstruction loss for self-supervised training. On nuScenes this lowers average L2 displacement from 0.61 m to 0.50 m and collision rate from 0.30% to 0.16% relative to LAW; on NavSim it reaches a closed-loop driving score of 85.1, above several perception-supervised baselines. The ablation showing that intentions without the world model degrade to 0.61 m L2 and 0.36% collision is the paper's key evidence that the selector's future-latent comparison, rather than the intention vocabulary alone, produces the safety gain.

Load-bearing premise

The load-bearing premise is that the depth and semantic priors supplied by the frozen foundation models act as scene knowledge rather than as hidden perception supervision, and that the authors' reimplementation of LAW is a faithful stand-in for the published baseline; if either fails, the reported gains over LAW shrink.

Editorial extensions

If this is right

  • Training data for end-to-end planners no longer needs 3D-box annotations or HD maps; raw driving logs with expert trajectories could be sufficient.
  • The reported 3.75x faster convergence means a given planning quality can be reached in roughly a quarter of the training epochs, lowering the compute cost of iterating on a driving policy.
  • Because the selector ranks trajectories by consistency between imagined and observed future latents, the planner receives a self-supervised safety signal in addition to the imitation loss.
  • The closed-loop NavSim results indicate that the learned latent dynamics transfer beyond the open-loop training distribution, which is what a world model should provide if it has learned scene evolution rather than memorized examples.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'perception annotation-free' label applies to the driving model itself, not the full system: the depth and segmentation models were trained on labeled data, so the pipeline relocates supervision into frozen foundation models rather than eliminating it. A fair test would measure how much of the gain survives when those priors come from purely self-supervised sources.
  • The generate-and-select loop is a general recipe: any sequential decision problem with an observable next state could use a latent world model to score candidate actions by the distance between predicted and observed future latents.
  • Because the trajectory vocabulary is fixed to 8192 templates with six intentions per command, the planner's expressiveness is capped by that vocabulary; testing larger or learned intention sets would show whether the remaining gap to perception-based planners is a representation limit or a vocabulary limit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. World4Drive proposes an end-to-end autonomous driving framework that couples a latent world model with multi-modal driving intentions and spatial-semantic priors from frozen vision foundation models (Metric3D v2 and Grounded-SAM). The model generates K candidate trajectories, predicts future latent states for each intention, and uses a world-model selector to pick the trajectory with the lowest reconstruction error (training) or highest learned score (inference). The paper reports open-loop results on nuScenes (0.50 m average L2, 0.16% collision rate) and closed-loop results on NavSim (85.1 PDMS), claiming an 18.1% relative L2 reduction, a 46.7% collision-rate reduction, and 3.75x faster convergence compared to LAW.

Significance. If the reported results are reproducible and the comparison to LAW is properly controlled, the work would be a meaningful step toward perception-annotation-free end-to-end planning by showing that a latent world model can rank multi-modal trajectories and that frozen foundation-model priors can substantially accelerate convergence and improve safety. The paper includes a component ablation, a scalability study, and an analysis across weather and maneuver conditions, which are useful. The main significance, however, depends entirely on the validity of the LAW baseline comparison, which is currently not established due to backbone mismatches and reliance on an unverified reimplementation.

major comments (4)
  1. [§4.3, Table 1] The headline comparison against LAW is not controlled for backbone. The footnote to Table 1 explicitly states that LAW* uses Swin-Tiny while all other methods, including World4Drive, use ResNet-50. The claimed 18.1% L2 reduction and 46.7% collision-rate reduction may therefore be attributable to the stronger backbone rather than to the proposed intention-aware world model or the spatial-semantic priors. Please provide a backbone-matched LAW baseline (e.g., LAW with ResNet-50 under the same training setup) and report the comparison in a controlled table; alternatively, if the LAW* numbers are taken from the original paper, rerun the comparison with a matching backbone.
  2. [§4.4.1, Table 3] The ablation baseline in row 1 reports 0.61/0.30, identical to the LAW* row in Table 1, but the table does not state which backbone is used for this baseline. If row 1 uses Swin-Tiny while rows 2-6 use ResNet-50 (as specified in §4.2), then the component attributions in rows 2-6 are confounded with backbone choice. Please specify the backbone for every row of Table 3 and, if row 1 is the Swin-Tiny LAW, re-run it with the same backbone as the other rows to make the ablation valid.
  3. [§4.2, Table 2] The closed-loop NavSim comparison relies on a self-reimplementation of LAW because the original is not open-sourced. The paper states 'we reimplement and evaluate it under settings identical to ours,' but no details of this reimplementation are given, and its fidelity to the published LAW is unverified. Please provide the architecture, training hyperparameters, and any deviations from the original LAW, and ideally report how close the reimplementation's numbers are to the original LAW's published results on a shared benchmark. Without this, the 83.8 vs. 85.1 PDMS gap cannot be attributed to the proposed method.
  4. [§1, §3.2.2] The paper describes World4Drive as 'perception annotation-free' and 'self-supervised,' but the method relies on frozen vision foundation models (Grounded-SAM and Metric3D v2) trained on external labeled data. While these models are not fine-tuned on nuScenes perception labels, their outputs provide semantic and metric-depth ground truth that directly supervise the latent representation. Please clarify that the claim is limited to not using nuScenes or NavSim perception annotations, and discuss the potential information leakage from these externally supervised priors, since this is central to the 'no perception supervision' framing.
minor comments (5)
  1. [Figure 1 caption] The caption says 'compared to PerAct' but the surrounding text and the rest of the paper compare against LAW; this appears to be a typo.
  2. [Table 3] The column headers of Table 3 are difficult to parse; the grouping of 'Depth', 'Semantic', 'WM', and 'Intentions' under the two main columns is unclear. Please reformat the table so each component has its own column and the exact configuration of each row is unambiguous.
  3. [Table 2 caption] The caption reads 'NavSim benchmark [2]' but the NavSim reference is [14] in the bibliography; this should be corrected.
  4. [Abstract and §1] The phrase '3.75 faster training convergence' should be '3.75x faster' for consistency with Figure 1.
  5. [§3.3.2] The selector is trained with a focal loss on the index of the minimum-distance modality, but the paper does not specify the focal loss parameters; please report them for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training and inference chain is self-contained, and the LAW backbone/reimplementation concerns are experimental-comparison issues rather than in-derivation circularity.

full rationale

The paper's derivation chain does not reduce to its own inputs. World4Drive trains with Eq. (8): a semantic loss using external Grounded-SAM pseudo labels, a reconstruction loss between predicted future latents Lt+n and actual future latents L-hatt+n from future frames, a focal loss for the ScoreNet, and an L1 trajectory loss against expert trajectories. Each supervision signal is either an external prior, an observed future state, or an expert trajectory; none is defined in terms of the reported L2 or collision-rate metrics. The World Model Selector (Sec. 3.3.2) selects the modality with minimum latent distance during training and learns a ScoreNet to imitate that selection, but the target index comes from a comparison against actual future observations, so it is self-supervised rather than definitionally circular. The admitted baseline issues—LAW is reimplemented because it is not open-sourced, and Table 1's footnote says LAW uses Swin-Tiny while other methods use ResNet-50—are controlled-comparison and reproducibility concerns, not circular reductions of a prediction to a fitted parameter. The self-citations ([17], [48]) are incidental context and are not load-bearing for any claim. No quoted equation or design step exhibits self-definition, renamed fitting, or uniqueness imported from the authors' own prior work. Therefore the correct circularity finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 3 assumptions · 1 invented entities

The system relies on several hand-set hyper-parameters (K, n, loss weights, vocabulary size) and on pretrained priors as an external scaffold. It introduces no new sensors or labeled perception outputs; the only invented entity is the intention-conditioned future latent state, which is evaluated only through the final planning metrics.

free parameters (5)
  • K (number of intentions per command) = 6
    Chosen by default in Section 3.2.1; system behavior and trajectory selection depend on it.
  • n (future timestamp interval) = 3
    Set to 3 by default in Section 3.3.1; ablation deferred to supplementary material.
  • Loss weights alpha, beta, gamma, eta = 0.2, 0.2, 0.5, 1.0
    Set by hand in Section 3.4 with no sensitivity analysis on these coefficients.
  • N (trajectory vocabulary size) = 8192
    Taken from VADv2; free choice of the vocabulary size influences the k-means intention clustering.
  • Grounded-SAM confidence threshold = high confidence only
    Threshold for keeping pseudo semantic labels in Section 3.2.2 is not specified quantitatively and affects the semantic loss.
assumptions (3)
  • domain assumption The learned latent features of Metric3D and Grounded-SAM, both trained on external labeled data, are reusable priors that transfer to nuScenes and NavSim without retraining.
    All gains from the physical latent encoder are attributed to these pretrained priors in Section 3.2.2, but the paper provides no transfer analysis.
  • ad hoc to paper Latent reconstruction of future states is a sufficient self-supervisory signal for driving-relevant scene understanding.
    The entire 'perception-free' claim assumes that minimizing MSE between predicted and actual future latents yields features that support safe planning; this is the architectural premise of Section 3.3 and is not validated independently of the planning metrics.
  • domain assumption k-means on trajectory endpoints produces a meaningful discretization of driving intentions.
    Adopted from VADv2 in Section 3.2.1; no analysis of sensitivity to K or to the clustering.
invented entities (1)
  • Intention-aware world latent state
    purpose: A per-intention predicted future latent representation used to select trajectories and to provide reconstruction supervision.
    The predicted future latents are only compared against the model's own actual latents; there is no external predictive benchmark on which the model latents are demonstrated to be physically meaningful.

how reviews work

0 comments
Cite this review

Pith. "Pith review of World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model." pith.science (2026). https://pith.science/paper/JH7R3SZX

@misc{pith2026250700603,
  author       = {Pith},
  title        = {Pith review of: World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JH7R3SZX}},
  note         = {Machine review of arXiv:2507.00603}
}
read the original abstract

End-to-end autonomous driving directly generates planning trajectories from raw sensor data, yet it typically relies on costly perception supervision to extract scene information. A critical research challenge arises: constructing an informative driving world model to enable perception annotation-free, end-to-end planning via self-supervised learning. In this paper, we present World4Drive, an end-to-end autonomous driving framework that employs vision foundation models to build latent world models for generating and evaluating multi-modal planning trajectories. Specifically, World4Drive first extracts scene features, including driving intention and world latent representations enriched with spatial-semantic priors provided by vision foundation models. It then generates multi-modal planning trajectories based on current scene features and driving intentions and predicts multiple intention-driven future states within the latent space. Finally, it introduces a world model selector module to evaluate and select the best trajectory. We achieve perception annotation-free, end-to-end planning through self-supervised alignment between actual future observations and predicted observations reconstructed from the latent space. World4Drive achieves state-of-the-art performance without manual perception annotations on both the open-loop nuScenes and closed-loop NavSim benchmarks, demonstrating an 18.1\% relative reduction in L2 error, 46.7% lower collision rate, and 3.75 faster training convergence. Codes will be accessed at https://github.com/ucaszyp/World4Drive.

Figures

Figures reproduced from arXiv: 2507.00603 by the authors.

Figure 1
Figure 1. Our proposed World4Drive demonstrates superior con [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We propose World4Drive, a novel approach that constructs an intention-aware latent world model to generate, evaluate, and rank [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Detailed pipeline of context encoder. It consists of a 3D [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Detailed pipeline of World Model Selector. By comput [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Visualization of World4Drive. Since World4Drive does not predict explicit perception results, we render the ground truth [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OmniNWM: Omniscient Driving Navigation World Models

    cs.CV 2025-10 conditional novelty 6.0 of 10

    OmniNWM jointly generates long panoramic multi-modal driving videos, controls them precisely via normalized Plücker ray-maps, and derives dense driving rewards from generated 3D occupancy.

  2. Depth-Regularized JEPA World Models Learn More Transferable Representations from Real Outdoor Robot Data

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Adding a training-only depth-alignment loss to a compact LeWM JEPA world model improves downstream representation quality and out-of-domain transfer on real agricultural robot data.

Reference graph

Works this paper leans on

48 extracted references · 33 canonical work pages · cited by 2 Pith papers

  1. [1]

    Vavim and vavam: Autonomous driving through video gen- erative modeling

    Florent Bartoccioni, Elias Ramzi, Victor Besnier, Shashanka Venkataramanan, Tuan-Hung Vu, Yihong Xu, Loick Cham- bon, Spyros Gidaris, Serkan Odabas, David Hurych, et al. Vavim and vavam: Autonomous driving through video gen- erative modeling. arXiv preprint arXiv:2502.15672, 2025. 1, 3

    arXiv 2025

  2. [2]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020. 2, 5, 6

    work page 2020

  3. [3]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13243,

    arXiv

  4. [4]

    Ppad: Iterative interactions of prediction and planning for end-to-end autonomous driving

    Zhili Chen, Maosheng Ye, Shuangjie Xu, Tongyi Cao, and Qifeng Chen. Ppad: Iterative interactions of prediction and planning for end-to-end autonomous driving. In European Conference on Computer Vision. Springer, 2024. 6

    work page 2024

  5. [5]

    Openscene: The largest up-to- date 3d occupancy prediction benchmark in autonomous driving

    OpenScene Contributors. Openscene: The largest up-to- date 3d occupancy prediction benchmark in autonomous driving. https://github.com/OpenDriveLab/ OpenScene, 2023. 5

    work page 2023

  6. [6]

    Vista: A generalizable driving world model with high fidelity and versatile controllability

    Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. arXiv preprint arXiv:2405.17398, 2024. 2

    arXiv 2024

  7. [7]

    Dome: Tam- ing diffusion model into high-fidelity controllable occupancy world model

    Songen Gu, Wei Yin, Bu Jin, Xiaoyang Guo, Junming Wang, Haodong Li, Qian Zhang, and Xiaoxiao Long. Dome: Tam- ing diffusion model into high-fidelity controllable occupancy world model. arXiv preprint arXiv:2410.10429, 2024. 2

    arXiv 2024

  8. [8]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, 2016. 5, 6

    work page 2016

Show all 48 references
  1. [9]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33, 2020. 2

    2020

  2. [10]

    Gaia-1: A generative world model for au- tonomous driving

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gian- luca Corrado. Gaia-1: A generative world model for au- tonomous driving. arXiv preprint arXiv:2309.17080, 2023. 2

    2023 arXiv

  3. [11]

    Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analys...

    2024

  4. [12]

    St-p3: End-to-end vision-based au- tonomous driving via spatial-temporal feature learning

    Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to-end vision-based au- tonomous driving via spatial-temporal feature learning. In European Conference on Computer Vision . Springer, 2022. 6

    2022

  5. [13]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023. 1, 2, 5, 6

    2023

  6. [14]

    Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking

    NA V IM. Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking. arXiv preprint arXiv:2406.15349, 2024. 2, 5

    2024 arXiv

  7. [15]

    Drivetransformer: Unified transformer for scalable end-to- end autonomous driving

    Xiaosong Jia, Junqi You, Zhiyuan Zhang, and Junchi Yan. Drivetransformer: Unified transformer for scalable end-to- end autonomous driving. In The Thirteenth International Conference on Learning Representations, 2025. 2, 7

    2025

  8. [16]

    Vad: Vectorized scene representation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. ICCV, 2023. 1, 2, 5, 6

    2023

  9. [17]

    Tod3cap: Towards 3d dense captioning in out- door scenes

    Bu Jin, Yupeng Zheng, Pengfei Li, Weize Li, Yuhang Zheng, Sujie Hu, Xinyu Liu, Jinwei Zhu, Zhijie Yan, Haiyang Sun, et al. Tod3cap: Towards 3d dense captioning in out- door scenes. In European Conference on Computer Vision. Springer, 2024. 1

    2024

  10. [18]

    Enhancing end-to-end au- tonomous driving with latent world model

    Yingyan Li, Lue Fan, et al. Enhancing end-to-end au- tonomous driving with latent world model. ICLR, 2025. 1, 2, 3, 4, 5, 6, 7

    2025

  11. [19]

    Hydra-mdp: End-to-end multimodal planning with multi- target hydra-distillation

    Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, et al. Hydra-mdp: End-to-end multimodal planning with multi- target hydra-distillation. arXiv preprint arXiv:2406.06978 ,

    arXiv

  12. [20]

    Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Qiao Yu, and Jifeng Dai. Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2024. 2

    2024

  13. [21]

    Is ego status all you need for open- loop end-to-end autonomous driving? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M Alvarez. Is ego status all you need for open- loop end-to-end autonomous driving? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 6

    2024

  14. [22]

    Maptr: Structured modeling and learning for online vectorized hd map construction

    Bencheng Liao, Shaoyu Chen, Xinggang Wang, Tianheng Cheng, Qian Zhang, Wenyu Liu, and Chang Huang. Maptr: Structured modeling and learning for online vectorized hd map construction. arXiv preprint arXiv:2208.14437, 2022. 2

    2022 arXiv

  15. [23]

    Diffusiondrive: Trun- cated diffusion model for end-to-end autonomous driving

    Bencheng Liao, Shaoyu Chen, et al. Diffusiondrive: Trun- cated diffusion model for end-to-end autonomous driving. CVPR, 2025. 2, 6

    2025

  16. [24]

    Sparse4d: Multi-view 3d object detec- tion with sparse spatial-temporal fusion

    Xuewu Lin, Tianwei Lin, Zixiang Pei, Lichao Huang, and Zhizhong Su. Sparse4d: Multi-view 3d object detec- tion with sparse spatial-temporal fusion. arXiv preprint arXiv:2211.10581, 2022. 2

    2022 arXiv

  17. [25]

    Petr: Position embedding transformation for multi-view 3d object detection

    Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection. In European conference on computer vi- sion. Springer, 2022. 4

    2022

  18. [26]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, 2021. 6

    2021

  19. [27]

    Lingoqa: Visual question answering for autonomous driv- ing

    Ana-Maria Marcu, Long Chen, Jan H ¨unermann, Alice Karn- sund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Visual question answering for autonomous driv- ing. In European Conference on Computer Vision. Springer,

  20. [28]

    Driveworld: 4d pre-trained scene understanding via world models for autonomous driving

    Chen Min, Dawei Zhao, Liang Xiao, Jian Zhao, Xinli Xu, Zheng Zhu, Lei Jin, Jianshu Li, Yulan Guo, Junliang Xing, et al. Driveworld: 4d pre-trained scene understanding via world models for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  21. [29]

    Vlp: Vision language planning for autonomous driving

    Chenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Abhirup Mallik, Alessandro G Allievi, Senem Velipasalar, and Liu Ren. Vlp: Vision language planning for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  22. [30]

    Multi-modal fusion transformer for end-to-end autonomous driving

    Aditya Prakash, Kashyap Chitta, et al. Multi-modal fusion transformer for end-to-end autonomous driving. In CVPR,

  23. [31]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 ,

    arXiv

  24. [32]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. In European Conference on Computer Vision. Springer, 2024. 1, 2

    2024

  25. [33]

    Sparsedrive: End-to-end au- tonomous driving via sparse scene representation

    Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Hao- ran Wu, and Sifa Zheng. Sparsedrive: End-to-end au- tonomous driving via sparse scene representation. arXiv preprint arXiv:2405.19620, 2024. 1, 2, 6

    2024 arXiv

  26. [34]

    Tokenize the world into object-level knowledge to address long-tail events in autonomous driving

    Ran Tian, Boyi Li, Xinshuo Weng, Yuxiao Chen, Edward Schmerling, Yue Wang, Boris Ivanovic, and Marco Pavone. Tokenize the world into object-level knowledge to address long-tail events in autonomous driving. arXiv preprint arXiv:2407.00959, 2024. 2

    2024 arXiv

  27. [35]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289, 2024. 2

    2024 arXiv

  28. [36]

    Scene as occupancy

    Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al. Scene as occupancy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 6

    2023

  29. [37]

    Drivedreamer: Towards real-world- drive world models for autonomous driving

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jia- gang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world- drive world models for autonomous driving. In European Conference on Computer Vision. Springer, 2024. 2

    2024

  30. [38]

    Driving into the future: Multiview visual forecasting and planning with world model for au- tonomous driving

    Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for au- tonomous driving. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  31. [39]

    Para-drive: Parallelized architecture for real- time autonomous driving

    Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Parallelized architecture for real- time autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,

  32. [40]

    Goalflow: Goal- driven flow matching for multimodal trajectories generation in end-to-end autonomous driving

    Zebin Xing, Xingyu Zhang, Yang Hu, Bo Jiang, Tong He, Qian Zhang, Xiaoxiao Long, and Wei Yin. Goalflow: Goal- driven flow matching for multimodal trajectories generation in end-to-end autonomous driving. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025. 2

    2025

  33. [41]

    Bevformer v2: Adapting modern image backbones to bird’s-eye-view recognition via perspective su- pervision

    Chenyu Yang, Yuntao Chen, Hao Tian, Chenxin Tao, Xizhou Zhu, Zhaoxiang Zhang, Gao Huang, Hongyang Li, Yu Qiao, Lewei Lu, et al. Bevformer v2: Adapting modern image backbones to bird’s-eye-view recognition via perspective su- pervision. In Proceedings of the IEEE/CVF Conference...

    2023

  34. [42]

    Uncad: Towards safe end-to-end au- tonomous driving via online map uncertainty

    Pengxuan Yang, Yupeng Zheng, Qichao Zhang, Kefei Zhu, Zebin Xing, Qiao Lin, Yun-Fu Liu, Zhiguo Su, and Dongbin Zhao. Uncad: Towards safe end-to-end au- tonomous driving via online map uncertainty. arXiv preprint arXiv:2504.12826, 2025. 1

    2025

  35. [43]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, et al. Metric3d: Towards zero-shot metric 3d prediction from a single image. In ICCV, 2023. 4

    2023

  36. [44]

    Copilot4d: Learning unsupervised world models for autonomous driving via discrete diffusion

    Lunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas, Rui Hu, and Raquel Urtasun. Copilot4d: Learning unsupervised world models for autonomous driving via discrete diffusion. In The Twelfth International Conference on Learning Repre- sentations, 2024. 2

    2024

  37. [45]

    Drivedreamer-2: Llm-enhanced world models for diverse driving video generation

    Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang. Drivedreamer-2: Llm-enhanced world models for diverse driving video generation. arXiv preprint arXiv:2403.06845,

    arXiv

  38. [46]

    Occworld: Learning a 3d occupancy world model for autonomous driving

    Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. In Eu- ropean conference on computer vision. Springer, 2024. 2

    2024

  39. [47]

    Genad: Generative end-to-end au- tonomous driving

    Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Generative end-to-end au- tonomous driving. In European Conference on Computer Vision. Springer, 2024. 2, 6

    2024

  40. [48]

    Preliminary investigation into data scaling laws for imitation learning-based end-to-end autonomous driving

    Yupeng Zheng, Zhongpu Xia, Qichao Zhang, Teng Zhang, Ben Lu, Xiaochuang Huo, Chao Han, Yixian Li, Mengjie Yu, Bu Jin, et al. Preliminary investigation into data scaling laws for imitation learning-based end-to-end autonomous driving. arXiv preprint arXiv:2412.02689, 2024. 2

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.