Pith. sign in

REVIEW 4 major objections 5 minor 82 references

The paper claims an end-to-end driving planner can be made more progressive and no less safe by grounding decisions in ego-centered 3D geometry and latent tokens that anticipate how that geometry will change over the next two seconds.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:42 UTC pith:M5KCPHHT

load-bearing objection Well-built geometry-grounded planning paper; future-geometry causal claim is unproven due to a confounded ablation and missing future-depth eval. the 4 major comments →

arxiv 2607.17521 v2 pith:M5KCPHHT submitted 2026-07-20 cs.RO

GeoWorldAD: Geometry World Action Model for Autonomous Driving

Songyan Zhang , Jinyuan Tian , Hanbing Li , Daqi Liu , Hao Chen , Wenhui Huang , Fang Li , Guang Chen
show 4 more authors
Hangjun Ye Long Chen Kuiyuan Yang Chen Lv
This is my paper
classification cs.RO
keywords autonomous drivingend-to-end planninggeometry world modelego-aligned 3D geometrylatent future tokensfuture depth predictiontrajectory refinementclosed-loop evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that autonomous driving planners do not have to choose between being safe and being forceful: explicitly representing the current 3D scene in the car's own coordinate frame, and additionally learning compact tokens that anticipate how that scene geometry will evolve over the next two seconds, lets the planner make more progress without suffering more collisions. The central comparison is between GeoAD, which plans from multi-scale present geometry alone, and GeoWorldAD, which adds latent future geometry tokens; the future tokens improved ego progress by 3.3 points on NAVSIM v1 and 2.8 points on v2 while collision and time-to-collision metrics were maintained or slightly improved. A sympathetic reader should care because end-to-end driving systems often become overly conservative under uncertainty, and this is a concrete proposal for injecting anticipatory spatial information without expensive pixel-level video prediction. The paper also argues that coordinate consistency is part of the effect: representing geometry in an ego-aligned frame beats a fixed anchor frame.

Core claim

On the paper's own terms, the discovery is that future geometry can be carried by a small set of latent tokens that are trained purely through a future-depth loss and then attend to trajectory queries, and that this is enough to shift planning behavior from conservative to progressive while keeping safety metrics flat. Across the NAVSIM v1 and v2 closed-loop benchmarks, every reported metric improves or stays level once the latent future tokens are added to the present-geometry planner. The largest gains are in ego progress, and on v2 the extended driving score EPDMS rises from 87.6 to 90.4. A second, supporting discovery is that the geometry backbone must speak the planner's coordinate lang

What carries the argument

The load-bearing object is the latent future geometry token set Qfut: K=4 chunks of M=64 learnable tokens covering the next two seconds, which are refined through a Q-Former-style module that cross-attends to multi-scale present geometry tokens and then applies causal self-attention, and are decoded via a shared DPT depth head into future depth maps under the loss in Eq. (5). This token set is the channel through which the geometry world model injects anticipatory spatial cues into the planner. The second piece is EgoStreamVGGT, a video geometry backbone modified so that point maps are expressed in each frame's own ego-camera coordinates and camera poses are relative transforms between adjac

Load-bearing premise

The claim rests on the assumption that the latent future tokens are genuinely encoding future scene evolution, not just acting as extra trainable parameters — the paper never reports the future-depth prediction accuracy of the geometry world model, and on the non-reactive NAVSIM v1 benchmark 'future' may be almost fully determined by the present state and scripted traffic.

What would settle it

Train GeoWorldAD on NAVSIM v2 with the same total parameter count and training budget but (a) replace the future-depth loss with a reconstruction loss on random noise, or (b) freeze the future tokens and only fine-tune the planner; if the EP and EPDMS gains over GeoAD persist, the claim that future geometric anticipation drives the improvement is falsified. Conversely, report the future-depth Abs Rel / δ<1.25 on held-out NAVSIM future frames; if the world model predicts future depth no better than repeating the current depth frame, the 'future' in the ablation is not geometric anticipation.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Future geometric anticipation can be learned as latent tokens supervised only by future depth, avoiding the cost and redundancy of predicting future RGB frames.
  • Ego-aligned coordinate representation is a design requirement, not a nicety: moving from a fixed anchor frame to ego-frame point maps changed PDMS from 84.8 to 87.3 even without 4D supervision.
  • Multi-scale geometry with iterative trajectory refinement matters: single-layer geometry aggregation yields lower collision metrics, while four-scale iterative refinement raises NC, DAC, TTC, and EP together.
  • Closed-loop driving metrics on NAVSIM v1 and v2 can be improved simultaneously on safety and progress, contradicting the assumption that progress gains must trade off against collision avoidance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An untested implication of treating future geometry as latent tokens is that the tokens might also serve as a scene-evolution bottleneck for other downstream tasks, e.g., decoding them into future occupancy or other-agent trajectories; the paper does not do this, but the architecture would support it.
  • On the non-reactive NAVSIM v1 split, 'future' geometry is largely determined by the current state and scripted traffic, so the v1 ablation may overstate the value of genuine anticipation; the reactive v2 results are the stronger evidence.
  • The paper does not report the future-depth accuracy of the geometry world model, so an external check of whether the EP gains vanish when future-depth supervision is removed but the added parameters and training schedule are kept would test whether the gains come from anticipation or from extra model capacity.
  • If the effect transfers to real-world reactive driving, the same recipe — ego-aligned geometry plus latent future tokens — could be applied to trajectory planners that currently rely on object lists and HD maps, potentially reducing their dependence on structured annotations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GeoWorldAD, an end-to-end video geometry world action model for autonomous driving. It adapts the StreamVGGT geometry foundation model to an ego-aligned coordinate system (EgoStreamVGGT), extracts multi-scale present-geometry tokens, and introduces a Q-Former-style geometry world model that learns latent future geometry tokens under future-depth supervision (Eq. 5). A geometry-conditioned action model iteratively refines trajectory proposals by first aggregating present geometry and then incorporating the latent future tokens. Closed-loop planning is evaluated on NAVSIM v1 and v2, where the method reports state-of-the-art PDMS (91.0) and EPDMS (90.4). Ablations in Table 3 compare a present-geometry-only planner (GeoAD) with the full GeoWorldAD and attribute consistent gains to future geometry anticipation.

Significance. If the claims are fully supported, the work is a meaningful step toward grounding trajectory planning in explicit 3D geometry and future scene evolution. Strengths include the use of an external closed-loop benchmark (NAVSIM), the ego-aligned geometry analysis in Table 4, and the explicit supervision of future geometry through a depth prediction loss rather than through the trajectory label. The main contribution—latent future geometry tokens improving planning—is intuitively appealing and the reported gains on both NAVSIM v1 and v2 are consistent. However, the evidence for this central claim is weakened by a confounded ablation, a missing quantitative evaluation of the future-depth supervision, and the absence of error bars or multi-seed results. Because the central mechanism is not conclusively isolated, the paper requires additional experimental work before it can be accepted.

major comments (4)
  1. [§4.2, Table 3] The key ablation (GeoAD vs. GeoWorldAD) is not controlled for training budget and added parameters. GeoAD is trained for 32K steps (Stage 2), whereas GeoWorldAD is initialized from that checkpoint, adds a Q-Former block and latent tokens Qfut (K×M×C), and is trained for an additional 64K steps at a lower learning rate (Stage 3). The reported improvements in EP (+3.3/+2.8) and PDMS/EPDMS could therefore be due to the extra optimizer steps or the extra model capacity, not to future-geometric anticipation. The zero-initialized output projection preserves initial behavior but does not neutralize the continued training. A matched control—e.g., GeoAD fine-tuned for the same 64K steps and LR without future tokens—is required to attribute the gains to future geometry. Please add such an ablation or explicitly qualify the conclusion.
  2. [Eq. (5), §4.3.2] The geometry world model loss supervises future depth maps, but the quality of those predictions is never quantitatively reported. The paper only shows qualitative visualizations (Figs. 8–9). Without metrics such as Abs Rel, δ<1.25, or a comparison against a no-future baseline with identical capacity, there is no evidence that Qfut genuinely encodes future scene geometry rather than acting as an additional learned feature extractor. Please report future-depth evaluation on a held-out split, and ideally a probing experiment showing that future tokens predict future depth better than present tokens alone.
  3. [Tables 1–3] All planning results are single-run point estimates with no error bars or significance tests. Many of the comparisons hinge on small differences—e.g., NC 98.9 vs. 99.0, TTC 95.7 vs. 95.8—which may be within run-to-run noise. At minimum, report variance across multiple seeds for the main GeoAD/GeoWorldAD comparison, and ideally for the headline SOTA numbers. This is especially important because the main ablation's differences are small on safety metrics and the paper claims that safety is 'maintained or slightly improved.'
  4. [§4.1, §4.3.1] The NAVSIM v1 environment is non-reactive, so 'future' geometry for v1 is largely determined by the present state and scripted traffic. The paper's conclusion that future geometry reduces conservative behavior relies on both v1 and v2; the reactive v2 results mitigate this concern, but the paper should still acknowledge the limitation and interpret the v1 gains accordingly. This does not invalidate the v2 evidence, but it should be discussed explicitly.
minor comments (5)
  1. [Abstract / Section 1] Typographical issues: 'NA VSIM' appears with a space in the abstract and elsewhere; the standard name is 'NAVSIM'. Also, 'Future States' and 'Dense & Future Geo.' in Table 1 are inconsistent with the terminology used in the text.
  2. [Appendix, Tables 5 and 7] Tables 5 and 7 are identical (video depth estimation results). Duplicating the same table in the main text and appendix is confusing; please consolidate or refer to one of them.
  3. [Appendix, Table 8] In Table 8, EgoStreamVGGT improves ATE, RPE trans, and RPE rot on OpenScene, but on nuScenes the RPE rot is worse than StreamVGGT (1.31 vs. 0.47). The text states that ego-aligned finetuning improves temporal geometry consistency without mentioning this exception. Please acknowledge and discuss the rotation-error increase.
  4. [Section 3.2, Eq. (3)] The formulation of the cross-attention keys uses '[Gℓt; Eego]', but the text later says '[·;·]' represents concatenation. There is a minor formatting issue with extra commas and the notation is not defined before use. Please clarify.
  5. [Section 4.2] The Stage 1 sampling ratio '10:10:1:1' is stated without explanation of which dataset corresponds to which value. Please label the ratio explicitly.

Circularity Check

0 steps flagged

No significant circularity: the headline result is measured against external NAVSIM benchmarks and future-geometry tokens are supervised by future depth, not by the trajectory metric; the main ablation is confounded but not circular.

full rationale

The paper's central claim is that latent future geometry tokens improve planning, evidenced by closed-loop NAVSIM v1/v2 scores. The future tokens are trained with a future-depth loss (Eq. 5) using ground-truth future depth, not with the trajectory labels or the reported PDMS/EPDMS scores, so the metric gains are not forced by construction. The GeoAD-vs-GeoWorldAD ablation (Table 3) is not fully controlled: GeoWorldAD continues from the GeoAD checkpoint with an added Q-Former future block, new parameters, 64K additional optimizer steps, and a lower learning rate. This weakens causal attribution of the EP/PDMS gains to future geometric anticipation, but it is a confound/validity issue rather than a circular reduction: no equation in the paper makes the ablated metric equal to a training objective or to a fitted parameter. The paper also never reports future-depth accuracy, so the claim that Qfut encodes future scene geometry rather than extra capacity is under-supported; again, this is missing evidence, not circularity. Self-citations (e.g., [6], [14], [16], [71]) and overlapping-author DVGT-2 [19] appear only as baseline/related-work context and are not load-bearing for the derivation. EgoStreamVGGT is evaluated with independent depth and pose metrics. No self-definitional, fitted-input-called-prediction, uniqueness-imported, ansatz-smuggled, or renaming circularity was found.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

The paper's central contribution is a trained system, not a derivation. The main free choices are architectural capacities and loss weights; none are fitted to PDMS, but several are chosen by hand and never ablated. The key invented entity, the latent future token, has no independent quantitative validation beyond the authors' own planner ablation.

free parameters (4)
  • Stage-wise trajectory loss weights lambda_j = not specified
    Eq. 7 weights five refinement stages with exponential down-weighting; values are chosen by hand and never reported or ablated.
  • Confidence regularization alpha = not specified
    Appears in depth and point-map losses (Eq. 5, Eq. 12, Eq. 13); value is not stated.
  • Number of latent future tokens per chunk M=64 = 64
    Architectural capacity chosen by hand; directly controls how much future information the world model can store.
  • Stage 1 data sampling ratio = 10:10:1:1
    OpenScene:nuScenes:ParallelDomain:RealDriveSim ratio is chosen without ablation or sensitivity analysis.
axioms (5)
  • domain assumption StreamVGGT pretrained checkpoint provides reliable 4D geometry features.
    GeoWorldAD is built directly on StreamVGGT; the quality of the present-geometry tokens rests on this external model.
  • domain assumption NAVSIM closed-loop simulator metrics are a faithful proxy for real-world driving safety and progress.
    All conclusions about safe and efficient planning are drawn from PDMS/EPDMS scores.
  • domain assumption Ground-truth future depth maps are available and correct for training the world model.
    Eq. 5 supervises latent future tokens with future depth; the paper does not evaluate the quality of these future-depth predictions.
  • domain assumption Ego-aligned relative-pose formulation preserves temporal geometry consistency.
    EgoStreamVGGT changes the coordinate frame from anchor-frame to per-timestep ego-frame; pose evaluation in Tab. 8 is supportive but limited.
  • domain assumption OpenScene geometry pretraining does not leak NAVSIM navtest scenes into training.
    NAVSIM is described as a subset of OpenScene, but the paper never states that navtest scenes are excluded from Stage 1 geometry pretraining on OpenScene.
invented entities (1)
  • Latent future geometry tokens Qfut no independent evidence
    purpose: Compact anticipatory representation of future scene evolution, used to refine trajectory proposals.
    No direct metric of future-depth prediction accuracy is reported; only the planning ablation and qualitative visualizations support the claim that these tokens encode future geometry.

pith-pipeline@v1.3.0-alltime-deepseek · 16825 in / 12351 out tokens · 107450 ms · 2026-08-01T17:42:56.980589+00:00 · methodology

0 comments
read the original abstract

Autonomous driving requires both safe and efficient planning decisions in dynamic 3D environments. Although recent Vision/Video-Action models learn policies directly from visual observations and scale well with advances in vision transformers and large-scale training data, they often lack explicit geometric grounding and future-aware spatial guidance, limiting their ability to balance collision avoidance and driving progress. In this work, we propose GeoWorldAD, a geometry world action model that grounds trajectory planning in ego-aligned 3D space and anticipates short-horizon scene evolution with latent future geometry tokens. Present geometry provides essential spatial constraints for safe planning, while future geometry reveals how surrounding agents and ego-centric free space may evolve, reducing overly conservative decisions without sacrificing safety. To efficiently exploit these geometric cues, GeoWorldAD progressively aggregates multi-scale present geometry and latent future geometry through iterative trajectory refinement. Experiments on NAVSIM v1 and v2 demonstrate state-of-the-art performance, highlighting the effectiveness of explicit 3D geometry grounding and future geometry world modeling for safe and efficient autonomous driving.

Figures

Figures reproduced from arXiv: 2607.17521 by Chen Lv, Daqi Liu, Fang Li, Guang Chen, Hanbing Li, Hangjun Ye, Hao Chen, Jinyuan Tian, Kuiyuan Yang, Long Chen, Songyan Zhang, Wenhui Huang.

Figure 1
Figure 1. Figure 1: An intuitive comparison between our video geometry world action model and previous [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the GeoWorldAD framework. Given an input video sequence, GeoWorldAD [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of three different geometry aggregation strategies for trajectory planning. [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visual comparison of StreamVGGT and our EgoStreamVGGT for 4D reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualized comparison of StreamVGGT and our EgoStreamVGGT for 4D reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualized comparison of StreamVGGT and our EgoStreamVGGT for 4D reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualized comparison of StreamVGGT and our EgoStreamVGGT for 4D reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p021_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of future depth prediction. [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of future depth prediction. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 33 linked inside Pith

  1. [1]

    Jiang, S

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang. Vad: Vectorized scene representation for efficient autonomous driving. InProc. IEEE Int. Conf. Comp. Vis., pages 8306–8316, 2023

    2023

  2. [2]

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, L. Lu, X. Jia, Q. Liu, J. Dai, Y . Qiao, and H. Li. Planning-oriented autonomous driving. InProc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 17853–17862, 2023

    2023

  3. [3]

    S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. InProc. Eur . Conf. Comp. Vis., pages 533–549, 2022

    2022

  5. [5]

    X. Weng, B. Ivanovic, Y . Wang, Y . Wang, and M. Pavone. Para-drive: Parallelized architecture for real-time autonomous driving. InProc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 15449– 15458, 2024

    2024

  6. [6]

    K. Guo, H. Liu, X. Wu, J. Pan, and C. Lv. ipad: Iterative proposal-centric end-to-end autonomous driving, 2025. URLhttps://arxiv.org/abs/2505.15111

    Pith/arXiv arXiv 2025

  7. [7]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. InProc. Int. Conf. Learn. Representations, pages 8748–8763. PMLR, 2021

    2021

  8. [8]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    Caron, H

    M. Caron, H. Touvron, I. Misra, H. J´egou, J. Mairal, P. Bojanowski, and A. Joulin. Emerging properties in self-supervised vision transformers. InICCV, pages 9650–9660, 2021

    2021

  10. [10]

    Contributors

    O. Contributors. Openscene: The largest up-to-date 3d occupancy predic- tion benchmark in autonomous driving. GitHub-OpenDriveLab/OpenScene: 3DOccupancyPredictionBenchmarkinAutonomousDriving, 2023

    2023

  11. [11]

    P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V . Patnaik, P. Tsui, J. Guo, Y . Zhou, Y . Chai, B. Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. InProc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 2446–2454, 2020

    2020

  12. [12]

    Caesar, V

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. InProc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 11621–11631, 2020

    2020

  13. [13]

    X. Jia, J. You, Z. Zhang, and J. Yan. Drivetransformer: Unified transformer for scalable end-to-end autonomous driving.arXiv preprint arXiv:2503.07656, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Zhang, W

    S. Zhang, W. Huang, Z. Gao, H. Chen, and C. Lv. Wisead: Knowledge augmented end-to-end autonomous driving with vision-language model.arXiv preprint arXiv:2412.09951, 2024. 10

    Pith/arXiv arXiv 2024

  15. [15]

    Zhang, W

    S. Zhang, W. Huang, Z. Chen, C. J. Collister, Q. Huang, and C. Lv. Openread: Reinforced open-ended reasoning for end-to-end autonomous driving with llm-as-critic, 2025. URL https://arxiv.org/abs/2512.01830

    arXiv 2025

  16. [16]

    Huang, S

    W. Huang, S. Zhang, Q. Huang, Z. Wang, Z. Mao, C. Chua, Z. Chen, L. Chen, and C. Lv. Automot: A unified vision-language-action model with asynchronous mixture-of-transformers for end-to-end autonomous driving, 2026. URLhttps://arxiv.org/abs/2603.14851

    Pith/arXiv arXiv 2026

  17. [17]

    Z. Zhou, T. Cai, S. Z. Zhao, Y . Zhang, Z. Huang, B. Zhou, and J. Ma. Autovla: A vision- language-action model for end-to-end autonomous driving with adaptive reasoning and rein- forcement fine-tuning.arXiv preprint arXiv:2506.13757, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Y . Li, K. Xiong, X. Guo, F. Li, S. Yan, G. Xu, L. Zhou, L. Chen, H. Sun, B. Wang, et al. Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving.arXiv preprint arXiv:2506.08052, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    S. Zuo, Z. Xie, W. Zheng, S. Xu, F. Li, H. Li, L. Chen, Z.-X. Yang, and J. Lu. Dvgt-2: Vision- geometry-action model for autonomous driving at scale, 2026. URL https://arxiv.org/ abs/2604.00813

    Pith/arXiv arXiv 2026

  20. [20]

    Zhang, Z

    K. Zhang, Z. Tang, X. Hu, X. Pan, X. Guo, Y . Liu, J. Huang, L. Yuan, Q. Zhang, X.-X. Long, X. Cao, and W. Yin. Epona: Autoregressive diffusion world model for autonomous driving. In ICCV, 2025

    2025

  21. [21]

    T. Xia, Y . Li, L. Zhou, J. Yao, K. Xiong, H. Sun, B. Wang, K. Ma, H. Ye, W. Liu, et al. Drivelaw: Unifying planning and video generation in a latent driving world.arXiv preprint arXiv:2512.23421, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Y . Li, S. Shang, W. Liu, B. Zhan, H. Wang, Y . Wang, Y . Chen, X. Wang, Y . An, C. Tang, et al. Drivevla-w0: World models amplify data scaling law in autonomous driving.arXiv preprint arXiv:2510.12796, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    D. Zhuo, W. Zheng, J. Guo, Y . Wu, J. Zhou, and J. Lu. Streaming 4d visual geometry transformer. arXiv preprint arXiv:2507.11539, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    W. Sun, X. Lin, Y . Shi, C. Zhang, H. Wu, and S. Zheng. SparseDrive: End-to-end autonomous driving via sparse scene representation. InIEEE Int. Conf. Robot. Autom., pages 8795–8801, 2025

    2025

  25. [25]

    Y . Li, Y . Wang, Y . Liu, J. He, L. Fan, and Z. Zhang. End-to-end driving with online trajectory evaluation via bev world model, 2025. URLhttps://arxiv.org/abs/2504.01941

    Pith/arXiv arXiv 2025

  26. [26]

    B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y . Zhang, Q. Zhang, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. InProc. IEEE Conf. Comp. Vis. Patt. Recogn., pages 12037–12047, 2025

    2025

  27. [27]

    W. Yao, Z. Li, S. Lan, Z. Wang, X. Sun, J. M. Alvarez, and Z. Wu. Drivesuprim: Towards precise trajectory selection for end-to-end planning, 2025. URL https://arxiv.org/abs/ 2506.06659

    arXiv 2025

  28. [28]

    Y . Luo, F. Li, S. Xu, Z. Lai, L. Yang, Q. Chen, Z. Luo, Z. Xie, S. Jiang, J. Liu, et al. Ada- thinkdrive: Adaptive thinking via reinforcement learning for autonomous driving.arXiv preprint arXiv:2509.13769, 2025

    arXiv 2025

  29. [29]

    Strong, W.-J

    M. Strong, W.-J. Chang, Q. Herau, J. Yang, Y . Hu, C. Peng, and W. Zhan. Learning to drive is a free gift: Large-scale label-free autonomy pretraining from unposed in-the-wild videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. 11

    2026

  30. [30]

    X. Gui, M. Zhang, T. Yan, W. Han, J. Gong, F. Tan, C. zhong Xu, and J. Shen. Bridging scene generation and planning: Driving with world model via unifying vision and motion representation, 2026. URLhttps://arxiv.org/abs/2603.14948

    arXiv 2026

  31. [31]

    J. Xu, Z. Zhong, Z. Shu, M. Jia, M. Li, J.-W. Bian, Q. Zhang, K. Zhang, J. Xie, J. Yang, and W. Yin. Eponav2: Driving world model with comprehensive future reasoning, 2026. URL https://arxiv.org/abs/2605.14696

    Pith/arXiv arXiv 2026

  32. [32]

    Y . Hong, K. Zhang, J. Gu, S. Bi, Y . Zhou, H. Liu, K. Liu, S. Soatto, C. Fowlkes, and H. Tan. Lrm: Large reconstruction model for single image to 3d.arXiv preprint arXiv:2311.04400, 2023

    Pith/arXiv arXiv 2023

  33. [33]

    P. Wang, H. Tan, S. Bi, Y . Xu, F. Luan, K. Sunkavalli, W. Wang, Z. Xu, and K. Zhang. Pf- lrm: Pose-free large reconstruction model for joint pose and shape prediction.arXiv preprint arXiv:2311.12024, 2023

    Pith/arXiv arXiv 2023

  34. [34]

    J. Li, H. Tan, K. Zhang, Z. Xu, F. Luan, Y . Xu, Y . Hong, K. Sunkavalli, G. Shakhnarovich, and S. Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023

    Pith/arXiv arXiv 2023

  35. [35]

    Z. Wang, Y . Wang, Y . Chen, C. Xiang, S. Chen, D. Yu, C. Li, H. Su, and J. Zhu. Crm: Single image to 3d textured mesh with convolutional reconstruction model.arXiv preprint arXiv:2403.05034, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    Tochilkin, D

    D. Tochilkin, D. Pankratz, Z. Liu, Z. Huang, A. Letts, Y . Li, D. Liang, C. Laforte, V . Jampani, and Y .-P. Cao. Triposr: Fast 3d object reconstruction from a single image.arXiv preprint arXiv:2403.02151, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    J. Xu, W. Cheng, Y . Gao, X. Wang, S. Gao, and Y . Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models.arXiv preprint arXiv:2404.07191, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Y . Xu, Z. Shi, W. Yifan, H. Chen, C. Yang, S. Peng, Y . Shen, and G. Wetzstein. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation. InProceedings of the European Conference on Computer Vision (ECCV), 2024

    2024

  39. [39]

    J. Tang, Z. Chen, X. Chen, T. Wang, G. Zeng, and Z. Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Zhang, S

    K. Zhang, S. A. Bi, H. Tan, Y . Xiangli, N. Zhao, K. Sunkavalli, and Z. Xu. Gs-lrm: Large reconstruction model for 3d gaussian splatting.arXiv preprint arXiv:2404.19702, 2024

    Pith/arXiv arXiv 2024

  41. [41]

    Szymanowicz, C

    S. Szymanowicz, C. Rupprecht, and A. Vedaldi. Splatter image: Ultra-fast single-view 3d reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10208–10217, 2024

    2024

  42. [42]

    R. Gao, A. Holynski, P. Henzler, A. Brussee, R. Martin-Brualla, P. Srinivasan, J. T. Barron, and B. Poole. Cat3d: Create anything in 3d with multi-view diffusion models. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  43. [43]

    Charatan, S

    D. Charatan, S. L. Li, A. Tagliasacchi, and V . Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19457–19467, 2024

    2024

  44. [44]

    Y . Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T.-J. Cham, and J. Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. InProceedings of the European Conference on Computer Vision (ECCV), pages 370–386, 2024. doi:10.1007/ 978-3-031-72664-4 21. 12

    2024

  45. [45]

    A. Chen, H. Xu, S. Esposito, S. Tang, and A. Geiger. Lara: Efficient large-baseline radiance fields. InProceedings of the European Conference on Computer Vision (ECCV), 2024

    2024

  46. [46]

    B. Ye, S. Liu, H. Xu, X. Li, M. Pollefeys, M.-H. Yang, and S. Peng. No pose, no prob- lem: Surprisingly simple 3d gaussian splats from sparse unposed images.arXiv preprint arXiv:2410.24207, 2024

    Pith/arXiv arXiv 2024

  47. [47]

    Smart, C

    B. Smart, C. Zheng, I. Laina, and V . A. Prisacariu. Splatt3r: Zero-shot gaussian splatting from uncalibrated image pairs.arXiv preprint arXiv:2408.13912, 2024

    Pith/arXiv arXiv 2024

  48. [48]

    Jiang, Y

    L. Jiang, Y . Mao, L. Xu, T. Lu, K. Ren, Y . Jin, X. Xu, M. Yu, J. Pang, F. Zhao, D. Lin, and B. Dai. Anysplat: Feed-forward 3d gaussian splatting from unconstrained views.arXiv preprint arXiv:2505.23716, 2025

    arXiv 2025

  49. [49]

    Weinzaepfel, R

    P. Weinzaepfel, R. Br´egier, T. Combaluzier, Y . Cabon, and J. Revaud. Croco: Cross-view completion pre-training for 3d vision. InNIPS, pages 3216–3229, 2022

    2022

  50. [50]

    S. Wang, V . Leroy, Y . Cabon, B. Chidlovskii, and J. Revaud. Dust3r: Geometric 3d vision made easy. InCVPR, pages 20697–20709, 2024

    2024

  51. [51]

    Leroy, Y

    V . Leroy, Y . Cabon, and J. Revaud. Grounding image matching in 3d with mast3r.arXiv preprint arXiv:2406.09756, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Zhang, C

    J. Zhang, C. Herrmann, J. Hur, V . Jampani, T. Darrell, F. Cole, D. Sun, and M.-H. Yang. Monst3r: A simple approach for estimating geometry in the presence of motion.arXiv preprint arXiv:2410.03825, 2024

    Pith/arXiv arXiv 2024

  53. [53]

    Zhang, J

    S. Zhang, J. Wang, Y . Xu, N. Xue, C. Rupprecht, X. Zhou, Y . Shen, and G. Wetzstein. Flare: Feed-forward geometry, appearance and camera estimation from uncalibrated sparse views. In CVPR, 2025

    2025

  54. [54]

    Z. Tang, Y . Fan, D. Wang, H. Xu, R. Ranjan, A. Schwing, and Z. Yan. Mv-dust3r+: Single- stage scene reconstruction from sparse views in 2 seconds. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5283–5293, 2025

    2025

  55. [55]

    Cabon, L

    Y . Cabon, L. Stoffl, L. Antsfeld, G. Csurka, B. Chidlovskii, J. Revaud, and V . Leroy. Must3r: Multi-view network for stereo 3d reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1050–1060, 2025

    2025

  56. [56]

    J. C. Yang, A. Sax, K. J. Liang, M. Henaff, H. Tang, A. Cao, J. Chai, F. Meier, and M. Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. InCVPR, 2025

    2025

  57. [57]

    J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. Vggt: Visual geometry grounded transformer. InCVPR, pages 5294–5306, 2025

    2025

  58. [58]

    J. Wang, J. Sch¨onberger, M. Chen, S. Zhang, N. Karaev, P. Labatut, A. Vedaldi, P. Bojanowski, C. Rupprecht, and D. Novotny. Vggt-ω.arXiv preprint arXiv:2605.15195, 2026

    Pith/arXiv arXiv 2026

  59. [59]

    Z. Teed, L. Lipson, and J. Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. InAdvances in Neural Information Processing Systems (NeurIPS), volume 34, pages 16558–16569, 2021

    2021

  60. [60]

    Z. Teed, L. Lipson, and J. Deng. Deep patch visual odometry. InAdvances in Neural Information Processing Systems (NeurIPS), volume 36, 2023

    2023

  61. [61]

    Z. Zhu, S. Peng, V . Larsson, Z. Cui, M. R. Oswald, A. Geiger, and M. Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12786–12796, 2022. 13

    2022

  62. [62]

    H. Wang, J. Wang, and L. Agapito. Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13293–13302, 2023

    2023

  63. [63]

    Keetha, J

    N. Keetha, J. Karhade, K. M. Jatavallabhula, G. Yang, S. Scherer, D. Ramanan, and J. Luiten. Splatam: Splat track & map 3d gaussians for dense rgb-d slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21357–21366, 2024

    2024

  64. [64]

    Matsuki, R

    H. Matsuki, R. Murai, P. H. J. Kelly, and A. J. Davison. Gaussian splatting slam. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18039–18048, 2024

    2024

  65. [65]

    Murai, E

    R. Murai, E. Dexheimer, and A. J. Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors. InCVPR, pages 16695–16705, 2025

    2025

  66. [66]

    Y . Liu, S. Dong, S. Wang, Y . Yin, Y . Yang, Q. Fan, and B. Chen. Slam3r: Real-time dense scene reconstruction from monocular rgb videos. InCVPR, pages 16651–16662, 2025

    2025

  67. [67]

    Wang and L

    H. Wang and L. Agapito. Spann3r: 3d reconstruction with spatial memory. InInternational Conference on 3D Vision (3DV), 2025

    2025

  68. [68]

    Q. Wang, Y . Zhang, A. Holynski, A. A. Efros, and A. Kanazawa. Continuous 3d perception model with persistent state. InCVPR, 2025

    2025

  69. [69]

    Y . Wu, W. Zheng, J. Zhou, and J. Lu. Point3r: Streaming 3d reconstruction with explicit spatial pointer memory. InNIPS, 2025

    2025

  70. [70]

    Z. Chen, M. Qin, T. Yuan, Z. Liu, and H. Zhao. Long3r: Long sequence streaming 3d reconstruction.arXiv preprint arXiv:2507.18255, 2025

    Pith/arXiv arXiv 2025

  71. [71]

    Zhang, Y

    S. Zhang, Y . Ge, J. Tian, G. Xu, H. Chen, C. Lv, and C. Shen. Pomato: Marrying pointmap matching with temporal motion for dynamic 3d reconstruction, 2025. URL https://arxiv. org/abs/2504.05692

    Pith/arXiv arXiv 2025

  72. [72]

    Y . Lan, Y . Luo, F. Hong, S. Zhou, H. Chen, Z. Lyu, S. Yang, B. Dai, C. C. Loy, and X. Pan. Stream3r: Scalable sequential 3d reconstruction with causal transformer.arXiv preprint arXiv:2508.10893, 2025

    Pith/arXiv arXiv 2025

  73. [73]

    S. Yuan, Y . Yang, X. Yang, X. Zhang, Z. Zhao, L. Zhang, and Z. Zhang. Infinitevggt: Visual geometry grounded transformer for endless streams.arXiv preprint arXiv:2601.02281, 2026

    arXiv 2026

  74. [74]

    Lu, P.-T

    S.-Y . Lu, P.-T. Chen, H.-C. Hsu, S.-Y . Jhong, W.-H. Cheng, and Y .-Y . Chen. Ovggt: O(1) constant-cost streaming visual geometry transformer.arXiv preprint arXiv:2603.05959, 2026

    Pith/arXiv arXiv 2026

  75. [75]

    X. Liu, C. Yu, D. Ji, Q. Zhu, L. Sun, X. Li, J. Ma, T. Chen, and L. Zhu. Streamcachevggt: Streaming visual geometry transformers with robust scoring and hybrid cache compression. arXiv preprint arXiv:2604.15237, 2026

    Pith/arXiv arXiv 2026

  76. [76]

    S. Zuo, Z. Xie, W. Zheng, S. Xu, F. Li, S. Jiang, L. Chen, Z.-X. Yang, and J. Lu. Dvgt: Driving visual geometry transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14658–14668, 2026

    2026

  77. [77]

    Ranftl, A

    R. Ranftl, A. Bochkovskiy, and V . Koltun. Vision transformers for dense prediction. InICCV, pages 12179–12188, 2021

    2021

  78. [78]

    Dauner, M

    D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, A. Geiger, and K. Chitta. NA VSIM: Data-driven non-reactive autonomous vehicle simulation and benchmarking. InAdv. Neural Inf. Process. Syst., volume 37, pages 28706– 28719, 2024. 14

    2024

  79. [79]

    Parallel domain.https://paralleldomain.com/, 2024

    2024

  80. [80]

    Jadon, H

    A. Jadon, H. Wang, P. Thomas, M. Stanley, S. N. Cibik, R. Laurat, O. Maher, L. Hoyer, O. Unal, and D. Dai. Realdrivesim: A realistic multi-modal multi-task synthetic dataset for autonomous driving, 2025. URLhttps://arxiv.org/abs/2506.16319

    Pith/arXiv arXiv 2025

Showing first 80 references.