Pith. sign in

REVIEW 3 major objections 4 minor 41 references

Monocular long-tail driving videos can be converted into multi-view policy training data that nearly matches real multi-camera capture.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 01:25 UTC pith:LDTSUZFS

load-bearing objection Solid engineering that turns monocular long-tail video into multi-view policy data, with real closed-loop gains that approach GT; the shared-pose/AlpaSim confound is real but does not erase the contribution. the 3 major comments →

arxiv 2607.09655 v1 pith:LDTSUZFS submitted 2026-07-10 cs.CV

OpenLongTail: Generative Scaling of Long-Tail Driving Data

Lulin Liu , Nuo Chen , Yan Wang , Bangya Liu , Wenyan Cong , Hezhen Hu , Boris Ivanovic , Hao Wang
show 10 more authors
Ziyao Zeng Xinyu Gong Yang Zhou Zixiang Xiong Dilin Wang Zhangyang Wang Weisong Shi Ruohan Zhang Marco Pavone Zhiwen Fan
This is my paper
classification cs.CV
keywords Autonomous DrivingVideo Diffusion ModelVLA ModelLong-Tail EventsExtrapolative View SynthesisMulti-View GenerationEgo-Trajectory Recovery
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Scaling autonomous driving policies is limited by scarce long-tail edge cases in curated multi-camera datasets. OpenLongTail claims that heterogeneous monocular videos—dashcams and incomplete logs—can be turned into pose-aligned, temporally coherent multi-view assets by recovering metric ego-motion and synthesizing the missing side and rear views with geometry-conditioned diffusion. Fine-tuning a vision-language-action policy on these synthetic assets raises closed-loop long-tail scores from 0.534 to 0.748 with zero collisions, close to the 0.764 obtained from ground-truth multi-view data. The paper further argues that effective generative scaling depends on aligning source distributions with the target long-tail slice, not merely on adding more samples. If correct, ubiquitous real-world monocular footage becomes a usable training source rather than remaining unused.

Core claim

OpenLongTail establishes that pose-informed extrapolative view synthesis—recovering a metric ego-trajectory from monocular video, then generating missing target-rig views with Plücker-ray geometry, temporal depth warps, and a cross-view memory bank—produces multi-view assets whose closed-loop training value approaches that of real multi-camera long-tail logs.

What carries the argument

Pose-informed extrapolative view synthesis: a video diffusion engine conditioned on Plücker ray geometry, temporal depth warping from the observed front view, and a topological cross-view memory bank that converts a monocular front stream into synchronized surround views under a target camera rig.

Load-bearing premise

The recovered camera trajectory and depth-based warps must be accurate enough that the synthetic side and rear views do not teach the policy geometric or appearance habits that only work in the simulator.

What would settle it

Fine-tune the same policy on OpenLongTail assets versus real multi-camera logs for identical long-tail events, then compare closed-loop collision rate and AlpaSim score on a held-out real multi-camera long-tail suite; a large gap favoring real data would falsify the claim that synthetic assets are a near substitute.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Heterogeneous monocular long-tail videos can be ingested as training data for multi-view VLA policies without requiring new multi-camera fleets.
  • Closed-loop long-tail collision rates can be driven toward zero by fine-tuning on synthesized surround-view assets.
  • Generative scaling of driving data is limited by source–target distribution alignment, not merely by sample count.
  • Visual fidelity and cross-view geometric consistency metrics can serve as intermediate checks before expensive closed-loop evaluation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same conversion pipeline could open web-scale consumer dashcam corpora for simulation and policy training beyond the evaluated sources.
  • Residual camera-rig biases in synthesis may still limit transfer to fleets whose extrinsics differ sharply from the generation target rig.
  • Coupling the engine with active source selection that matches specific long-tail slices could raise sample efficiency beyond uniform data pooling.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. OpenLongTail proposes a generative data engine that converts heterogeneous monocular long-tail driving videos into pose-grounded, synchronized multi-view assets under a target camera rig. The pipeline recovers metric ego-trajectories with MapAnything plus Kalman/RTS smoothing, then synthesizes missing side/rear views with a Wan2.1-VACE diffusion backbone conditioned on Plücker-ray geometry, temporal depth warps (including lookback for non-overlapping rear cameras), and a cross-view dense/semantic memory bank. The central empirical claim is that fine-tuning Alpamayo-R1 on these synthesized assets raises closed-loop AlpaSim average score from 0.534 to 0.748 at 0% collision rate on 53 long-tail events, approaching ground-truth multi-view fine-tuning (AS 0.764), with supporting gains in GeoKPM and trajectory smoothness.

Significance. If the closed-loop gains transfer beyond the shared geometric pipeline, the work would meaningfully expand usable long-tail training data for VLA driving policies by unlocking ubiquitous dashcam and monocular sources. Strengths include open release of data/checkpoints/code, clear factorization of trajectory recovery and geometry-grounded generation (Eqs. 1–2), competitive view-synthesis and pose tables against recent baselines, and an explicit distribution-alignment analysis of external sources (Appendix B). The contribution is engineering-heavy but addresses a real data bottleneck in end-to-end driving.

major comments (3)
  1. [§4.2, Table 1] §4.2 and Table 1: The same recovered ego-motion is used both for generation conditioning and as policy input, while AlpaSim re-renders via NuRec/3DGS NVS from those poses. This shared geometric loop can reward policies that exploit residual artifacts of MapAnything+smoothing and DepthCrafter warps rather than genuine long-tail robustness. A control that freezes generation poses but substitutes independent GT or alternative pose estimators for policy input (or evaluates on a real multi-camera closed-loop setup) is needed to support the transfer claim.
  2. [Table 1] Table 1 reports near-zero collision rates on only 53 long-tail events with two rollouts each, without error bars, confidence intervals, or statistical tests. Given residual metric ATE ~2.2 m and non-zero jerk in Table 3, the AS 0.534→0.748 / CR→0% result is fragile; the manuscript should quantify variance across seeds/events and report failure modes more systematically.
  3. [§4.3, Table 2] Table 2 GeoKPM and qualitative Figs. 4–6 measure internal cross-view consistency of generated views, not fidelity of synthesized side/rear cameras to real multi-camera observations under the same rig. Without a held-out real multi-view reconstruction or photometric comparison on overlapping regions beyond front-conditioned warps, the claim that synthetic assets are interchangeable with GT multi-view data for policy training remains under-supported.
minor comments (4)
  1. [§3.3] §3.3: View-specific temporal lookback offsets Δt and rear-tele loss weight λ_rear are free parameters; report chosen values and any sensitivity analysis.
  2. [Appendix C] Appendix C: Memory-gate init bias and sigma-dependent gates are described; a short ablation of dense vs. semantic memory and of Plücker injection would strengthen the method claims.
  3. [Figure 1, Table 1] Figure 1 and Table 1 notation (NV-OOD, Ex., Syn/GT) should be fully expanded in the main caption for readers who skip the appendix.
  4. [§5] §5 Limitations correctly notes inference cost and residual temporal artifacts; quantify wall-clock cost per 41-frame clip and typical failure modes (e.g., dynamic agents, night/rain).

Circularity Check

0 steps flagged

Empirical systems paper with external closed-loop, pose, and view-synthesis benchmarks; no derivation reduces to its inputs by construction.

full rationale

OpenLongTail is an engineering/data-engine paper, not a first-principles derivation. Its load-bearing claims are experimental: (i) closed-loop AlpaSim AS/CR under long-tail events (Table 1), (ii) extrapolative view fidelity and GeoKPM vs third-party generators (Table 2), and (iii) metric/Sim(3) pose metrics vs independent estimators (Table 3). None of these quantities is defined from the training objective or from a fitted scalar that is then re-reported as a prediction. Scene-ID and temporal splits, held-out Waymo E2E and Nexar sources, and frozen external modules (MapAnything, DepthCrafter, Wan2.1-VACE) keep the evaluation chain externally falsifiable. Overlap with Alpamayo-R1 / AlpaSim authorship is expected because those systems are the evaluation vehicle; the central comparison is data composition (GT vs Syn vs nominal), not a self-proving uniqueness theorem. Shared use of recovered ego-motion for generation conditioning and policy input is a possible confound for transfer, not circularity: AS still varies freely across recipes (0.534 → 0.748 → 0.764) rather than being forced by construction. Score 1 reflects only routine self-use of the authors’ policy stack, with no circular step that collapses a claimed result to its inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central claim rests on standard video-diffusion and SE(3) geometry plus several domain assumptions about monocular metric recovery and simulator fidelity; free parameters are ordinary training hyperparameters; invented entities are engineering modules rather than new physical objects.

free parameters (5)
  • LoRA rank / alpha = 32 / 16
    Chosen as 32/16 for Wan-DiT/VACE adapters; affects capacity of the fine-tuned generator.
  • rear-tele loss weight λ_rear
    Hand-set higher weight for the hardest extrapolative view in the flow-matching objective.
  • memory-gate init bias = −1.4
    Initialized to −1.4 so gates start near 0.2; stabilizes early training of residual memory pathways.
  • view-specific temporal lookback offsets Δt
    Chosen per rear camera to maximize spatial coverage from past front frames; not derived from first principles.
  • learning rates and training schedule = 1e-5, 20k steps
    1e-5 for new modules and LoRA, 20k steps, cosine warmup; standard but free choices that affect final fidelity.
axioms (5)
  • domain assumption MapAnything recovers metrically useful SE(3) trajectories from monocular driving video that remain accurate after Kalman/RTS smoothing.
    Invoked in §3.2 and used for all geometry conditions and policy ego-motion; Table 3 supports smoothness but absolute ATE remains multi-meter.
  • domain assumption Frozen DepthCrafter depths plus analytic warps supply sufficiently correct pixel priors for unobserved side/rear regions.
    §3.3 Temporal Depth Warp; no trainable parameters, so any depth bias propagates directly into generation.
  • domain assumption AlpaSim NuRec/3DGS closed-loop rendering is a faithful enough proxy for real multi-camera long-tail robustness.
    §4.2 and Appendix H; all policy gains are measured inside this simulator.
  • domain assumption Flow-matching velocity prediction with LoRA adapters on a frozen Wan 2.1 VAE/DiT backbone can learn extrapolative multi-view consistency from the curated multi-view corpus.
    §3.4 training objective and §4.1 implementation.
  • standard math Standard SE(3) rigid-body geometry and Plücker ray algebra hold for the target camera rig.
    Eqs. 5–8; ordinary projective geometry.
invented entities (3)
  • OpenLongTail pose-informed extrapolative synthesis pipeline no independent evidence
    purpose: End-to-end conversion of monocular long-tail video into target-rig multi-view assets.
    The overall system is the paper’s primary contribution; independent evidence is the released (promised) code and the reported metrics.
  • Cross-view memory bank with dense + semantic pathways and sigma-dependent gates no independent evidence
    purpose: Propagate spatial and semantic context across autoregressively generated target cameras.
    §3.3; engineering construct without external physical referent.
  • Temporal lookback depth-warp schedule for non-overlapping rear cameras no independent evidence
    purpose: Supply geometry-aligned RGB priors when same-frame overlap is zero.
    Eq. 8 and surrounding text; design choice specific to this paper.

pith-pipeline@v1.1.0-grok45 · 28547 in / 3521 out tokens · 38275 ms · 2026-07-13T01:25:16.344673+00:00 · methodology

0 comments
read the original abstract

Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these ubiquitous observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing views. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Pl\"ucker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 31 linked inside Pith

  1. [1]

    World simulation with video foundation models for physical ai.arXiv preprint arXiv:2511.00062,

    Arslan Ali, Junjie Bai, Maciej Bala, Yogesh Balaji, Aaron Blakeman, Tiffany Cai, Jiaxin Cao, Tianshi Cao, Eliza- beth Cha, Yu-Wei Chao, et al. World simulation with video foundation models for physical ai.arXiv preprint arXiv:2511.00062,

    Pith/arXiv arXiv

  2. [2]

    nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles.arXiv preprint arXiv:2106.11810,

    Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles.arXiv preprint arXiv:2106.11810,

    Pith/arXiv arXiv

  3. [3]

    Skyreels-v2: Infinite-length film generative model.arXiv preprint arXiv:2504.13074,

    Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Junchen Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengcheng Ma, et al. Skyreels-v2: Infinite-length film generative model.arXiv preprint arXiv:2504.13074,

    Pith/arXiv arXiv

  4. [4]

    Unimlvg: Unified framework for multi-view long video generation with comprehensive control capabilities for autonomous driving.arXiv preprint arXiv:2412.04842,

    Rui Chen, Zehuan Wu, Yichen Liu, Yuxin Guo, Jingcheng Ni, Haifeng Xia, and Siyu Xia. Unimlvg: Unified framework for multi-view long video generation with comprehensive control capabilities for autonomous driving.arXiv preprint arXiv:2412.04842,

    Pith/arXiv arXiv

  5. [5]

    Verdi: Vlm-embedded reasoning for autonomous driving.arXiv preprint arXiv:2505.15925,

    Bowen Feng, Zhiting Mei, Julian Ost, Filippo Ghilotti, Baiang Li, Roger Girgis, Anirudha Majumdar, and Felix Heide. Verdi: Vlm-embedded reasoning for autonomous driving.arXiv preprint arXiv:2505.15925,

    Pith/arXiv arXiv

  6. [6]

    Magicdrive: Street view generation with diverse 3d geometry control.arXiv preprint arXiv:2310.02601,

    Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, and Qiang Xu. Magicdrive: Street view generation with diverse 3d geometry control.arXiv preprint arXiv:2310.02601,

    Pith/arXiv arXiv

  7. [7]

    Steervla: Steering vision-language-action models in long-tail driving scenarios.arXiv preprint arXiv:2602.08440,

    Tian Gao, Celine Tan, Catherine Glossop, Timothy Gao, Jiankai Sun, Kyle Stachowicz, Shirley Wu, Oier Mees, Dorsa Sadigh, Sergey Levine, et al. Steervla: Steering vision-language-action models in long-tail driving scenarios.arXiv preprint arXiv:2602.08440,

    arXiv

  8. [8]

    Infinitydrive: Breaking time limits in driving world models.arXiv preprint arXiv:2412.01522,

    Xi Guo, Chenjing Ding, Haoxuan Dou, Xin Zhang, Weixuan Tang, and Wei Wu. Infinitydrive: Breaking time limits in driving world models.arXiv preprint arXiv:2412.01522,

    Pith/arXiv arXiv

  9. [9]

    Gaia-1: A generative world model for autonomous driving.arXiv preprint arXiv:2309.17080,

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving.arXiv preprint arXiv:2309.17080,

    Pith/arXiv arXiv

  10. [10]

    Drivingworld: Constructing world model for autonomous driving via video gpt.arXiv preprint arXiv:2412.19505,

    Xiaotao Hu, Wei Yin, Mingkai Jia, Junyuan Deng, Xiaoyang Guo, Qian Zhang, Xiaoxiao Long, and Ping Tan. Drivingworld: Constructing world model for autonomous driving via video gpt.arXiv preprint arXiv:2412.19505,

    Pith/arXiv arXiv

  11. [11]

    Vipe: Video pose engine for 3d geometric perception

    Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, Jiawei Ren, Kevin Xie, Joydeep Biswas, Laura Leal-Taixe, and Sanja Fidler. Vipe: Video pose engine for 3d geometric perception. InNVIDIA Research Whitepapers arXiv:2508.10934,

    Pith/arXiv arXiv

  12. [12]

    Emma: End-to-end multimodal model for autonomous driving.arXiv preprint arXiv:2410.23262,

    Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, et al. Emma: End-to-end multimodal model for autonomous driving.arXiv preprint arXiv:2410.23262,

    Pith/arXiv arXiv

  13. [13]

    Diffvla: Vision-language guided diffusion planning for autonomous driving.arXiv preprint arXiv:2505.19381, 2025a

    Anqing Jiang, Yu Gao, Zhigang Sun, Yiru Wang, Jijun Wang, Jinghao Chai, Qian Cao, Yuweng Heng, Hao Jiang, Yunda Dong, et al. Diffvla: Vision-language guided diffusion planning for autonomous driving.arXiv preprint arXiv:2505.19381, 2025a. Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Se...

    Pith/arXiv arXiv

  14. [14]

    Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning.arXiv preprint arXiv:2503.07608, 2025b

    Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, and Xinggang Wang. Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning.arXiv preprint arXiv:2503.07608, 2025b. Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. InProceedings of the IEEE/...

    Pith/arXiv arXiv

  15. [15]

    Coda: A real-world road corner case dataset for object detection in autonomous driving

    Kaican Li, Kai Chen, Haoyu Wang, Lanqing Hong, Chaoqiang Ye, Jianhua Han, Yukuai Chen, Wei Zhang, Chunjing Xu, Dit-Yan Yeung, et al. Coda: A real-world road corner case dataset for object detection in autonomous driving. arXiv preprint arXiv:2203.07724,

    Pith/arXiv arXiv

  16. [16]

    Droid-slam in the wild.arXiv preprint arXiv:2603.19076, 2026a

    Moyang Li, Zihan Zhu, Marc Pollefeys, and Daniel Barath. Droid-slam in the wild.arXiv preprint arXiv:2603.19076, 2026a. Xiaofan Li, Yifu Zhang, and Xiaoqing Ye. Drivingdiffusion: Layout-guided multi-view driving scenarios video generation with latent diffusion model. InEuropean Conference on Computer Vision, pages 469–485. Springer,

    arXiv

  17. [17]

    Far-drive: Frame-autoregressive video generation in closed-loop autonomous driving.arXiv preprint arXiv:2603.14938, 2026b

    Yaoru Li, Federico Landi, Marco Godi, Xin Jin, Ruiju Fu, Yufei Ma, Muyang Sun, Heyu Si, and Qi Guo. Far-drive: Frame-autoregressive video generation in closed-loop autonomous driving.arXiv preprint arXiv:2603.14938, 2026b. Yingyan Li, Shuyao Shang, Weisong Liu, Bing Zhan, Haochen Wang, Yuqi Wang, Yuntao Chen, Xiaoman Wang, Yasong An, Chufeng Tang, et al. ...

    arXiv

  18. [18]

    Seeing beyond views: Multi-view driving scene video generation with holistic attention.arXiv preprint arXiv:2412.03520, 2024a

    Hannan Lu, Xiaohe Wu, Shudong Wang, Xiameng Qin, Xinyu Zhang, Junyu Han, Wangmeng Zuo, and Ji Tao. Seeing beyond views: Multi-view driving scene video generation with holistic attention.arXiv preprint arXiv:2412.03520, 2024a. Jiachen Lu, Ze Huang, Zeyu Yang, Jiahui Zhang, and Li Zhang. Wovogen: World volume-aware diffusion for controllable multi-camera dr...

    Pith/arXiv arXiv

  19. [19]

    One million scenes for autonomous driving: Once dataset.arXiv preprint arXiv:2106.11037,

    Jiageng Mao, Minzhe Niu, Chenhan Jiang, Hanxue Liang, Jingheng Chen, Xiaodan Liang, Yamin Li, Chaoqiang Ye, Wei Zhang, Zhenguo Li, et al. One million scenes for autonomous driving: Once dataset.arXiv preprint arXiv:2106.11037,

    Pith/arXiv arXiv

  20. [20]

    https://arxiv.org/abs/2503.03848. NVIDIA, Yulong Cao, Riccardo de Lutio, Sanja Fidler, Guillermo Garcia Cobo, Zan Gojcic, Maximilian Igl, Boris Ivanovic, Peter Karkus, Janick Martinez Esturo, Marco Pavone, Aaron Smith, Ellie Tanimura, Michal Tyszkiewicz, Michael Watson, Qi Wu, and Le Zhang. Alpasim: A modular, lightweight, and data-driven research simulat...

    Pith/arXiv arXiv 2025

  21. [21]

    Cosmos-drive-dreams: Scalable synthetic driving data generation with world foundation models.arXiv preprint arXiv:2506.09042, 2025a

    Xuanchi Ren, Yifan Lu, Tianshi Cao, Ruiyuan Gao, Shengyu Huang, Amirmojtaba Sabour, Tianchang Shen, Tobias Pfaff, Jay Zhangjie Wu, Runjian Chen, et al. Cosmos-drive-dreams: Scalable synthetic driving data generation with world foundation models.arXiv preprint arXiv:2506.09042, 2025a. Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin N...

    Pith/arXiv arXiv

  22. [22]

    Gaia-2: A controllable multi-view generative world model for autonomous driving.arXiv preprint arXiv:2503.20523,

    Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving.arXiv preprint arXiv:2503.20523,

    Pith/arXiv arXiv

  23. [23]

    Lmad: Integrated end-to-end vision-language model for explainable autonomous driving.arXiv preprint arXiv:2508.12404,

    Nan Song, Bozhou Zhang, Xiatian Zhu, Jiankang Deng, and Li Zhang. Lmad: Integrated end-to-end vision-language model for explainable autonomous driving.arXiv preprint arXiv:2508.12404,

    Pith/arXiv arXiv

  24. [24]

    Tokenize the world into object-level knowledge to address long-tail events in autonomous driving.arXiv preprint arXiv:2407.00959,

    Ran Tian, Boyi Li, Xinshuo Weng, Yuxiao Chen, Edward Schmerling, Yue Wang, Boris Ivanovic, and Marco Pavone. Tokenize the world into object-level knowledge to address long-tail events in autonomous driving.arXiv preprint arXiv:2407.00959,

    Pith/arXiv arXiv

  25. [25]

    Towards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717,

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717,

    Pith/arXiv arXiv

  26. [26]

    Anyview: Synthesizing any novel view in dynamic scenes.arXiv preprint arXiv:2601.16982,

    12 Basile Van Hoorick, Dian Chen, Shun Iwase, Pavel Tokmakov, Muhammad Zubair Irshad, Igor Vasiljevic, Swati Gupta, Fangzhou Cheng, Sergey Zakharov, and Vitor Campagnolo Guizilini. Anyview: Synthesizing any novel view in dynamic scenes.arXiv preprint arXiv:2601.16982,

    arXiv

  27. [27]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314,

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314,

    Pith/arXiv arXiv

  28. [28]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025a. Wenhai Wang, Jiangwei Xie, ChuanYang Hu, Haoming Zou, Jianan Fan, Wenwen Tong, Yang Wen, Silei Wu, Hanming Deng, Zhi...

    arXiv

  29. [29]

    Longdwm: Cross-granularity distillation for building a long-term driving world model.arXiv preprint arXiv:2506.01546, 2025b

    Xiaodong Wang, Zhirong Wu, and Peixi Peng. Longdwm: Cross-granularity distillation for building a long-term driving world model.arXiv preprint arXiv:2506.01546, 2025b. Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world- drive world models for autonomous driving. InEuropean conference on computer v...

    Pith/arXiv arXiv

  30. [30]

    Argoverse 2: Next generation datasets for self-driving perception and forecasting.arXiv preprint arXiv:2301.00493,

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting.arXiv preprint arXiv:2301.00493,

    Pith/arXiv arXiv

  31. [31]

    Wod-e2e: Waymo open dataset for end-to-end driving in challenging long-tail scenarios.arXiv preprint arXiv:2510.26125,

    Runsheng Xu, Hubert Lin, Wonseok Jeon, Hao Feng, Yuliang Zou, Liting Sun, John Gorman, Ekaterina Tolstaya, Sarah Tang, Brandyn White, et al. Wod-e2e: Waymo open dataset for end-to-end driving in challenging long-tail scenarios.arXiv preprint arXiv:2510.26125,

    arXiv

  32. [32]

    Vlm-ad: End-to-end autonomous driving through vision-language model supervision.arXiv preprint arXiv:2412.14446, 2024a

    Yi Xu, Yuxin Hu, Zaiwei Zhang, Gregory P Meyer, Siva Karthik Mustikovela, Siddhartha Srinivasa, Eric M Wolff, and Xin Huang. Vlm-ad: End-to-end autonomous driving through vision-language model supervision.arXiv preprint arXiv:2412.14446, 2024a. Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Driveg...

    Pith/arXiv arXiv

  33. [33]

    Neoverse: Enhancing 4d world model with in-the-wild monocular videos.arXiv preprint arXiv:2601.00393,

    13 Yuxue Yang, Lue Fan, Ziqi Shi, Junran Peng, Feng Wang, and Zhaoxiang Zhang. Neoverse: Enhancing 4d world model with in-the-wild monocular videos.arXiv preprint arXiv:2601.00393,

    arXiv

  34. [34]

    Geniedrive: Towards physics-aware driving world model with 4d occupancy guided video generation.arXiv preprint arXiv:2512.12751,

    Zhenya Yang, Zhe Liu, Yuxiang Lu, Liping Hou, Chenxuan Miao, Siyi Peng, Bailan Feng, Xiang Bai, and Hengshuang Zhao. Geniedrive: Towards physics-aware driving world model with 4d occupancy guided video generation.arXiv preprint arXiv:2512.12751,

    arXiv

  35. [35]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072,

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072,

    Pith/arXiv arXiv

  36. [36]

    Mygo: Consistent and controllable multi-view driving video generation with camera control.arXiv preprint arXiv:2409.06189,

    Yining Yao, Xi Guo, Chenjing Ding, and Wei Wu. Mygo: Consistent and controllable multi-view driving video generation with camera control.arXiv preprint arXiv:2409.06189,

    Pith/arXiv arXiv

  37. [37]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048,

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048,

    Pith/arXiv arXiv

  38. [38]

    Autodrive-r2: Incentivizing reasoning and self-reflection capacity for vla model in autonomous driving.arXiv preprint arXiv:2509.01944,

    Zhenlong Yuan, Chengxuan Qian, Jing Tang, Rui Chen, Zijian Song, Lei Sun, Xiangxiang Chu, Yujun Cai, Dapeng Zhang, and Shuo Li. Autodrive-r2: Incentivizing reasoning and self-reflection capacity for vla model in autonomous driving.arXiv preprint arXiv:2509.01944,

    Pith/arXiv arXiv

  39. [39]

    Autovla: A vision- language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning

    Zewei Zhou, Tianhui Cai, Seth Z Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. Autovla: A vision- language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. arXiv preprint arXiv:2506.13757,

    Pith/arXiv arXiv

  40. [40]

    Most training clips are collected from PAV, which provides diverse large-scale driving logs and rich long-tail driving scenarios

    14 A Dataset Curation and Splits A.1 Dataset Sources We curate driving video logs from NVIDIA PhysicalAI-Autonomous-Vehicles (PAV) (NVIDIA Corporation, 2025), PandaSet (Xiao et al., 2021), and nuScenes (Caesar et al., 2020), resulting in approximately 200K 41-frame clips from about 50K scenes. Most training clips are collected from PAV, which provides div...

    2025

  41. [41]

    provides a useful external source for generative scaling. OpenLongTail converts Waymo front-camera videos into policy-compatible multi-view assets under the target camera rig, enabling these out-of-source videos to be used for downstream VLA training. Beyond qualitative conversion quality, we further evaluate whether the converted assets provide practical...

    2025