Pith. sign in

REVIEW 4 major objections 6 minor 5 cited by

Epona: Autoregressive Diffusion World Model for Autonomous Driving

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Epona claims a single autoregressive diffusion model can generate two minutes of consistent driving video and, by disabling the video branch, plan trajectories in real time.

desk verdict A capable driving world model with a genuinely useful autoregressive-diffusion design, but the headline FVD claim is not verifiable as reported; the NAVSIM planning result is the more solid part. read the letter →

arxiv 2506.24113 v1 pith:XO2YJ7CM submitted 2025-06-30 cs.CV

Kaiwen Zhang , Zhenyu Tang , Xiaotao Hu , Xingang Pan , Xiaoyang Guo , Yuan Liu , Jingwei Huang , Li Yuan
show 4 more authors
Qian Zhang Xiao-Xiao Long Xun Cao Wei Yin
This is my paper
classification cs.CV
keywords worldmodelautonomousdrivingdiffusionautoregressivegenerationtrajectoryplanningvideorectifiedflowNAVSIM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that driving world models should predict the next frame one at a time rather than generating a fixed block of future frames at once. It introduces Epona, an autoregressive diffusion model that combines a causal transformer for temporal context with two diffusion transformers: one renders the next frame, the other outputs a multi-second trajectory. Because the temporal and spatial parts are decoupled, the model can generate high-resolution video for over two minutes and, when video generation is switched off, plan trajectories in real time. On the nuScenes benchmark it reports an FVD of 82.8, below the 89.4 of the strongest prior open world model, and on NAVSIM it reports a PDMS of 86.2, above several end-to-end planners, while using only a front camera. The claim, if it holds, would unify long-horizon world simulation and motion planning in a single self-supervised framework.

What carries the argument

The load-bearing machinery is the decoupling of temporal dynamics from spatial rendering. A multimodal spatiotemporal transformer (MST) interleaves spatial attention and causal temporal attention, compressing history into a compact latent $F$; a next-frame prediction DiT (VisDiT) denoises the next image latent conditioned on $F$ and an action, while a trajectory planning DiT (TrajDiT) denoises a full future trajectory conditioned on the same $F$. Chain-of-forward training is a training-time rollout that feeds the model its own one-step-velocity-predicted frames back as context every 10 steps, so the model sees its own errors during training. The temporal-aware DCAE decoder adds spatiotemporal self-attention before image decoding to reduce frame-to-frame flicker.

What would settle it

Run Epona and Vista (plus InfinityDrive) through the same evaluation code with identical clip length and conditioning prior on the nuScenes validation set; if Epona's FVD is not below Vista's under that shared protocol, the central SOTA claim collapses.

Watch

Extended reading notes

Core claim

Epona's central claim is that the global joint-distribution formulation of video diffusion world models is the bottleneck for driving, and that world modeling should instead be written as a sequential distribution over the next camera frame, $p(O_{T+1}\mid \{O_t,a_{t-1\to t}\}_{t=1}^T, a_{T\to T+1})$, plus a separate policy over future trajectories $\pi(\{a_{T\to T+i}\}_{i=1}^n \mid \cdot)$. The paper's contribution is an architecture that makes this formulation practical: a causal multimodal spatiotemporal transformer compresses history into a compact latent, a next-frame DiT renders the future frame in continuous latent space under flow-matching, and a small trajectory DiT generates an entire 3-second pose trajectory in one denoising pass. A chain-of-forward training procedure periodically runs the model on its own one-step-denoised predictions so that training matches autoregressive inference, and a temporal-aware decoder suppresses flicker across frames. The result is claimed to be the first driving world model that is simultaneously a long-horizon high-fidelity video simulator (up to 120 seconds, 600 frames) and a real-time motion planner reaching 20 Hz.

Load-bearing premise

The 'state-of-the-art 7.4% FVD improvement' claim assumes every method's published FVD was measured under the same evaluation protocol (clip length, rollout procedure, and evaluation script), which the paper does not show; if protocols differ, the FVD ranking may not be meaningful.

Editorial extensions

If this is right

  • Long-horizon simulation: the 120-second autoregressive generation capability makes closed-loop, high-fidelity simulation of driving scenes feasible for test-time evaluation and data augmentation.
  • Real-time planning without perception stacks: with video prediction deactivated, MST plus TrajDiT plans at up to 20 Hz on a single consumer GPU.
  • Controllable scenario generation: user-specified pose trajectories condition the video stream, allowing synthesis of diverse and extreme driving scenarios.
  • Self-supervised traffic-rule learning: the model learns behaviors such as stopping at red lights purely from next-frame and trajectory prediction without annotation.
  • A unified pretraining objective: joint trajectory and image prediction in a shared latent can serve as a foundation for downstream planning or reinforcement learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported FVD holds under a matched protocol, the decoupled autoregressive-diffusion design is a general recipe: any fixed-length video diffusion world model could be converted to frame-wise diffusion with a trajectory stream, even outside driving.
  • The chain-of-forward training trick (feeding one-step-denoised latents back as context) is architecture-agnostic and could be tested on other autoregressive video diffusion models, e.g., general long-video generators, as a cheap drift remedy.
  • The paper's own supplementary comparison with InfinityDrive (FVD 70.06 vs Epona 82.83) suggests that the SOTA claim is sensitive to which concurrent works are included; a fair reader should treat 'state of the art' as conditional on the list of baselines.
  • A natural next step is evaluating the generated minutes-long videos under closed-loop driving metrics (collisions, rule violations) rather than FVD, which would test whether the world model's world knowledge is actually usable for control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Epona, an autoregressive diffusion world model for autonomous driving. It factorizes spatiotemporal modeling into a causal multimodal spatiotemporal transformer (MST) over historical context and two diffusion transformers: TrajDiT for future trajectories and VisDiT for next-frame latents, trained jointly with rectified-flow losses (Eqs. 4-5). A chain-of-forward training strategy (Sec. 3.4) feeds one-step denoised self-predictions back into training to reduce autoregressive drift, and a temporal-aware DCAE decoder (Sec. 3.5) adds temporal attention to the image decoder. The headline claims are state-of-the-art nuScenes FVD (82.8 vs. Vista's 89.4, a 7.4% improvement, Table 1), generation of up to 120s/600 frames, and NAVSIM PDMS 86.2, outperforming UniAD (83.4) and LAW/PARA-Drive (84.6) (Table 4).

Significance. The architectural combination is timely and interesting: continuous-latent autoregression avoids discrete tokenization, joint trajectory-video training is a plausible route to world-model-based planning, and the chain-of-forward strategy directly addresses a known failure mode. The internal ablations in Tables 5-7 and Fig. 8 are consistent and support the design choices of joint training, chain-of-forward, and temporal-aware decoding. If the SOTA claims were backed by protocol-matched comparisons, this would be a strong contribution. At present, however, the central video-generation SOTA claim is not verifiable from the reported numbers, and Supplementary B contains an internal inconsistency that directly affects that claim. The planning result (PDMS 86.2) is more internally consistent, though it would benefit from additional evaluation details.

major comments (4)
  1. [Sec. 4.2, Table 1] The nuScenes FVD comparison is not protocol-controlled. The paper never states the number of generated frames, the frame rate, or the FVD evaluation window used for Epona's Table 1 number, and it compares against FVD scores reported by other papers without a shared evaluation script. This matters because Tables 6-7 show FVD strongly depends on sequence length on NuPlan (50.77 at 10 frames vs. 74.88 at 40 frames). The 7.4% improvement over Vista cannot be assessed unless the authors state the exact protocol and provide matched evaluations, for example by running Vista with the same rollout length and using the same FVD implementation.
  2. [Supplementary B] Supplementary B reports InfinityDrive's nuScenes FVD as 70.06 and MagicDriveDiT's as 94.84, and then states that Epona (82.83) "exhibits competitive visual generation performance" and that MagicDriveDiT "achieves slightly better visual quality." Lower FVD is better, so InfinityDrive at 70.06 is better than Epona at 82.83, while MagicDriveDiT at 94.84 is worse than Epona. This is internally inconsistent; if InfinityDrive's number was measured under a comparable protocol, it invalidates the main-text claim of state-of-the-art FVD. The authors must either include these concurrent methods in Table 1 with protocol-matched numbers or clearly specify the protocol differences; the current text does not resolve the contradiction.
  3. [Table 1 footnote and Sec. 4.2] The max-duration claim of 120s/600 frames is not backed by a quantitative definition. The footnote only says "the horizon that produces plausible results," with no stated frame rate, no criterion for plausibility, and no metric for drift. If the frame rate is 5 fps the duration is 120s, but if it is 10 fps it is 60s; Vista's 15s/150 implies 10 fps. The "minutes longer" comparison may therefore reflect a frame-rate convention rather than model capability. The authors should state the frame rate and report a drift/quality metric as a function of generated frames, ideally with sample videos.
  4. [Sec. 4.3, Table 4] The NAVSIM PDMS is reported as a single number without stating the exact simulation setting, such as the number of runs, the closed-loop policy used, or whether this is the standard NAVSIM data-driven benchmark. Because the planning claim is load-bearing, the evaluation protocol should be specified precisely, and it would be useful to report per-split variance or multiple seeds.
minor comments (6)
  1. [Sec. 3.2] There are typos in this section: "world moels" should be "world models" and "distortes" should be "distorts."
  2. [References] Reference [46] for UniAD points to an ACM Transactions on Database Systems paper by Shi et al.; the correct UniAD citation (Li et al., CVPR 2023) should be used.
  3. [Supplementary B] Reference [19] (InfinityDrive) is discussed only in the supplementary material; it should be cited in the main text or the discussion should be moved to the main comparison.
  4. [Sec. 3.4, Eq. (7)] The one-step denoised latent in Eq. (7) is treated as a proxy for a real next frame, but there is no discussion of the approximation error or an ablation on the number of chain steps beyond the fixed setting of three forward passes every 10 training steps.
  5. [Main text vs. Supplementary Fig. 10] The main text emphasizes generation of "2 minutes" (120s), while Supplementary Fig. 10 shows 140 seconds; the stated maximum duration should be made consistent.
  6. [Table 2 and Sec. 4.1] The "real-time" planning claim should clarify that it applies to planning-only mode with MST and TrajDiT at 10 sampling steps; the full video-generation path at 100 steps takes about 2.3s per frame on a 4090, which is not real-time.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: central results are external benchmark evaluations; only minor non-load-bearing self-citations exist.

full rationale

Epona's claimed derivation chain is a standard train-then-evaluate pipeline. The losses in Eqs. 4-6 are rectified-flow objectives on trajectory and next-frame latent targets, and all model parameters are trained on NuPlan data plus 700 nuScenes scenes, then evaluated on held-out nuScenes validation, NuPlan test, and NAVSIM test splits. No free parameter is fitted to the FVD, PDMS, or L2 columns that support the headline claims, and the baselines in Tables 1, 3, and 4 are externally reported numbers, not quantities derived from Epona's own outputs. The self-citations ([25], [18], [60]) are used for architectural inspiration and related-work placement, not as load-bearing evidence: no central conclusion depends on an unverified theorem or prior conclusion by the same authors. The causal-temporal design is explicitly acknowledged as similar to Diffusion Forcing and FIFO-Diffusion and is validated by in-paper ablations (Fig. 8, Table 5). The Supplementary B discrepancy (InfinityDrive FVD 70.06 below Epona's 82.83 while the abstract claims SOTA) and the unspecified FVD clip-length protocol are correctness and comparability concerns, not circularity: Epona's numbers are not constructed from the baseline numbers. Therefore no claimed prediction reduces by construction to its own input; the score of 1 merely flags minor, non-load-bearing self-citation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on empirical benchmark evaluations rather than a mathematical derivation. The model depends on chosen hyperparameters (context length, sampling steps, CoF schedule) and on assumptions that flow matching, DCAE latents, one-step CoF estimates, and cross-paper FVD comparisons are valid. No invented entities are introduced.

free parameters (3)
  • Conditioning context length T = 10 frames
    Table 7 shows FVD monotonically improves with T (2, 5, 10), and the paper sets 10 because of memory limits; headline FVD numbers therefore depend on this user-chosen cap.
  • Diffusion sampling steps = 100 per DiT
    Section 4.1 sets DiT sampling to 100 steps for all reported quality metrics; no ablation on steps is shown, so results are tied to this choice.
  • Chain-of-Forward schedule = every 10 training steps, 3 forward passes
    Introduced in Sec 4.1 without sensitivity analysis; Fig. 8's drift improvement depends on this specific schedule.
assumptions (4)
  • standard math Rectified flow (Eqs. 2-3) with linear interpolation between noise and data is a valid generative training objective for both image latents and trajectories.
    Invoked in Sec. 3.1 and used for Lvis (Eq. 5) and Ltraj (Eq. 4); rests on flow-matching results from [2,34,35].
  • domain assumption The DCAE latent space preserves enough visual detail for high-fidelity next-frame prediction and for FVD to reflect video quality.
    Adopted in Sec. 3.1/3.5 without a reconstruction-quality comparison against other autoencoders; the paper acknowledges visual trade-offs caused by DCAE in Supplementary B.
  • ad hoc to paper A one-step denoised latent (Eq. 7) approximates a real sample closely enough that training on chained self-predicted frames transfers to inference-time autoregressive sampling.
    This is the basis of chain-of-forward training in Sec. 3.4; no analysis is given of the approximation error or of why one step suffices.
  • domain assumption FVD values reported by different papers are comparable even when video lengths, clip counts, and metric implementations differ.
    Underlies the SOTA comparison in Table 1; the paper does not specify Epona's nuScenes FVD clip length, and its own Supplementary B shows a concurrent method with a lower FVD.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Epona: Autoregressive Diffusion World Model for Autonomous Driving." pith.science (2026). https://pith.science/paper/XO2YJ7CM

@misc{pith2026250624113,
  author       = {Pith},
  title        = {Pith review of: Epona: Autoregressive Diffusion World Model for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XO2YJ7CM}},
  note         = {Machine review of arXiv:2506.24113}
}
read the original abstract

Diffusion models have demonstrated exceptional visual quality in video generation, making them promising for autonomous driving world modeling. However, existing video diffusion-based world models struggle with flexible-length, long-horizon predictions and integrating trajectory planning. This is because conventional video diffusion models rely on global joint distribution modeling of fixed-length frame sequences rather than sequentially constructing localized distributions at each timestep. In this work, we propose Epona, an autoregressive diffusion world model that enables localized spatiotemporal distribution modeling through two key innovations: 1) Decoupled spatiotemporal factorization that separates temporal dynamics modeling from fine-grained future world generation, and 2) Modular trajectory and video prediction that seamlessly integrate motion planning with visual modeling in an end-to-end framework. Our architecture enables high-resolution, long-duration generation while introducing a novel chain-of-forward training strategy to address error accumulation in autoregressive loops. Experimental results demonstrate state-of-the-art performance with 7.4\% FVD improvement and minutes longer prediction duration compared to prior works. The learned world model further serves as a real-time motion planner, outperforming strong end-to-end planners on NAVSIM benchmarks. Code will be publicly available at \href{https://github.com/Kevin-thu/Epona/}{https://github.com/Kevin-thu/Epona/}.

Figures

Figures reproduced from arXiv: 2506.24113 by the authors.

Figure 1
Figure 1. Versatile capabilities of Epona. Given historical driving context, our Epona can generate consistent minutes-long future driving scenes at high resolution (A). It can be controlled by diverse trajectories (B), and understand real-world traffic knowledge (C). In addition, our world model can predict future trajectories and serve as an end-to-end real-time motion planner (D). Abstract Diffusion models have demonstrate… view at source ↗
Figure 2
Figure 2. Overview of Epona. Our world model utilizes a multimodal spatiotemporal transformer to process the historical context of the first T frames and employs a next-frame prediction DiT to generate the frame at T + 1 and a trajectory planning DiT to forecast the future N-frame pose trajectory. By adopting a chain-of-forward strategy, our approach enables high-quality and long-horizon video generation with an autoregressiv… view at source ↗
Figure 3
Figure 3. Comparison of Different World Modeling Formula￾tion. Up: Conventional autoregressive pipeline quantizes conti￾nous images into discrete tokens and perform next-token predic￾tion iteratively. Middle: The video-diffusion-based methods gen￾erate future n frames simultaneously. Down: Our method autore￾gressively predicts fine-grained future frames in continuous space. tailed in Sec.3.4. Additionally, to enhance video qu… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Concept illustration of our training process. Here x can be either image latents or trajectories. After training, this embedding encapsulates the historical context {Ot, at−1→t} T t=1. Trajectory Planning Diffusion Transformer (TrajDiT). TrajDiT predicts future traject…
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Trajectory-controlled video generation. Our world model can generate controllable videos based on predefined trajectories [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative Comparison between long videos generated by models w/ and w/o Chain-of-Forward training. Left: Visual quality deteriorates rapidly after 10–20 seconds. Right: The same driving scenes with Chain-of-Forward training maintain high visual quality, generating mi…
Figure 8
Figure 8. Figure 8: Effect of Chain-of-Forward training. FID comparison in NuPlan test set between models w/ and w/o Chain-of-Forward training strategy. mance, we conduct an ablation study comparing results with and without this strategy. Given that our model iter￾atively generates the ne…
Figure 9
Figure 9. Figure 9: Detailed architecture of dual-stram DiT and single stream DiT blocks. We use nearly identical architecures for both TrajDiT and VisDiT, modified from text-image and video DiT architecture from [26, 31]. Action control is only for VisDiT. 2 [PITH_FULL_IMAGE:figures/ful…
Figure 10
Figure 10. Figure 10: Visualization of Longer Videos. Our world model is capable of generating extended videos (140 seconds) while maintaining [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation

    cs.GR 2026-07 conditional novelty 6.0 of 10

    A feed-forward model reconstructs a layered, simulation-ready 3D Gaussian world from multi-view driving video in ~1.5 s, with quality approaching per-scene optimized reconstruction.

  2. OmniNWM: Omniscient Driving Navigation World Models

    cs.CV 2025-10 conditional novelty 6.0 of 10

    OmniNWM jointly generates long panoramic multi-modal driving videos, controls them precisely via normalized Plücker ray-maps, and derives dense driving rewards from generated 3D occupancy.

  3. UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A single mask-modulated DiT that co-trains future video and trajectories yields stronger autonomous-driving action generalization and 4.3× faster trajectory-only inference than dual-DiT designs.

  4. UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving

    cs.CV 2026-01 conditional novelty 5.0 of 10

    A unified VLM for autonomous driving that couples trajectory planning with future-frame image generation improves open- and closed-loop planning metrics on Bench2Drive and nuScenes.

  5. MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization

    cs.CV 2025-07 conditional novelty 4.0 of 10

    Splitting quantization across multiple small sub-codebooks with nested masking raises VQ-VAE reconstruction fidelity, giving MGVQ rFID 0.49 and PSNR 24.70 on ImageNet at 16 times downsampling.

Reference graph

Works this paper leans on

69 extracted references · 41 canonical work pages · cited by 5 Pith papers

  1. [1]

    Cosmos world foun- dation model platform for physical ai

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foun- dation model platform for physical ai. arXiv preprint arXiv:2501.03575, 2025. 2, 3

    arXiv 2025

  2. [2]

    Albergo and Eric Vanden-Eijnden

    Michael S. Albergo and Eric Vanden-Eijnden. Building nor- malizing flows with stochastic interpolants, 2023. 2, 4

    work page 2023

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3, 4

    arXiv 2023

  4. [4]

    Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis

    A. Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent dif- fusion models. CVPR, pages 22563–22575, 2023. 5

    work page 2023

  5. [5]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS) , 33:1877–1901,

    work page 1901

  6. [6]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 2, 6, 7, 8

    work page 2020

  7. [7]

    nuplan: A closed-loop ml-based plan- ning benchmark for autonomous vehicles

    Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based plan- ning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021. 6, 9

    arXiv 2021

  8. [8]

    Diffusion forcing: Next-token prediction meets full-sequence diffu- sion, 2024

    Boyuan Chen, Diego Marti Monso, Yilun Du, Max Sim- chowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffu- sion, 2024. 3

    2024

Show all 69 references
  1. [9]

    Deep compression autoencoder for efficient high-resolution diffu- sion models

    Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffu- sion models. arXiv preprint arXiv:2410.10733, 2024. 4, 6

    2024 arXiv

  2. [10]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning. arXiv preprint arXiv:2402.13243,

    arXiv

  3. [11]

    Driving- gpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers

    Yuntao Chen, Yuqi Wang, and Zhaoxiang Zhang. Driving- gpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers. arXiv preprint arXiv:2412.18607, 2024. 2, 3, 4, 1

    2024 arXiv

  4. [12]

    Navsim: Data-driven non- reactive autonomous vehicle simulation and benchmarking

    Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, Andreas Geiger, and Kashyap Chitta. Navsim: Data-driven non- reactive autonomous vehicle simulation and benchmarking. In NeurIPS,...

    2024

  5. [13]

    Understanding world or predict- ing future? a comprehensive survey of world models

    Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Ze- fang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, et al. Understanding world or predict- ing future? a comprehensive survey of world models. arXiv preprint arXiv:2411.14499, 2024. 2, 3

    2024

  6. [14]

    Scaling rectified flow trans- formers for high-resolution image synthesis, 2024

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yan- nik Marek, and Robin Rombach. Scaling rectified flow tr...

    2024

  7. [15]

    Magicdrivedit: High-resolution long video generation for autonomous driving with adaptive con- trol

    Ruiyuan Gao, Kai Chen, Bo Xiao, Lanqing Hong, Zhen- guo Li, and Qiang Xu. Magicdrivedit: High-resolution long video generation for autonomous driving with adaptive con- trol. arXiv preprint arXiv:2411.13807, 2024. 2, 3, 1

    2024 arXiv

  8. [16]

    MagicDrive: Street view generation with diverse 3d geometry control

    Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, and Qiang Xu. MagicDrive: Street view generation with diverse 3d geometry control. In ICLR, 2024. 3

    2024

  9. [17]

    Vista: A generalizable driving world model with high fidelity and versatile controllability

    Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. arXiv preprint arXiv:2405.17398, 2024. 2, 3, 4, 6, 7

    2024 arXiv

  10. [18]

    Dome: Tam- ing diffusion model into high-fidelity controllable occupancy world model

    Songen Gu, Wei Yin, Bu Jin, Xiaoyang Guo, Junming Wang, Haodong Li, Qian Zhang, and Xiaoxiao Long. Dome: Tam- ing diffusion model into high-fidelity controllable occupancy world model. ArXiv, abs/2410.10429, 2024. 3

    2024 arXiv

  11. [19]

    Infinitydrive: Breaking time limits in driving world models, 2024

    Xi Guo, Chenjing Ding, Haoxuan Dou, Xin Zhang, Weixuan Tang, and Wei Wu. Infinitydrive: Breaking time limits in driving world models, 2024. 1

    2024

  12. [20]

    World models

    David Ha and J ¨urgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2018. 2

    2018 arXiv

  13. [21]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 7

    2017

  14. [22]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. In Advances in Neural Infor- mation Processing Systems, pages 6840–6851. Curran Asso- ciates, Inc., 2020. 2, 4

    2020

  15. [23]

    Gaia-1: A generative world model for au- tonomous driving

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gian- luca Corrado. Gaia-1: A generative world model for au- tonomous driving. arXiv preprint arXiv:2309.17080, 2023. 2, 3, 4, 1

    2023 arXiv

  16. [24]

    St-p3: End-to-end vision-based au- tonomous driving via spatial-temporal feature learning

    Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to-end vision-based au- tonomous driving via spatial-temporal feature learning. In European Conference on Computer Vision, pages 533–549. Springer, 2022. 7, 8

    2022

  17. [25]

    Driving- world: Constructingworld model for autonomous driving via video gpt

    Xiaotao Hu, Wei Yin, Mingkai Jia, Junyuan Deng, Xiaoyang Guo, Qian Zhang, Xiaoxiao Long, and Ping Tan. Driving- world: Constructingworld model for autonomous driving via video gpt. arXiv preprint arXiv:2412.19505, 2024. 2, 3, 4, 5, 6 10

    2024 arXiv

  18. [26]

    Hunyuanvideo: A systematic framework for large video generative models, 2024

    Tencent Hunyuan. Hunyuanvideo: A systematic framework for large video generative models, 2024. 4, 5, 1, 2

    2024

  19. [27]

    Adriver-i: A general world model for autonomous driving,

    Fan Jia, Weixin Mao, Yingfei Liu, Yucheng Zhao, Yuqing Wen, Chi Zhang, Xiangyu Zhang, and Tiancai Wang. Adriver-i: A general world model for autonomous driving,

  20. [28]

    Vad: Vectorized scene representa- tion for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representa- tion for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision ,...

    2023

  21. [29]

    Fifo-diffusion: Generating infinite videos from text without training

    Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. In NeurIPS, 2024. 3

    2024

  22. [30]

    Drivegan: Towards a controllable high-quality neural simulation

    Seung Wook Kim, Jonah Philion, Antonio Torralba, and Sanja Fidler. Drivegan: Towards a controllable high-quality neural simulation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 5820–5829, 2021. 6

    2021

  23. [31]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 2, 4, 5, 1

    2024

  24. [32]

    A path towards autonomous machine intelli- gence version 0.9

    Yann LeCun. A path towards autonomous machine intelli- gence version 0.9. 2, 2022-06-27. Open Review, 62(1):1–62,

    2022

  25. [33]

    Enhancing end-to-end autonomous driving with latent world model

    Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan. Enhancing end-to-end autonomous driving with latent world model. 2024. 9

    2024

  26. [34]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi- milian Nickel, and Matt Le. Flow matching for generative modeling, 2023. 2, 4

    2023

  27. [35]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022. 2, 4, 6

    2022 arXiv

  28. [36]

    WoV oGen: World V olume-Aware Diffusion for Con- trollable Multi-Camera Driving Scene Generation

    Jiachen Lu, Ze Huang, Jiahui Zhang, Zeyu Yang, and Li Zhang. WoV oGen: World V olume-Aware Diffusion for Con- trollable Multi-Camera Driving Scene Generation. arXiv preprint arXiv:2312.02934, 2023. 3, 6

    2023 arXiv

  29. [37]

    Latte: La- tent diffusion transformer for video generation, 2024

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: La- tent diffusion transformer for video generation, 2024. 5

    2024

  30. [38]

    Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation

    Yiyang Ma, Xingchao Liu, Xi aokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai Yu, Liang Zhao, Yisong Wang, Jiaying Liu, and Chong Ruan. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and g...

    2024 arXiv

  31. [39]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  32. [40]

    Multi- modal fusion transformer for end-to-end autonomous driv- ing

    Aditya Prakash, Kashyap Chitta, and Andreas Geiger. Multi- modal fusion transformer for end-to-end autonomous driv- ing. In CVPR, 2021. 9

    2021

  33. [41]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling, 2023

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xin- tao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling, 2023. 3

    2023

  34. [42]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. OpenAI blog, 2018. 2

    2018

  35. [43]

    Language models are unsuper- vised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsuper- vised multitask learners. OpenAI blog, 1(8):9, 2019. 2

    2019

  36. [44]

    Einops: Clear and reliable tensor manip- ulations with einstein-like notation

    Alex Rogozhnikov. Einops: Clear and reliable tensor manip- ulations with einstein-like notation. In International Confer- ence on Learning Representations, 2022. 5

    2022

  37. [45]

    High-resolution image syn- thesis with latent diffusion models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2022. 2, 3, 4

    2022

  38. [46]

    Uniad: A unified ad hoc data processing system.ACM Trans- actions on Database Systems (TODS), 42(1):1–42, 2016

    Xiaogang Shi, Bin Cui, Gillian Dobbie, and Beng Chin Ooi. Uniad: A unified ad hoc data processing system.ACM Trans- actions on Database Systems (TODS), 42(1):1–42, 2016. 7, 8, 9

    2016

  39. [47]

    History-guided video diffusion, 2025

    Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, and Vincent Sitzmann. History-guided video diffusion, 2025. 3

    2025

  40. [48]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In ICLR, 2021. 2, 4

    2021

  41. [49]

    Scene as occupancy

    Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al. Scene as occupancy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8406– 8415, 2023. 8

    2023

  42. [50]

    To- wards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 2, 7

    2018 arXiv

  43. [51]

    Diffusion models are real-time game engines,

    Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines,

  44. [52]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 2

    2017

  45. [53]

    Phenaki: Variable length video generation from open domain textual description, 2022

    Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kin- dermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description, 2022. 3

    2022

  46. [54]

    Gen-l-video: Multi-text to long video generation via temporal co-denoising

    Fu-Yun Wang, Wenshuo Chen, Guanglu Song, Han-Jia Ye, Yu Liu, and Hongsheng Li. Gen-l-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264, 2023. 3

    2023 arXiv

  47. [55]

    Zola: Zero-shot creative long animation generation with short video model

    Fu-Yun Wang, Zhaoyang Huang, Qiang Ma, Guanglu Song, Xudong Lu, Weikang Bian, Yijin Li, Yu Liu, and Hongsheng Li. Zola: Zero-shot creative long animation generation with short video model. In ECCV, 2024. 3

    2024

  48. [56]

    Drivedreamer: Towards real-world- driven world models for autonomous driving

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jia- gang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world- driven world models for autonomous driving. arXiv preprint arXiv:2309.09777, 2023. 2, 3, 6 11

    2023 arXiv

  49. [57]

    Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Au- tonomous Driving

    Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Au- tonomous Driving. In CVPR, 2024. 2, 3, 6

    2024

  50. [58]

    Loong: Generating minute-level long videos with autoregressive lan- guage models, 2024

    Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive lan- guage models, 2024. 3

    2024

  51. [59]

    Para-drive: Parallelized architecture for real- time autonomous driving

    Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Parallelized architecture for real- time autonomous driving. In CVPR, 2024. 9

    2024

  52. [60]

    Goalflow: Goal- driven flow matching for multimodal trajectories generation in end-to-end autonomous driving

    Zebin Xing, Xingyu Zhang, Yang Hu, Bo Jiang, Tong He, Qian Zhang, Xiaoxiao Long, and Wei Yin. Goalflow: Goal- driven flow matching for multimodal trajectories generation in end-to-end autonomous driving. ArXiv, abs/2503.05689,

  53. [61]

    Generalized Predictive Model for Autonomous Driving

    Jiazhi Yang, Shenyuan Gao, Yihang Qiu, Li Chen, Tianyu Li, Bo Dai, Kashyap Chitta, Penghao Wu, Jia Zeng, Ping Luo, Jun Zhang, Andreas Geiger, Yu Qiao, and Hongyang Li. Generalized Predictive Model for Autonomous Driving. In CVPR, 2024. 3, 6

    2024

  54. [62]

    Nuwa-xl: Diffusion over diffusion for extremely long video generation,

    Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Gong Ming, Lijuan Wang, Zicheng Liu, Houqiang Li, and Nan Duan. Nuwa-xl: Diffusion over diffusion for extremely long video generation,

  55. [63]

    Ang Jr au2

    Chengran Yuan, Zhanqi Zhang, Jiawei Sun, Shuo Sun, Zefan Huang, Christina Dao Wen Lee, Dongen Li, Yuhang Han, Anthony Wong, Keng Peng Tee, and Marcelo H. Ang Jr au2. Drama: An efficient end-to-end motion planner for au- tonomous driving with mamba, 2024. 9

    2024

  56. [64]

    Occworld: Learning a 3d occupancy world model for autonomous driving

    Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. In Euro- pean conference on computer vision, pages 55–72. Springer,

  57. [65]

    Genad: Generative end-to-end au- tonomous driving

    Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Generative end-to-end au- tonomous driving. In European Conference on Computer Vision, pages 87–104. Springer, 2024. 8

    2024

  58. [66]

    Doe-1: Closed-loop au- tonomous driving with large world model

    Wenzhao Zheng, Zetian Xia, Yuanhui Huang, Sicheng Zuo, Jie Zhou, and Jiwen Lu. Doe-1: Closed-loop au- tonomous driving with large world model. arXiv preprint arXiv:2412.09627, 2024. 2, 3, 7, 8

    2024 arXiv

  59. [67]

    Zettlemoyer, and Omer Levy

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke S. Zettlemoyer, and Omer Levy. Transfusion: Pre- dict the next token and diffuse images with one multi-modal model. ArXiv, abs/2408.11039, 2024. 1 12 Epona: Autor...

    2024 arXiv

  60. [68]

    single-step prediction

    Multi-step vs. single-step prediction. Unlike prior ap- proaches that interleave single-step image and action generation using transformers, our model predicts an en- tire N-step future trajectory in one shot. This is par- ticularly beneficial for real-time motion planning in ...

  61. [69]

    Continuous vs. discrete action representation.While ex- isting methods discretize continuous action spaces into tokens, our diffusion model generates high-resolution continuous trajectories directly, enabling more precise planning and control. Among these methods, only Driving...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.