Pith. sign in

REVIEW 4 major objections 8 minor 5 cited by

InfinityDrive: Breaking Time Limits in Driving World Models

T0 review · 4 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read InfinityDrive is presented as the first driving world model to generate over 1,500 frames of consistent 576×1024 video.

desk verdict A plausible architecture for minute-scale driving world models, but the core 1500-frame claim is supported by only 100-120 frames of quantitative evaluation; worth referee time, not belief yet. read the letter →

arxiv 2412.01522 v2 pith:KW2X4YQA submitted 2024-12-02 cs.CV

Xi Guo , Chenjing Ding , Haoxuan Dou , Xin Zhang , Weixuan Tang , Wei Wu This is my paper · ORCID
classification cs.CV
keywords drivingworldmodellongvideogenerationdiffusiontransformerspatio-temporalco-modelingmemoryinjectionerroraccumulationautonomoussimulationtext-to-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

InfinityDrive is presented as the first driving world model that can synthesize more than 1,500 frames of 576×1024 driving video, lasting over two minutes, while keeping the scene spatially and temporally coherent. The paper argues that existing driving world models fall short either by generating only a few seconds of video, by dropping resolution to reach longer horizons, or by drifting into blur when rolled out autoregressively. Its proposed remedy has three parts: an efficient spatio-temporal co-modeling module that shifts processing effort between spatial detail and temporal density depending on resolution, a curriculum that extends training windows to 128 frames, and memory injection/retention paired with a memory-curve loss that down-weights distant frames so errors do not compound. If the claim holds, world models become a practical source of minutes-long, diverse, out-of-distribution driving footage for training and testing autonomous systems.

What carries the argument

The load-bearing object is the memory block inside the autoregressive rollout. The input is split into a memory segment of the first $M$ frames and a future segment; during training the memory frames receive diffusion timestep $t=0$ (no noise), so the model conditions on clean history while the future frames undergo full denoising. The memory curve adaptive loss then weights each future frame by $w(T)=e^{-\lambda T}$, where $T$ is the distance from the memory segment: near frames are strongly held to the history, far frames are allowed generative freedom, and the single decay rate $\lambda$ controls how quickly that transition happens. Around this, the Efficient Spatio-Temporal Co-Modeling module dynamically changes temporal density with resolution, and skip-ROPE positional embeddings preserve the original frame-rate indices when training frames are subsampled, letting 128-frame windows fit in GPU memory at 576×1024. These mechanisms are what the paper credits with preventing cumulative error and with keeping FID/FVD flat over time.

What would settle it

Run the Table 2 protocol on InfinityDrive's actual 1,500-frame generations: compute FID and FVD on frames 1,300–1,500 with the same reference distribution used for the 100-frame scores, and compare with those scores. If the numbers climb steeply, or if a specific vehicle or building tracked in the released videos changes identity between frames 1,000 and 1,500, the extrapolated minute-scale claim fails; the model would still be a strong 100- to 120-frame world model, but not the 1,500-frame one advertised.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a diffusion-transformer driving world model can roll out high-resolution video over minute-scale horizons without the usual collapse into blur or scene drift. The central quantitative evidence is the long-term comparison in Table 2: at 100 frames, InfinityDrive reports FID 14.92 and FVD 113.91, lower than Vista (FID 34.61, FVD 234.60), SVD-XT (FID 42.99, FVD 275.08), and StreamingT2V-SVDXT (FID 181.22, FVD 1255.30), and its FID/FVD curves stay roughly flat across 40, 80, and 120 frames while those baselines climb. Qualitatively, the authors show rollouts of 1,200 and 1,500 frames in which buildings and vehicles persist and the scene continues to evolve rather than repeating the initial frames. The discovery, as the authors state it, is that the combination of an extended training schedule, clean-memory injection, and exponential loss weighting removes the error-accumulation barrier that previously limited driving world models to short clips.

Load-bearing premise

The paper's two-minute claim rests on extrapolating from quantitative evaluation on sequences of at most 120 frames; the load-bearing assumption is that the drift-control mechanisms that keep FID and FVD flat over those frames will continue to keep a 1,500-frame rollout coherent, since the longer generation is shown only in qualitative stills.

Editorial extensions

If this is right

  • Minutes-long, high-resolution rollouts would let a single conditioned clip generate an entire simulated trip, making world models a practical data engine for perception training rather than a source of short clips.
  • The roughly flat FID/FVD curves over 40–120 frames imply that the drift-control machinery should keep quality approximately constant over much longer rollouts, so simulation episodes can match the length of real drives.
  • Joint text-to-video and image-to-video training on re-captioned driving data means one model can sample diverse weather, layout, and traffic-rule scenarios from text while staying anchored to a given initial image.
  • Training windows of 128 frames let the model learn behaviors that unfold over tens of seconds, such as deceleration at intersections and yielding to turning vehicles, which short-window models cannot capture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own limitation note says the focus is single-view, long-duration, high-fidelity generation; the 1,500-frame result should therefore be read as a single-camera claim, not a multi-view or controllable-simulation claim.
  • The missing experiment is a quantitative FID/FVD measurement at 1,500 frames; the paper reports numbers only through 100 frames in Table 2 and 120 frames in Fig. 3, so running the same protocol over the full rollout would directly test the headline claim.
  • The single decay rate $\lambda$ in $w(T)=e^{-\lambda T}$ is a free parameter; a natural extension is a learned or scenario-dependent rate, since optimal retention may differ between highway cruising and dense intersections.
  • The $t=0$ clean-memory block plus exponential frame-distance weighting is not driving-specific; it could be transferred to other long-horizon video domains such as egocentric navigation or surveillance, with the same goal of keeping memory separate from generation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. InfinityDrive proposes a driving world model that generates 576×1024 videos at 10 Hz and claims consistent generation for over 1500 frames (more than 2 minutes). To achieve this it introduces: (i) a spatio-temporal co-modeling (STCM) module that dynamically varies resolution and temporal density and uses skip-ROPE positional embeddings; (ii) a curriculum that extends the training window to 128 frames; (iii) memory injection and retention mechanisms that feed clean historical frames with timestep t=0; and (iv) a memory curve adaptive loss with exponentially decaying weights w(T)=e^−λT. Experiments report FID/FVD at 25 and 100 frames against prior driving world models, together with ablations, qualitative long-rollout examples, and diversity demonstrations.

Significance. If the 1500-frame claim were quantitatively substantiated, this would be a major step for driving world models: the paper's own comparisons show baselines degrading within 80–120 frames while InfinityDrive preserves visible structure at 1200–1500 frames. The proposed components are plausible, and the ablation study in Table 3 isolates the contributions of ETST, STCM, and MCAL. However, the manuscript provides no quantitative evidence at the claimed horizon, omits its entire experimental setup by deferring to an absent Appendix, and reports no uncertainty or significance measures. The qualitative examples are impressive but do not by themselves support the 'state-of-the-art' and 'first' claims. The work is promising but not yet substantiated.

major comments (4)
  1. [§4.2, Table 2, Fig. 3] The central claim of 'consistent video generation lasting over 1500 frames' is not evaluated quantitatively. Table 2 reports FID/FVD only at 25 and 100 frames, and Fig. 3 plots metrics only up to 120 frames. No metric is reported at 500, 1000, or 1500 frames. Moreover, the text is internally inconsistent: Sec. 4.2 states 'sustaining quality up to 1,200 frames, approximately 2 minutes,' while Fig. 1 shows 1500 frames and Fig. 4's caption says 'up to 1200 frames.' At the stated 10 Hz, 1200 frames is 2 minutes and 1500 frames is 2.5 minutes. Please add quantitative long-horizon evaluations at the claimed frame counts or revise the claims to match the evaluated horizon.
  2. [§4.1 / Appendix] The 'Experimental Setup' section consists of a single sentence referring to an Appendix, but no appendix is present in this version of the manuscript. This omits information needed to reproduce or interpret the reported numbers: dataset splits (including the 'opendv2k' dataset of Fig. 5), the FID/FVD evaluation protocol (number of generated videos, resolution, frame sampling, and reference-video set), the text-captioning pipeline, and all training hyperparameters. Without this section, Tables 2–3 and the comparison claims cannot be verified. Please include the full appendix and state the essential setup in the main text.
  3. [§3.2.2, Eqs. (3) and (6)] The memory curve adaptive loss uses a single exponential decay rate λ in w(T)=e^−λT, and the memory segment length M (Sec. 3.2.1) is chosen by hand. Neither λ nor M is reported, and no sensitivity analysis or ablation over these hyperparameters is given. Because the loss assigns exponentially smaller weight to frames far from the memory block, the training signal is weakest precisely in the regime covered by the 1500-frame claim. This is a load-bearing design choice: please report λ and M, explain how they were selected, and provide a sensitivity study with long-horizon metrics.
  4. [Tables 2–3, Fig. 3] No error bars, confidence intervals, sample counts, or statistical significance tests are reported for any FID/FVD value. Several comparisons are close (for example, adding STCM in Table 3 changes FID by only 0.11, and the short-term FVD gap between Ours and Vista is about 19), so without uncertainty estimates it is impossible to know whether these differences are meaningful. Please report means and standard deviations over multiple evaluation runs or seeds, together with the number of evaluation videos.
minor comments (8)
  1. [Abstract and §1] The phrases 'the first driving world model with exceptional generalization capabilities' and 'state-of-the-art performance' are stronger than the current evidence supports; please qualify these claims to match the quantitative evaluation.
  2. [§2.1] The section heading contains a typo: 'Autonoumous' should be 'Autonomous.'
  3. [§3.1.1] The resolution scaling factor α is said to be 'randomly select[ed]' from [1,4], but the surrounding text implies discrete buckets and α=1 would mean no scaling; please clarify whether α is continuous or discrete and how the buckets are defined.
  4. [§3.2.1, Eqs. (1)–(2)] The symbol t is used both for the diffusion timestep in Eq. (1) and for the sampled timestep U(0,1000) in Eq. (2), which is confusing; please use distinct notation for the frame index and the diffusion timestep.
  5. [Fig. 4 and Fig. 1] The figure captions disagree with the abstract: Fig. 4 states 'up to 1200 frames' while Fig. 1 and the abstract claim 1500 frames; please reconcile these numbers.
  6. [References] Reference [1], listed as 'Anonymous. Imagen video: ... Submitted to TMLR, 2022. Rejected,' is not a standard citation and should be replaced with the proper published or preprint reference.
  7. [§4.5, Diversity] The diversity evaluation is entirely qualitative; a quantitative diversity metric (for example, LPIPS variance or FVD over multiple rollouts from the same condition) would substantially strengthen the claim.
  8. [Limitations and future work] The limitations paragraph does not mention the absence of quantitative long-horizon evaluation, which is the most important limitation of the current version.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the paper's long-video claim is an empirical extrapolation from shorter measured rollouts, not a claim forced by its own equations or by a self-citation chain.

full rationale

The paper contains no derivation that reduces to its inputs. The closest candidate is the memory curve adaptive loss, Eq. (6), which weights the MSE and VB losses by w(T)=e^{-lambda T}. This is a training objective, not a metric: the reported improvements (Table 2, Fig. 3) are FID/FVD values measured on 25-, 100-, and 120-frame rollouts, so they are not implied by the loss formula by construction. lambda and M are hyperparameters, but the paper does not state that they were fitted on the evaluation set or rename a fitted value as a prediction, so no fitted-input/called-prediction circularity can be substantiated. The same-author citation to DriveScape [51] appears only in the comparison table and related work; it is not used as the load-bearing justification for InfinityDrive's architecture or for the long-rollout claim, so it does not constitute self-citation circularity. The main validity concern is non-circular: the headline "over 1500 frames" is supported only by stills (Figs. 1 and 5), while quantitative evidence stops at 100 frames (Table 2) and 120 frames (Fig. 3), and Fig. 4's caption states 1200 frames; additionally, Sec. 4.1 defers all setup details to a missing appendix. These are evidential gaps, not circular reasoning.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard diffusion machinery, on an assumed extrapolation property of rotary embeddings, and on two undisclosed hyperparameters (lambda and M). The memory curve loss is a weighting scheme rather than an additional free entity, and no new physical or conceptual entities are proposed.

free parameters (2)
  • lambda, memory curve decay rate. = Not stated in text.
    Controls the exponential decay w(T)=e^(-lambda*T) in Eq. 3, tuning the balance between memory retention for near-future frames and generative freedom for distant frames. Its value is not disclosed and must be tuned on the validation set.
  • Memory segment length M. = Resolution-dependent, not stated.
    Section 3.2.1 states M depends on the resolution; it determines how many clean frames are injected as memory and therefore directly affects long-term consistency. The specific schedule is not given.
assumptions (3)
  • standard math The standard DDPM forward noising process and the DiT backbone are used as-is.
    Eq. 1 uses the standard diffusion noising schedule with alpha-bar, and the model builds on the DiT architecture from Peebles and Xie [34], so the paper inherits all assumptions of that framework.
  • domain assumption Rotary position embeddings extrapolate to sequence lengths beyond those seen in training.
    Section 3.1.2 states the curriculum starts at 16 frames 'leveraging the extrapolative capacity of ROPE' to later extend to 128 frames; this is an assumed property that is load-bearing for the extended temporal scope training.
  • domain assumption Clean historical frames at t=0 are a sufficient conditioning signal for generating the remaining frames.
    Section 3.2.1 sets the timestep to 0 for memory frames so they carry no noise; the design assumes these exact frames anchor the diffusion process without needing additional conditioning or correction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InfinityDrive: Breaking Time Limits in Driving World Models." pith.science (2026). https://pith.science/paper/KW2X4YQA

@misc{pith2026241201522,
  author       = {Pith},
  title        = {Pith review of: InfinityDrive: Breaking Time Limits in Driving World Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KW2X4YQA}},
  note         = {Machine review of arXiv:2412.01522}
}
abstract

Autonomous driving systems struggle with complex scenarios due to limited access to diverse, extensive, and out-of-distribution driving data which are critical for safe navigation. World models offer a promising solution to this challenge; however, current driving world models are constrained by short time windows and limited scenario diversity. To bridge this gap, we introduce InfinityDrive, the first driving world model with exceptional generalization capabilities, delivering state-of-the-art performance in high fidelity, consistency, and diversity with minute-scale video generation. InfinityDrive introduces an efficient spatio-temporal co-modeling module paired with an extended temporal training strategy, enabling high-resolution (576$\times$1024) video generation with consistent spatial and temporal coherence. By incorporating memory injection and retention mechanisms alongside an adaptive memory curve loss to minimize cumulative errors, achieving consistent video generation lasting over 1500 frames (more than 2 minutes). Comprehensive experiments in multiple datasets validate InfinityDrive's ability to generate complex and varied scenarios, highlighting its potential as a next-generation driving world model built for the evolving demands of autonomous driving. Our project homepage: https://metadrivescape.github.io/papers_project/InfinityDrive/page.html

Figures

Figures reproduced from arXiv: 2412.01522 by the authors.

Figure 1
Figure 1. InfinityDrive can generate long-term driving videos up to 1500 frames. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. InfinityDrive Pipeline: We introduce an efficient spatio-temporal co-modeling module enhanced with memory injection and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Comparison of long-term video generation results un [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: The curves of FID and FVD as world models evolve [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 5
Figure 5. Figure 5: Our long-term generation results on the opendv2k [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Our model’s long-term prediction capacity captures ex [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Our high-resolution (576×1024) video generation re￾sults. 4.4. Consistency Our model achieves coherence in both long-term and short￾term generation. As illustrated by the red boxes in the [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Comparison of diversity in I2V rollouts. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 10
Figure 10. Figure 10: Effectiveness of Spatio-Temporal Co-Modeling [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OpenLongTail: Generative Scaling of Long-Tail Driving Data

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Pose-informed diffusion with Plücker rays, depth warps, and cross-view memory converts monocular long-tail videos into multi-view assets that improve closed-loop driving robustness nearly to ground-truth multi-view levels.

  2. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

  3. 3D and 4D World Modeling: A Survey

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A survey that defines 3D/4D world modeling, organizes methods into VideoGen, OccGen, and LiDARGen categories, and compiles datasets, metrics, and benchmark numbers.

  4. Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model

    cs.RO 2024-12 conditional novelty 5.0 of 10

    A reactive closed-loop driving simulator that uses a diffusion renderer with retrieval from real recordings, plus a nuPlan behavioral controller, to generate sensor images in response to an end-to-end driving model's actions.

  5. A Survey of World Models for Autonomous Driving

    cs.RO 2025-01 conditional novelty 2.0 of 10

    A survey presenting a three-branch taxonomy of world models for autonomous driving, plus benchmark tables comparing representative generation and planning methods on nuScenes, Waymo, Occ3D, and CarlaSC.

Reference graph

Works this paper leans on

65 extracted references · 43 canonical work pages · cited by 5 Pith papers

  1. [1]

    Imagen video: High definition video gener- ation with diffusion models

    Anonymous. Imagen video: High definition video gener- ation with diffusion models. Submitted to Transactions on Machine Learning Research, 2022. Rejected. 3

    work page 2022

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling latent video diffusion models to large datasets, 2023. 1, 5, 6

    work page 2023

  3. [3]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2023. 3

    2023

  4. [4]

    Genie: Gen- erative interactive environments, 2024

    Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker- Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder S...

    work page 2024

  5. [5]

    Videocrafter1: Open diffusion models for high-quality video generation, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter1: Open diffusion models for high-quality video generation, 2023. 3

    2023

  6. [6]

    Livephoto: Real image animation with text-guided motion control, 2023

    Xi Chen, Zhiheng Liu, Mengting Chen, Yutong Feng, Yu Liu, Yujun Shen, and Hengshuang Zhao. Livephoto: Real image animation with text-guided motion control, 2023. 3

    work page 2023

  7. [7]

    SEINE: Short-to-long video diffu- sion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. SEINE: Short-to-long video diffu- sion model for generative transition and prediction. In The Twelfth International Conference on Learning Representa- tions, 2024. 3

    work page 2024

  8. [8]

    Animateanything: Fine- grained open domain image animation with motion guid- ance, 2023

    Zuozhuo Dai, Zhenghao Zhang, Yao Yao, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Animateanything: Fine- grained open domain image animation with motion guid- ance, 2023. 3

    2023

Show all 65 references
  1. [9]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Infor- mation Processing Systems, pages 8780–8794. Curran Asso- ciates, Inc., 2021. 3

    2021

  2. [10]

    Ebbinghaus

    H. Ebbinghaus. ¨Uber Das Ged ¨achtnis: Untersuchungen Zur Experimentellen Psychologie. Creative Media Partners, LLC, 2018. 2, 5

    2018

  3. [11]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7346–7356, 2023. 3

    2023

  4. [12]

    Vista: A generalizable driving world model with high fidelity and versatile controllability, 2024

    Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yi- hang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability, 2024. 1, 3, 5, 6

    2024

  5. [13]

    Emu video: Factoriz- ing text-to-video generation by explicit image conditioning,

    Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Du- val, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra. Emu video: Factoriz- ing text-to-video generation by explicit image conditioning,

  6. [14]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models, 2023. 3

    2023

  7. [15]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. In The Twelfth International Conference on Learning Representa- tio...

    2024

  8. [16]

    Dream to control: Learning behaviors by la- tent imagination, 2020

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Moham- mad Norouzi. Dream to control: Learning behaviors by la- tent imagination, 2020. 2

    2020

  9. [17]

    Latent video diffusion models for high-fidelity long video generation, 2023

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation, 2023. 3

    2023

  10. [18]

    Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text, 2024

    Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tadevosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text, 2024. 1, 3, 6

    2024

  11. [19]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. In Advances in Neural Infor- mation Processing Systems, pages 6840–6851. Curran Asso- ciates, Inc., 2020. 3

    2020

  12. [20]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffu- sion models, 2022. 3

    2022

  13. [21]

    Gaia-1: A generative world model for au- tonomous driving, 2023

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gian- luca Corrado. Gaia-1: A generative world model for au- tonomous driving, 2023. 1, 2, 3

    2023

  14. [22]

    Toward general-purpose robots via founda- tion models: A survey and meta-analysis, 2024

    Yafei Hu, Quanting Xie, Vidhi Jain, Jonathan Francis, Jay Patrikar, Nikhil Keetha, Seungchan Kim, Yaqi Xie, Tianyi Zhang, Hao-Shu Fang, Shibo Zhao, Shayegan Omidshafiei, 9 Dong-Ki Kim, Ali akbar Agha-mohammadi, Katia Sycara, Matthew Johnson-Roberson, Dhruv Batra, Xiaolong Wang...

    2024

  15. [23]

    Adriver-i: A general world model for autonomous driving,

    Fan Jia, Weixin Mao, Yingfei Liu, Yucheng Zhao, Yuqing Wen, Chi Zhang, Xiangyu Zhang, and Tiancai Wang. Adriver-i: A general world model for autonomous driving,

  16. [24]

    Text2video-zero: Text-to- image diffusion models are zero-shot video generators.arXiv preprint arXiv:2303.13439, 2023

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to- image diffusion models are zero-shot video generators.arXiv preprint arXiv:2303.13439, 2023. 3

    2023 arXiv

  17. [25]

    Learning to simulate dynamic envi- ronments with gamegan

    Seung Wook Kim, Yuhao Zhou, Jonah Philion, Antonio Tor- ralba, and Sanja Fidler. Learning to simulate dynamic envi- ronments with gamegan. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2020. 2

    2020

  18. [26]

    Drivegan: Towards a controllable high-quality neural simulation

    Seung Wook Kim, Jonah Philion, Antonio Torralba, and Sanja Fidler. Drivegan: Towards a controllable high-quality neural simulation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 5820–5829, 2021. 2, 3, 6

    2021

  19. [27]

    Drivingdif- fusion: Layout-guided multi-view driving scene video generation with latent diffusion model

    Xiaofan Li, Yifu Zhang, and Xiaoqing Ye. Drivingdif- fusion: Layout-guided multi-view driving scene video generation with latent diffusion model. arXiv preprint arXiv:2310.07771, 2023. 2, 3

    2023 arXiv

  20. [28]

    Arlon: Boosting diffusion transformers with au- toregressive models for long video generation, 2024

    Zongyi Li, Shujie Hu, Shujie Liu, Long Zhou, Jeongsoo Choi, Lingwei Meng, Xun Guo, Jinyu Li, Hefei Ling, and Furu Wei. Arlon: Boosting diffusion transformers with au- toregressive models for long video generation, 2024. 3

    2024

  21. [29]

    Videostudio: Generating consistent-content and multi-scene videos, 2024

    Fuchen Long, Zhaofan Qiu, Ting Yao, and Tao Mei. Videostudio: Generating consistent-content and multi-scene videos, 2024. 3

    2024

  22. [30]

    Wovogen: World volume-aware diffusion for con- trollable multi-camera driving scene generation, 2024

    Jiachen Lu, Ze Huang, Zeyu Yang, Jiahui Zhang, and Li Zhang. Wovogen: World volume-aware diffusion for con- trollable multi-camera driving scene generation, 2024. 1, 2, 3, 6

    2024

  23. [31]

    Free- long: Training-free long video generation with spectralblend temporal attention, 2024

    Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. Free- long: Training-free long video generation with spectralblend temporal attention, 2024. 3

    2024

  24. [32]

    Unleashing generalization of end-to-end autonomous driving with controllable long video generation

    Enhui Ma, Lijun Zhou, Tao Tang, Zhan Zhang, Dong Han, Junpeng Jiang, Kun Zhan, Peng Jia, Xianpeng Lang, Haiyang Sun, et al. Unleashing generalization of end-to-end autonomous driving with controllable long video generation. arXiv preprint arXiv:2406.01349, 2024. 2, 3

    2024 arXiv

  25. [33]

    Mevg: Multi-event video generation with text-to-video models,

    Gyeongrok Oh, Jaehwan Jeong, Sieun Kim, Wonmin Byeon, Jinkyu Kim, Sungwoong Kim, and Sangpil Kim. Mevg: Multi-event video generation with text-to-video models,

  26. [34]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  27. [35]

    Freenoise: Tuning- free longer video diffusion via noise rescheduling

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xin- tao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning- free longer video diffusion via noise rescheduling. In The Twelfth International Conference on Learning Representa- tions, 2024. 3

    2024

  28. [36]

    Consisti2v: Enhancing visual consistency for image-to-video generation, 2024

    Weiming Ren, Huan Yang, Ge Zhang, Cong Wei, Xinrun Du, Wenhao Huang, and Wenhu Chen. Consisti2v: Enhancing visual consistency for image-to-video generation, 2024. 3

    2024

  29. [37]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. 3

    2022

  30. [38]

    Learning a driving simulator,

    Eder Santana and George Hotz. Learning a driving simulator,

  31. [39]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. In The Eleventh International Conference on Lea...

    2023

  32. [40]

    Roformer: Enhanced transformer with rotary position embedding, 2023

    Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, 2023. 4

    2023

  33. [41]

    Richard S. Sutton. The quest for a common model of the intelligent decision maker, 2022. 2

    2022

  34. [42]

    Gen-l-video: Multi-text to long video generation via temporal co-denoising, 2023

    Fu-Yun Wang, Wenshuo Chen, Guanglu Song, Han-Jia Ye, Yu Liu, and Hongsheng Li. Gen-l-video: Multi-text to long video generation via temporal co-denoising, 2023. 3

    2023

  35. [43]

    Modelscope text-to-video technical report, 2023

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report, 2023. 3

    2023

  36. [44]

    Drivedreamer: Towards real-world- driven world models for autonomous driving, 2023

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jia- gang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world- driven world models for autonomous driving, 2023. 1, 2, 3, 6

    2023

  37. [45]

    Videocomposer: compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: compositional video synthesis with motion controllability. In Proceedings of the 37th Inter- national Conference on Neural Information Pro...

    2024

  38. [46]

    Driving into the future: Multiview visual forecasting and planning with world model for au- tonomous driving

    Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for au- tonomous driving. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CV...

    2024

  39. [47]

    Loong: Generating minute-level long videos with autoregressive lan- guage models, 2024

    Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive lan- guage models, 2024. 1, 3

    2024

  40. [48]

    Motionctrl: A unified and flexible motion controller for video generation,

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation,

  41. [49]

    Panacea: Panoramic and controllable video generation for autonomous driving, 2023

    Yuqing Wen, Yucheng Zhao, Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang, Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang. Panacea: Panoramic and controllable video generation for autonomous driving, 2023. 3 10

    2023

  42. [50]

    Art ·v: Auto-regressive text-to-video generation with diffusion models, 2023

    Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jian- min Bao, Yuhui Yuan, Chong Luo, Yueyi Zhang, and Zhiwei Xiong. Art ·v: Auto-regressive text-to-video generation with diffusion models, 2023. 3

    2023

  43. [51]

    Drivescape: To- wards high-resolution controllable multi-view driving video generation, 2024

    Wei Wu, Xi Guo, Weixuan Tang, Tingxuan Huang, Chiyu Wang, Dongyue Chen, and Chenjing Ding. Drivescape: To- wards high-resolution controllable multi-view driving video generation, 2024. 2, 3

    2024

  44. [52]

    Videogpt: Video generation using vq-vae and trans- formers, 2021

    Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and trans- formers, 2021. 3

    2021

  45. [53]

    Genad: Generalized predictive model for autonomous driving, 2024

    Jiazhi Yang, Shenyuan Gao, Yihang Qiu, Li Chen, Tianyu Li, Bo Dai, Kashyap Chitta, Penghao Wu, Jia Zeng, Ping Luo, Jun Zhang, Andreas Geiger, Yu Qiao, and Hongyang Li. Genad: Generalized predictive model for autonomous driving, 2024. 1, 2, 6

    2024

  46. [54]

    Direct-a-video: Customized video generation with user- directed camera movement and object motion

    Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. Direct-a-video: Customized video generation with user- directed camera movement and object motion. In Special Interest Group on Computer Graphics and Interactive Tech- niq...

  47. [55]

    Position: Video as the new language for real-world de- cision making

    Sherry Yang, Jacob C Walker, Jack Parker-Holder, Yilun Du, Jake Bruce, Andre Barreto, Pieter Abbeel, and Dale Schuur- mans. Position: Video as the new language for real-world de- cision making. In Proceedings of the 41st International Con- ference on Machine Learning, pages 56...

  48. [56]

    Nuwa-xl: Diffusion over diffusion for extremely long video generation,

    Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Gong Ming, Lijuan Wang, Zicheng Liu, Houqiang Li, and Nan Duan. Nuwa-xl: Diffusion over diffusion for extremely long video generation,

  49. [57]

    Make pixels dance: High- dynamic video generation, 2023

    Yan Zeng, Guoqiang Wei, Jiani Zheng, Jiaxin Zou, Yang Wei, Yuchen Zhang, and Hang Li. Make pixels dance: High- dynamic video generation, 2023. 3

    2023

  50. [58]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3836–3847, 2023. 3

    2023

  51. [59]

    Copilot4d: Learning unsupervised world models for autonomous driving via discrete diffusion

    Lunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas, Rui Hu, and Raquel Urtasun. Copilot4d: Learning unsupervised world models for autonomous driving via discrete diffusion. In The Twelfth International Conference on Learning Repre- sentations, 2024. 2

    2024

  52. [60]

    Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual input and output, 2024

    Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, J...

    2024

  53. [61]

    I2vgen-xl: High-quality image-to-video synthe- sis via cascaded diffusion models, 2023

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jin- gren Zhou. I2vgen-xl: High-quality image-to-video synthe- sis via cascaded diffusion models, 2023. 3

    2023

  54. [62]

    Moviedreamer: Hierarchical generation for coherent long vi- sual sequence, 2024

    Canyu Zhao, Mingyu Liu, Wen Wang, Weihua Chen, Fan Wang, Hao Chen, Bo Zhang, and Chunhua Shen. Moviedreamer: Hierarchical generation for coherent long vi- sual sequence, 2024. 3

    2024

  55. [63]

    Drivedreamer4d: World models are effective data ma- chines for 4d driving scene representation, 2024

    Guosheng Zhao, Chaojun Ni, Xiaofeng Wang, Zheng Zhu, Xueyang Zhang, Yida Wang, Guan Huang, Xinze Chen, Boyuan Wang, Youyi Zhang, Wenjun Mei, and Xingang Wang. Drivedreamer4d: World models are effective data ma- chines for 4d driving scene representation, 2024. 2

    2024

  56. [64]

    Drivedreamer-2: Llm-enhanced world models for diverse driving video generation, 2024

    Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang. Drivedreamer-2: Llm-enhanced world models for diverse driving video generation, 2024. 1, 2

    2024

  57. [65]

    Embodied understanding of driving scenarios,

    Yunsong Zhou, Linyan Huang, Qingwen Bu, Jia Zeng, Tianyu Li, Hang Qiu, Hongzi Zhu, Minyi Guo, Yu Qiao, and Hongyang Li. Embodied understanding of driving scenarios,

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.