Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that rendering the desired ego trajectory as a visual 3D condition, rather than as a numerical control vector, lets a driving world model follow actions precisely while staying geometrically consistent, cutting…

desk verdict GeoDrive's visual-trajectory conditioning is a genuinely new idea with strong FID/FVD numbers, but its headline ADE/FDE advantage is likely inflated by a circular MonST3R-based metric. read the letter →

arxiv 2505.22421 v2 pith:KFZADPHN submitted 2025-05-28 cs.CV cs.RO

Anthony Chen , Wenzhao Zheng , Yida Wang , Xueyang Zhang , Kun Zhan , Peng Jia , Kurt Keutzer , Shanghang Zhang This is my paper
classification cs.CVcs.RO
keywords drivingworldmodelaction-conditionedvideogeneration3Dgeometryconditioningmetricmonocularreconstructiondiffusiontransformertrajectoryfollowingnovelviewsynthesisautonomoussimulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a driving world model, a generative model that predicts future driving scenes from the current view, can follow a user-specified ego trajectory far more precisely when the trajectory is given as a visually rendered 3D condition rather than as numerical control vectors. The proposed system, GeoDrive, reconstructs a metric-scale 3D point cloud from a single input frame, renders the scene along the desired camera path, and feeds that rendered video into a frozen video diffusion transformer through a small trainable condition encoder. On the nuScenes benchmark, it reports a 42 percent lower average displacement error than the Vista baseline (1.62 versus 2.77, scaled by $10^{-2}$), better FID and FVD scores, and it trains on 5 hours of data versus Vista's 1,740 hours. If correct, this means explicit geometric rendering can substitute for large amounts of action-conditioned training data.

What carries the argument

The load-bearing device is visual trajectory conditioning: a rendered video sequence obtained by projecting a monocular 3D point-cloud reconstruction along the requested camera path, used as a visual prompt instead of a numeric control vector. The pipeline first builds a metric point cloud and camera poses from one frame, then uses projective rendering with z-buffering to produce geometrically consistent views along the trajectory. A dynamic editing module moves vehicles using 2D bounding-box tracks during training so the static reconstruction better matches moving traffic. These rendered videos are encoded and fused into a frozen video diffusion transformer through a condition encoder that is only about 6 percent of backbone parameters, with the fusion weights zero-initialized to avoid corrupting the pretrained model.

What would settle it

Run the trajectory-following evaluation using camera poses and trajectories from an independent source, such as LiDAR-inertial odometry or a different structure-from-motion pipeline, for both the condition rendering and the predicted frames, and compare GeoDrive against Vista on the same clips. If the ADE gap of 1.15 (2.77 versus 1.62) narrows or reverses, the control advantage is partly an artifact of sharing the same monocular estimator in conditioning and evaluation.

Watch

Extended reading notes

Core claim

The central claim is that explicit visual grounding of the action signal, rather than a larger training corpus, is what makes trajectory conditioning reliable. GeoDrive takes the first frame of a driving video, runs an off-the-shelf monocular dense-stereo estimator to obtain a metric-scale point cloud and camera poses, and projectively renders that point cloud along the user-specified ego trajectory to produce a dynamic-aware conditioning video. A lightweight condition encoder injects this rendered video into a frozen latent video diffusion transformer, constraining the generated frames to agree with the rendered geometry while retaining the backbone's photorealism. With this mechanism, the authors report trajectory-following ADE of 1.62 versus Vista's 2.77, FID of 4.1 versus 6.6, and FVD of 61.6 versus 167.7 on nuScenes, plus zero-shot generalization to shifted and reverse trajectories. The same geometry-conditioned pipeline also controls individual vehicles via 2D bounding-box tracks and synthesizes novel views on Waymo without retraining.

Load-bearing premise

Everything rests on the monocular metric-scale reconstruction being accurate enough to serve as geometric ground truth: if the depth or camera-pose estimates are biased, both the rendered condition and the trajectory-error measurement inherit that bias, so the reported action advantage could shrink or disappear when measured against independent ground truth.

Editorial extensions

If this is right

  • Trajectory-following accuracy no longer needs to scale with action-label dataset size: 5 hours of nuScenes data gives an ADE of 1.62 versus Vista's 2.77 on 1,740 hours, implying the rendered condition carries most of the control information.
  • The model can follow trajectories absent from training data, including reverse and laterally shifted paths, because the visual condition itself is geometrically consistent.
  • Conditioning on rendered geometry keeps static infrastructure fixed across generated frames, reducing the structural drift common in 2D-only world models.
  • Dynamic editing with 2D bounding-box tracks gives users control over non-ego vehicles, enabling object insertion, removal, and trajectory control in generated scenes.
  • The model can roll out candidate trajectories for a vision-language-action planner, letting the planner reject unsafe maneuvers before execution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism works as claimed, then improving the monocular metric reconstruction, whether through temporal smoothing, multi-frame input, or LiDAR fusion, should directly improve both trajectory control and evaluation accuracy, meaning the world model's ceiling is set by 3D reconstruction rather than by video generation.
  • A fair comparison that measures both the condition rendering and the predicted trajectories with an independent or ground-truth pose source would isolate whether the 42 percent improvement comes from the rendered geometry or from aligning both condition and evaluation through the same monocular estimator.
  • The rendering-as-condition idea could transfer to other single-image-to-video domains with known camera trajectories, such as robotics manipulation or human motion, though the paper does not test this.
  • Because the frozen backbone is large but only about 6 percent of parameters are trained, the geometry-conditioning pipeline may carry over to higher resolutions or longer horizons by swapping the backbone, an extension the paper does not demonstrate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. GeoDrive is a video-diffusion driving world model that uses the off-the-shelf dense stereo model MonST3R to reconstruct a 3D point cloud from a single initial frame, renders that point cloud along a user-specified ego-vehicle trajectory (with an optional dynamic-object editing module), and feeds the resulting rendered video as a visual condition to a frozen CogVideo diffusion transformer via a small trainable condition encoder. The paper claims that this explicit 3D visual conditioning reduces trajectory-following error by 42% relative to Vista on nuScenes (Table 1: ADE 1.62 vs. 2.77), improves generation quality metrics such as FID and FVD on nuScenes, yields zero-shot novel-view synthesis on Waymo, and supports applications including object editing, object trajectory control, and VLA-based planning.

Significance. If the trajectory-following numbers are trustworthy, the paper makes a significant practical contribution: it demonstrates that rendering the desired ego trajectory as a projective visual condition is a far stronger control signal than feeding raw numerical action vectors, and it does so while training only a 6%-parameter condition encoder on a small fine-tuning set. The dynamic-editing and VLA-planning applications are useful extensions of world-model controllability. However, the evaluation protocol currently couples the geometric estimator used to construct the condition with the estimator used to measure trajectory fidelity, and the quantitative tables report no error bars or significance tests. The headline claims therefore need independent validation before their significance can be fully assessed.

major comments (4)
  1. [Section B.4, Eq. (6); Section 3.1] The ADE/FDE evaluation is potentially circular. MonST3R produces both the point cloud and camera poses that condition GeoDrive (Eqs. (1)-(3)), and Eq. (6) computes ADE/FDE from trajectories "estimated via MonST3R". If MonST3R has any systematic scale or pose bias, a video that faithfully renders the MonST3R point cloud will appear accurately localized to MonST3R even if the true camera motion is wrong, whereas Vista, conditioned on raw control vectors, receives no such benefit. The text is ambiguous about whether y_t is the sensor/calibration pose or another MonST3R estimate. Please clarify the exact provenance of y_t and, more importantly, re-evaluate ADE/FDE for both GeoDrive and baselines using an independent trajectory estimator (e.g., LiDAR/INS odometry or a different monocular pose model), and report sensitivity of the results to MonST3R depth/pose error.
  2. [Tables 1-4] No error bars or significance tests are reported. Table 1's headline 42% ADE reduction is a point estimate on a 1087-video subset, and Table 3's novel-view synthesis rests on only 5 Waymo scenes. The conclusion of "significant" outperformance is not verifiable without measures of variance. Please provide standard errors or confidence intervals over multiple seeds or scene-level bootstraps, and for NVS report per-scene numbers or use a larger sample.
  3. [Table 4] The dual-branch ablation does not clearly support the method. The full model has ADE 1.62 versus 3.45 for w/o dual-branch, but its FVD is 85.22 versus 74.76 for w/o dual-branch; the w/o D.E. variant also differs. With no error bars, the ADE advantage could be noise or reflect a trade-off rather than a uniform improvement. Please report statistical significance and explain the FVD regression, or temper the conclusion that the dual-branch conditioning is uniformly beneficial.
  4. [Section 4.3, Table 3] The text states that GeoDrive "achieves lower (and thus better) FID & FVD scores" than StreetGaussians, but Table 3 contradicts this: on Left@3m, GeoDrive's FID is 67.13, worse than StreetGS's 63.84, although GeoDrive's FVD is better. Please correct the claim and discuss the trade-off, or adjust the evaluation to be consistent with the data.
minor comments (6)
  1. [Tables 1 and 2] The same methods report different metric values in Tables 1 and 2 (e.g., GeoDrive FID 7.17 vs. 4.1; Vista FID 8.35 vs. 6.6). Since Table 1 is on a subset and Table 2 on the full validation set, please clarify the subset composition and metric protocol to avoid confusion.
  2. [Throughout] The phrase "significantly outperforms" is used without statistical significance testing; please either add tests or use non-statistical wording such as "outperforms on the reported metrics".
  3. [Eq. (4)] The notation for the condition-encoder feature fusion, "\(\gamma^\phi_{enc}([z_t, z_R], t)_{i // M}^2\)", is garbled and does not parse; please define the layer indexing and the fusion operation precisely.
  4. [Section 4.2] The paper says the 1087-video subset has "balanced driving trajectories" but does not define "balanced" or report the trajectory distribution; please specify the balancing criterion.
  5. [Figure 10 / Section C.1] The claim of generalization to a reverse trajectory is supported only by a qualitative figure; if quantitative trajectory-error numbers for this setting exist, please include them.
  6. [References] The name "StreetGaussians" is spelled inconsistently as "StreetGaussian" in several places; standardize the spelling.

Circularity Check

1 steps flagged · score 6.0 of 10

ADE/FDE trajectory-following result is partially circular: MonST3R both constructs the conditioning geometry and measures the predicted trajectory, so the 42% improvement is partly self-consistency rather than independent ego-motion accuracy.

  1. self definitional [Sec. 4.2 (Benchmark and Baselines) and Appendix B.4 (Eq. 6)]
    "We estimate our condition camera poses by running MonST3R on GT video. ... the trajectory fidelity is quantified by Average Displacement Error (ADE) and Final Displacement Error (FDE) upon the trajectory pair {yt, ˆyt} estimated via MonST3R: ADE = 1/T Σ ∥yt − ˆyt∥2, FDE = ∥yT − ˆyT∥2, where yt denotes ground truth poses and ˆyt predicted positions."

    GeoDrive's geometric condition is built by MonST3R: Eqs. (1)-(3) produce the point cloud P0 and camera poses from the GT video, and Sec. 3.2 renders P0 along those same poses. At evaluation, Eq. (6) measures ADE/FDE using trajectories 'estimated via MonST3R', where yt is MonST3R's pose estimate on the GT video and ŷt is MonST3R's pose estimate on the generated frames. The same estimator therefore defines both the conditioning signal and the 'ground truth' for the action-fidelity metric. A generated video that faithfully reproduces the MonST3R-conditioned rendering will appear perfectly localized to MonST3R even if the true camera path is wrong, whereas Vista, conditioned on raw sensor/calibration trajectories, receives no such consistency bonus.

full rationale

There is no formal mathematical derivation in the paper, so classical derivation-circularity does not apply. The central circularity is evaluative and specific: the same monocular geometry estimator, MonST3R, is used to create GeoDrive's conditioning render (Eqs. 1-3 and Sec. 3.2) and to compute the ADE/FDE trajectory-fidelity scores (Eq. 6). The condition poses are 'estimated via MonST3R' and the trajectory pair is also 'estimated via MonST3R', so the metric rewards agreement with MonST3R's own reconstruction, which GeoDrive is explicitly trained and conditioned to reproduce. This makes the headline trajectory-following claim (Table 1: ADE 1.62 vs Vista's 2.77) partially circular, with the 42% improvement inflated by estimator self-consistency. Other results are independent: FID/FVD visual-quality numbers, the novel-view-synthesis comparison with StreetGaussian, and the ablations do not depend on the same estimator in their evaluation, giving the paper substantial non-circular content. The Limitations section candidly notes that performance depends on MonST3R's depth/pose accuracy, which is a relevant admission, but the dependence is not quantified or controlled for. Overall, one prediction (action fidelity) reduces by construction, so a score of 6 is appropriate for partial circularity rather than full circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The ledger is small: three hand-set thresholds and four domain assumptions. The method introduces no new physical entities or fitted constants; its central result is an empirical gain in generative fidelity and control, conditional on the accuracy of upstream MonST3R geometry.

free parameters (3)
  • confidence threshold tau = 0.65
    Points with confidence below 0.65 are discarded when forming the point cloud (Eq. 2, Section 3.1). The choice changes geometry and affects subsequent renderings.
  • depth range [0.1, 100.0] m = [0.1, 100.0]
    Only points within this depth range are projected during rendering (Section 3.2); near/far clipping affects occlusions and content.
  • condition encoder capacity (first two DiT layers, 6% parameters) = 2 layers
    The size of the condition encoder relative to the frozen DiT is a design choice that trades control strength against preserving pre-trained photorealism (Section B.1).
assumptions (4)
  • standard math The diffusion forward process and denoising objective (Eq. 5) describe a valid generative model.
    Standard latent diffusion theory inherited from Ho et al. and Rombach et al.; used without proof.
  • domain assumption Additive feature injection from the condition encoder into the frozen DiT (Eq. 4) preserves the pre-trained model's photorealism while steering output.
    The paper relies on this without theoretical justification; the ablation compares dual-branch vs single-branch but no formal guarantee.
  • domain assumption MonST3R's metric-scale 3D reconstructions and camera poses are accurate for urban driving scenes.
    The method takes them as ground truth; the paper states performance 'depends on the accuracy of depth and pose estimation from MonST3R' (Section 5).
  • domain assumption Moving vehicles can be represented by shifting their associated 3D points, and the diffusion model will render plausible motion after editing.
    Dynamic editing (Section 3.2) assumes bounding-box-driven point displacement produces realistic motion; no quantitative verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control." pith.science (2026). https://pith.science/paper/KFZADPHN

@misc{pith2026250522421,
  author       = {Pith},
  title        = {Pith review of: GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KFZADPHN}},
  note         = {Machine review of arXiv:2505.22421}
}
read the original abstract

Recent advancements in world models have revolutionized dynamic environment simulation, allowing systems to foresee future states and assess potential actions. In autonomous driving, these capabilities help vehicles anticipate the behavior of other road users, perform risk-aware planning, accelerate training in simulation, and adapt to novel scenarios, thereby enhancing safety and reliability. Current approaches exhibit deficiencies in maintaining robust 3D geometric consistency or accumulating artifacts during occlusion handling, both critical for reliable safety assessment in autonomous navigation tasks. To address this, we introduce GeoDrive, which explicitly integrates robust 3D geometry conditions into driving world models to enhance spatial understanding and action controllability. Specifically, we first extract a 3D representation from the input frame and then obtain its 2D rendering based on the user-specified ego-car trajectory. To enable dynamic modeling, we propose a dynamic editing module during training to enhance the renderings by editing the positions of the vehicles. Extensive experiments demonstrate that our method significantly outperforms existing models in both action accuracy and 3D spatial awareness, leading to more realistic, adaptable, and reliable scene modeling for safer autonomous driving. Additionally, our model can generalize to novel trajectories and offers interactive scene editing capabilities, such as object editing and object trajectory control.

Figures

Figures reproduced from arXiv: 2505.22421 by the authors.

Figure 1
Figure 1. GeoDrive enables precise trajectory following, correct novel view synthesis, and dynamic scene editing in autonomous driving scenarios. Our method integrates robust 3D conditions into driving world models, enhancing spatial understanding and action controllability. Abstract Recent advancements in world models have revolutionized dynamic environment simulation, allowing systems to foresee future states and assess pot… view at source ↗
Figure 2
Figure 2. Overview of our training pipeline. We use a pretrained dense stereo model to obtain 3D point clouds and camera trajectories. A dynamic video is rendered from the first-frame point cloud using our dynamic editing technique. The noisy latent representation and rendered video are encoded via a VAE and concatenated as input for our condition encoder, modulating the DiT model’s features. The DiT then generates photoreali… view at source ↗
Figure 3
Figure 3. Illustration of dynamic edit design. Compared with default rendering, it effectively reduces disparity between static rendering and dynamic real-world scenarios [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of action fidelity under the same conditional frame and action control. Our model precisely follows desired trajectory, while Vista [13] produce misaligned results. Vista Ours Vista Condition frame Ours Zoom in Prediction Quality World Understand…
Figure 5
Figure 5. Figure 5: Qualitative Comparisons: Left - Enhanced visual fidelity in our predictions; Right - Superior scene dynamics understanding. These renderings are processed through a lightweight condition encoder, which extracts essential background cues without duplicating extensive po…
Figure 6
Figure 6. Figure 6: Qualitative comparison on novel-view synthesis on Waymo validation subset. Our model generates sharp results for deviated trajectories in a zero-shot manner, whereas the reconstruction-based method StreetGaussian [73] produces significant artifacts. By limiting trainin…
Figure 7
Figure 7. Figure 7: Qualitative Results on Vehicle Manipulation. Our approach allows for the manipulation of vehicle movement directions within a scene by specifying different bounding boxes [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Qualitative Results on Scene Editing. Our approach enables the removal or replacement of vehicles within a scene, allowing for the prediction of seamless future scenarios. <Thinking…> Here are three possible action you could take, let me verify if they are paussible Pl…
Figure 9
Figure 9. Figure 9: Illustration of application to VLA planning. By simulating each possible planned trajectory, we can assist the VLA model in refining its decisions until it reaches the optimal decision. Quantitative Results. As demonstrated in [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Our model can faithfully follow given trajectory and predict consistent future, even when [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A five-aspect, 24-metric benchmark, a 26K human-annotated dataset, and an AI evaluator show that today's driving world models cannot simultaneously look real, respect geometry, and behave safely.

  2. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

  3. 3D and 4D World Modeling: A Survey

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A survey that defines 3D/4D world modeling, organizes methods into VideoGen, OccGen, and LiDARGen categories, and compiles datasets, metrics, and benchmark numbers.

Reference graph

Works this paper leans on

91 extracted references · 30 canonical work pages · cited by 3 Pith papers

  1. [1]

    Act-bench: Towards action controllable world models for autonomous driving, 2024

    Hidehisa Arai, Keishi Ishihara, Tsubasa Takahashi, and Yu Yamaguchi. Act-bench: Towards action controllable world models for autonomous driving, 2024

    2024

  2. [2]

    Videopainter: Any-length video inpainting and editing with plug-and-play context control, 2025

    Yuxuan Bian, Zhaoyang Zhang, Xuan Ju, Mingdeng Cao, Liangbin Xie, Ying Shan, and Qiang Xu. Videopainter: Any-length video inpainting and editing with plug-and-play context control, 2025

    2025

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

  4. [4]

    Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2023

    2023

  5. [5]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. CVPR, 2023

    2023

  6. [6]

    Muvo: A multimodal generative world model for autonomous driving with geometric representations

    Daniel Bogdoll, Yitian Yang, and J Marius Zöllner. Muvo: A multimodal generative world model for autonomous driving with geometric representations. arXiv preprint arXiv:2311.11762, 2023

    arXiv 2023

  7. [7]

    Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yuxin Pan, Giancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yuxin Pan, Giancarlo Baldan, and Oscar Beijbom. nuScenes: A Multimodal Dataset for Autonomous Driving. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020

    2020

  8. [8]

    Training-free regional prompting for diffusion transformers, 2024

    Anthony Chen, Jianjin Xu, Wenzhao Zheng, Gaole Dai, Yida Wang, Renrui Zhang, Haofan Wang, and Shanghang Zhang. Training-free regional prompting for diffusion transformers, 2024

    2024

Show all 91 references
  1. [9]

    Eva: An embodied world model for future video anticipation, 2024

    Xiaowei Chi, Hengyuan Zhang, Chun-Kai Fan, Xingqun Qi, Rongyu Zhang, Anthony Chen, Chi min Chan, Wei Xue, Wenhan Luo, Shanghang Zhang, and Yike Guo. Eva: An embodied world model for future video anticipation, 2024

    2024

  2. [10]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proc. IEEE Int. Conf. Comput. Vis., 2023. 10

    2023

  3. [11]

    Pointrnn: Point recurrent neural network for moving point cloud processing

    Hehe Fan and Yi Yang. Pointrnn: Point recurrent neural network for moving point cloud processing. arXiv preprint arXiv:1910.08287, 2019

    1910 arXiv

  4. [12]

    Cardreamer: Open-source learning platform for world model based autonomous driving

    Dechen Gao, Shuangyu Cai, Hanchu Zhou, Hang Wang, Iman Soltani, and Junshan Zhang. Cardreamer: Open-source learning platform for world model based autonomous driving. arXiv preprint arXiv:2405.09111, 2024

    2024 arXiv

  5. [13]

    Vista: A generalizable driving world model with high fidelity and versatile controllability

    Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. Proc. Adv. Neural Inf. Process. Syst., 2024

    2024

  6. [14]

    Imagine-2-drive: High-fidelity world modeling in carla for au- tonomous vehicles

    Anant Garg and K Madhava Krishna. Imagine-2-drive: High-fidelity world modeling in carla for au- tonomous vehicles. arXiv preprint arXiv:2411.10171, 2024

    2024 arXiv

  7. [15]

    Dome: Taming diffusion model into high-fidelity controllable occupancy world model

    Songen Gu, Wei Yin, Bu Jin, Xiaoyang Guo, Junming Wang, Haodong Li, Qian Zhang, and Xiaoxiao Long. Dome: Taming diffusion model into high-fidelity controllable occupancy world model. arXiv preprint arXiv:2410.10429, 2024

    2024 arXiv

  8. [16]

    AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning. InProc. Int. Conf. Learn. Represent., 2024

    2024

  9. [17]

    Vdt-auto: End-to-end autonomous driving with vlm-guided diffusion transformers, 2025

    Ziang Guo, Konstantin Gubernatorov, Selamawit Asfaw, Zakhar Yagudin, and Dzmitry Tsetserukou. Vdt-auto: End-to-end autonomous driving with vlm-guided diffusion transformers, 2025

    2025

  10. [18]

    Gem: A generalizable ego-vision multimodal world model for fine-grained ego-motion, object dynamics, and scene composition control

    Mariam Hassan, Sebastian Stapf, Ahmad Rahimi, Pedro Rezende, Yasaman Haghighi, David Brüggemann, Isinsu Katircioglu, Lin Zhang, Xiaoran Chen, Suman Saha, et al. Gem: A generalizable ego-vision multimodal world model for fine-grained ego-motion, object dynamics, and scene compo...

    2024 arXiv

  11. [19]

    Cameractrl: Enabling camera control for text-to-video generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101, 2024

    2024 arXiv

  12. [20]

    GANs trained by a two time-scale update rule converge to a local Nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  13. [21]

    Video Diffusion Models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video Diffusion Models. arXiv preprint arXiv:2204.03458, 2022

    2022 arXiv

  14. [22]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022

    2022 arXiv

  15. [24]

    GAIA-1: A Generative World Model for Autonomous Driving

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. GAIA-1: A Generative World Model for Autonomous Driving. arXiv preprint arXiv:2309.17080, 2023

    2023 arXiv

  16. [25]

    Drivingworld: Constructingworld model for autonomous driving via video gpt

    Xiaotao Hu, Wei Yin, Mingkai Jia, Junyuan Deng, Xiaoyang Guo, Qian Zhang, Xiaoxiao Long, and Ping Tan. Drivingworld: Constructingworld model for autonomous driving via video gpt. arXiv preprint arXiv:2412.19505, 2024

    2024 arXiv

  17. [26]

    Vad: Vectorized scene representation for autonomous driving

    Bencheng Huang, Shaoyu Liu, Tianheng Chen, Xinggang Shen, Zeming Zhu, Zhe Wang, et al. Vad: Vectorized scene representation for autonomous driving. arXiv preprint arXiv:2303.12077, 2023

    2023 arXiv

  18. [27]

    Neural lidar fields for novel view synthesis

    Shengyu Huang, Zan Gojcic, Zian Wang, Francis Williams, Yoni Kasten, Sanja Fidler, Konrad Schindler, and Or Litany. Neural lidar fields for novel view synthesis. In Proc. IEEE Int. Conf. Comput. Vis., pages 18236–18246, 2023

    2023

  19. [28]

    Neural volumetric world models for autonomous driving

    Zanming Huang, Jimuyang Zhang, and Eshed Ohn-Bar. Neural volumetric world models for autonomous driving. In Proc. Eur . Conf. Comput. Vis., pages 195–213. Springer, 2025

    2025

  20. [29]

    Perceptual losses for real-time style transfer and super-resolution

    Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Proc. Eur . Conf. Comput. Vis., 2016. 11

    2016

  21. [30]

    Differentiable raycasting for self-supervised occupancy forecasting

    Tarasha Khurana, Peiyun Hu, Achal Dave, Jason Ziglar, David Held, and Deva Ramanan. Differentiable raycasting for self-supervised occupancy forecasting. In Proc. Eur . Conf. Comput. Vis., pages 353–369. Springer, 2022

    2022

  22. [31]

    DriveGAN: Towards a Controllable High-Quality Neural Simulation

    Seung Wook Kim, Jonah Philion, Antonio Torralba, and Sanja Fidler. DriveGAN: Towards a Controllable High-Quality Neural Simulation. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2021

    2021

  23. [32]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  24. [33]

    Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2024

    2024

  25. [34]

    Drivingdiffusion: Layout-guided multi-view driving scene video generation with latent diffusion model

    Xiaofan Li, Yifu Zhang, and Xiaoqing Ye. Drivingdiffusion: Layout-guided multi-view driving scene video generation with latent diffusion model. arXiv preprint arXiv:2310.07771, 2023

    2023 arXiv

  26. [35]

    Enhancing end-to-end autonomous driving with latent world model

    Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan. Enhancing end-to-end autonomous driving with latent world model. arXiv preprint arXiv:2406.08481, 2024

    2024 arXiv

  27. [36]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2023

    2023

  28. [37]

    Lidar-based 4d occupancy completion and forecasting

    Xinhao Liu, Moonjun Gong, Qi Fang, Haoyu Xie, Yiming Li, Hang Zhao, and Chen Feng. Lidar-based 4d occupancy completion and forecasting. In Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst. , pages 11102–11109. IEEE, 2024

    2024

  29. [38]

    Monet: Motion-based point cloud prediction network

    Fan Lu, Guang Chen, Zhijun Li, Lijun Zhang, Yinlong Liu, Sanqing Qu, and Alois Knoll. Monet: Motion-based point cloud prediction network. IEEE Trans. Intell. Transp. Syst., 23(8):13794–13804, 2021

    2021

  30. [39]

    Wovogen: World volume-aware diffusion for controllable multi-camera driving scene generation

    Jiachen Lu, Ze Huang, Zeyu Yang, Jiahui Zhang, and Li Zhang. Wovogen: World volume-aware diffusion for controllable multi-camera driving scene generation. In Proc. Eur . Conf. Comput. Vis., pages 329–345. Springer, 2025

    2025

  31. [40]

    Unleashing generalization of end-to-end autonomous driving with controllable long video generation

    Enhui Ma, Lijun Zhou, Tao Tang, Zhan Zhang, Dong Han, Junpeng Jiang, Kun Zhan, Peng Jia, Xianpeng Lang, Haiyang Sun, et al. Unleashing generalization of end-to-end autonomous driving with controllable long video generation. arXiv preprint arXiv:2406.01349, 2024

    2024 arXiv

  32. [41]

    Cam4docc: Benchmark for camera-only 4d occupancy forecasting in autonomous driving applications

    Junyi Ma, Xieyuanli Chen, Jiawei Huang, Jingyi Xu, Zhen Luo, Jintao Xu, Weihao Gu, Rui Ai, and Hesheng Wang. Cam4docc: Benchmark for camera-only 4d occupancy forecasting in autonomous driving applications. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 21486–21495, 2024

    2024

  33. [42]

    Dreamdrive: Generative 4d scene modeling from street view images

    Jiageng Mao, Boyi Li, Boris Ivanovic, Yuxiao Chen, Yan Wang, Yurong You, Chaowei Xiao, Danfei Xu, Marco Pavone, and Yue Wang. Dreamdrive: Generative 4d scene modeling from street view images. arXiv preprint arXiv:2501.00601, 2024

    2024 arXiv

  34. [43]

    Latent-nerf for shape- guided generation of 3d shapes and textures

    Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape- guided generation of 3d shapes and textures. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2023

    2023

  35. [44]

    Driveworld: 4d pre-trained scene understanding via world models for autonomous driving

    Chen Min, Dawei Zhao, Liang Xiao, Jian Zhao, Xinli Xu, Zheng Zhu, Lei Jin, Jianshu Li, Yulan Guo, Junliang Xing, et al. Driveworld: 4d pre-trained scene understanding via world models for autonomous driving. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 15522–15533, 2024

    2024

  36. [45]

    T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In AAAI Conf. Artif. Intell., 2024

    2024

  37. [46]

    Multidiff: Consistent novel view synthesis from a single image

    Norman Müller, Katja Schwarz, Barbara Rössle, Lorenzo Porzi, Samuel Rota Bulò, Matthias Nießner, and Peter Kontschieder. Multidiff: Consistent novel view synthesis from a single image. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2024

    2024

  38. [47]

    Recondreamer: Crafting world models for driving scene reconstruction via online restoration

    Chaojun Ni, Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Wenkang Qin, Guan Huang, Chen Liu, Yuyin Chen, Yida Wang, Xueyang Zhang, et al. Recondreamer: Crafting world models for driving scene reconstruction via online restoration. arXiv preprint arXiv:2411.19548, 2024

    2024 arXiv

  39. [48]

    Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model

    Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang, Ying Shan, and Yinqiang Zheng. Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model. arXiv preprint arXiv:2405.20222, 2024. 12

    2024 arXiv

  40. [49]

    Vase: Object-centric appearance and shape manipulation of real videos

    Elia Peruzzo, Vidit Goel, Dejia Xu, Xingqian Xu, Yifan Jiang, Zhangyang Wang, Humphrey Shi, and Nicu Sebe. Vase: Object-centric appearance and shape manipulation of real videos. arXiv preprint arXiv:2401.02473, 2024

    2024 arXiv

  41. [50]

    Mitigating covariate shift in imitation learning for autonomous vehicles using latent space generative world models

    Alexander Popov, Alperen Degirmenci, David Wehr, Shashank Hegde, Ryan Oldja, Alexey Kamenev, Bertrand Douillard, David Nistér, Urs Muller, Ruchi Bhargava, et al. Mitigating covariate shift in imitation learning for autonomous vehicles using latent space generative world models...

    2024 arXiv

  42. [51]

    Dust3r: Geometric 3d vision made easy

    Jérôme Revaud, Vincent Leroy, Philippe Weinzaepfel, Boris Chidlovskii, and Gabriela Csurka. Dust3r: Geometric 3d vision made easy. arXiv preprint arXiv:2312.14132, 2023

    2023 arXiv

  43. [52]

    Gaia-2: A controllable multi-view generative world model for autonomous driving, 2025

    Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving, 2025

    2025

  44. [53]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Proc. Int. Conf. Learn. Represent., 2021

    2021

  45. [54]

    Probing multimodal llms as world models for driving

    Shiva Sreeram, Tsun-Hsuan Wang, Alaa Maalouf, Guy Rosman, Sertac Karaman, and Daniela Rus. Probing multimodal llms as world models for driving. arXiv preprint arXiv:2405.05956, 2024

    2024 arXiv

  46. [55]

    Street-view image generation from a bird’s-eye view layout

    Alexander Swerdlow, Runsheng Xu, and Bolei Zhou. Street-view image generation from a bird’s-eye view layout. IEEE Robot. Autom. Lett. , 2024

    2024

  47. [56]

    Drivevlm: The convergence of autonomous driving and large vision-language models, 2024

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models, 2024

    2024

  48. [57]

    Towards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. 2018

    2018

  49. [58]

    ModelScope Text-to-Video Technical Report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. ModelScope Text-to-Video Technical Report. arXiv preprint arXiv:2308.06571, 2023

    2023 arXiv

  50. [59]

    Occ- sora: 4d occupancy generation models as world simulators for autonomous driving

    Lening Wang, Wenzhao Zheng, Yilong Ren, Han Jiang, Zhiyong Cui, Haiyang Yu, and Jiwen Lu. Occ- sora: 4d occupancy generation models as world simulators for autonomous driving. arXiv preprint arXiv:2405.20337, 2024

    2024 arXiv

  51. [60]

    Drivedreamer: Towards real-world-driven world models for autonomous driving

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world-driven world models for autonomous driving. Proc. Eur . Conf. Comput. Vis., 2024

    2024

  52. [61]

    Worlddreamer: Towards general world models for video generation via predicting masked tokens

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Boyuan Wang, Xinze Chen, and Jiwen Lu. Worlddreamer: Towards general world models for video generation via predicting masked tokens. arXiv preprint arXiv:2401.09985, 2024

    2024 arXiv

  53. [62]

    Drivingdojo dataset: Advancing interactive and knowledge-enriched driving world model

    Yuqi Wang, Ke Cheng, Jiawei He, Qitai Wang, Hengchen Dai, Yuntao Chen, Fei Xia, and Zhaoxiang Zhang. Drivingdojo dataset: Advancing interactive and knowledge-enriched driving world model. arXiv preprint arXiv:2410.10738, 2024

    arXiv

  54. [63]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004

    2004

  55. [64]

    Mo- tionCtrl: A Unified and Flexible Motion Controller for Video Generation.arXiv preprint arXiv:2312.03641, 2023

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Mo- tionCtrl: A Unified and Flexible Motion Controller for Video Generation.arXiv preprint arXiv:2312.03641, 2023

    2023 arXiv

  56. [65]

    Oc- cllama: An occupancy-language-action generative world model for autonomous driving

    Julong Wei, Shanshuai Yuan, Pengfei Li, Qingda Hu, Zhongxue Gan, and Wenchao Ding. Oc- cllama: An occupancy-language-action generative world model for autonomous driving. arXiv preprint arXiv:2409.03272, 2024

    2024 arXiv

  57. [66]

    Fast point cloud generation with straight flows

    Lemeng Wu, Dilin Wang, Chengyue Gong, Xingchao Liu, Yunyang Xiong, Rakesh Ranjan, Raghuraman Krishnamoorthi, Vikas Chandra, and Qiang Liu. Fast point cloud generation with straight flows. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 9445–9454, 2023

    2023

  58. [67]

    Holodrive: Holistic 2d-3d multi-modal street scene generation for autonomous driving

    Zehuan Wu, Jingcheng Ni, Xiaodong Wang, Yuxin Guo, Rui Chen, Lewei Lu, Jifeng Dai, and Yuwen Xiong. Holodrive: Holistic 2d-3d multi-modal street scene generation for autonomous driving. arXiv preprint arXiv:2412.01407, 2024. 13

    2024 arXiv

  59. [68]

    Tooncrafter: Generative cartoon interpolation

    Jinbo Xing, Hanyuan Liu, Menghan Xia, Yong Zhang, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Tooncrafter: Generative cartoon interpolation. arXiv preprint arXiv:2405.17933, 2024

    2024 arXiv

  60. [69]

    Make-your-video: Customized video generation using textual and structural guidance

    Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, et al. Make-your-video: Customized video generation using textual and structural guidance. IEEE Trans. Vis. Comput. Graph., 2024

    2024

  61. [70]

    Dy- namicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Xintao Wang, Tien-Tsin Wong, and Ying Shan. Dy- namicrafter: Animating open-domain images with video diffusion priors. arXiv preprint arXiv:2310.12190, 2023

    2023 arXiv

  62. [71]

    Camco: Camera-controllable 3d-consistent image-to-video generation

    Dejia Xu, Weili Nie, Chao Liu, Sifei Liu, Jan Kautz, Zhangyang Wang, and Arash Vahdat. Camco: Camera-controllable 3d-consistent image-to-video generation. arXiv preprint arXiv:2406.02509, 2024

    2024 arXiv

  63. [72]

    Drivingsphere: Building a high-fidelity 4d world for closed-loop simulation

    Tianyi Yan, Dongming Wu, Wencheng Han, Junpeng Jiang, Xia Zhou, Kun Zhan, Cheng-zhong Xu, and Jianbing Shen. Drivingsphere: Building a high-fidelity 4d world for closed-loop simulation. arXiv preprint arXiv:2411.11252, 2024

    2024 arXiv

  64. [73]

    Street gaussians: Modeling dynamic urban scenes with gaussian splatting

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians: Modeling dynamic urban scenes with gaussian splatting. In ECCV, 2024

    2024

  65. [74]

    Renderworld: World model with self-supervised 3d label.arXiv preprint arXiv:2409.11356, 2024

    Ziyang Yan, Wenzhen Dong, Yihua Shao, Yuhang Lu, Liu Haiyang, Jingwen Liu, Haozhe Wang, Zhe Wang, Yan Wang, Fabio Remondino, et al. Renderworld: World model with self-supervised 3d label.arXiv preprint arXiv:2409.11356, 2024

    2024 arXiv

  66. [75]

    Generalized Predictive Model for Autonomous Driving

    Jiazhi Yang, Shenyuan Gao, Yihang Qiu, Li Chen, Tianyu Li, Bo Dai, Kashyap Chitta, Penghao Wu, Jia Zeng, Ping Luo, Jun Zhang, Andreas Geiger, Yu Qiao, and Hongyang Li. Generalized Predictive Model for Autonomous Driving. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2024

    2024

  67. [76]

    Drivearena: A closed-loop generative simulation platform for autonomous driving

    Xuemeng Yang, Licheng Wen, Yukai Ma, Jianbiao Mei, Xin Li, Tiantian Wei, Wenjie Lei, Daocheng Fu, Pinlong Cai, Min Dou, et al. Drivearena: A closed-loop generative simulation platform for autonomous driving. arXiv preprint arXiv:2408.00415, 2024

    2024 arXiv

  68. [77]

    Visual point cloud forecasting enables scalable autonomous driving

    Zetong Yang, Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scalable autonomous driving. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 14673–14684, 2024

    2024

  69. [78]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  70. [79]

    Drag- nuwa: Fine-grained control in video generation by integrating text, image, and trajectory

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Drag- nuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089, 2023

    2023 arXiv

  71. [80]

    Nerf-lidar: Generating realistic lidar point clouds with neural radiance fields

    Junge Zhang, Feihu Zhang, Shaochen Kuang, and Li Zhang. Nerf-lidar: Generating realistic lidar point clouds with neural radiance fields. In AAAI Conf. Artif. Intell., volume 38, pages 7178–7186, 2024

    2024

  72. [81]

    Monst3r: A simple approach for estimating geometry in the presence of motion

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arxiv:2410.03825, 2024

    arXiv

  73. [82]

    Copilot4d: Learning unsupervised world models for autonomous driving via discrete diffusion

    Lunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas, Rui Hu, and Raquel Urtasun. Copilot4d: Learning unsupervised world models for autonomous driving via discrete diffusion. In Proc. Int. Conf. Learn. Represent., 2024

    2024

  74. [83]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proc. IEEE Int. Conf. Comput. Vis., 2023

    2023

  75. [84]

    Bevworld: A multimodal world model for autonomous driving via unified bev latent space

    Yumeng Zhang, Shi Gong, Kaixin Xiong, Xiaoqing Ye, Xiao Tan, Fan Wang, Jizhou Huang, Hua Wu, and Haifeng Wang. Bevworld: A multimodal world model for autonomous driving via unified bev latent space. arXiv preprint arXiv:2407.05679, 2024

    2024 arXiv

  76. [85]

    Drivedreamer4d: World models are effective data machines for 4d driving scene representation

    Guosheng Zhao, Chaojun Ni, Xiaofeng Wang, Zheng Zhu, Xueyang Zhang, Yida Wang, Guan Huang, Xinze Chen, Boyuan Wang, Youyi Zhang, et al. Drivedreamer4d: World models are effective data machines for 4d driving scene representation. arXiv preprint arXiv:2410.13571, 2024

    arXiv

  77. [86]

    DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation

    Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang. DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation. arXiv preprint arXiv:2403.06845, 2024. 14

    2024 arXiv

  78. [87]

    Sce2drivex: A generalized mllm framework for scene-to-drive learning, 2025

    Rui Zhao, Qirui Yuan, Jinyu Li, Haofeng Hu, Yun Li, Chengyuan Zheng, and Fei Gao. Sce2drivex: A generalized mllm framework for scene-to-drive learning, 2025

    2025

  79. [88]

    Occworld: Learning a 3d occupancy world model for autonomous driving

    Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. In Proc. Eur . Conf. Comput. Vis., pages 55–72. Springer, 2025

    2025

  80. [89]

    Doe-1: Closed-loop autonomous driving with large world model

    Wenzhao Zheng, Zetian Xia, Yuanhui Huang, Sicheng Zuo, Jie Zhou, and Jiwen Lu. Doe-1: Closed-loop autonomous driving with large world model. arXiv preprint arXiv: 2412.09627 , 2024

    2024 arXiv

  81. [90]

    Xingcheng Zhou, Xuyuan Han, Feng Yang, Yunpu Ma, and Alois C. Knoll. Opendrivevla: Towards end-to-end autonomous driving with large vision language action model, 2025

    2025

  82. [91]

    Simgen: Simulator-conditioned driving scene generation

    Yunsong Zhou, Michael Simon, Zhenghao Peng, Sicheng Mo, Hongzi Zhu, Minyi Guo, and Bolei Zhou. Simgen: Simulator-conditioned driving scene generation. arXiv preprint arXiv:2406.09386, 2024

    2024 arXiv

  83. [92]

    Gaussianworld: Gaussian world model for streaming 3d occupancy prediction

    Sicheng Zuo, Wenzhao Zheng, Yuanhui Huang, Jie Zhou, and Jiwen Lu. Gaussianworld: Gaussian world model for streaming 3d occupancy prediction. arXiv preprint arXiv:2412.10373, 2024. 15 Appendix A Preliminary Details A.1 Video Diffusion Models A diffusion model [53] is built fro...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.