Pith. sign in

REVIEW 5 major objections 6 minor 30 references

DriveCamSim: Generalizable Camera Simulation via Explicit Camera Modeling for Autonomous Driving

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that replacing implicit 2D-space attention with explicit 3D pixel correspondence makes a diffusion-based driving simulator generalize to arbitrary camera configurations and frame rates.

desk verdict Genuinely new camera-modeling mechanism with solid controlled results, but the headline generalization claim needs a baseline-comparison experiment before it's established. read the letter →

arxiv 2505.19692 v1 pith:ILYOEG7A submitted 2025-05-26 cs.CV

Wenchao Sun , Xuewu Lin , Keyu Chen , Zixiang Pei , Yining Shi , Chuang Zhang , Sifa Zheng This is my paper · ORCID
classification cs.CV
keywords autonomousdrivingcamerasimulationdiffusionmodelsexplicitmodelingmulti-viewvideogenerationtemporalgeneralizationcontrollabledepthanchors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes DriveCamSim, a camera-simulation framework for autonomous driving that generates multi-view video from 3D scene conditions. Its central claim is that a generative model should not learn cross-view and cross-frame relationships in 2D image space, because that overfits to the exact camera rig and frame rate used during training. Instead, DriveCamSim projects each query pixel into 3D along several depth anchors and re-projects the resulting points into other views and frames, building explicit pixel-wise correspondences. The authors argue this decouples the model from specific intrinsic and extrinsic parameters, number of views, and temporal sampling rates, and they report that it improves visual quality, controllability, and generalization to novel camera poses and video frequencies. If true, this makes camera simulation usable for testing driving systems under varied sensor configurations.

What carries the argument

The central object is the depth-anchor pixel correspondence inside Explicit Camera Modeling (ECM): $p_q$ in the query view is unprojected to $D=10$ points $\{P_{qi}\}$ at linearly spaced depths in $[1,60]$ meters, then re-projected into each target view as $\{p_{ki}\}$, and the query feature is updated by a softmax-weighted sum of the target features. This machinery converts multi-view and multi-frame interaction into 3D geometric correspondence, so the model is no longer tied to a fixed camera rig or a fixed time step. Two supporting mechanisms carry the argument further: overlap-based target view matching selects target views by the fraction of projected points that land inside the view, and random frame sampling during training breaks chronological order so the model must learn geometric rather than copy adjacent frames. The condition side uses a scatter-based injection that projects keypoints around each 3D box onto the image latent and adds the box embedding at those locations, preserving depth and relative-pose information that the paper says perspective-based and attention-based control lose.

What would settle it

Generate a scene with a foreground vehicle placed at 70 meters ahead (outside the 1–60 m anchor range) and compare the generated cross-view images to the same scene with the vehicle at 30 meters; if the depth-anchor approximation is the load-bearing mechanism, the far vehicle should show visibly worse cross-view alignment or ghosting, whereas a model with true per-pixel depth would degrade gracefully.

Watch

Extended reading notes

Core claim

DriveCamSim's core discovery is that replacing implicit attention in 2D image space with explicit 3D back-projection makes a diffusion-based multi-view video generator generalize across cameras and frame rates. For each pixel in a query view, the model samples 10 depth anchors linearly spaced in [1,60] meters, back-projects the pixel to 3D points, projects those points into target views via their camera matrices, and aggregates target features with attention weights predicted from the query feature. This ECM mechanism is used for cross-view, reference, and temporal attention, and it is paired with overlap-based target view matching and random frame sampling so the model learns geometric transformation rather than copying adjacent frames. The paper further claims that a scatter-based condition injection, which preserves 3D information through encoding and injection and can incorporate identity-aware appearance features, improves controllability beyond existing perspective-based and attention-based control. On nuScenes, DriveCamSim reports lower FID than prior baselines and better downstream perception and planning scores, and it demonstrates generation with rotated or translated cameras, inserted virtual cameras, higher frame rates, and reverse temporal order.

Load-bearing premise

The model assumes that every pixel's true 3D location can be approximated by a weighted combination of 10 fixed depth planes spanning 1 to 60 meters; objects or geometry outside that range, or lying between planes, are correlated to the wrong target pixels.

Editorial extensions

If this is right

  • Trained only on 2Hz nuScenes data, the model can generate video at higher frame rates and even in reverse temporal order, so simulation frequency is no longer locked to training data.
  • Virtual cameras can be inserted or existing cameras can be rotated or translated at inference time, enabling evaluation of driving systems under sensor configurations never seen in training.
  • Downstream perception and planning metrics on generated images improve over prior generative simulators, which the paper attributes to geometrically consistent cross-view and cross-frame interaction.
  • The identity-aware control extension, which injects appearance features sampled from reference frames, improves foreground tracking consistency in crowded scenes.
  • Ablations removing overlap-based view matching or random frame sampling degrade downstream metrics, indicating both mechanisms are needed for the reported gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is closed-loop evaluation: whether the generalization to arbitrary cameras and frame rates persists when the simulator is driven reactively by a policy that changes the ego trajectory, rather than replaying logged scenes.
  • Because ECM relies only on depth anchors rather than learned geometry, the same mechanism might transfer to other sensor modalities, such as radar or lidar projections, or to non-driving scenes with known camera rigs.
  • The authors' reported failure under large translations and rotations along x and z suggests the depth-anchor prior (fixed [1,60] m range, linear spacing) is a likely bottleneck; increasing the number of anchors or making them depth-aware could be a direct extension.
  • The information-preserving control pipeline may also improve text-to-3D or layout-to-image generation outside driving, since it is agnostic to the specific scene representation and preserves 3D structure through conditioning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes DriveCamSim, a multi-view driving video generation framework built on a pretrained latent diffusion model. The central novelty is Explicit Camera Modeling (ECM), which replaces vanilla implicit attention with pixel-wise correspondences established by back-projecting query pixels onto fixed depth anchors and projecting them into target views via camera matrices, together with overlap-based target view matching and random frame sampling. A second contribution is an information-preserving conditional injection mechanism that encodes 3D boxes and maps directly, with an identity-aware extension using reference appearance features. Experiments on nuScenes report improved FID and downstream perception metrics (BEVFusion, StreamPETR, UniAD) over MagicDrive, Panacea, Panacea+, and Bench2Drive-R, plus ablations for ECM, target matching, sampling, and control. Generalization to camera perturbations and frame-rate changes is claimed on the basis of a perturbed-dataset study (Table 6) and qualitative examples.

Significance. If the generalization claim holds, the work would be a useful step: it replaces 2D-image-space attention with geometrically grounded cross-view/cross-frame attention, and it diagnoses a real information-loss issue in condition encoding and injection. The main quantitative results are favorable, and the ablations are internally consistent. The paper also evaluates with multiple independent perception heads (BEVFusion, StreamPETR, UniAD), which grounds the realism and controllability claims. However, the load-bearing generalization claim is not yet supported quantitatively against implicit-camera baselines, and the temporal part remains unquantified; these gaps must be closed before the paper's central novelty can be accepted.

major comments (5)
  1. [Appendix B, Table 6] The spatial-generalization experiment compares only the proposed control variants (ours, perspective-based, attention-based) under the same perturbation protocol; it does not include any implicit-camera baseline such as MagicDrive, Panacea, or DreamForge. Since the abstract and Section 1 claim state-of-the-art generalization relative to prior methods, the reader cannot tell whether the observed robustness reflects a genuine advantage of ECM and the proposed control mechanism or simply the tolerance of the SparseDrive evaluator to perturbations. Please add the same nuScenes-to-nuScenes-Perturb metric-difference evaluation for at least the strongest implicit-camera baselines, or otherwise justify why such a comparison is impossible.
  2. [Appendix B] The perturbation protocol is explicitly matched to the training augmentation of SparseDrive (Section 4.3 and Appendix B), and SparseDrive is a model from the same research group. This makes the generalization evidence partly circular: the experiment measures the combined robustness of the generator and this particular evaluator, not the generator's generalization to camera parameters in general. Please either use an independent detector under the same perturbations, or provide a protocol motivated by camera-parameter ranges rather than by the evaluator's augmentation, and report the sensitivity of the conclusion to the choice of evaluator.
  3. [Appendix B, Figs. 15-17] Temporal generalization (training at 2 Hz, inference at 12 Hz, and generation in reverse chronological order) is supported only by qualitative visualizations. There are no FID, detection, tracking, or planning metrics for these settings, and no quantitative comparison with DreamForge or other video baselines under an identical frame-rate protocol. Because temporal generalization is one of the two halves of the paper's central claim, quantitative results (at minimum perception metrics on generated 12 Hz sequences and on reverse-order sequences) are needed.
  4. [Sec. 3.3; Appendix A] Building Pixel Correspondence relies on ten fixed depth anchors linearly spaced in [1, 60] m, with correspondence weights predicted by an MLP over these anchors. The paper never analyzes sensitivity to this range or to the number of anchors, although pixels whose true surface depth lies outside the range or between anchors receive an approximate correspondence and may fuse features from wrong target pixels. Please include an ablation over anchor ranges and counts, and discuss failure modes in scenes with nearby or distant objects.
  5. [Sec. 5, Future Work] The Future Work paragraph concedes that large rotations and translations around x and z axes produce poor results. This is a substantial scope restriction on the word 'generalizable' as used in the title and abstract, and it should be stated in the experiments section with a precise range of perturbations for which the method is intended to work. Currently the evaluation in Table 6 is limited to 'slight perturbation,' but the paper does not define this range quantitatively.
minor comments (6)
  1. [Throughout] There are several typos, including 'avaliable' in the abstract, 'scenss' in Section 1, 'generalizing' in Section 2.2, 'tempoarl-level' in the Appendix heading, and 'flexiable' in Section 5.
  2. [Table 1] 'Panecea+' should be spelled 'Panacea+'.
  3. [Section 4.1] The symbol Nf is used in 'Nr + Nf = 3' but is never defined; please clarify whether it is the number of historical frames or the total number of non-reference frames.
  4. [Table 5] The row labels ID-2, ID-3, and ID-4 are reused across Tables 4 and 5, and the relationship between the two tables is not explained; please label the rows in each table independently or add an explicit pointer.
  5. [Figure 2 caption] The phrase 'The (a) proposed method is built upon...' is awkward; it should read 'The proposed method (a) is built upon...'.
  6. [Table 6 caption] The caption uses 'nuScenes-Pertube' for what the text elsewhere calls 'nuScenes-Perturb'.

Circularity Check

1 steps flagged · score 4.0 of 10

Spatial-generalization evidence is partly self-referential: the only quantitative perturbed-camera evaluation uses the authors' own SparseDrive model as the evaluator and defines the perturbation distribution by SparseDrive's training augmentation, while the headline realism and controllability claims are independently grounded.

  1. self citation load bearing [Sec. 4.3 and Appendix B, Table 6]
    "There is an important reason to choose SparseDrive[17], that the model is trained with random augmentation in image space and 3D space, thus can generalize to perturbations in camera parameters to some extent. ... The random perturbation is consistent with training augmentation of SparseDrive, so we can compare the metric difference between nuScenes and nuScenes-Perturb to validate the generalization ability of the generative model."

    SparseDrive (ref. [17]) shares five authors with the present paper and is the sole quantitative evaluation backbone for the camera-parameter generalization claim (Table 6 and the ablation tables). The perturbed-camera protocol is explicitly defined by SparseDrive's own training augmentation, and SparseDrive was selected precisely because that augmentation makes it robust to such perturbations. The reported 'smaller difference' therefore measures how well an augmentation-matched, same-group evaluator maintains its metrics, rather than providing an external, independent test of DriveCamSim's extrapolation.

full rationale

The main realism and controllability results (FID, BEVFusion, StreamPETR, UniAD, Tables 1-2) are against external benchmarks with standard metrics and do not reduce to any fitted parameter or self-citation; these are independent support for the visual-quality and controllability parts of the claim. The ECM equations (pixel back-projection through depth anchors, projection to target views, weighted aggregation) are explicitly defined and are not circular by construction. The depth-anchor approximation is an acknowledged modeling assumption, not a circular step. The notable concern is the spatial-generalization evaluation: the quantitative perturbed-camera experiment uses SparseDrive, a model from the same research group, and the perturbation distribution is chosen to match SparseDrive's training augmentation. This makes the generalization evidence partly self-referential and load-bearing for the 'generalizable camera simulation' claim, though it is not a by-construction equivalence. Temporal generalization (2Hz training to 12Hz inference, reverse order) is supported only by qualitative figures and unquantified, which is an evidence gap rather than circularity. The future-work admission that large rotations and translations fail further bounds the claim but does not make it circular. Overall, the central realism/controllability claims survive with independent content; the generalization claim is weakened by a self-referential evaluation protocol, yielding a score of 4 rather than a higher circularity score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The model's central mechanism rests on a small set of hand-chosen hyperparameters (depth anchors, target view count, frame sampling window) and assumptions about the sufficiency of depth discretization, overlap-based view selection, and the SD1.5 backbone. No new physical entities are introduced.

free parameters (3)
  • Depth anchors = 10 anchors linearly spaced in [1, 60] meters
    Used in ECM to back-project each pixel along depth candidates; the true depth distribution is replaced by a learned weighting over these fixed planes. This hand-set range and count determine what geometry the attention can express.
  • Target view count = 2 for cross-view; 2*(Nr+Nf) for reference/temporal
    Overlap-based view matching selects a fixed number of target views; this hyperparameter controls how much context is aggregated.
  • Random frame sampling window = 4 frames sampled within 12 consecutive frames
    Training samples the generation frame and three context frames from a 12-frame window, defining the temporal offsets the model sees.
assumptions (4)
  • ad hoc to paper Each pixel's true 3D surface point can be approximated by one of D fixed depth planes; the learned depth distribution is sufficient for pixel correspondence.
    Introduced in Sec 3.3 Building Pixel Correspondence; if true depths fall outside [1, 60] m or between anchors, correspondence is approximate, and no mechanism corrects this.
  • ad hoc to paper Overlap percentage of back-projected points is a reliable proxy for relevance between views.
    Sec 3.3 Overlap-based Target View Matching; assumes maximum projected-point hit rate selects the most informative context, with no learning or validation of this proxy.
  • domain assumption Pretrained Stable Diffusion v1.5 features are a suitable backbone for driving-scene generation after full fine-tuning.
    Sec 4.1 Model Setup; the entire framework inherits SD1.5 latent space and text conditioning.
  • domain assumption Downstream perception metrics (SparseDrive, BEVFusion, StreamPETR, UniAD) are valid proxies for generation quality and controllability.
    Used throughout Sec 4; no human evaluation or direct rendering-quality metric beyond FID.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DriveCamSim: Generalizable Camera Simulation via Explicit Camera Modeling for Autonomous Driving." pith.science (2026). https://pith.science/paper/ILYOEG7A

@misc{pith2026250519692,
  author       = {Pith},
  title        = {Pith review of: DriveCamSim: Generalizable Camera Simulation via Explicit Camera Modeling for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ILYOEG7A}},
  note         = {Machine review of arXiv:2505.19692}
}
read the original abstract

Camera sensor simulation serves as a critical role for autonomous driving (AD), e.g. evaluating vision-based AD algorithms. While existing approaches have leveraged generative models for controllable image/video generation, they remain constrained to generating multi-view video sequences with fixed camera viewpoints and video frequency, significantly limiting their downstream applications. To address this, we present a generalizable camera simulation framework DriveCamSim, whose core innovation lies in the proposed Explicit Camera Modeling (ECM) mechanism. Instead of implicit interaction through vanilla attention, ECM establishes explicit pixel-wise correspondences across multi-view and multi-frame dimensions, decoupling the model from overfitting to the specific camera configurations (intrinsic/extrinsic parameters, number of views) and temporal sampling rates presented in the training data. For controllable generation, we identify the issue of information loss inherent in existing conditional encoding and injection pipelines, proposing an information-preserving control mechanism. This control mechanism not only improves conditional controllability, but also can be extended to be identity-aware to enhance temporal consistency in foreground object rendering. With above designs, our model demonstrates superior performance in both visual quality and controllability, as well as generalization capability across spatial-level (camera parameters variations) and temporal-level (video frame rate variations), enabling flexible user-customizable camera simulation tailored to diverse application scenarios. Code will be avaliable at https://github.com/swc-17/DriveCamSim for facilitating future research.

Figures

Figures reproduced from arXiv: 2505.19692 by the authors.

Figure 1
Figure 1. Instead of (a) implicit camera modeling in 2D image space, we propose (b) explicit camera [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of DriveCamSim. The (a) proposed method is built upon a pretrained [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Frame sampling strategy for training and inference. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Our control mechanism preserves information in encoding and injection stage, and support [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results for spatial-level generalization. Rotate front camera 20 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Visualization results for rotating front camera along x-axis. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Visualization results for rotating front camera along y-axis. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Visualization results for rotating front camera along z-axis. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Visualization results for translating front camera along x-axis. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Visualization results for translating front camera along y-axis. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Visualization results for translating front camera along z-axis. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Visualization results for scaling focal length. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Visualization results for inserting 3 virtual cameras on both sides of the front camera [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Failure cases of large rotation of front camera. [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: Qualitative results for temporal-level generalization. Trained on 2Hz data, our model can [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Qualitative results for temporal-level generalization. Our model can generate videos in [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]
Figure 17
Figure 17. Figure 17: Qualitative results for temporal-level generalization of baseline model DreamForge[ [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 4 canonical work pages

  1. [1]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020

    2020

  2. [2]

    Magicdrive: Street view generation with diverse 3d geometry control

    Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, and Qiang Xu. Magicdrive: Street view generation with diverse 3d geometry control. arXiv preprint arXiv:2310.02601, 2023

    arXiv 2023

  3. [3]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  4. [4]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17853–17862, 2023

    2023

  5. [5]

    Senna: Bridging large vision-language models and end-to-end autonomous driving

    Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Senna: Bridging large vision-language models and end-to-end autonomous driving. arXiv preprint arXiv:2410.22313, 2024

    arXiv

  6. [6]

    Vad: Vectorized scene representation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340–8350, 2023

    2023

  7. [7]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1, 2023

    2023

  8. [8]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving

    Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. arXiv preprint arXiv:2411.15139, 2024

    arXiv 2024

Show all 30 references
  1. [9]

    Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion

    Xuewu Lin, Tianwei Lin, Zixiang Pei, Lichao Huang, and Zhizhong Su. Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion. arXiv preprint arXiv:2211.10581, 2022

    2022 arXiv

  2. [10]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation

    Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation. In 2023 IEEE international conference on robotics and automation (ICRA), pages 2774–2781. IEEE, 2023

    2023

  3. [11]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  4. [12]

    Dreamforge: Motion-aware autoregressive video generation for multi-view driving scenes

    Jianbiao Mei, Tao Hu, Xuemeng Yang, Licheng Wen, Yu Yang, Tiantian Wei, Yukai Ma, Min Dou, Botian Shi, and Yong Liu. Dreamforge: Motion-aware autoregressive video generation for multi-view driving scenes. arXiv preprint arXiv:2409.04003, 2024

    2024 arXiv

  5. [13]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021

    2021

  6. [14]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  7. [15]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part ...

    2015

  8. [16]

    Gaia-2: A controllable multi-view generative world model for autonomous driving

    Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving. arXiv preprint arXiv:2503.20523, 2025

    2025 arXiv

  9. [17]

    Sparsedrive: End-to- end autonomous driving via sparse scene representation

    Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng. Sparsedrive: End-to- end autonomous driving via sparse scene representation. arXiv preprint arXiv:2405.19620, 2024

    2024 arXiv

  10. [18]

    Center3d: Center-based monocular 3d object detection with joint depth understanding

    Yunlei Tang, Sebastian Dorn, and Chiragkumar Savani. Center3d: Center-based monocular 3d object detection with joint depth understanding. In DAGM German Conference on Pattern Recognition, pages 289–302. Springer, 2020. 10

    2020

  11. [19]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289, 2024

    2024 arXiv

  12. [20]

    Exploring object-centric temporal modeling for efficient multi-view 3d object detection

    Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3621–3631, 2023

    2023

  13. [21]

    Drivedreamer: Towards real-world-drive world models for autonomous driving

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world-drive world models for autonomous driving. In European Conference on Computer Vision, pages 55–72. Springer, 2024

    2024

  14. [22]

    Dilu: A knowledge-driven approach to autonomous driving with large language models

    Licheng Wen, Daocheng Fu, Xin Li, Xinyu Cai, Tao Ma, Pinlong Cai, Min Dou, Botian Shi, Liang He, and Yu Qiao. Dilu: A knowledge-driven approach to autonomous driving with large language models. arXiv preprint arXiv:2309.16292, 2023

    2023 arXiv

  15. [23]

    Panacea+: Panoramic and controllable video generation for autonomous driving

    Yuqing Wen, Yucheng Zhao, Yingfei Liu, Binyuan Huang, Fan Jia, Yanhui Wang, Chi Zhang, Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang. Panacea+: Panoramic and controllable video generation for autonomous driving. arXiv preprint arXiv:2408.07605, 2024

    2024 arXiv

  16. [24]

    Panacea: Panoramic and controllable video generation for autonomous driving

    Yuqing Wen, Yucheng Zhao, Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang, Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang. Panacea: Panoramic and controllable video generation for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2024

  17. [25]

    Bevcontrol: Accurately controlling street-view elements with multi-perspective consistency via bev sketch layout

    Kairui Yang, Enhui Ma, Jibin Peng, Qing Guo, Di Lin, and Kaicheng Yu. Bevcontrol: Accurately controlling street-view elements with multi-perspective consistency via bev sketch layout. arXiv preprint arXiv:2308.01661, 2023

    2023 arXiv

  18. [26]

    Drivearena: A closed-loop generative simulation platform for autonomous driving

    Xuemeng Yang, Licheng Wen, Yukai Ma, Jianbiao Mei, Xin Li, Tiantian Wei, Wenjie Lei, Daocheng Fu, Pinlong Cai, Min Dou, et al. Drivearena: A closed-loop generative simulation platform for autonomous driving. arXiv preprint arXiv:2408.00415, 2024

    2024 arXiv

  19. [27]

    Bench2drive-r: Turning real world data into reactive closed-loop autonomous driving benchmark by generative model

    Junqi You, Xiaosong Jia, Zhiyuan Zhang, Yutao Zhu, and Junchi Yan. Bench2drive-r: Turning real world data into reactive closed-loop autonomous driving benchmark by generative model. arXiv preprint arXiv:2412.09647, 2024

    2024 arXiv

  20. [28]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  21. [29]

    Drivedreamer-2: Llm-enhanced world models for diverse driving video generation

    Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang. Drivedreamer-2: Llm-enhanced world models for diverse driving video generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 10412–10420, 2025

    2025

  22. [30]

    Unipc: A unified predictor-corrector framework for fast sampling of diffusion models

    Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: A unified predictor-corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36:49842–49869, 2023. 11 Table 6: Metric difference between original nuScene...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.