Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

COME: Adding Scene-Centric Forecasting Control to Occupancy World Model

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Disentangling ego motion from scene evolution lets a diffusion occupancy world model predict future driving scenes more accurately, with reported gains of 26.3% mIoU over DOME and 23.7% over UniScene on Occ3D-nuScenes.

desk verdict A genuinely useful scene-centric conditioning idea, but the abstract's 'more accurate' claim is undercut by the paper's own Table 2, where the final model scores lower all-voxel mIoU than its own Stage 2 forecasting module. read the letter →

arxiv 2506.13260 v1 pith:QXWEUP6G submitted 2025-06-16 cs.CV

Yining Shi , Kun Jiang , Qiang Meng , Ke Wang , Jiabao Wang , Wenchao Sun , Tuopu Wen , Mengmeng Yang
show 1 more author
Diange Yang
This is my paper · ORCID
classification cs.CV
keywords occupancyworldmodel4Dforecastingscene-centriccoordinatesControlNetdiffusiontransformerautonomousdrivingnuScenes-Occ3Dego-motiondisentanglement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Future occupancy prediction in autonomous driving mixes two very different sources of change: the ego vehicle's own motion, which shifts the perspective, and the natural evolution of the scene. This paper claims that explicitly separating the two, by forecasting occupancy in a scene-centric coordinate frame where static structure stays fixed, makes a diffusion-based occupancy world model more accurate and more controllable. The proposed framework, COME, does this in three stages—a pose-conditioned diffusion world model, a simple U-Net that forecasts the scene in a shared coordinate system, and a ControlNet-style module that injects those scene-centric forecasts back into the world model. On the Occ3D-nuScenes benchmark, COME reports consistent improvements across input types and prediction horizons, including 26.3% higher mIoU than DOME and 23.7% higher mIoU than UniScene under matching settings.

What carries the argument

The load-bearing mechanism is the pairing of a scene-centric forecasting branch with a ControlNet-style conditioner. Historical occupancy frames are transformed into the current ego pose via rigid-body transformations, so static voxels stay aligned across time, and a U-Net produces future occupancy in that fixed frame. These forecasts are warped back to their future ego poses, encoded with the same variational occupancy auto-encoder used by the world model, and passed through COME ControlNet—trainable copies of the world model's early spatial-temporal blocks followed by zero convolutions—which inject control features into the second half of the diffusion world model through residual additions and skip connections. A visibility mask multiplies the control features by a binary indicator of whether each spatial pillar was observable in the historical occupancy, preventing the forecasting branch from forcing noise into regions it never saw.

What would settle it

Evaluate COME against DOME on the Occ3D-nuScenes validation set restricted to dynamic voxels only (voxels whose occupancy label changes within the horizon), and repeat the comparison under a metric that rewards correctly generating previously invisible regions. If the scene-centric module is mainly copying static structure, the dynamic-only mIoU gap should collapse toward zero, and the all-voxel ranking between Stage 2 and Stage 3 could invert.

Watch

Extended reading notes

Core claim

The paper's central claim is that an occupancy world model generates better future occupancy when ego motion is factored out before forecasting and then re-injected as an explicit control signal. COME first transforms historical occupancy frames into a common ego-aligned coordinate frame using rigid-body transformations, predicts the future scene in that frame with a U-Net, and converts the forecast into control features through a trainable copy of half the world model's blocks with zero-convolution output layers, following the ControlNet recipe. A visibility mask suppresses control features that correspond to pillars whose voxels were not historically observable, so the model can copy reliable static structure without letting the forecasting module's speculation degrade generation. The resulting system outperforms the base world model and state-of-the-art occupancy generators on the nuScenes-Occ3D validation set across camera, fusion, and ground-truth occupancy inputs and over 3-second and 8-second horizons, with the largest margins in settings where both occupancy and trajectories are known.

Load-bearing premise

The evaluation assumes a benchmark dominated by static voxels (92.7%) is the appropriate test bed, so the gains from copying static background outweigh the cost of reduced imaginative accuracy; if the benchmark emphasized moving agents, or the metric punished unobserved-region errors more heavily, the reported advantage could vanish.

Editorial extensions

If this is right

  • Under the paper's settings, COME outperforms DOME and UniScene on average mIoU and IoU across 1s, 2s, and 3s horizons, as well as on an 8s long-horizon rollout.
  • The gains appear with predicted trajectories as well as ground-truth trajectories; with alignment of predicted and ground-truth occupancy to the same future waypoint, the drop from predicted poses is only 0.23 mIoU.
  • Smaller world models benefit disproportionately: adding the ControlNet raises a small model's mIoU from 7.78 to 32.00, exceeding the standalone large world model's 23.49.
  • The framework supports flexible inference, such as using the ControlNet only for the first roll-out of a super-long 20-second sequence and then continuing with the base model alone.
  • Injecting scene-centric control preserves road structure over long horizons, avoiding spurious intersections that the base world model tends to hallucinate in static areas.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The margin over DOME likely depends on the benchmark's high static-voxel ratio (92.7% reported in the paper); on a scene dominated by moving agents the scene-centric branch would have little static structure to copy, so the advantage could shrink or vanish.
  • Because the paper's own ablation shows the final three-stage model's all-voxel mIoU (34.23) is lower than its Stage 2 forecasting module's (39.12), the headline comparison is sensitive to metric choice; a metric that weights dynamic voxels or penalizes unobserved-region errors would rank the stages differently.
  • The same fixed-frame forecast-as-control idea could transfer to video or point-cloud world models, where a scene-centric prior computed from any sensor modality could serve as an analogous control signal.
  • One direct test of the mechanism would be to train COME on a sequence with a forceful ego maneuver (sharp turn or fast lane change) and measure whether the static-background copying still yields the same relative improvement, since the scene-centric alignment is exactly the component doing the work there.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes COME, a three-stage framework for occupancy world models on nuScenes-Occ3D. Stage 1 is a DOME-style diffusion transformer that generates future occupancy in an ego-centric frame. Stage 2 trains a UNet in a scene-centric coordinate system, where past and future frames are transformed into a common ego frame, producing ego-invariant forecasts. Stage 3 injects these forecasts into the frozen world model through a ControlNet-style adapter with a visibility-aware masking strategy. The paper reports large improvements over DOME and UniScene on several input/trajectory configurations and over DOME on 8-second generation, along with ablations of model size, denoising steps, and masking strategies.

Significance. The core idea of disentangling ego motion from scene evolution is well motivated and potentially useful for occupancy world models. The paper provides a reasonably detailed architecture description, multi-stage training details, and a broad experimental sweep over input modalities, trajectory sources, model sizes, and horizons. The promised code release is also a plus. However, the paper's own ablations undercut the headline accuracy claim: the final COME model has a lower all-voxel mIoU than its own Stage 2 scene-centric forecasting module. Since static voxels are reported to be 92.7% of occupied voxels, the reported gains over prior work may be substantially attributable to the scene-centric static prior rather than to the proposed ControlNet. The abstract's claim of "consistent" improvements is also not literally true in the predicted-trajectory setting. These issues are fixable with revised claims and additional ablations, so the paper is not beyond salvage, but the current presentation overstates what is demonstrated.

major comments (4)
  1. [Sec. 4.4, Table 2] The paper's own ablation contradicts the abstract's central "more accurate" claim. In Table 2, the Stage 3 COME model achieves 34.23 all-voxel mIoU, while its own Stage 2 scene-centric forecasting module achieves 39.12 mIoU; the visible-voxel mIoU is also lower (40.06 vs 42.74). The only meaningful improvement is in invisible-voxel mIoU (5.56 vs 0.09), and the text admits that "the overall mIoU experiences a slight decline." Therefore, the final COME is not more accurate than the scene-centric branch it uses as input. The abstract and conclusion should either qualify this explicitly or reframe the contribution as improving invisible-voxel generation and controllability at a cost in overall accuracy.
  2. [Table 1, 3D-Occ Pred row] The abstract claims "consistent and significant improvements over SOTA methods across diverse configurations," but in the 3D-Occ with predicted-trajectory setting COME achieves 21.29 mIoU and 29.03 IoU, below DFIT-OccWorld (22.71 mIoU, 32.27 IoU) and Occ-LLM (32.52 IoU). The paper's explanation that COME is sensitive to planning errors is plausible, but it does not reconcile with the unqualified abstract claim. The authors should either remove "consistent" or explicitly state which configurations do not show improvements.
  3. [Table 1 vs Table 2] The main-results table never reports the Stage 2 scene-centric forecasting module, even though Table 2 shows it achieves 39.12 all-voxel mIoU, higher than the final COME. Without a Stage 2 row in Table 1 (or an equivalent protocol-matched comparison), the reader cannot determine whether the reported gains over DOME and UniScene are produced by the proposed COME ControlNet or by the scene-centric forecasting prior alone. This is load-bearing for the paper's central claim and should be addressed with an explicit ablation row or table.
  4. [Sec. 3.2 and Table 2] Because static voxels account for 92.7% of occupied voxels, the all-voxel mIoU is dominated by static background. The high Stage 2 score (39.12) likely reflects the trivial copy of static voxels after ego-compensation, and the reported SOTA improvements (26.3% over DOME, 23.7% over UniScene) may be carried by that static prior rather than by the control injection. The paper should report separate metrics for dynamic and static voxels, or at least per-class mIoU for moving-object classes, to establish that the proposed mechanism actually helps with scene dynamics.
minor comments (4)
  1. [Sec. 4.1] The sentence "All models are trained on 4 H20 GPUs and use a learning rate of 1e-4 is not stated specifically" is grammatically broken and ambiguous about whether the learning rate is 1e-4.
  2. [Table 6] In the Random Dropout row, the listed average values are inconsistent with the per-timestamp numbers: the mIoU values 40.32, 30.55, and 36.69 do not average to 31.97, and the IoU values 48.78, 41.78, and 25.04 do not average to 42.42.
  3. [Appendix A.3.2] The network-statistics paragraph says "The base world model has 362.31M parameters" and then later "The base world model has 45.83M parameters," which appears to be a typo for the small model; the same applies to the ControlNet parameter counts.
  4. [References] Several references are duplicated: [1] and [2] are the same UnO paper, [9] and [10] are the same point-cloud-forecasting paper, and [32] and [33] are the same visual-point-cloud-forecasting paper. These should be consolidated.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the scene-centric branch is a separately trained forecaster whose output is injected as a condition, not a re-fit of the evaluation metric.

full rationale

The paper's derivation chain is not circular. The scene-centric forecasting module (Sec. 3.2) is a UNet trained with cross-entropy on transformed occupancy frames; its predictions are then encoded with the frozen Occ-VAE and injected through a separately trained COME ControlNet (Sec. 3.3) into a frozen world model. The final occupancy predictions are evaluated on the external Occ3D-nuScenes benchmark with IoU/mIoU computed against ground-truth labels, so the reported gains are measured, not defined, by the method's own outputs. The internal comparison in Table 2 shows Stage 2 alone (39.12 mIoU) beating final COME (34.23 mIoU) on all voxels, and Table 6 shows an unselected 'No Mask' variant with higher mIoU (36.19) than the selected 'Mask Control' variant (34.23); these are correctness and metric-selection concerns about the headline 'more accurate' claim, not circularity. The only self-citation, EFFOcc [21], is used as an external input generator in one input-source experiment and does not carry the central claim. No load-bearing step reduces to its inputs by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced; the scene condition is a learned feature map and the COME ControlNet is a neural module, with no independent falsifiable handle. The central claim rests on the static-dominance assumption, the rigid-body transform, and the architecture choices listed above.

free parameters (4)
  • Visibility mask threshold (epsilon) = 0.5
    Eq. (1) drops control features when fewer than epsilon of voxels in a pillar were observed historically; with 92.7% static voxels this threshold governs how much static prior is injected.
  • Guidance scale = 7.5
    Classifier-free guidance scale used in all reported DDPM generations (Sec. A.3.3); it trades fidelity against diversity and affects every mIoU number in the tables.
  • Inference denoising steps = 20
    Tab. 4a shows 20 steps are used for the reported values; dropping to 10 steps costs little, but 5 steps collapses performance from 34.23 to 4.95 mIoU.
  • Pose-condition probability during training = 0.9
    During training, pose is dropped as a condition with probability 0.9; this affects how strongly trajectory conditioning is learned (Sec. A.3.3).
assumptions (4)
  • domain assumption Static voxels account for over 92.7% of occupied voxels in Occ3D-nuScenes.
    Section 3.2 uses this figure to justify the scene-centric forecasting design; the main numerical improvements over DOME and UniScene are concentrated in the resulting static and visible region.
  • domain assumption Rigid-body transformation with nearest-neighbor search and grid sampling preserves occupancy enough for forecasting.
    Section 3.2 transforms each historical frame into the current frame (x'_i = T(x_i, p_i, p_t)) and later transforms predictions back; discretization artifacts are not quantified.
  • ad hoc to paper A trainable copy of the first half of the diffusion transformer, with zero-convolution outputs, can inject scene conditions without destabilizing the frozen world model.
    Section 3.3 and appendix A.1 motivate this design by analogy to ControlNet and HunyuanDiT, but no stability analysis is given; the choice is validated only empirically.
  • domain assumption The pretrained Occ-VAE from DOME compresses occupancy without losing information relevant to the metric.
    The pipeline encodes and decodes through this VAE (Sec. 3.1, A.3.2); reconstruction error is not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of COME: Adding Scene-Centric Forecasting Control to Occupancy World Model." pith.science (2026). https://pith.science/paper/QXWEUP6G

@misc{pith2026250613260,
  author       = {Pith},
  title        = {Pith review of: COME: Adding Scene-Centric Forecasting Control to Occupancy World Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QXWEUP6G}},
  note         = {Machine review of arXiv:2506.13260}
}
read the original abstract

World models are critical for autonomous driving to simulate environmental dynamics and generate synthetic data. Existing methods struggle to disentangle ego-vehicle motion (perspective shifts) from scene evolvement (agent interactions), leading to suboptimal predictions. Instead, we propose to separate environmental changes from ego-motion by leveraging the scene-centric coordinate systems. In this paper, we introduce COME: a framework that integrates scene-centric forecasting Control into the Occupancy world ModEl. Specifically, COME first generates ego-irrelevant, spatially consistent future features through a scene-centric prediction branch, which are then converted into scene condition using a tailored ControlNet. These condition features are subsequently injected into the occupancy world model, enabling more accurate and controllable future occupancy predictions. Experimental results on the nuScenes-Occ3D dataset show that COME achieves consistent and significant improvements over state-of-the-art (SOTA) methods across diverse configurations, including different input sources (ground-truth, camera-based, fusion-based occupancy) and prediction horizons (3s and 8s). For example, under the same settings, COME achieves 26.3% better mIoU metric than DOME and 23.7% better mIoU metric than UniScene. These results highlight the efficacy of disentangled representation learning in enhancing spatio-temporal prediction fidelity for world models. Code and videos will be available at https://github.com/synsin0/COME.

Figures

Figures reproduced from arXiv: 2506.13260 by the authors.

Figure 1
Figure 1. COME with both scene-centric and ego-centric representation. Compared to ego-centric [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The proposed COME framework comprises three main modules: (1) an [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results of 3-s 4D occupancy generation. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Visualization examples demonstrate the pose control alignment ability of COME generation. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Visualization examples of occupancy generation with BEV layouts. COME generates [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Visualization examples demonstrate the spatiotemporal consistency of differnet stages in [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Visualization examples that takes 3D-Occ, vision-based BEVDet and fusion-based EFFOcc [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Visualization examples that uses different masking strategies during training and inference. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Visualization examples that uses small models. In this right turning example, the small [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Visualization examples that uses repeated roll-out to generate super-long scenarios (20 [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

  2. 3D and 4D World Modeling: A Survey

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A survey that defines 3D/4D world modeling, organizes methods into VideoGen, OccGen, and LiDARGen categories, and compiles datasets, metrics, and benchmark numbers.

Reference graph

Works this paper leans on

37 extracted references · 15 canonical work pages · cited by 2 Pith papers

  1. [2]

    Uno: Unsupervised occupancy fields for perception and forecasting

    Ben Agro, Quinlan Sykora, Sergio Casas, Thomas Gilles, and Raquel Urtasun. Uno: Unsupervised occupancy fields for perception and forecasting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14487–14496, 2024

    work page 2024

  2. [3]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020

    2020

  3. [4]

    Dome: Taming diffusion model into high-fidelity controllable occupancy world model.arXiv preprint arXiv:2410.10429, 2024

    Songen Gu, Wei Yin, Bu Jin, Xiaoyang Guo, Junming Wang, Haodong Li, Qian Zhang, and Xiaoxiao Long. Dome: Taming diffusion model into high-fidelity controllable occupancy world model.arXiv preprint arXiv:2410.10429, 2024

    arXiv 2024

  4. [5]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InProceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY , USA, 2020. Curran Associates Inc

    2020

  5. [6]

    Gaia-1: A generative world model for autonomous driving.arXiv preprint arXiv:2309.17080, 2023

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving.arXiv preprint arXiv:2309.17080, 2023

    arXiv 2023

  6. [7]

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view.arXiv preprint arXiv:2112.11790, 2021

    Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view.arXiv preprint arXiv:2112.11790, 2021

    arXiv 2021

  7. [8]

    Occvar: Scalable 4d occupancy prediction via next-scale prediction

    Bu Jin, Xiaotao Hu, Yupeng Zheng, Xiaoyang Guo, Qian Zhang, Yao Yao, Diming Zhang, Xiaoxiao Long, Wei Yin, et al. Occvar: Scalable 4d occupancy prediction via next-scale prediction

    work page

  8. [10]

    Point cloud forecasting as a proxy for 4d occupancy forecasting

    Tarasha Khurana, Peiyun Hu, David Held, and Deva Ramanan. Point cloud forecasting as a proxy for 4d occupancy forecasting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1116–1124, 2023

    work page 2023

Show all 37 references
  1. [11]

    Uniscene: Unified occupancy-centric driving scene generation.arXiv preprint arXiv:2412.05435, 2024

    Bohan Li, Jiazhe Guo, Hongsi Liu, Yingshuang Zou, Yikang Ding, Xiwu Chen, Hu Zhu, Feiyang Tan, Chi Zhang, Tiancai Wang, et al. Uniscene: Unified occupancy-centric driving scene generation.arXiv preprint arXiv:2412.05435, 2024

    2024 arXiv

  2. [12]

    Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo

    Yinhao Li, Han Bao, Zheng Ge, Jinrong Yang, Jianjian Sun, and Zeming Li. Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1486–1494, 2023

    2023

  3. [13]

    Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding.arXiv preprint arXiv:2405.08748, 2024

    Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, et al. Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding.arXiv preprint arXiv:2405.08748, 2024

    2024 arXiv

  4. [14]

    Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M. Alvarez. Is ego status all you need for open-loop end-to-end autonomous driving? In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14864–14873, 2024

    2024

  5. [15]

    Lidar-based 4d occupancy completion and forecasting

    Xinhao Liu, Moonjun Gong, Qi Fang, Haoyu Xie, Yiming Li, Hang Zhao, and Chen Feng. Lidar-based 4d occupancy completion and forecasting. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11102–11109. IEEE, 2024

    2024

  6. [16]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9992–10002, 2021

    2021

  7. [17]

    Cam4docc: Benchmark for camera-only 4d occupancy forecasting in autonomous driving applications

    Junyi Ma, Xieyuanli Chen, Jiawei Huang, Jingyi Xu, Zhen Luo, Jintao Xu, Weihao Gu, Rui Ai, and Hesheng Wang. Cam4docc: Benchmark for camera-only 4d occupancy forecasting in autonomous driving applications. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  8. [18]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  9. [19]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part I...

    2015

  10. [20]

    Con- volutional lstm network: A machine learning approach for precipitation nowcasting.Advances in neural information processing systems, 28, 2015

    Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Con- volutional lstm network: A machine learning approach for precipitation nowcasting.Advances in neural information processing systems, 28, 2015

    2015

  11. [21]

    Effocc: A minimal baseline for efficient fusion-based 3d occupancy network

    Yining Shi, Kun Jiang, Ke Wang, Kangan Qian, Yunlong Wang, Jiusi Li, Tuopu Wen, Mengmeng Yang, Yiliang Xu, and Diange Yang. Effocc: A minimal baseline for efficient fusion-based 3d occupancy network. arXiv preprint arXiv:2406.07042, 2024

    2024 arXiv

  12. [22]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurélien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang,...

    2020

  13. [23]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36:64318–64330, 2023

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36:64318–64330, 2023

    2023

  14. [24]

    Neural discrete representation learning

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6309–6318, Red Hook, NY , USA, 2017. Curran Associates Inc

    2017

  15. [25]

    Drivedreamer: Towards real-world-drive world models for autonomous driving

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world-drive world models for autonomous driving. InEuropean Conference on Computer Vision, pages 55–72. Springer, 2024

    2024

  16. [26]

    Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving

    Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1...

    2024

  17. [27]

    Oc- cllama: An occupancy-language-action generative world model for autonomous driving.arXiv preprint arXiv:2409.03272, 2024

    Julong Wei, Shanshuai Yuan, Pengfei Li, Qingda Hu, Zhongxue Gan, and Wenchao Ding. Oc- cllama: An occupancy-language-action generative world model for autonomous driving.arXiv preprint arXiv:2409.03272, 2024

    2024 arXiv

  18. [28]

    Occ-llm: Enhancing autonomous driving with occupancy-based large language models.arXiv preprint arXiv:2502.06419, 2025

    Tianshuo Xu, Hao Lu, Xu Yan, Yingjie Cai, Bingbing Liu, and Yingcong Chen. Occ-llm: Enhancing autonomous driving with occupancy-based large language models.arXiv preprint arXiv:2502.06419, 2025

    2025 arXiv

  19. [29]

    Drivingsphere: Building a high-fidelity 4d world for closed-loop simulation.arXiv preprint arXiv:2411.11252, 2024

    Tianyi Yan, Dongming Wu, Wencheng Han, Junpeng Jiang, Xia Zhou, Kun Zhan, Cheng-zhong Xu, and Jianbing Shen. Drivingsphere: Building a high-fidelity 4d world for closed-loop simulation.arXiv preprint arXiv:2411.11252, 2024

    2024 arXiv

  20. [30]

    Renderworld: World model with self-supervised 3d label.arXiv preprint arXiv:2409.11356, 2024

    Ziyang Yan, Wenzhen Dong, Yihua Shao, Yuhang Lu, Liu Haiyang, Jingwen Liu, Haozhe Wang, Zhe Wang, Yan Wang, Fabio Remondino, et al. Renderworld: World model with self-supervised 3d label.arXiv preprint arXiv:2409.11356, 2024

    2024 arXiv

  21. [31]

    Driving in the occupancy world: Vision-centric 4d occupancy forecasting and planning via world models for autonomous driving

    Yu Yang, Jianbiao Mei, Yukai Ma, Siliang Du, Wenqing Chen, Yijie Qian, Yuxiang Feng, and Yong Liu. Driving in the occupancy world: Vision-centric 4d occupancy forecasting and planning via world models for autonomous driving. InProceedings of the AAAI Conference on Artificial I...

    2025

  22. [32]

    Visual point cloud forecasting enables scalable autonomous driving

    Zetong Yang, Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scalable autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14673–14684, 2024

    2024

  23. [33]

    Visual point cloud forecasting enables scalable autonomous driving

    Zetong Yang, Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scalable autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14673–14684, 2024. 12

    2024

  24. [34]

    An efficient occupancy world model via decoupled dynamic flow and image-assisted training.arXiv preprint arXiv:2412.13772, 2024

    Haiming Zhang, Ying Xue, Xu Yan, Jiacheng Zhang, Weichao Qiu, Dongfeng Bai, Bingbing Liu, Shuguang Cui, and Zhen Li. An efficient occupancy world model via decoupled dynamic flow and image-assisted training.arXiv preprint arXiv:2412.13772, 2024

    2024 arXiv

  25. [35]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  26. [36]

    Occworld: Learning a 3d occupancy world model for autonomous driving

    Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. InEuropean conference on computer vision, pages 55–72. Springer, 2024

    2024

  27. [37]

    Class-balanced grouping and sampling for point cloud 3d object detection.arXiv preprint arXiv:1908.09492, 2019

    Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, and Gang Yu. Class-balanced grouping and sampling for point cloud 3d object detection.arXiv preprint arXiv:1908.09492, 2019

    1908 arXiv

  28. [38]

    Lidardm: Generative lidar simulation in a generated world.arXiv preprint arXiv:2404.02903, 2024

    Vlas Zyrianov, Henry Che, Zhijian Liu, and Shenlong Wang. Lidardm: Generative lidar simulation in a generated world.arXiv preprint arXiv:2404.02903, 2024. 13 A Technical Appendices and Supplementary Material A.1 Discussion on the Design Philosophy of External Occupancy Control...

    2024

  29. [39]

    For different size of world models, corresponding controlnet has half of the depth of the world model and uses the same parameters in each block

    The topk is set as 10. For different size of world models, corresponding controlnet has half of the depth of the world model and uses the same parameters in each block. Network statistics.The following statistics is tested with the standard task of generation future three seco...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.