arxiv: 2603.28980 · v2 · submitted 2026-03-30 · 💻 cs.CV

Recognition: unknown

Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas

Christian Rupprecht, Daniel Cremers, Fabian Manhardt, Federico Tombari, Felix Wimbauer, Michael Oechsle, Nikolai Kalischek

Authors on Pith no claims yet

Pith reviewed 2026-05-14 21:10 UTC · model grok-4.3

classification 💻 cs.CV

keywords text-to-3Dimmersive scene generationpanorama expansionmultiview diffusion3D reconstructionconsistent scene synthesis

0 comments

The pith

Stepper generates immersive 3D scenes from text by expanding consistent multiview panoramas one step at a time.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that text-driven 3D scene synthesis can avoid the usual trade-off between visual quality and explorability by replacing autoregressive or video-based methods with controlled stepwise expansion of panoramic views. It introduces a multi-view 360-degree diffusion model trained on a new large-scale panorama dataset to keep geometry and appearance aligned across expansions, then feeds the results into a reconstruction pipeline that enforces 3D coherence. The central result is that this combination produces higher-fidelity and more structurally consistent scenes than prior work, directly addressing context drift and resolution limits. If the approach holds, it opens a practical route to generating large, navigable environments suitable for AR/VR and world modeling without manual stitching or post-processing.

Core claim

Stepper is a unified framework for text-driven immersive 3D scene synthesis that performs stepwise panoramic scene expansion using a novel multi-view 360 diffusion model for consistent high-resolution output, coupled with a geometry reconstruction pipeline that enforces geometric coherence, all trained on a new large-scale multi-view panorama dataset to achieve state-of-the-art fidelity and structural consistency.

What carries the argument

The novel multi-view 360° diffusion model that produces geometrically and visually consistent high-resolution panorama expansions across multiple steps.

If this is right

High-fidelity panoramic scenes can be generated directly from text without suffering context drift.
The same diffusion model supports arbitrary scene sizes while preserving resolution and coherence.
A single trained model replaces separate initialization and expansion stages used in earlier pipelines.
Reconstructed geometry from the expanded panoramas is immediately usable for downstream navigation and rendering tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If expansion consistency scales without drift, the method could support on-demand generation of city-scale environments from simple text descriptions.
The released multi-view panorama dataset may serve as a training resource for other tasks that require aligned 360-degree views.
The stepwise design suggests a natural way to add user control at each expansion step for iterative scene editing.

Load-bearing premise

The multi-view 360 diffusion model can maintain geometric and visual consistency across any number of expansion steps without drift or accumulating artifacts.

What would settle it

Run the model on a sequence of ten or more expansion steps from a single text prompt and measure whether structural misalignment or visual artifacts appear in the reconstructed 3D scene.

Figures

Figures reproduced from arXiv: 2603.28980 by Christian Rupprecht, Daniel Cremers, Fabian Manhardt, Federico Tombari, Felix Wimbauer, Michael Oechsle, Nikolai Kalischek.

**Figure 1.** Figure 1: Stepper sets a new state-of-the-art quality level of generated explorable 3D scenes. Its core innovation is a novel cubemap-based multi-view panorama diffusion model that ensures high-resolution scene synthesis while facilitating step-wise, coherent scene expansion and high-quality scene reconstruction. Please check out our project page at: fwmb.github.io/stepper Abstract The synthesis of immersive 3D scen… view at source ↗

**Figure 2.** Figure 2: Method overview. a) Our model generates a new panoramic image from a previously unobserved viewpoint based on a given input panorama. To ensure high quality, we utilize a pre-trained diffusion model with expanded multi-view attention that is instrumental for jointly denoising the high-resolution cubefaces of the newly generated novel-view panorama. b) Our ability to generate novel-view panoramas enables au… view at source ↗

**Figure 3.** Figure 3: Dataset Samples. The dataset generated with Infinigen consists of a diverse set of high quality synthetic panoramas of indoor and outdoor scenes. For every panorama, we rendered a pair from a novel viewpoint enabling the training of the multiview panorama generation model. All panoramas are aligned to the horizontal line. 4. Experiments In the following, we first describe the design of our training and t… view at source ↗

**Figure 4.** Figure 4: 3D Scene Generation. We provide visual example of generated novel-view panoramas on the left side. The details of the initial panorama are well preserved and previously unseen regions are filled in. On the right side we show novel-view renderings of the reconstructed scenes indicating the 3D consistency of the generated panoramas. lar LDM [45] model. It processes twelve individual cubemap faces at a resol… view at source ↗

**Figure 5.** Figure 5: Comparison with Baselines. Given a high quality input panorama, we observe that our approach achieves consistent scene generation while showing significantly more details and sharpness in the rendered novel view images in comparison to the baselines. we align the scenes to the ground-truth scale by comparing rendered and ground-truth depth maps. In [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

**Figure 6.** Figure 6: Single vs multiple panoramas to 3DGS. The multi panorama input to the 3DGS reconstruction consistently fills in the unobserved regions in the initial panorama without sacrificing the quality of the input panorama. Adjustable step (forward / backward) vs. fixed step (forward) Step to the right Input pano & crop 1x 0.25m step 2x 0.25m step 1x 0.5m step a) b) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

**Figure 7.** Figure 7: Effect of step size. Novel panos. generated by a model with a) adjustable step direction, b) a larger step size d = 0.5m. step size of d = 0.5m (vs the default d = 0.25m). As can be seen in [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗

**Figure 8.** Figure 8: Impact of Multi-View Panoramas We depict the effect of using various options of panorama input for MapAnything and found that the auto-regressive expansion and sampling yield the most complete and accurate outputs. positions are sampled for panoramic cameras. For an indoor scene, the pipeline places up to n = 20 cameras at random locations in the room. To reduce the numbers of assets that have to be gener… view at source ↗

**Figure 9.** Figure 9: Auto-regressive steps. We perform a number of auto-regressive steps to generate novel-view panoramas from the initial panorama to the left and right for several examples. From the generated panoramas, we visualize the forward-facing view to highlight the generated details. A single step corresponds to a step length of 0.25m. 4 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗

**Figure 10.** Figure 10: Multi-view panoramas pairs generated with Infinigen [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗

**Figure 11.** Figure 11: Input panoramas used for testing. 5 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗

**Figure 12.** Figure 12: User study form. Participants fill out this randomized form and cast a single vote per category. 6 [PITH_FULL_IMAGE:figures/full_fig_p017_12.png] view at source ↗

read the original abstract

The synthesis of immersive 3D scenes from text is rapidly maturing, driven by novel video generative models and feed-forward 3D reconstruction, with vast potential in AR/VR and world modeling. While panoramic images have proven effective for scene initialization, existing approaches suffer from a trade-off between visual fidelity and explorability: autoregressive expansion suffers from context drift, while panoramic video generation is limited to low resolution. We present Stepper, a unified framework for text-driven immersive 3D scene synthesis that circumvents these limitations via stepwise panoramic scene expansion. Stepper leverages a novel multi-view 360{\deg} diffusion model that enables consistent, high-resolution expansion, coupled with a geometry reconstruction pipeline that enforces geometric coherence. Trained on a new large-scale, multi-view panorama dataset, Stepper achieves state-of-the-art fidelity and structural consistency, outperforming prior approaches, thereby setting a new standard for immersive scene generation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Stepper introduces a stepwise multi-view 360 diffusion approach for consistent text-to-3D panoramic scene expansion, but the abstract supplies no metrics to back the SOTA claims.

read the letter

The main thing here is a stepwise framework that expands high-resolution panoramas one step at a time with a dedicated multi-view 360 diffusion model, then uses geometry reconstruction to keep things coherent. This is positioned as a way around the drift that hits autoregressive methods and the resolution limits of panoramic video generation. They trained on a new large-scale multi-view panorama dataset, which gives the model a concrete base to work from. The high-level design looks coherent for practical immersive use cases in AR/VR or world modeling. It directly targets the fidelity-versus-explorability trade-off that the field has been stuck on. The paper does a decent job laying out the pipeline in plain terms without overcomplicating the motivation. On the soft spots, the abstract asserts state-of-the-art fidelity and structural consistency yet shows zero numbers, baselines, ablations, or error analysis. That leaves the central claim about long-horizon consistency untested in the summary we have. The assumption that the model avoids drift or artifacts over many expansion steps is plausible on paper but needs the actual results and failure cases to hold up. Minor issues like dataset details or exact training setup would also matter for anyone trying to reproduce it. This is for people working on generative 3D vision and immersive scene synthesis who want a practical expansion method rather than pure theory. A reader focused on applications would get value from the framework description and the dataset mention. It deserves a serious referee to check the experiments and see whether the consistency claims survive scrutiny.

Referee Report

3 major / 1 minor

Summary. The paper introduces Stepper, a unified framework for text-driven immersive 3D scene synthesis via stepwise panoramic expansion. It proposes a novel multi-view 360° diffusion model for consistent high-resolution scene expansion, paired with a geometry reconstruction pipeline to enforce coherence. The approach is trained on a newly introduced large-scale multi-view panorama dataset and claims to deliver state-of-the-art fidelity and structural consistency, outperforming prior autoregressive and panoramic video methods.

Significance. If the empirical results hold, Stepper would meaningfully advance immersive scene generation for AR/VR and world modeling by mitigating context drift while maintaining high resolution and geometric consistency. The stepwise multi-view design and new dataset represent a coherent response to documented trade-offs in existing pipelines. However, the absence of any quantitative metrics, baselines, or ablation studies in the manuscript as presented substantially weakens the ability to confirm the SOTA claim or assess generalizability across arbitrary expansion steps.

major comments (3)

[Abstract] Abstract: The central claim that Stepper 'achieves state-of-the-art fidelity and structural consistency, outperforming prior approaches' is presented without any quantitative metrics, comparison tables, error analysis, or evaluation protocol. This evidentiary gap directly undermines verification of the primary contribution.
[Methods (inferred from abstract)] The manuscript provides no equations, architectural diagrams, or training details for the multi-view 360° diffusion model, making it impossible to evaluate how geometric consistency is maintained across multiple expansion steps or whether the approach is parameter-free as implied by the high-level description.
[Dataset (inferred from abstract)] No description of the new large-scale multi-view panorama dataset is supplied (size, diversity, annotation protocol, or train/test split), which is load-bearing for the claim that the model is generalizable and sets a new standard.

minor comments (1)

[Abstract] The notation '360{°}' in the abstract should be rendered as 360° for standard readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed review. We agree that the current manuscript presentation requires strengthening through the addition of quantitative evidence, methodological specifics, and dataset information to fully support our claims. We will revise accordingly.

read point-by-point responses

Referee: [Abstract] Abstract: The central claim that Stepper 'achieves state-of-the-art fidelity and structural consistency, outperforming prior approaches' is presented without any quantitative metrics, comparison tables, error analysis, or evaluation protocol. This evidentiary gap directly undermines verification of the primary contribution.

Authors: We acknowledge this gap in the presented version. The revised manuscript will include a concise summary of key quantitative results in the abstract (e.g., FID and geometric consistency scores versus baselines), with explicit references to new comparison tables, ablation studies on expansion steps, and the evaluation protocol (including metrics for fidelity, structural coherence, and user studies) in the main text. revision: yes
Referee: [Methods (inferred from abstract)] The manuscript provides no equations, architectural diagrams, or training details for the multi-view 360° diffusion model, making it impossible to evaluate how geometric consistency is maintained across multiple expansion steps or whether the approach is parameter-free as implied by the high-level description.

Authors: The full manuscript contains a methods section, but we agree it requires expansion for clarity and reproducibility. We will add explicit equations for the multi-view diffusion process and consistency losses, additional architectural diagrams showing the stepwise expansion and geometry integration, training details (hyperparameters, loss weights), and a clear explanation of how the geometry reconstruction pipeline enforces coherence across steps. We will also clarify that the model relies on learned parameters rather than being parameter-free. revision: yes
Referee: [Dataset (inferred from abstract)] No description of the new large-scale multi-view panorama dataset is supplied (size, diversity, annotation protocol, or train/test split), which is load-bearing for the claim that the model is generalizable and sets a new standard.

Authors: We will add a dedicated dataset section describing its scale (number of multi-view panoramas), diversity (scene types, lighting conditions, and viewpoints), annotation protocol for ensuring multi-view geometric consistency, and the train/validation/test splits. This will directly support the generalizability and SOTA claims. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper introduces a new multi-view 360° diffusion model and a new large-scale multi-view panorama dataset for stepwise panoramic scene expansion, followed by a geometry reconstruction stage. No equations, derivations, or fitted parameters are presented in the abstract or described structure that reduce a claimed prediction or result back to the inputs by construction. The central claims of improved fidelity and consistency rest on the novelty of the model architecture and training data rather than any self-referential loop, self-citation chain, or renaming of prior fitted quantities. This is a standard case of a self-contained empirical method paper whose validity depends on external benchmarks and ablations, not internal definitional equivalence.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

Only abstract available; ledger reflects high-level assumptions in the described pipeline. The work depends on the effectiveness of the new diffusion model and dataset whose training details and validation are not shown.

axioms (2)

domain assumption Multi-view 360 diffusion models can produce consistent high-resolution panoramas across expansion steps
Central to the framework described in the abstract
domain assumption Geometry reconstruction pipeline enforces structural coherence
Stated as coupled with the diffusion model

invented entities (1)

Stepper framework no independent evidence
purpose: Unified text-driven immersive 3D scene synthesis via stepwise panoramic expansion
New system introduced in the paper

pith-pipeline@v0.9.0 · 5478 in / 1348 out tokens · 49300 ms · 2026-05-14T21:10:10.310871+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

80 extracted references · 13 canonical work pages · 7 internal anchors

[1]

Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields

Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields. InICCV, pages 5855–5864, 2021. 3

2021
[2]

Zip-nerf: Anti-aliased grid- based neural radiance fields

Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid- based neural radiance fields. InICCV, pages 19697–19705,
[3]

ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth

Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023. 2

work page internal anchor Pith review arXiv 2023
[4]

Diffdreamer: Towards consistent unsupervised single-view scene extrapolation with conditional diffusion models

Shengqu Cai, Eric Ryan Chan, Songyou Peng, Mohamad Shahbazi, Anton Obukhov, Luc Van Gool, and Gordon Wetzstein. Diffdreamer: Towards consistent unsupervised single-view scene extrapolation with conditional diffusion models. InICCV, pages 2139–2150, 2023. 1, 3

2023
[5]

Luciddreamer: Domain-free genera- tion of 3d gaussian splatting scenes.IEEE TVCG, 2025

Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free genera- tion of 3d gaussian splatting scenes.IEEE TVCG, 2025. 1, 3

2025
[6]

Latentpaint: Image inpainting in latent space with diffusion models

Ciprian Corneanu, Raghudeep Gadde, and Aleix M Mar- tinez. Latentpaint: Image inpainting in latent space with diffusion models. InWACV, pages 4334–4343, 2024. 2

2024
[7]

Depth map prediction from a single image using a multi-scale deep net- work.NeurIPS, 27, 2014

David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work.NeurIPS, 27, 2014. 2

2014
[8]

Scaling rec- tified flow transformers for high-resolution image synthesis

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rec- tified flow transformers for high-resolution image synthesis. InICML. JMLR.org, 2024. 2

2024
[9]

Diffusion360: Seamless 360 degree panoramic image generation based on diffusion models.CoRR, 2023

Mengyang Feng, Jinlin Liu, Miaomiao Cui, and Xuansong Xie. Diffusion360: Seamless 360 degree panoramic image generation based on diffusion models.CoRR, 2023. 2

2023
[10]

Scenescape: Text-driven consistent scene generation

Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel. Scenescape: Text-driven consistent scene generation. NeurIPS, 36:39897–39914, 2023. 1, 3

2023
[11]

Opa-ma: Text guided mamba for 360-degree image out-painting.arXiv preprint arXiv:2407.10923, 2024

Penglei Gao, Kai Yao, Tiandi Ye, Steven Wang, Yuan Yao, and Xiaofeng Wang. Opa-ma: Text guided mamba for 360-degree image out-painting.arXiv preprint arXiv:2407.10923, 2024. 2

work page arXiv 2024
[12]

Cat3d: create anything in 3d with multi-view diffusion models

Ruiqi Gao, Aleksander Hoły ´nski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: create anything in 3d with multi-view diffusion models. InNeurIPS, pages 75468–75494, 2024. 3, 4

2024
[13]

Cameractrl: Enabling camera control for text-to-video generation

Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. InICLR, 2025. 2

2025
[14]

Classifier-Free Diffusion Guidance

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 2

work page internal anchor Pith review Pith/arXiv arXiv 2022
[15]

Denoising dif- fusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. InNeurIPS, pages 6840–6851,
[16]

Text2room: Extracting textured 3d meshes from 2d text-to-image models

Lukas H ¨ollein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. InICCV, pages 7909– 7920, 2023. 1, 3

2023
[17]

Cogvideo: Large-scale pretraining for text-to-video generation via transformers

Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. InICLR, 2023. 2

2023
[18]

LoRA: Low-rank adaptation of large language models

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 2

2022
[19]

Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation.IEEE TPAMI, 2024

Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation.IEEE TPAMI, 2024. 3

2024
[20]

Cubediff: Repurposing diffusion-based image models for panorama generation

Nikolai Kalischek, Michael Oechsle, Fabian Manhardt, Philipp Henzler, Konrad Schindler, and Federico Tombari. Cubediff: Repurposing diffusion-based image models for panorama generation. InICLR, 2025. 2, 3, 4, 5

2025
[21]

Repurpos- ing diffusion-based image generators for monocular depth estimation

Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. InCVPR, pages 9492–9502, 2024. 2

2024
[22]

MapAnything: Universal Feed-Forward Metric 3D Reconstruction

Nikhil Keetha, Norman M ¨uller, Johannes Sch ¨onberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weber, Nelson Antunes, et al. Mapanything: Universal feed-forward metric 3d re- construction.arXiv preprint arXiv:2509.13414, 2025. 2, 3, 5

work page internal anchor Pith review Pith/arXiv arXiv 2025
[23]

3d gaussian splatting for real-time radiance field rendering.ACM TOG, 42(4), 2023

Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM TOG, 42(4), 2023. 2, 3, 4

2023
[24]

3d gaussian splat- ting as markov chain monte carlo.NeurIPS, 37:80965– 80986, 2024

Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Wei- wei Sun, Yang-Che Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splat- ting as markov chain monte carlo.NeurIPS, 37:80965– 80986, 2024. 5

2024
[25]

HunyuanVideo: A Systematic Framework For Large Video Generative Models

Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024. 2

work page internal anchor Pith review Pith/arXiv arXiv 2024
[26]

Edgs: Eliminating densification for efficient convergence of 3dgs, 2025

Dmytro Kotovenko, Olga Grebenkova, and Bj ¨orn Ommer. Edgs: Eliminating densification for efficient convergence of 3dgs, 2025. 5

2025
[27]

Flux.1 kontext: Flow matching for in-context image generation and editing in latent space,

Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, Sumith Ku- lal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas M¨uller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context i...
[28]

Deeper depth prediction with fully convolutional residual networks

Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Fed- erico Tombari, and Nassir Navab. Deeper depth prediction with fully convolutional residual networks. In3DV, pages 239–248. IEEE, 2016. 2

2016
[29]

Wonderland: Navi- gating 3d scenes from a single image

Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Konstantinos N Pla- taniotis, Sergey Tulyakov, and Jian Ren. Wonderland: Navi- gating 3d scenes from a single image. InCVPR, pages 798– 810, 2025. 3

2025
[30]

Pinco: Position-induced consistent adapter for diffu- sion transformer in foreground-conditioned inpainting

Guangben Lu, Yuzhen Du, Yizhe Tang, Zhimin Sun, Ran Yi, Yifan Qi, Tianyi Wang, Lizhuang Ma, and Fangyuan Zou. Pinco: Position-induced consistent adapter for diffu- sion transformer in foreground-conditioned inpainting. In ICCV, pages 15266–15276, 2025. 2

2025
[31]

Autoregressive omni-aware outpainting for open- vocabulary 360-degree image generation

Zhuqiang Lu, Kun Hu, Chaoyue Wang, Lei Bai, and Zhiy- ong Wang. Autoregressive omni-aware outpainting for open- vocabulary 360-degree image generation. InAAAI, pages 14211–14219, 2024. 2

2024
[32]

Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021

Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 3

2021
[33]

T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. InAAAI, pages 4296–4304, 2024. 2

2024
[34]

Radsplat: Radiance field-informed gaussian splat- ting for robust real-time rendering with 900+ fps

Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakoto- saona, Michael Oechsle, Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominik Kaeser, and Federico Tombari. Radsplat: Radiance field-informed gaussian splat- ting for robust real-time rendering with 900+ fps. In3DV,
[35]

Sora: Video generation models as world simulators

OpenAI. Sora: Video generation models as world simulators. https://openai.com/sora/, 2024. 2

2024
[36]

Sora 2.https://openai.com/index/sor a-2/, 2025

OpenAI. Sora 2.https://openai.com/index/sor a-2/, 2025. 2

2025
[37]

UniDepth: Universal monocular metric depth estimation

Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. InCVPR,
[38]

UniK3D: Universal camera monocular 3d estimation

Luigi Piccinelli, Christos Sakaridis, Mattia Segu, Yung- Hsu Yang, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniK3D: Universal camera monocular 3d estimation. In CVPR, 2025

2025
[39]

UniDepthV2: Universal monocular metric depth estimation made simpler, 2025

Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mat- tia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniDepthV2: Universal monocular metric depth estimation made simpler, 2025. 3

2025
[40]

SDXL: Improving latent diffusion models for high-resolution image synthesis

Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. InICLR, 2024. 2

2024
[41]

Infinite photore- alistic worlds using procedural generation

Alexander Raistrick, Lahav Lipson, Zeyu Ma, Lingjie Mei, Mingzhe Wang, Yiming Zuo, Karhan Kayan, Hongyu Wen, Beining Han, Yihan Wang, Alejandro Newell, Hei Law, Ankit Goyal, Kaiyu Yang, and Jia Deng. Infinite photore- alistic worlds using procedural generation. InCVPR, pages 12630–12641, 2023. 2, 5

2023
[42]

Infinigen indoors: Photorealistic indoor scenes using procedural generation

Alexander Raistrick, Lingjie Mei, Karhan Kayan, David Yan, Yiming Zuo, Beining Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, Zeyu Ma, and Jia Deng. Infinigen indoors: Photorealistic indoor scenes using procedural generation. InCVPR, pages 21783–21794,
[43]

Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE TPAMI, 44(3), 2022

Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE TPAMI, 44(3), 2022. 2

2022
[44]

Accelerating 3d deep learning with pytorch3d

Nikhila Ravi, Jeremy Reizenstein, David Novotny, Tay- lor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501, 2020. 5

work page arXiv 2007
[45]

High-resolution image syn- thesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InCVPR, pages 10684– 10695, 2022. 2, 4, 6

2022
[46]

Worldexplorer: Towards generating fully naviga- ble 3d scenes

Manuel-Andreas Schneider, Lukas H ¨ollein, and Matthias Nießner. Worldexplorer: Towards generating fully naviga- ble 3d scenes. InSIGGRAPH Asia, 2025. 1, 2, 3, 6

2025
[47]

Controlroom3d: Room generation using semantic proxy rooms

Jonas Schult, Sam Tsai, Lukas H ¨ollein, Bichen Wu, Jialiang Wang, Chih-Yao Ma, Kunpeng Li, Xiaofang Wang, Felix Wimbauer, Zijian He, Peizhao Zhang, Bastian Leibe, Peter Vajda, and Ji Hou. Controlroom3d: Room generation using semantic proxy rooms. InCVPR, 2024. 1, 3

2024
[48]

A recipe for generating 3d worlds from a single image

Katja Schwarz, Denis Rozumny, Samuel Rota Bul`o, Lorenzo Porzi, and Peter Kontschieder. A recipe for generating 3d worlds from a single image. InICCV, pages 3520–3530,
[49]

Mvdream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023

Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023. 4

work page arXiv 2023
[50]

Deep unsupervised learning using nonequilibrium thermodynamics

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. InICML, pages 2256–
[51]

Denois- ing diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. InICLR, 2021. 2

2021
[52]

Mochi 1.https://github.com/gen moai/models, 2024

Genmo Team. Mochi 1.https://github.com/gen moai/models, 2024. 2

2024
[53]

Wan: Open and Advanced Large-Scale Video Generative Models

Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025. 2

work page internal anchor Pith review Pith/arXiv arXiv 2025
[54]

360-degree panorama generation from few unregis- tered nfov images

Jionghao Wang, Ziyu Chen, Jun Ling, Rong Xie, and Li Song. 360-degree panorama generation from few unregis- tered nfov images. InACM MM, pages 6811–6821, 2023. 2

2023
[55]

Vggt: Vi- sual geometry grounded transformer

Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Vi- sual geometry grounded transformer. InCVPR, pages 5294– 5306, 2025. 3 10

2025
[56]

Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision

Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. InCVPR, pages 5261–5271, 2025. 3

2025
[57]

Dust3r: Geometric 3d vi- sion made easy

Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InCVPR, pages 20697–20709, 2024. 3

2024
[58]

Motionctrl: A unified and flexible motion controller for video generation

Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InSIGGRAPH, pages 1–11, 2024. 2

2024
[59]

Video models are zero-shot learners and reasoners

Thadd ¨aus Wiedemer, Yuxuan Li, Paul Vicol, Shixiang Shane Gu, Nick Matarese, Kevin Swersky, Been Kim, Priyank Jaini, and Robert Geirhos. Video models are zero-shot learn- ers and reasoners.arXiv preprint arXiv:2509.20328, 2025. 2

work page internal anchor Pith review Pith/arXiv arXiv 2025
[60]

Panodif- fusion: 360-degree panorama outpainting via diffusion

Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. Panodif- fusion: 360-degree panorama outpainting via diffusion. In ICLR, 2024. 2

2024
[61]

Dream-to-recon: Monocular 3d reconstruction with diffusion-depth distillation from single images

Philipp Wulff, Felix Wimbauer, Dominik Muhle, and Daniel Cremers. Dream-to-recon: Monocular 3d reconstruction with diffusion-depth distillation from single images. In ICCV, pages 9352–9362, 2025. 3

2025
[62]

Depth anything: Unleashing the power of large-scale unlabeled data

Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InCVPR, 2024. 2

2024
[63]

Depth Anything V2

Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2.arXiv:2406.09414, 2024. 2

work page internal anchor Pith review Pith/arXiv arXiv 2024
[64]

Layerpano3d: Lay- ered 3d panorama for hyper-immersive scene generation

Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. Layerpano3d: Lay- ered 3d panorama for hyper-immersive scene generation. In SIGGRAPH, pages 1–10, 2025. 1, 6

2025
[65]

Matrix-3d: Omnidirectional explorable 3d world generation.arXiv preprint arXiv:2508.08086, 2025

Zhongqi Yang, Wenhang Ge, Yuqi Li, Jiaqi Chen, Haoyuan Li, Mengyin An, Fei Kang, Hua Xue, Baixin Xu, Yuyang Yin, et al. Matrix-3d: Omnidirectional explorable 3d world generation.arXiv preprint arXiv:2508.08086, 2025. 1, 2, 3, 6

work page arXiv 2025
[66]

Cogvideox: Text-to-video diffusion models with an expert transformer

Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. InICLR, 2025. 2

2025
[67]

gsplat: An open-source library for gaussian splatting.JMLR, 26(34): 1–17, 2025

Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, and Angjoo Kanazawa. gsplat: An open-source library for gaussian splatting.JMLR, 26(34): 1–17, 2025. 3

2025
[68]

Metric3d: Towards zero-shot metric 3d prediction from a single image

Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. InICCV, pages 9043–9053, 2023. 3

2023
[69]

Wonderjourney: Going from anywhere to everywhere

Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, De- qing Sun, Noah Snavely, Jiajun Wu, et al. Wonderjourney: Going from anywhere to everywhere. InCVPR, pages 6658– 6667, 2024. 3

2024
[70]

Wonderworld: Interactive 3d scene generation from a single image

Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. InCVPR, pages 5916–5926,
[71]

Tra- jectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models

Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan. Tra- jectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models. InICCV, 2025. 2

2025
[72]

Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.IEEE TPAMI,

Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.IEEE TPAMI,
[73]

Adding conditional control to text-to-image diffusion models

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 2

2023
[74]

Matrix-game: Interactive world foundation model.arXiv preprint arXiv:2506.18701, 2025

Yifan Zhang, Chunli Peng, Boyang Wang, Puyi Wang, Qingcheng Zhu, Fei Kang, Biao Jiang, Zedong Gao, Eric Li, Yang Liu, et al. Matrix-game: Interactive world foundation model.arXiv preprint arXiv:2506.18701, 2025. 2

work page arXiv 2025
[75]

Layoutdiffusion: Controllable diffu- sion model for layout-to-image generation

Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffu- sion model for layout-to-image generation. InCVPR, pages 22490–22499, 2023. 2

2023
[76]

Holodreamer: Holistic 3d panoramic world generation from text descriptions.arXiv preprint arXiv:2407.15187, 2024

Haiyang Zhou, Xinhua Cheng, Wangbo Yu, Yonghong Tian, and Li Yuan. Holodreamer: Holistic 3d panoramic world generation from text descriptions.arXiv preprint arXiv:2407.15187, 2024. 2, 3 11 Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas Supplementary Material A. Overview This supplementary material will provideadditional re- sultsin ...

work page arXiv 2024
[77]

Gen.n= 8novel panosn×30s WorldExplorer 7h
[78]

MapAnything 20s Matrix-3D 40min
[79]

Filtering 2min LayerPano3D 32min
[80]

textureless regions with cameras facing downwards

3DGS optimization 8min Ours 15min Table 3.Inference time - Panorama to 3D scene. textureless regions with cameras facing downwards. This is fixed by again sampling more views from the different generated panoramas (d). Runtime analysis.While our pipeline is not optimized for inference speed yet, it is significantly faster than the baselines, as shown in d...

2048