Pith. sign in

REVIEW 3 major objections 6 minor 3 cited by

DreamCube: 3D Panorama Generation via Multi-plane Synchronization

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single synchronization of self-attention, convolutions, and normalization lets existing 2D diffusion models generate seam-consistent cubemap panoramas, and DreamCube extends this to joint RGB-D and 3D scene generation.

desk verdict A practical, mostly sound recipe for lifting 2D diffusion models to cubemaps; the geometry claims need an evaluation fix. read the letter →

arxiv 2506.17206 v1 pith:SD3A37LM submitted 2025-06-20 cs.GR cs.CVcs.LG

Yukun Huang , Yanning Zhou , Jianan Wang , Kaiyi Huang , Xihui Liu This is my paper
classification cs.GRcs.CVcs.LG
keywords panoramagenerationcubemapmulti-planesynchronizationRGB-Ddiffusionpanoramicdepthestimation3Dscenelatentmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that pre-trained 2D diffusion models can be turned into seamless omnidirectional generators without fine-tuning or field-of-view overlap, by synchronizing three spatial operators across the six faces of a cube map. On top of that, it introduces DreamCube, a joint RGB-D cubemap diffusion model that generates appearance and depth from a single view. If true, this would remove two known obstacles in panorama generation: polar distortion from equirectangular projections and seam artifacts from separately generated views. The resulting RGB-D cubemaps lift directly into 3D point clouds, meshes, and Gaussian splats, making single-view-to-3D scene generation practical in seconds.

What carries the argument

The central mechanism is multi-plane synchronization, which adapts the spatial operators of a 2D diffusion U-Net to a six-face cubemap representation: synced self-attention changes the token layout from (BM)×(HW)×C to B×(MHW)×C so attention sees all faces simultaneously; synced 2D convolutions replace zero-padding with pixels projected from adjacent cube faces; and synced group normalization computes statistics across all planes rather than per view. DreamCube adds joint RGB-D latents with masked noise injection, Z-depth rescaling to create a margin for out-of-range depth values, and XYZ positional encoding based on normalized unit-sphere coordinates that avoids the discontinuity artifacts of UV encoding. Together, these operators carry the argument that translation equivariance in the omnidirectional domain is sufficient for seam consistency.

What would settle it

Re-run the depth comparison on generated panoramas using human-annotated or sensor-captured ground-truth depth for the same scenes, then check whether DreamCube still beats PanoDiffusion and the depth-estimation baseline on δ-1.25, AbsRel, RMSE, and MAE.

Watch

Extended reading notes

Core claim

The paper's central claim is that existing 2D diffusion models can generate seam-consistent panoramic multi-planes without requiring fine-tuning or FoV overlapping. The reason standard multi-plane generation fails is that neural operators such as self-attention, convolutions, and group normalization are not translation-equivalent on the omnidirectional domain: boundary pixels of one cube face are padded with zeros instead of information from adjacent faces, and attention never sees tokens across faces. Multi-plane synchronization fixes this by reshaping attention tokens to span all six faces at once, replacing zero-padding in convolutions with geometrically projected pixels from neighboring faces, and computing group-normalization statistics globally. DreamCube then applies this synchronization to a masked RGB-D cubemap diffusion model, uses Z-depth rather than Euclidean depth to stay closer to 2D image priors, adds XYZ positional encoding from unit-sphere geometry, and jointly denoises RGB and depth latents. The paper reports that this outperforms equirectangular-based RGB-D panorama methods on Structured3D for both appearance and depth metrics, and that the same synchronization makes panoramic depth estimation seamless.

Load-bearing premise

The reported depth superiority depends on the pseudo ground truth from a monocular depth estimator being an unbiased referee; if that estimator is biased toward the kind of geometry DreamCube tends to produce, the depth advantage could be an artifact of the evaluation pipeline.

Editorial extensions

If this is right

  • Pre-trained 2D diffusion models can be repurposed as omnidirectional generators at inference time, with no weight updates, removing the need for large curated panorama fine-tuning datasets.
  • Because no field-of-view overlap is needed, the full resolution of each cube face is used for content and compute is not spent rendering duplicate regions.
  • Joint RGB-D cubemap generation lifts directly to a metric 3D point cloud, mesh, or Gaussian representation, so a single input view becomes a 3D scene in about ten seconds.
  • The same operator synchronization transfers to monocular depth estimation, removing seam discontinuities in 360-degree depth without retraining the depth model.
  • The method generalizes to out-of-domain inputs such as generated images and real sensor captures, although extreme elevation angles remain a known failure mode.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A likely extension that the paper hints at but does not test is that operator synchronization should transfer to transformer-based diffusion backbones, making the recipe architecture-agnostic beyond U-Nets.
  • If synchronization restores translation equivariance in the omnidirectional domain, the same idea should reduce seam artifacts in other multi-view settings such as camera arrays, light fields, or tiled high-resolution generation, where overlapping frusta are currently used.
  • A testable refinement of the depth claim is to swap the pseudo ground-truth depth estimator for an independent estimator or measured depth; if DreamCube's depth advantage in Table 3 shrinks, part of that advantage is estimator-specific rather than geometric.
  • The depth-rescaling margin (sampled in [0.2, 1.0] at training, fixed to 0.6 at inference) acts as an unexplored user knob that could trade depth range against consistency in very deep scenes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Multi-plane Synchronization, a modification of pre-trained 2D diffusion models that adapts self-attention, convolution padding, and group normalization to operate jointly over the six faces of a cubemap. The authors argue that this enables seam-consistent cubemap generation without fine-tuning or overlapping fields of view, and they build DreamCube on top of it: a fine-tuned Stable Diffusion v2 model that generates RGB-D cubemaps from a single RGB-D view using XYZ positional encoding and Z-depth rather than Euclidean depth. Experiments cover RGB panorama generation (FID/IS), depth panorama generation (delta-1.25, AbsRel, RMSE, MAE against Depth Anything v2 pseudo-labels), panoramic depth estimation, 3D scene reconstruction, ablations, generalization/robustness, and efficiency.

Significance. The core architectural observation is plausible and potentially useful: synchronizing a small set of spatial operators extends 2D diffusion priors to cubemaps without retraining or overlapping FoVs, and the qualitative demonstrations on SD2, SDXL, and Marigold are compelling. The paper contributes a systematic ablation of the three synchronized operators and an efficiency analysis, which are valuable. However, the quantitative claim of superior geometry over prior RGB-D panorama methods rests on a depth-evaluation protocol that is not shown to be fair: the pseudo-labels are affine-invariant and scale-sensitive metrics are used without an explicit alignment, and the compared methods output different depth parametrizations. Until this is fixed, the geometry comparison and the 'accurate geometry' claim are not established.

major comments (3)
  1. [Section 5.3, Table 3] The depth evaluation protocol is not sufficiently specified. The reference depth comes from Depth Anything v2, which produces affine-invariant disparity-like predictions, but the paper does not state how scale and shift were aligned before computing AbsRel, RMSE, and MAE; these metrics are not invariant to such an alignment. In addition, DreamCube is designed to output Z-depth while LDM3D-Pano and PanoDiffusion output Euclidean depth, and the paper motivates this distinction in Figure 2; if projected perspective depths are compared without converting to a common parametrization, DreamCube has a structural advantage because the DA2 reference is based on ray-axis depth. The training depth labels are also produced by a learned pipeline (Depth Anywhere + PromptDA), so the evaluation reference from another learned estimator may share common biases. Please report the exact alignment procedure, use scale/shift-invariant metrics or align all outputs with a single least-squares fit, and validate against ground-truth depth on Structured3D if possible.
  2. [Section 3.2, Table 4] The central claim of seam-consistent multi-plane generation without fine-tuning or FoV overlap is supported only qualitatively, through Figures 3 and 10. No numeric seam-consistency metric (e.g., discontinuity at face boundaries, color or geometry agreement across adjacent faces) is reported. Because this is the paper's main distinguishing contribution relative to CubeDiff and MVDiffusion, quantitative evidence of seam consistency would strengthen the claim considerably.
  3. [Tables 2, 3, and 4] No error bars or multiple-run statistics are reported. FID differences of a few points and depth differences of 0.01-0.03 in Table 3 may be within run-to-run variation, especially given the noisy pseudo-label protocol. Please report confidence intervals or results over multiple seeds, and state the number of generated samples and the resolution used for FID/IS computation.
minor comments (6)
  1. [Section 3.2] The terms 'translation-equivalence' and 'translation invariance' are used interchangeably in the analysis; the mathematical statement should be made consistent.
  2. [Section 3.2] There is a typo in the first paragraph: 'U-Net-based iffusion models' should read 'diffusion models'.
  3. [Section 5.4] Panoramic depth estimation is only evaluated qualitatively. A quantitative comparison with DAC and Depth Anywhere on a standard benchmark would be needed to support the claim of 'minimal performance loss'.
  4. [Section 4.2] The depth rescaling parameter s is sampled from [0.2, 1.0] at training and fixed at 0.6 at inference, but no sensitivity analysis of s is provided, although it is a free parameter of the method.
  5. [Section 5.2] The general dataset mixes synthetic and real HDRIs and uses pseudo-depth labels from a learned pipeline; the paper does not analyze how label noise from this depth-annotation pipeline affects the trained model.
  6. [Table 2] The FID/IS computation is not fully specified; including the number of generated samples per method and the Inception checkpoint used would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DreamCube's synchronization is an architectural contribution and its evaluations use external datasets and external pseudo-label models.

full rationale

The paper's two central claims are (1) that synchronizing self-attention, convolutions, and group normalization across cubemap faces lets pre-trained 2D diffusion models generate seam-consistent multi-planes without fine-tuning or FoV overlap, and (2) that DreamCube extends this to joint RGB-D generation. Neither claim is defined in terms of its own evaluation. Section 3.2 gives a constructive description of the synced operators: attention tokens are reshaped from (BM) x (HW) x C to B x (M HW) x C, convolutions replace zero-padding with geometrically projected pixels from adjacent faces, and group norm statistics are computed globally across planes. These are architectural modifications, not fitted quantities. DreamCube's training objective in Section 4.1 is a standard v-prediction loss on RGB and depth latents, and the depth rescaling parameter s is sampled in [0.2, 1.0] during training and fixed to 0.6 at inference; no reported metric reduces to the value of s. Evaluations are anchored to external resources: Structured3D and SUN360 for RGB metrics, and Depth Anything v2 as pseudo ground truth for depth metrics. The depth evaluation protocol is open to validity concerns, since DA2 predicts affine-invariant disparity and the paper does not state its scale/shift alignment, and since DreamCube outputs Z-depth while some baselines output Euclidean depth. However, this is an evaluation-design issue, not circularity: DA2 was not used to fit DreamCube's parameters, and for the Structured3D split used in Table 3 the model is trained on dataset-provided ground-truth depth rather than on DA2 pseudo-labels. The few references involving the current authors (DDP [20] and Hyperhuman [34]) appear only as context citations in related-work surveys and are not load-bearing premises; no uniqueness theorem is imported from the authors' own prior work. Therefore no step in the paper's derivation chain is equivalent by construction to its inputs, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The method depends on several unproven modeling choices: the transferability of 2D diffusion priors to cubemap faces, the diagnosis that seams stem from translational inequivalence of spatial operators, the preference for Z-depth over Euclidean depth, and the use of Depth Anything v2 as pseudo ground truth. The depth rescaling scale s is a tuned free parameter. No invented entities are introduced.

free parameters (1)
  • Depth rescaling scale s = 0.6 at inference; sampled from U[0.2, 1.0] during training
    Selected to keep generated depth values inside the diffusion VAE's in-domain range; it directly controls the scale of all generated depth and downstream 3D reconstruction.
assumptions (4)
  • domain assumption Six cube faces of a 360-degree panorama are close enough to perspective images that pre-trained 2D diffusion priors transfer to the cubemap domain.
    Used throughout Sections 3.2 and 4.2 to justify extending Stable Diffusion and Marigold without retraining the image generator.
  • domain assumption Seam artifacts in multi-plane generation are caused primarily by translation inequivalence of spatial operators (zero padding, per-face attention, per-face group norm) rather than by insufficient data or global scene priors.
    Motivates the proposed operator synchronization; the paper provides qualitative evidence but no formal proof (Section 3.2).
  • domain assumption Depth Anything v2 provides a reliable pseudo-ground-truth for evaluating depth of generated panoramas.
    The depth evaluation protocol in Section 5.3 compares against Depth Anything v2 predictions; if this estimator is biased, the reported depth metrics are not trustworthy.
  • ad hoc to paper Z-distance is more compatible with 2D diffusion image priors than Euclidean distance.
    Modeling choice unique to this paper (Section 4.2, Figure 2) that underpins the joint RGB-D design and the claimed depth advantages.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DreamCube: 3D Panorama Generation via Multi-plane Synchronization." pith.science (2026). https://pith.science/paper/SD3A37LM

@misc{pith2026250617206,
  author       = {Pith},
  title        = {Pith review of: DreamCube: 3D Panorama Generation via Multi-plane Synchronization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SD3A37LM}},
  note         = {Machine review of arXiv:2506.17206}
}
read the original abstract

3D panorama synthesis is a promising yet challenging task that demands high-quality and diverse visual appearance and geometry of the generated omnidirectional content. Existing methods leverage rich image priors from pre-trained 2D foundation models to circumvent the scarcity of 3D panoramic data, but the incompatibility between 3D panoramas and 2D single views limits their effectiveness. In this work, we demonstrate that by applying multi-plane synchronization to the operators from 2D foundation models, their capabilities can be seamlessly extended to the omnidirectional domain. Based on this design, we further introduce DreamCube, a multi-plane RGB-D diffusion model for 3D panorama generation, which maximizes the reuse of 2D foundation model priors to achieve diverse appearances and accurate geometry while maintaining multi-view consistency. Extensive experiments demonstrate the effectiveness of our approach in panoramic image generation, panoramic depth estimation, and 3D scene generation.

Figures

Figures reproduced from arXiv: 2506.17206 by the authors.

Figure 1
Figure 1. In this work, we introduce Multi-plane Synchronization to generalize 2D diffusion models to multi-plane omnidirectional representations (i.e., cubemaps), and DreamCube for RGB-D cubemap generation. The proposed approaches can be applied to different tasks including RGB-D panorama generation, panorama depth estimation, and 3D scene generation. Abstract 3D panorama synthesis is a promising yet challenging task that de… view at source ↗
Figure 2
Figure 2. Motivation. Previous works [50, 55] on RGB-D panorama generation is based on equirectangular representations and only supports Euclidean depth instead of the more popular Z-depth. However, the distribution of Euclidean depth is quite different from that of RGB images (e.g., circles on flat surfaces as highlighted by orange dashed boxes), which hinders the use of pre-trained 2D diffusion priors. Multi-plane methods [… view at source ↗
Figure 3
Figure 3. Results of Multi-plane Synchronization on pre-trained 2D diffusion models: SD2 [41], SDXL [39], and Marigold [22]. Our method enables 2D diffusion to generate multi-plane synchronized omnidirectional image representations without fine-tuning. are architecture-agnostic and can be adapted to other diffu￾sion frameworks, such as DiT [37] and alternative panorama representation with minimal modifications. Analysis of sp… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Training and inference framework of DreamCube for RGB-D cube map generation. At training time, RGB-D cube faces are encoded by synced VAE and injected masked Gaussian noises, obtaining image and depth latents. These latents are concatenated with positional encoding and…
Figure 5
Figure 5. Figure 5: Comparison between UV encoding and XYZ encoding [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of the proposed DreamCube on RGB-D panorama generation compared with RGB-D panorama generation methods: LDM3D-Pano [50] and PanoDiffusion [55]. The green dashed boxes highlight the input image condition. Marigold + Multi-plane Sync. (Ours) w/ multi-…
Figure 7
Figure 7. Figure 7: Qualitative results of the proposed Multi-plane Synchronization on panoramic depth estimation compared with recent panoramic depth estimation methods: Depth Any Camera (DAC) [12] and Depth Anywhere [54]. 5.4. Panoramic Depth Estimation Our proposed Multi-plane Synchron…
Figure 8
Figure 8. Figure 8: Panorama-to-3D scene reconstruction. Based on the RGB-D cubemap generated by DreamCube, we can reconstruct the corresponding 3D scenes in seconds and obtain both 3D mesh and 3D Gaussian [23] representations. Generated RGB-D Images 3D Point Cloud From Equirectangular 3D…
Figure 9
Figure 9. Figure 9: Qualitative comparison of 3D point clouds recon￾structed from equirectangular-based and cubemap-based RGB-D panoramas. Equirectangular panoramas produce an uneven, ring￾shaped 3D point distribution dense near the poles, while cubemap panoramas yield a more uniform and …
Figure 10
Figure 10. Figure 10: Ablation analysis of Multi-plane Synchronization. We adopt Stable-Diffusion v2 as baseline model for multi-plane genera￾tion, and the text prompt used is “The vast cosmos in the style of Van Gogh, with swirling patterns and vibrant colors.” [PITH_FULL_IMAGE:figures/f…
Figure 11
Figure 11. Figure 11: Ablation analysis of XYZ Positional Encoding. We present the qualitative results of the back view of cubemap, where the UV positional encoding introduces discontinuous numerical steps. This leads to line artifacts (Case 1) and incoherent visual contents (Case 2), as i…
Figure 12
Figure 12. Figure 12: Out-domain RGB-D panorama generation. The RGB-D inputs are obtained by Flux.1-dev [27] and Depth Anything v2 [59]. DreamCube demonstrates generalization ability on diverse inputs, maintaining high-quality RGB appearance and geometric consistency. panoramas, demonstrat…
Figure 13
Figure 13. Figure 13: Robustness analysis of DreamCube to out-domain RGB-D inputs from real world and extreme viewing angles. and latency (ms) by 113.1% than no synchronization (“No Sync.”). This accounts for 86.0% of the latency cost and almost 100% of the TFLOPs cost incurred by our appr…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EmoSpace: Immersive Affective Image Generation Guided by Fine-Grained Emotion Prototypes

    cs.CV 2026-02 conditional novelty 6.0 of 10

    EmoSpace generates emotion-controlled images and VR panoramas via a dynamic bank of 1,024 CLIP-space emotion prototypes, reporting higher fine-grained emotional alignment than baseline diffusion models.

  2. OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenes

    cs.CV 2025-10 conditional novelty 6.0 of 10

    OmniX trains separate LoRA adapters on FLUX.1-dev so one framework handles panorama generation, intrinsic perception (depth, normals, albedo, roughness, metallic), and completion, then feeds the maps into PBR-ready 3D scenes.

  3. Gimbal360: Canonicalizing Planar Diffusion for Spherical Panorama Completion

    cs.CV 2026-03 reject novelty 5.0 of 10

    Gimbal360 completes 360° panoramas from unposed perspective images by rigidly auto-leveling inputs and training diffusion with a Siamese shift-equivariance loss to preserve ERP seam continuity.

Reference graph

Works this paper leans on

65 extracted references · 44 canonical work pages · cited by 3 Pith papers

  1. [1]

    Diverse Plausible 360-Degree Image Outpainting for Ef- ficient 3DCG Background Creation

    Naofumi Akimoto, Yuhi Matsuo, and Yoshimitsu Aoki. Diverse Plausible 360-Degree Image Outpainting for Ef- ficient 3DCG Background Creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 1, 2, 7

    work page 2022

  2. [2]

    MultiDiffusion: Fusing Diffusion Paths for Controlled Im- age Generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. MultiDiffusion: Fusing Diffusion Paths for Controlled Im- age Generation. In International Conference on Machine Learning, pages 1737–1752. PMLR, 2023. 3

    work page 2023

  3. [3]

    ARK- 12 itScenes - A Diverse Real-World Dataset for 3D Indoor Scene Understanding Using Mobile RGB-D Data

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. ARK- 12 itScenes - A Diverse Real-World Dataset for 3D Indoor Scene Understanding Using Mobile RGB-D Data. In Neural Information Processing Systems, 2021. 11, 12

    work page 2021

  4. [4]

    Text2light: Zero-shot text-driven hdr panorama generation

    Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Text2light: Zero-shot text-driven hdr panorama generation. ACM Trans- actions on Graphics (TOG), 41(6):1–16, 2022. 2, 3

    work page 2022

  5. [5]

    Depth esti- mation via affinity learned with convolutional spatial propa- gation network

    Xinjing Cheng, Peng Wang, and Ruigang Yang. Depth esti- mation via affinity learned with convolutional spatial propa- gation network. In Proceedings of the European conference on computer vision (ECCV), pages 103–119, 2018. 7

    work page 2018

  6. [6]

    Diffusiondepth: Diffusion denoising approach for monocular depth estima- tion

    Yiquan Duan, Xianda Guo, and Zheng Zhu. Diffusiondepth: Diffusion denoising approach for monocular depth estima- tion. In European Conference on Computer Vision , pages 432–449. Springer, 2024. 3

    work page 2024

  7. [7]

    Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture

    David Eigen and Rob Fergus. Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture. In Proceedings of the IEEE inter- national conference on computer vision , pages 2650–2658,

  8. [8]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning,

Show all 65 references
  1. [9]

    Diffusion360: Seamless 360 Degree Panoramic Im- age Generation based on Diffusion Models

    Mengyang Feng, Jinlin Liu, Miaomiao Cui, and Xuansong Xie. Diffusion360: Seamless 360 Degree Panoramic Im- age Generation based on Diffusion Models. arXiv preprint arXiv:2311.13141, 2023. 1, 2, 7

    2023 arXiv

  2. [10]

    Geowiz- ard: Unleashing the diffusion priors for 3d geometry esti- mation from a single image

    Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowiz- ard: Unleashing the diffusion priors for 3d geometry esti- mation from a single image. In European Conference on Computer Vision, pages 241–258. Springer, 2024. 2, 3

    2024

  3. [11]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 3

    2020

  4. [12]

    Depth any camera: Zero-shot met- ric depth estimation from any camera

    Yuliang Guo, Sparsh Garg, S Mahdi H Miangoleh, Xinyu Huang, and Liu Ren. Depth any camera: Zero-shot met- ric depth estimation from any camera. arXiv preprint arXiv:2501.02464, 2025. 3, 7, 8

    2025 arXiv

  5. [13]

    hdri skies. HDRIs. https://hdri-skies.com/, accessed 02/2025. 6

    2025

  6. [14]

    hdri skies. HDRIs. https://www.ihdri.com/hdri-skies- outdoor/, accessed 02/2025. 6

    2025

  7. [15]

    Lotus: Diffusion-based visual foundation model for high-quality dense prediction

    Jing He, Haodong Li, Wei Yin, Yixun Liang, Leheng Li, Kaiqiang Zhou, Hongbo Zhang, Bingbing Liu, and Ying- Cong Chen. Lotus: Diffusion-based visual foundation model for high-quality dense prediction. arXiv preprint arXiv:2409.18124, 2024. 2

    2024 arXiv

  8. [16]

    ScaleCrafter: Tuning-free Higher- Resolution Visual Generation with Diffusion Models

    Yingqing He, Shaoshu Yang, Haoxin Chen, Xiaodong Cun, Menghan Xia, Yong Zhang, Xintao Wang, Ran He, Qifeng Chen, and Ying Shan. ScaleCrafter: Tuning-free Higher- Resolution Visual Generation with Diffusion Models. In In- ternational Conference on Learning Representations , 2024. 4

    2024

  9. [17]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 7

    2017

  10. [18]

    Denoising Diffu- sion Probabilistic Models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffu- sion Probabilistic Models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 6

    2020

  11. [19]

    Joint learning of depth and appearance for portrait image an- imation

    Xinya Ji, Gaspard Zoss, Prashanth Chandran, Lingchen Yang, Xun Cao, Barbara Solenthaler, and Derek Bradley. Joint learning of depth and appearance for portrait image an- imation. arXiv preprint arXiv:2501.08649, 2025. 3

    2025 arXiv

  12. [20]

    Ddp: Diffusion model for dense visual prediction

    Yuanfeng Ji, Zhe Chen, Enze Xie, Lanqing Hong, Xihui Liu, Zhaoqiang Liu, Tong Lu, Zhenguo Li, and Ping Luo. Ddp: Diffusion model for dense visual prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 21741–21752, 2023. 3

    2023

  13. [21]

    Cubediff: Repurposing diffusion-based image models for panorama generation, 2025

    Nikolai Kalischek, Michael Oechsle, Fabian Manhardt, Philipp Henzler, Konrad Schindler, and Federico Tombari. Cubediff: Repurposing diffusion-based image models for panorama generation, 2025. 1, 2, 3, 4, 6

    2025

  14. [22]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024...

    2024

  15. [23]

    3D Gaussian Splatting for Real-Time Radiance Field Rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics, 42(4), 2023. 6, 9

    2023

  16. [24]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. arXiv preprint arXiv:2408.12569, 2024. 3

    2024 arXiv

  17. [25]

    SPEC: Seeing People in the Wild With an Estimated Camera

    Muhammed Kocabas, Chun-Hao P Huang, Joachim Tesch, Lea M ¨uller, Otmar Hilliges, and Michael J Black. SPEC: Seeing People in the Wild With an Estimated Camera. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11035–11045, 2021. 6

    2021

  18. [26]

    Orchid: Image latent diffusion for joint appearance and geometry generation

    Akshay Krishnan, Xinchen Yan, Vincent Casser, and Abhijit Kundu. Orchid: Image latent diffusion for joint appearance and geometry generation. arXiv preprint arXiv:2501.13087,

    arXiv

  19. [27]

    Flux.1-dev

    Black Forest Labs. Flux.1-dev. https : / / huggingface . co / black - forest - labs / FLUX.1-dev, 2025. Accessed: 2025-01-19. 10, 11

    2025

  20. [28]

    Black Forest Labs. Flux. https://github.com/ black- forest- labs/flux, 2025. Accessed: 2025- 01-19. 1

    2025

  21. [29]

    Depth and surface normal estimation from monocular images using regression on deep features and hi- erarchical crfs

    Bo Li, Chunhua Shen, Yuchao Dai, Anton Van Den Hengel, and Mingyi He. Depth and surface normal estimation from monocular images using regression on deep features and hi- erarchical crfs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1119–1127,

  22. [30]

    PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and- Language Navigation

    Jialu Li and Mohit Bansal. PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and- Language Navigation. In International Conference on Neural Information Processing Systems, 2023. 1, 3

    2023

  23. [31]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 7

    2023

  24. [32]

    Prompting depth anything for 4k resolution accurate metric depth estimation

    Haotong Lin, Sida Peng, Jingxiao Chen, Songyou Peng, Ji- aming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou, and Bingyi Kang. Prompting depth anything for 4k resolution accurate metric depth estimation. arXiv preprint arXiv:2412.14015, 2024. 7

    2024

  25. [33]

    Panofree: Tuning-free holistic multi- view image generation with cross-view self-guidance

    Aoming Liu, Zhong Li, Zhang Chen, Nannan Li, Yi Xu, and Bryan A Plummer. Panofree: Tuning-free holistic multi- view image generation with cross-view self-guidance. In European Conference on Computer Vision, pages 146–164. Springer, 2024. 3

    2024

  26. [34]

    Hyperhuman: Hyper-realistic human gener- ation with latent structural diffusion

    Xian Liu, Jian Ren, Aliaksandr Siarohin, Ivan Skorokhodov, Yanyu Li, Dahua Lin, Xihui Liu, Ziwei Liu, and Sergey Tulyakov. Hyperhuman: Hyper-realistic human gener- ation with latent structural diffusion. arXiv preprint arXiv:2310.08579, 2023. 3

    arXiv

  27. [35]

    Depthlab: From partial to complete

    Zhiheng Liu, Ka Leong Cheng, Qiuyu Wang, Shuzhe Wang, Hao Ouyang, Bin Tan, Kai Zhu, Yujun Shen, Qifeng Chen, and Ping Luo. Depthlab: From partial to complete. arXiv preprint arXiv:2412.18153, 2024. 5

    2024

  28. [36]

    CubeGAN: Omnidirec- tional Image Synthesis Using Generative Adversarial Net- works

    Christopher May and Daniel Aliaga. CubeGAN: Omnidirec- tional Image Synthesis Using Generative Adversarial Net- works. In Computer Graphics Forum, pages 213–224. Wiley Online Library, 2023. 3, 4

    2023

  29. [37]

    Scalable Diffusion Models with Transformers

    William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  30. [38]

    Texture from Humus

    Emil Persson. Texture from Humus. https://www.humus.name/index.php?page=Textures, ac- cessed 02/2025. 6

    2025

  31. [39]

    SDXL: Improving Latent Diffusion Mod- els for High-Resolution Image Synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: Improving Latent Diffusion Mod- els for High-Resolution Image Synthesis. In International Conference on Learning Representations, 2024. 4

    2024

  32. [40]

    polyhaven.com. HDRIs. https://polyhaven.com/hdris, ac- cessed 02/2025. 6

    2025

  33. [41]

    High-Resolution Image Synthesis with Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 1, 2, 3, 4, 5, 6, 11

    2022

  34. [42]

    Progressive Distillation for Fast Sampling of Diffusion Models

    Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffusion Models. In International Con- ference on Learning Representations, 2022. 5

    2022

  35. [43]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016. 7

    2016

  36. [44]

    Saurabh Saxena, Charles Herrmann, Junhwa Hur, Abhishek Kar, Mohammad Norouzi, Deqing Sun, and David J. Fleet. The surprising effectiveness of diffusion models for optical flow and monocular depth estimation. InThirty-seventh Con- ference on Neural Information Processing System...

    2023

  37. [45]

    Monocular depth estimation using diffusion models

    Saurabh Saxena, Abhishek Kar, Mohammad Norouzi, and David J Fleet. Monocular depth estimation using diffusion models. arXiv preprint arXiv:2302.14816, 2023. 3

    2023 arXiv

  38. [46]

    MVDream: Multi-view Diffusion for 3D Generation

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi-view Diffusion for 3D Generation. In International Conference on Learning Rep- resentations, 2024. 4

    2024

  39. [47]

    Hdr environment map estimation for real-time augmented reality

    Gowri Somanath and Daniel Kurz. Hdr environment map estimation for real-time augmented reality. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11298–11306, 2021. 1

    2021

  40. [48]

    Denois- ing Diffusion Implicit Models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing Diffusion Implicit Models. In International Conference on Learning Representations, 2021. 6

    2021

  41. [49]

    LDM3D-VR: Latent Diffusion Model for 3D VR

    Gabriela Ben Melech Stan, Diana Wofk, Estelle Aflalo, Shao-Yen Tseng, Zhipeng Cai, Michael Paulitsch, and Va- sudev Lal. LDM3D-VR: Latent Diffusion Model for 3D VR. arXiv preprint arXiv:2311.03226, 2023. 2, 3, 5, 7

    2023 arXiv

  42. [50]

    LDM3D: Latent Diffusion Model for 3D

    Gabriela Ben Melech Stan, Diana Wofk, Scottie Fox, Alex Redden, Will Saxton, Jean Yu, Estelle Aflalo, Shao-Yen Tseng, Fabio Nonato, Matthias Muller, et al. LDM3D: Latent Diffusion Model for 3D. arXiv preprint arXiv:2305.10853, 2023. 2, 7, 8

    2023 arXiv

  43. [51]

    MVDiffusion: Enabling Holistic Multi- view Image Generation with Correspondence-Aware Diffu- sion

    Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. MVDiffusion: Enabling Holistic Multi- view Image Generation with Correspondence-Aware Diffu- sion. In Proceedings of the International Conference on Neu- ral Information Processing Systems, 2023. 1, 2, 3, 7

    2023

  44. [52]

    Customizing 360-degree panoramas through text-to-image diffusion models

    Hai Wang, Xiaoyu Xiang, Yuchen Fan, and Jing-Hao Xue. Customizing 360-degree panoramas through text-to-image diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 4933–4943, 2024. 1, 3

    2024

  45. [53]

    360-degree panorama generation from few unregis- tered nfov images

    Jionghao Wang, Ziyu Chen, Jun Ling, Rong Xie, and Li Song. 360-degree panorama generation from few unregis- tered nfov images. In Proceedings of the 31st ACM Interna- tional Conference on Multimedia , pages 6811–6821, 2023. 1, 3

    2023

  46. [54]

    Depth Any- where: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation

    Ning-Hsu Albert Wang and Yu-Lun Liu. Depth Any- where: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation. Advances in Neural Information Processing Systems , 37: 127739–127764, 2024. 7, 8

    2024

  47. [55]

    PanoD- iffusion: 360-degree Panorama Outpainting via Diffusion

    Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. PanoD- iffusion: 360-degree Panorama Outpainting via Diffusion. In International Conference on Learning Representations ,

  48. [56]

    Recognizing scene viewpoint using panoramic place representation

    Jianxiong Xiao, Krista A Ehinger, Aude Oliva, and Anto- nio Torralba. Recognizing scene viewpoint using panoramic place representation. In 2012 IEEE conference on computer vision and pattern recognition , pages 2695–2702. IEEE,

    2012

  49. [57]

    Pad-net: Multi-tasks guided prediction-and-distillation net- work for simultaneous depth estimation and scene parsing

    Dan Xu, Wanli Ouyang, Xiaogang Wang, and Nicu Sebe. Pad-net: Multi-tasks guided prediction-and-distillation net- work for simultaneous depth estimation and scene parsing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 675–684, 2018. 3

    2018

  50. [58]

    Dreamspace: Dreaming your room space with text-driven panoramic texture propa- gation

    Bangbang Yang, Wenqi Dong, Lin Ma, Wenbo Hu, Xiao Liu, Zhaopeng Cui, and Yuewen Ma. Dreamspace: Dreaming your room space with text-driven panoramic texture propa- gation. In 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pages 650–660. IEEE, 2024. 1

    2024

  51. [59]

    Depth Any- thing V2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth Any- thing V2. Advances in Neural Information Processing Sys- tems, 37:21875–21911, 2024. 7, 10, 11

    2024

  52. [60]

    Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion

    Weicai Ye, Chenhao Ji, Zheng Chen, Junyao Gao, Xiaoshui Huang, Song-Hai Zhang, Wanli Ouyang, Tong He, Cairong Zhao, and Guofeng Zhang. Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion. arXiv preprint arXiv:2410.24203, 2024. 1, 3

    arXiv

  53. [61]

    Wonderworld: Interactive 3d scene generation from a single image

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. arXiv preprint arXiv:2406.09394, 2024. 6

    2024 arXiv

  54. [62]

    Taming Stable Diffusion for Text to 360 Panorama Image Generation

    Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xi- aoshui Huang, Dinh Phung, Wanli Ouyang, and Jianfei Cai. Taming Stable Diffusion for Text to 360 Panorama Image Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6347– 6...

    2024

  55. [63]

    Unleashing text-to-image diffu- sion models for visual perception

    Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, and Jiwen Lu. Unleashing text-to-image diffu- sion models for visual perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5729–5739, 2023. 3

    2023

  56. [64]

    Structured3D: A Large Photo-Realistic Dataset for Structured 3D Modeling

    Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou. Structured3D: A Large Photo-Realistic Dataset for Structured 3D Modeling. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part IX 16 , pages 519–5...

    2020

  57. [65]

    Point cloud matters: Rethinking the impact of different observation spaces on robot learn- ing

    Haoyi Zhu, Yating Wang, Di Huang, Weicai Ye, Wanli Ouyang, and Tong He. Point cloud matters: Rethinking the impact of different observation spaces on robot learn- ing. Advances in Neural Information Processing Systems , 37:77799–77830, 2024. 1 15

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.