Pith. sign in

REVIEW 5 major objections 6 minor 86 references

CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generation

T0 review · 5 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CGGS claims ego-centric text-to-3D generation succeeds by chaining a consistency-augmented multi-view diffusion generator, a flow-tracked depth layout, and a mutual-information-depth Gaussian refiner, outperforming prior text-to-3D baseline

desk verdict A sensible systems recipe for ego-centric text-to-3D, but the 'accurate geometry' claim leans on self-supervised depth and circular metrics; worth a serious look, not for the claimed reason. read the letter →

arxiv 2607.03819 v3 pith:22C7ISMD submitted 2026-07-04 cs.GR cs.AI

Zhenyu Sun , Xiaohan Zhang , Qi Liu , Huan Wang This is my paper
classification cs.GRcs.AI
keywords text-to-3Dscenegenerationego-centric3Dmulti-viewlatentdiffusionconsistency-augmentedlossflow-baseddepthestimationGaussiansplattingmutualinformationnovelviewsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that ego-centric text-to-3D scene generation — producing a 360° world from a text prompt and an outward-facing camera path — should be built as three coupled stages, and that this decomposition fixes the failure modes of both panoramic and independent multi-view pipelines. The first stage fine-tunes a multi-view latent diffusion model with a new consistency-augmented loss so the generated views agree with each other and with the prompt. The second treats those views as a short video and uses optical flow plus long-range point tracks to estimate consistent depth maps, which are back-projected into a dense point cloud, avoiding conventional structure-from-motion, which fails under low view overlap. The third stage optimizes a 3D Gaussian splatting scene under a mutual-information depth loss and a hierarchical camera schedule that sharpens geometry. If the central claim holds, text-driven ego-centric 3D scene generation can achieve both semantic alignment and geometric structure without the polar distortion of panoramas or the cross-view drift of unregularized multi-view generation, and it extends to out-of-domain scenes not seen in training.

What carries the argument

The load-bearing mechanism is a three-stage pipeline. In the Ego-centric Generator, the central object is the consistency-augmented loss term: a frozen, randomly initialized multi-layer CNN whose Jacobian acts as a subspace projection, pulling per-view denoising gradients into a common subspace and preventing gradient conflicts. In the Layout Decorator, the central object is the Flow-Depth Estimator: dense optical flow and long-range point tracks supply relative geometric constraints; a depth estimation network is optimized against those correspondences plus back-projection under known camera poses, yielding per-frame depth maps that fuse into a unified point cloud. In the Geometric Refiner,

What would settle it

Render a scene with the full pipeline, then take the point cloud produced by the Layout Decorator alone and check it for geometric self-consistency against a known canonical layout, such as a prompt describing a rectangular room; if the back-projected floor and walls fail to be coplanar or relatable by a rigid transformation across adjacent views, the flow-supervised depth fusion does not provide the claimed global alignment. A second check: compare MID-refined renderings against ground-truth depth from a real captured ego-centric trajectory; if the rendering gains disappear when measured agai

Watch

Extended reading notes

Core claim

The paper's central claim is that the ego-centric text-to-3D problem decomposes into three subproblems, each with an identified failure mode that the framework's components explicitly target. Multi-view diffusion trained with an ordinary score-matching objective gives each view its own gradient direction, so the paper adds a consistency-augmented loss: a frozen, randomly initialized hierarchical CNN projects per-view noise errors into a shared feature space, aligning gradients across views and harmonizing structure. The Layout Decorator then takes the generated views as a video stream and supervises a depth network with optical-flow and point-track correspondences, producing globally aligned

Load-bearing premise

The load-bearing assumption is that the synthetically generated multi-view images form a geometrically coherent video stream: if those views contain hallucinated or mutually inconsistent geometry, the optical-flow-and-point-track depth estimator will faithfully reconstruct the wrong structure, and the mutual-information depth loss will reinforce that hallucination as ground truth during refinement.

Editorial extensions

If this is right

  • Text-to-3D scene generation becomes a concrete three-stage recipe: generate consistent views, lift them with flow-and-point-track depth, then refine with mutual-information depth supervision.
  • Conventional structure-from-motion is unnecessary for ego-centric scenes; the flow-tracked depth estimator builds the initial layout directly from generated priors.
  • Mutual-information depth supervision can replace scale-invariant losses such as Pearson-correlation losses whenever preserving sharp geometric edges in Gaussian splatting matters.
  • Because the Gaussian optimizer is view-regularized by hierarchical cameras, minor inconsistencies in the generated 2D priors can be repaired downstream before the final render.
  • Out-of-domain prompt types remain synthesizable, since only the generator is fine-tuned on indoor data while the rest of the pipeline is domain-general.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the consistency-augmented loss is a generic regularizer for any multi-view diffusion model; its conflict-harmonization argument does not depend on ego-centric 360-degree layout, so it could transfer to object-level multi-view generation or video diffusion.
  • Editorial inference: since the Layout Decorator already models motion between views via flow and point tracks, the framework points toward dynamic ego-centric scene generation — the paper's own future-work direction — by reinterpreting tracked motion as temporal change rather than camera parallax.
  • Editorial inference: the known-camera-trajectory assumption is the convenient boundary; extending the pipeline to estimate poses jointly with depth and Gaussians would open it to arbitrary captured video, not just trajectory-specified generation.
  • Editorial inference: the MID-loss claim that statistical dependence preserves high-frequency detail is testable outside 3D generation, for instance in monocular depth refinement where scale-invariant losses struggle with thick-versus-thin structures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes CGGS, a three-stage pipeline for text-driven ego-centric 3D scene generation. The Ego-centric Generator fine-tunes a Multi-View Latent Diffusion Model with a consistency-augmented loss (L_aug) to synthesize multi-view 2D priors. The Layout Decorator estimates depth from optical flow and point tracks, back-projects the views into a coarse point cloud, and refines it with a correspondence loss. The Geometric Refiner fits 3D Gaussians with a mutual-information depth loss (MID) and a hierarchical camera-expansion scheme. The paper claims that CGGS outperforms prior methods in generating coherent, semantically aligned, and geometrically accurate 3D scenes.

Significance. If the geometric claims were fully supported, CGGS would be a useful systems-level contribution, combining a consistency-augmented multi-view diffusion generator with flow-guided depth and 3DGS refinement. The paper has clear strengths: a well-motivated problem, a complete pipeline, extensive qualitative results across indoor/outdoor and out-of-domain scenes, and ablation studies for each main component. The use of point tracks to mitigate drift in monocular depth lifting is a reasonable design. However, the evidence is not yet sufficient. The quantitative evaluation does not measure geometric accuracy against any external ground truth, the MID loss is trained against the same estimated depths that provide its supervision, and the main comparison is asymmetric. These are load-bearing gaps for the central 'accurate 3D scenes' claim. No code or data are provided, and the limitations section does not acknowledge the missing geometric validation.

major comments (5)
  1. [Sec. V-D, Table I] The reconstruction-quality metrics (PSNR, SSIM, LPIPS) are computed against the synthesized multi-view images X' that the pipeline itself fits. These scores measure how well the 3D Gaussians overfit the generator's 2D priors, not whether the recovered 3D geometry is accurate. A globally distorted but internally consistent point cloud can still achieve high PSNR/SSIM. To support the claim of 'accurate geometric structures,' the authors should report a geometry-aware evaluation: e.g., depth error against Matterport3D/ScanNet RGB-D, Chamfer distance or normal consistency against a reconstructed surface, or cross-view reprojection error on held-out cameras.
  2. [Sec. IV-C, Eqs. (12)-(13)] The MID loss treats the Layout Decorator's estimated depth maps d_i as ground truth and maximizes mutual information between rendered and estimated depth. Since mutual information is invariant under invertible transformations, minimizing Eq. (13) can be satisfied by a monotonic warp of the true depth; it does not enforce metric accuracy. Moreover, if the Ego-centric Generator hallucinates inconsistent geometry, the flow/track depth estimates inherit that corruption, and the MID loss then reinforces it. The paper needs independent geometric supervision or at least an evaluation with known 3D geometry to demonstrate that MID improves actual structure, not just statistical dependence.
  3. [Sec. V-D, Table I] The comparison is asymmetric: Text2Room is evaluated only on indoor scenes because its outdoor results contain large black artifacts; Director3D has no reconstruction metrics because it does not use intermediate reference images; and LucidDreamer is seeded with CGGS's first generated image, which may bias the comparison. These are reasonable practical choices, but they make the headline 'outperforms previous methods' less direct. The paper should report per-scene results with error bars, statistical significance tests, and a clear statement of which scenes/metrics are used for each baseline.
  4. [Sec. IV-A, Eqs. (5)-(6)] The central mechanism of L_aug is a frozen, randomly initialized VGG-16 claimed to act as a 'conflict harmonizer' that projects per-view gradients into a shared subspace. The paper does not justify why a random convolutional network yields an isotropic/stationary feature metric, nor does it provide empirical validation of the gradient-alignment claim. The ablation in Table II shows small differences without significance tests, and Fig. 7 is qualitative. A comparison against a pretrained VGG, a simpler fixed filter bank, or a direct measurement of gradient alignment would be needed to substantiate this component.
  5. [Table IV] The full setting reports SSIM of 0.997 and LPIPS of 0.0193, which are unusually high and likely reflect overfitting to the same images used for optimization. The ablation table has no error bars or repeated runs; the difference between PD+HO (36.251) and MID+HO (37.345) could be noise. The authors should report variance across scenes and possibly separate the hierarchical optimization (HO) and MID contributions more cleanly, since HO alone already improves over the baseline.
minor comments (6)
  1. [Sec. V-A] The subsection is titled 'Geometric Decorator' but should be 'Geometric Refiner'.
  2. [Eq. (12)] The double sum 'X X' should be a double integral or sum over the appropriate support, and the density estimation (histogram, kernel density, or neural estimator) should be specified. The paper also calls the MID loss 'entropy-based,' but mutual information is not entropy; please correct the terminology.
  3. [Sec. IV-C] Equations (14) and (16) are the same loss written twice; remove the duplication and clarify the relationship between the theoretical and detailed forms (Eq. 17).
  4. [Sec. IV-B, Eq. (8)] The set F includes f_{N'} mapping x_{N'} to x_1, but the textual description says 'between neighboring viewpoints'; please make the indexing consistent.
  5. [Sec. V-F, Table V] The out-of-domain evaluation uses only 4 scenes. This is too few for stable conclusions; add more scenes and report per-scene results or confidence intervals.
  6. [Limitations and Future Work] The limitations paragraph mentions only per-scene optimization cost. Given the paper's central geometric-accuracy claim, the absence of external 3D validation should be acknowledged as a limitation, and the reproducibility statement should include code/data availability.

Circularity Check

2 steps flagged · score 6.0 of 10

The pipeline's forward derivation is largely compositional, but the central geometric-accuracy claim is supported by self-referential fits: reconstruction metrics are computed against the same generated images the 3D Gaussians are optimized to reproduce, and the MID 'ground truth' depth is the Layout Decorator's own estimate.

  1. fitted input called prediction [Sec. V-D (Table I); Sec. IV-C Eq. (16)-(17)]
    "PSNR, SSIM, and LPIPS are used to assess view rendering quality in methods that generate scenes with reference images. ... CGGS exhibits exceptional performance in reconstruction fidelity, delivering a PSNR of 37.345 and an LPIPS of 0.0193, which underscores its high structural accuracy."

    The reference images for CGGS are the generated multi-view images that the 3D Gaussian optimization is trained to reproduce: Eq. (16)-(17) minimize L_rgb between rendered and 'GT images' (the Ego-centric Generator's outputs). Reporting PSNR/SSIM/LPIPS against these same training images therefore measures how well the Gaussians overfit the 2D priors, not how accurate the recovered 3D geometry is. Calling these numbers 'reconstruction fidelity' and 'structural accuracy' presents a training-set fit as an independent prediction of 3D quality.

  2. fitted input called prediction [Sec. IV-C Eq. (12)-(14); Sec. IV-B]
    "Specifically, for the perspective of the i-th camera, the depth map d_i from the Layout Decorator is considered as the ground truth depth, with its rendered depth map d_i^render calculated from the differential rasterization of 3DGS. ... LMID = 1 - I(D_render; D_gt)."

    The 'ground truth' depth D_gt is the output of the Layout Decorator, which estimates depth from the same generated ego-centric images via flow and point tracks; the same d_i maps are back-projected to form the initial point cloud that initializes the 3D Gaussians. Thus MID supervises the Geometric Refiner with the very same geometric estimate that produced the initialization. Optimizing Eq. (14) can only enforce consistency with the model's own prior, not independent geometric accuracy, and since mutual information is invariant under invertible monotone transformations, Eq. (13) does not even fix metric depth.

full rationale

No self-citation chain or definitional equation makes the forward pipeline itself circular: the Ego-centric Generator is a fine-tuned MVDiffusion with an auxiliary consistency loss, and the Layout Decorator is an independently trained flow/point-track depth estimator. The circularity is localized to the evidence for the central 'accurate geometric structures' claim. The quantitative 'Reconstruction Quality' metrics are computed on the same generated images the 3D Gaussians are optimized against, so high PSNR/SSIM/LPIPS are fit scores, not external 3D reconstruction accuracy. The MID loss uses the Layout Decorator's own depth estimate as ground truth, which is also the source of the initial point cloud, so the geometric refinement is a self-consistency loop. CLIP score, Q-Align, and qualitative comparisons provide some external grounding for semantic and perceptual quality, but they do not validate the claimed geometric superiority over DreamScene360/LucidDreamer. Consequently the core geometric claim rests substantially on self-referential fits, warranting a score of 6 rather than a lower score.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The central claim depends on several hand-chosen loss weights, a trained depth estimator used as geometric ground truth, and the assumption that diffusion-generated views are consistent enough for flow-based correspondence. No new physical entities are introduced.

free parameters (7)
  • lambda_aug = 0.5
    Weight on consistency-augmented loss in Eq. 7; chosen by hand (Sec. V-A).
  • lambda_MID = 0.05
    Weight on mutual-information depth loss in Eq. 16; chosen by hand (Sec. V-A).
  • lambda_SSIM = 0.2
    Weight on SSIM term in reconstruction loss Eq. 17; chosen by hand.
  • Generator view count and spacing = N=8, theta=45 deg, FOV=90 deg
    Design choices for Ego-centric Generator (Sec. V-A).
  • Interpolation count and FOV = N'=20, FOV=60 deg
    Design choices for Layout Decorator (Sec. V-A).
  • Hierarchical stages and cameras = n=3 stages, m=4 additional cameras
    Design choices for Geometric Refiner (Sec. V-A/IV-C).
  • Flow-Depth Estimator weights = unknown (trained on RealEstate-10k and CO3Dv2)
    Learned component that produces the depth maps used as ground truth by MID; no architecture or weights are released.
assumptions (6)
  • domain assumption Generated multi-view images can be modeled as a coherent video stream for optical flow and point tracking
    Layout Decorator treats X' as a short video and assumes RAFT/CoTracker correspondences are reliable (Sec. IV-B, Eq. 8-9).
  • domain assumption Camera trajectory and intrinsics are known for generated views
    Eq. 10 back-projects using K, T_i, T_j; no pose estimation is performed for generated scenes.
  • domain assumption Depth estimates from the Flow-Depth Estimator are valid ground truth for geometric supervision
    Sec. IV-C calls d_i 'ground truth depth' though it comes from an estimator trained on RE10k/CO3Dv2 and aligned via flow.
  • ad hoc to paper Frozen random VGG-16 provides an isotropic, stationary feature metric that harmonizes cross-view gradients
    Sec. IV-A asserts that He initialization and freezing make the feature space unbiased; the 'shared singular subspace' interpretation is informal and not proven.
  • ad hoc to paper Mutual information between rendered and estimated depth is computable and preserves sharp discontinuities
    Eq. 12-13 assume continuous MI can be evaluated as a double sum over p(r,g) and that maximizing MI enforces strict structural alignment; no density estimator is given.
  • standard math Standard diffusion score matching and 3DGS rasterization are valid background
    Preliminaries in Sec. III, Eqs. 1 and 3DGS rendering; accepted literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generation." pith.science (2026). https://pith.science/paper/22C7ISMD

@misc{pith2026260703819,
  author       = {Pith},
  title        = {Pith review of: CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/22C7ISMD}},
  note         = {Machine review of arXiv:2607.03819}
}
read the original abstract

Challenges remain in ego-centric 3D scene generation due to limited view overlap and the dominant influence of individual perspectives on scene interpretation. These factors hinder the creation of viewpoint-consistent and semantically aligned visual content, as well as the construction of accurate geometric structures. In this paper, we propose CGGS, a text-to-3D framework aiming to enhance 3D-content-awareness and address geometric distortions in ego-centric scene generation. Firstly, the Ego-centric Generator is proposed by fine-tuning a Multi-View Latent Diffusion Model with consistency-augmented loss to generate consistent, high-fidelity 2D content aligned with textual descriptions. Then, Layout Decorator leverages optical flow and point-track correspondence to estimate depth, therefore producing dense point clouds as coarse layouts from the ego-centric 2D priors. Building on this initialization, Geometric Refiner is proposed to enhance 3D Gaussian reconstruction via an entropy-based Mutual Information Depth Loss (MID) combined with a hierarchical optimization scheme for improving visual quality and geometric structure. Comprehensive experiments demonstrate that CGGS outperforms previous methods in generating coherent and accurate text-driven 3D scenes.

Figures

Figures reproduced from arXiv: 2607.03819 by the authors.

Figure 1
Figure 1. Visualization of geometric distortions in panoramic generation (using DreamScene360 [1] as an example). 1) Insufficient Text-Content Alignment: Significant textual details are omitted in the generation, such as the absence of ”mismatched frames” on the gallery wall despite being explicitly specified in the prompt. 2) Polar Geometric Distortions: Due to the inherent nature of equirectangular projection, severe radial… view at source ↗
Figure 2
Figure 2. With text prompts as input, CGGS employs three core components: the Ego-centric Generator creates ego-centric 2D priors, the Layout Decorator proposes additional scene details, and the Geometric Refiner further enhances the geometric structure and visual quality. Despite improvements for egocentric scenarios, these methods still lead to geometric and textural artifacts due to inherent inpainting limitations and dept… view at source ↗
Figure 3
Figure 3. Pipeline of CGGS. It primarily comprises the Consistency-Augmented MV-LDM as Ego-centric Generator, the Flow-Depth Estimator as Layout Decorator, and the 3D Gaussian Optimization combined with MID Loss and the hierarchical optimization strategy, serving as Geometric Refiner. on the 2D displacement between s l ∗ and s, which provides relative location within local neighborhoods. CAA blocks are integrated into the pre… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Generation results of CGGS for ego-centric multi-view priors, gaussian point clouds, novel view synthesis, and depth maps. Our method generates harmonious, domain-free 3D scenes from ego-centric views, highly aligned with complex textual descriptions. C. Geometric Refi…
Figure 5
Figure 5. Figure 5: Qualitative comparison between CGGS with other baselines. Our CGGS produces multi-view images with rich detail and superior semantic coherence, showcasing domain-agnosticity. Our results outperform other works with an accurately detailed description and unified 3D cons…
Figure 6
Figure 6. Figure 6: Poor quality of the rendered views from Text2Room [13] on outdoor scenes. There are large black artifacts and missing geometry. from the multi-view sequence generated by CGGS as its initial input. As for Director3D, since it does not utilize any intermediate reference …
Figure 7
Figure 7. Figure 7: Ablation study on consistency-augmented loss Laug. Without Laug, cross-view texture discrepancies become pronounced, with abrupt background artifacts (e.g., exposed ceilings in bedroom scenes) and physically implausible anomalies (e.g., floating, distorted trees on bea…
Figure 8
Figure 8. Figure 8: The qualitative comparison demonstrates that incorpo [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 8
Figure 8. Figure 8: Ablation studies of Geometric-Refiner on MID loss and hierarchical optimization. Here we demonstrate the qualitative comparison between the ground [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Additional generation results from CGGS. Our work can generate richly detailed, high-fidelity scenes with considerable diversity while ensuring cohesive semantic content and a harmonized visual style that faithfully reflects even the most intricate textual descriptions…
Figure 10
Figure 10. Figure 10: Additional generation results from CGGS. Our work can generate richly detailed, high-fidelity scenes with considerable diversity while ensuring cohesive semantic content and a harmonized visual style that faithfully reflects even the most intricate textual description…

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

86 extracted references · 22 linked inside Pith

  1. [1]

    Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting,

    S. Zhou, Z. Fan, D. Xu, H. Chang, P. Chari, T. Bharadwaj, S. You, Z. Wang, and A. Kadambi, “Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting,” inECCV, 2024

    2024

  2. [2]

    Glide: Towards photorealistic image gener- ation and editing with text-guided diffusion models,

    A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image gener- ation and editing with text-guided diffusion models,”arXiv:2112.10741, 2021

    arXiv 2021

  3. [3]

    Hierarchical text-conditional image generation with clip latents,

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,”arXiv:2204.06125, 2022

    arXiv 2022

  4. [4]

    Photoreal- istic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. G. Lopes, B. K. Ayan, T. Salimanset al., “Photoreal- istic text-to-image diffusion models with deep language understanding,” inNeurIPS, 2022

    2022

  5. [5]

    High resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High resolution image synthesis with latent diffusion models,” inCVPR, 2022

    2022

  6. [6]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” inECCV, 2020

    2020

  7. [7]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, and et al., “Learning transferable visual models from natural language supervision,” inICML, 2021

    2021

  8. [8]

    Zero-shot text-guided object generation with dream fields,

    A. Jain, B. Mildenhall, J. T. Barron, P. Abbeel, and B. Poole, “Zero-shot text-guided object generation with dream fields,” inCVPR, 2022

    2022

Show all 86 references
  1. [9]

    Dreamfusion: Text- to-3d using 2d diffusion,

    B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, “Dreamfusion: Text- to-3d using 2d diffusion,” inICLR, 2023

    2023

  2. [10]

    Magic3d: High-resolution text-to-3d content creation,

    C.-H. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng, X. Huang, K. Kreis, S. Fidler, M.-Y . Liu, and T.-Y . Lin, “Magic3d: High-resolution text-to-3d content creation,” inCVPR, 2023

    2023

  3. [11]

    Mvdream: Multi- view diffusion for 3d generation,

    Y . Shi, P. Wang, J. Ye, L. Mai, K. Li, and X. Yang, “Mvdream: Multi- view diffusion for 3d generation,” inICLR, 2024

    2024

  4. [12]

    Text2room: Extracting textured 3d meshes from 2d text-to-image models,

    L. H ¨ollein, A. Cao, A. Owens, J. Johnson, and M. Nießner, “Text2room: Extracting textured 3d meshes from 2d text-to-image models,” inICCV, 2023

    2023

  5. [13]

    Lucid- dreamer: Domain-free generation of 3d gaussian splatting scenes,

    J. Chung, S. Lee, H. Nam, J. Lee, and K. M. Lee, “Lucid- dreamer: Domain-free generation of 3d gaussian splatting scenes,” arXiv:2311.13384, 2023

    2023 arXiv

  6. [14]

    Text2nerf: Text-driven 3d scene generation with neural radiance fields,

    J. Zhang, X. Li, Z. Wan, C. Wang, and J. Liao, “Text2nerf: Text-driven 3d scene generation with neural radiance fields,”IEEE Transactions on Visualization and Computer Graphics, 2024

    2024

  7. [15]

    Wonderjourney: Going from anywhere to everywhere,

    H.-X. Yu, H. Duan, J. Hur, K. Sargent, M. Rubinstein, W. T. Freeman, F. Cole, D. Sun, N. Snavely, J. Wuet al., “Wonderjourney: Going from anywhere to everywhere,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  8. [16]

    Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion,

    J. Shriram, A. Trevithick, L. Liu, and R. Ramamoorthi, “Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion,” in 3DV, 2025

    2025

  9. [17]

    Recurrent diffusion for 3d point cloud generation from a single image,

    Y . Zhou, D. Ye, H. Zhang, X. Xu, H. Sun, Y . Xu, X. Liu, and Y . Zhou, “Recurrent diffusion for 3d point cloud generation from a single image,” IEEE Transactions on Image Processing, vol. 34, pp. 1753–1765, 2025

    2025

  10. [18]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,”ACM Transactions on Graphics, 2023

    2023

  11. [19]

    Scene- dreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting,

    W. Li, F. Cai, Y . Mi, Z. Yang, W. Zuo, X. Wang, and X. Fan, “Scene- dreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting,”arXiv:2408.13711, 2024

    2024 arXiv

  12. [20]

    Holodreamer: Holistic 3d panoramic world generation from text descriptions,

    H. Zhou, X. Cheng, W. Yu, Y . Tian, and L. Yuan, “Holodreamer: Holistic 3d panoramic world generation from text descriptions,” arXiv:2407.15187, 2024

    2024 arXiv

  13. [21]

    Director3d: Real-world camera trajectory and 3d scene generation from text,

    X. Li, Z. Lai, L. Xu, Y . Qu, L. Cao, S. Zhang, B. Dai, and R. Ji, “Director3d: Real-world camera trajectory and 3d scene generation from text,” inNeurIPS, 2024

    2024

  14. [22]

    Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation,

    Y . Yang, J. Shao, X. Li, Y . Shen, A. Geiger, and Y . Liao, “Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation,”arXiv:2412.21117, 2024

    2024 arXiv

  15. [23]

    Matterport3D: Learning from RGB-D data in indoor environments,

    A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y . Zhang, “Matterport3D: Learning from RGB-D data in indoor environments,” in3DV, 2017

    2017

  16. [24]

    Stereo magnification: Learning view synthesis using multiplane images,

    T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely, “Stereo magnification: Learning view synthesis using multiplane images,”ACM Transactions on Graphics, vol. 37, no. 4, pp. 65:1–65:12, 2018

    2018

  17. [25]

    Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction,

    J. Reizenstein, R. Shapovalov, P. Henzler, L. Sbordone, P. Labatut, and D. Novotny, “Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction,” inICCV, 2021. 15

    2021

  18. [26]

    Mvdiffusion: Enabling holistic multi-view image generation with correspondence- aware diffusion,

    S. Tang, F. Zhang, J. Chen, P. Wang, and Y . Furukawa, “Mvdiffusion: Enabling holistic multi-view image generation with correspondence- aware diffusion,”arXiv:2307.01097, 2023

    2023 arXiv

  19. [27]

    Stable diffusion 2,

    StabilityAI, “Stable diffusion 2,” https://huggingface.co/stabilityai/ stable-diffusion-2, 2023

    2023

  20. [28]

    Structure-from-motion revisited,

    J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” inCVPR, 2016

    2016

  21. [29]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” in NeurIPS, 2014

    2014

  22. [30]

    Stylegan2 distillation for feed-forward image manipulation,

    Y . Viazovetskyi, V . Ivashkin, and E. Kashin, “Stylegan2 distillation for feed-forward image manipulation,” inECCV, 2020, pp. 170–186

    2020

  23. [31]

    Unsupervised representation learning with deep convolutional generative adversarial networks,

    A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv:1511.06434, 2015

    2015 arXiv

  24. [32]

    Image-to-image translation with conditional adversarial networks,

    P. Isola, J.-Y . Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” inCVPR, 2017, pp. 1125–1134

    2017

  25. [33]

    A style-based generator architecture for generative adversarial networks,

    T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” inCVPR, 2019, pp. 4401–4410

    2019

  26. [34]

    Generative modeling by estimating gradients of the data distribution,

    Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” inNeurIPS, 2019

    2019

  27. [35]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” inNeurIPS, 2020

    2020

  28. [36]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” inICLR, 2021

    2021

  29. [37]

    Score-based generative modeling through stochastic differ- ential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differ- ential equations,” inICLR, 2021

    2021

  30. [38]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv:2207.12598, 2022

    2022 arXiv

  31. [39]

    Boosting latent diffusion with perceptual objectives,

    T. Berrada, P. Astolfi, M. Hall, M. Havasi, Y . Benchetrit, A. Romero- Soriano, K. Alahari, M. Drozdzal, and J. Verbeek, “Boosting latent diffusion with perceptual objectives,”arXiv:2411.04873, 2024

    2024 arXiv

  32. [40]

    Diffusion model with perceptual loss,

    S. Lin and X. Yang, “Diffusion model with perceptual loss,” arXiv:2401.00110, 2024

    2024 arXiv

  33. [41]

    Multidiffusion: Fusing diffusion paths for controlled image generation,

    O. Bar-Tal, L. Yariv, Y . Lipman, and T. Dekel, “Multidiffusion: Fusing diffusion paths for controlled image generation,”arXiv:2302.08113, 2023

    2023 arXiv

  34. [42]

    Panogen: Text-conditioned panoramic environ- ment generation for vision-and-language navigation,

    J. Li and M. Bansal, “Panogen: Text-conditioned panoramic environ- ment generation for vision-and-language navigation,”arXiv:2305.19195, 2023

    2023 arXiv

  35. [43]

    Diffcollage: Parallel generation of large content with diffusion models,

    Q. Zhang, J. Song, X. Huang, Y . Chen, and M. yu Liu, “Diffcollage: Parallel generation of large content with diffusion models,” inCVPR, 2023

    2023

  36. [44]

    Customizing 360-degree panoramas through text-to-image diffusion models,

    H. Wang, X. Xiang, Y . Fan, and J.-H. Xue, “Customizing 360-degree panoramas through text-to-image diffusion models,” inWACV, 2024

    2024

  37. [45]

    Taming stable diffusion for text to 360◦ panorama image generation,

    C. Zhang, Q. Wu, C. Cruz Gambardella, X. Huang, D. Phung, W. Ouyang, and J. Cai, “Taming stable diffusion for text to 360◦ panorama image generation,” inCVPR, 2024

    2024

  38. [46]

    Text2mesh: Text-driven neural stylization for meshes,

    O. Michel, R. Bar-On, R. Liu, S. Benaim, and R. Hanocka, “Text2mesh: Text-driven neural stylization for meshes,” inCVPR, 2022

    2022

  39. [47]

    Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation,

    R. Chen, Y . Chen, N. Jiao, and K. Jia, “Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation,” inICCV, 2023

    2023

  40. [48]

    Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation,

    H. Wang, X. Du, J. Li, R. A. Yeh, and G. Shakhnarovich, “Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation,” in CVPR, 2023

    2023

  41. [49]

    Prolific- dreamer: High-fidelity and diverse text-to-3d generation with variational score distillation,

    Z. Wang, C. Lu, Y . Wang, F. Bao, C. Li, H. Su, and J. Zhu, “Prolific- dreamer: High-fidelity and diverse text-to-3d generation with variational score distillation,” inNeurIPS, 2023

    2023

  42. [50]

    3dtopia: Large text-to-3d generation model with hybrid diffusion priors,

    F. Hong, J. Tang, Z. Cao, M. Shi, T. Wu, Z. Chen, T. Wang, L. Pan, D. Lin, and Z. Liu, “3dtopia: Large text-to-3d generation model with hybrid diffusion priors,”arXiv:2403.02234, 2024

    2024 arXiv

  43. [51]

    Rgbd2: Generative scene synthesis via incremental view inpainting using rgbd diffusion models,

    J. Zhang, Y . Xu, W. Wang, J. Yang, Y . Shen, X. Li, L. Xie, and F. Yu, “Rgbd2: Generative scene synthesis via incremental view inpainting using rgbd diffusion models,” inCVPR, 2023

    2023

  44. [52]

    Text2immersion: Generative immersive scene with 3d gaussians,

    H. Ouyang, K. Heal, S. Lombardi, and T. Sun, “Text2immersion: Generative immersive scene with 3d gaussians,”arXiv: 2312.09242, 2023

    2023 arXiv

  45. [53]

    Scenescape: Text- driven consistent scene generation,

    R. Fridman, A. Abecasis, Y . Kasten, and T. Dekel, “Scenescape: Text- driven consistent scene generation,”arXiv:2302.01133, 2023

    2023 arXiv

  46. [54]

    Vistadream: Sampling multiview consistent images for single-view scene reconstruc- tion,

    H. Wang, Y . Liu, Z. Liu, W. Wang, Z. Dong, and B. Yang, “Vistadream: Sampling multiview consistent images for single-view scene reconstruc- tion,”arXiv:2410.16892, 2024

    2024 arXiv

  47. [55]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross- dataset transfer,

    R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V . Koltun, “Towards robust monocular depth estimation: Mixing datasets for zero-shot cross- dataset transfer,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 3, pp. 1623–1637, 2022

    2022

  48. [56]

    Zoedepth: Zero-shot transfer by combining relative and metric depth,

    S. F. Bhat, R. Birkl, D. Wofk, P. Wonka, and M. M ¨uller, “Zoedepth: Zero-shot transfer by combining relative and metric depth,” arXiv:2302.12288, 2023

    2023 arXiv

  49. [57]

    Depth anything: Unleashing the power of large-scale unlabeled data,

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” inCVPR, 2024

    2024

  50. [58]

    Learning a locally unified 3d point cloud for view synthesis,

    M. You, M. Guo, X. Lyu, H. Liu, and J. Hou, “Learning a locally unified 3d point cloud for view synthesis,”IEEE Transactions on Image Processing, vol. 32, pp. 5610–5622, 2023

    2023

  51. [59]

    pixelnerf: Neural radiance fields from one or few images,

    A. Yu, V . Ye, M. Tancik, and A. Kanazawa, “pixelnerf: Neural radiance fields from one or few images,” inCVPR, 2021

    2021

  52. [60]

    Nope-nerf: Optimising neural radiance field with no pose prior,

    W. Bian, Z. Wang, K. Li, J. Bian, and V . A. Prisacariu, “Nope-nerf: Optimising neural radiance field with no pose prior,” inCVPR, 2023

    2023

  53. [61]

    Latent-nerf for shape-guided generation of 3d shapes and textures,

    G. Metzer, E. Richardson, O. Patashnik, R. Giryes, and D. Cohen-Or, “Latent-nerf for shape-guided generation of 3d shapes and textures,” in CVPR, 2023

    2023

  54. [62]

    Diffusionerf: Regularizing neural radiance fields with denoising diffusion models,

    J. Wynn and D. Turmukhambetov, “Diffusionerf: Regularizing neural radiance fields with denoising diffusion models,” inCVPR, 2023

    2023

  55. [63]

    Global structure-from-motion revisited,

    L. Pan, D. Bar ´ath, M. Pollefeys, and J. L. Sch ¨onberger, “Global structure-from-motion revisited,” inECCV, 2024

    2024

  56. [64]

    Flowmap: High- quality camera poses, intrinsics, and depth via gradient descent,

    C. Smith, D. Charatan, A. Tewari, and V . Sitzmann, “Flowmap: High- quality camera poses, intrinsics, and depth via gradient descent,” arXiv:2404.15259, 2024

    2024 arXiv

  57. [65]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation,

    J. Tang, Z. Chen, X. Chen, T. Wang, G. Zeng, and Z. Liu, “Lgm: Large multi-view gaussian model for high-resolution 3d content creation,” in ECCV, 2024

    2024

  58. [66]

    Gs-lrm: Large reconstruction model for 3d gaussian splatting,

    K. Zhang, S. Bi, H. Tan, Y . Xiangli, N. Zhao, K. Sunkavalli, and Z. Xu, “Gs-lrm: Large reconstruction model for 3d gaussian splatting,” inECCV, 2024

    2024

  59. [67]

    Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation,

    Y . Xu, Z. Shi, W. Yifan, H. Chen, C. Yang, S. Peng, Y . Shen, and G. Wetzstein, “Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation,” inECCV, 2024

    2024

  60. [68]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images,

    Y . Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T.-J. Cham, and J. Cai, “Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images,” inECCV, 2024

    2024

  61. [69]

    Pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction,

    D. Charatan, S. Li, A. Tagliasacchi, and V . Sitzmann, “Pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction,” inCVPR, 2024

    2024

  62. [70]

    Splatter image: Ultra- fast single-view 3d reconstruction,

    S. Szymanowicz, C. Rupprecht, and A. Vedaldi, “Splatter image: Ultra- fast single-view 3d reconstruction,” inCVPR, 2024

    2024

  63. [71]

    Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization,

    J. Li, J. Zhang, X. Bai, J. Zheng, X. Ning, J. Zhou, and L. Gu, “Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization,” inCVPR, 2024

    2024

  64. [72]

    Fsgs: Real-time few-shot view synthesis using gaussian splatting,

    Z. Zhu, Z. Fan, Y . Jiang, and Z. Wang, “Fsgs: Real-time few-shot view synthesis using gaussian splatting,” inECCV, 2024

    2024

  65. [73]

    Wonderworld: Interactive 3d scene generation from a single image,

    H.-X. Yu, H. Duan, C. Herrmann, W. T. Freeman, and J. Wu, “Wonderworld: Interactive 3d scene generation from a single image,” arXiv:2406.09394, 2024

    2024 arXiv

  66. [74]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in ICLR, 2014

    2014

  67. [75]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” inICLR, 2015

    2015

  68. [76]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,

    K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in ICCV, 2015

    2015

  69. [77]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and F.-F. Li, “Imagenet: A large-scale hierarchical image database,” inCVPR, 2009

    2009

  70. [78]

    Raft: Recurrent all-pairs field transforms for optical flow,

    Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” inECCV, 2020

    2020

  71. [79]

    Cotracker: It is better to track together,

    N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rup- precht, “Cotracker: It is better to track together,” inECCV, 2024

    2024

  72. [80]

    Nexusgs: Sparse view synthesis with epipolar depth priors in 3d gaussian splatting,

    Y . Zheng, Z. Jiang, S. He, Y . Sun, J. Dong, H. Zhang, and Y . Du, “Nexusgs: Sparse view synthesis with epipolar depth priors in 3d gaussian splatting,”arXiv:2503.18794, 2025

    2025 arXiv

  73. [81]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,”arXiv: 2303.08774, 2023

    2023 arXiv

  74. [82]

    Tanks and temples: Benchmarking large-scale scene reconstruction,

    A. Knapitsch, J. Park, Q.-Y . Zhou, and V . Koltun, “Tanks and temples: Benchmarking large-scale scene reconstruction,”ACM Transactions on Graphics, 2017

    2017

  75. [83]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes,

    A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” inCVPR, 2017

    2017

  76. [84]

    CLIPScore: a reference-free evaluation metric for image captioning,

    J. Hessel, A. Holtzman, M. Forbes, R. L. Bras, and Y . Choi, “CLIPScore: a reference-free evaluation metric for image captioning,” inEMNLP, 2021. 16

    2021

  77. [85]

    Q-align: Teaching lmms for visual scoring via discrete text-defined levels,

    H. Wu, Z. Zhang, W. Zhang, C. Chen, L. Liao, C. Li, Y . Gao, A. Wang, E. Zhang, W. Sunet al., “Q-align: Teaching lmms for visual scoring via discrete text-defined levels,” inICML, 2024

    2024

  78. [86]

    Exploring clip for assessing the look and feel of images,

    J. Wang, K. C. K. Chan, and C. C. Loy, “Exploring clip for assessing the look and feel of images,” inAAAI, 2023

    2023

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.