Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Diorama claims that a single RGB image of a cluttered indoor room can be turned into a structured, editable 3D scene built from CAD objects, with no end-to-end training and no human annotations.

desk verdict A real first in training-free holistic CAD scene modeling whose feasibility claim holds, though the abstract's 'significantly outperform' overstates the numbers and the depth-dependence concern deserves one extra experiment. read the letter →

arxiv 2411.19492 v2 pith:2OKANKZZ submitted 2024-11-29 cs.CV cs.LG

Qirui Wu , Denys Iliash , Daniel Ritchie , Manolis Savva , Angel X. Chang This is my paper · ORCID
classification cs.CVcs.LG
keywords zero-shot3Dscenemodelingsingle-viewreconstructionCADmodelretrieval9-DoFposeestimationgraphgenerationplanararchitecturelayoutoptimizationopen-worldperception
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single RGB image of a cluttered indoor room is enough to produce a structured, editable 3D scene built from CAD (computer-aided design) objects, without training a new network or collecting human annotations. The proposed system, Diorama, splits the task into four zero-shot subtasks: detecting and describing objects, reconstructing planar architecture such as floors and walls, retrieving matching CAD models from a large database, and estimating and refining each object's position, orientation, and scale. Each subtask is handled by off-the-shelf pretrained components connected by prompting and optimization, so the system can accept internet photos and even images generated from text prompts. The authors evaluate on synthetic scenes, real scans, and web images, reporting that the pipeline outperforms prior modular baselines and handles out-of-distribution object categories those baselines cannot. If the claim holds, an ordinary photograph could become a compact, rearrangeable scene representation rather than a fixed triangle mesh.

What carries the argument

The load-bearing mechanism is a modular pipeline of four zero-shot subtasks. PlainRecon reconstructs architecture by segmenting objects, inpainting them out, lifting the empty room to a point cloud with metric depth and normals, and fitting bounded planes through normal-based clustering. Shape retrieval uses a text query to select a semantic category and an image query to re-rank candidates inside a joint text-image-shape embedding space. Pose estimation computes visual patch correspondences between the object crop and multiview renders of the retrieved model, picks the most similar view, lifts 2D matches to 3D with depth, and solves for rotation, translation, and uniform scale with a RANSAC-based least-squares rigid transform, aided by a small scale-prediction network. A four-stage semantic-aware scene optimization (orientation, placement, space, refinement) then enforces the scene graph's support relations, keeps objects out of one another, and anchors them to their supports.

What would settle it

Run the full pipeline on a set of single-view images where a monocular depth estimator is known to fail, such as rooms with large mirrors, transparent surfaces, or extreme unusual layouts, while keeping every other module fixed. If the reconstructed architecture planes and object poses deviate beyond the paper's tolerances (relative translation or rotation deviation over 20 degrees, or relative scale error over 0.2), that identifies the depth-estimation dependency as the decisive failure point. A complementary check is to compare scene-aware alignment accuracy between images with occlusion ratio below and above 5% of object pixels.

Watch

Extended reading notes

Core claim

The central claim is that holistic 3D scene modeling from a monocular image does not require end-to-end training or 3D annotations, because the problem can be decomposed into subtasks that existing foundation models and lightweight optimization already solve. Diorama is presented as the first zero-shot open-world system to complete the full chain: it builds a scene graph of support relations, reconstructs the architecture as bounded 3D planes, retrieves a CAD object for every detected instance, and refines the coarse poses through a stage-wise optimizer. In the authors' own terms, the discovery is feasibility: the combination of open-vocabulary detection, metric depth estimation, multimodal shape embedding, visual-correspondence pose solving, and semantic-aware layout optimization is enough to model cluttered indoor scenes with plausible support hierarchies, including small objects and objects mounted on walls.

Load-bearing premise

The whole scene inherits the metric accuracy of a pretrained monocular depth estimator run on an inpainted, object-free version of the image; the paper notes that translation and scale estimates are significantly influenced by poor depth estimation and by large object occlusion, so if that depth is wrong, the architecture and object poses collapse.

Editorial extensions

If this is right

  • A single pretrained-component pipeline with no end-to-end training can model cluttered indoor scenes well enough to beat modular baselines that rely on physics-engine simulation.
  • Because retrieval is open-vocabulary and training-free, the system handles out-of-distribution categories such as laptops and backpacks that supervised baselines cannot process.
  • The same pipeline extends to text-to-scene generation by first synthesizing an image from a prompt and then modeling that image.
  • Stage-wise layout optimization materially improves the output: the full four-stage procedure reduces collisions and raises support-relation accuracy relative to the unoptimized poses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because every module is zero-shot and loosely coupled, stronger future foundation models (improved depth, detection, segmentation, or vision-language grounding) can be swapped in component by component, and each improvement should transfer without retraining the rest of the chain.
  • Beyond the paper: the pipeline's reliance on inpainting before depth estimation suggests a testable extension: inpaint the defurnished image multiple times and average the resulting point clouds, or let the depth estimator see the original image with masked object regions, to reduce the blur the paper identifies as a bottleneck.
  • Beyond the paper: the scene-graph stage restricts support relations to 'placed on' and 'mounted on,' so hanging, leaning, or interlocked arrangements are likely out of reach; extending the relation vocabulary would be a direct test of how far the zero-shot paradigm can go.
  • Beyond the paper: at roughly $0.12 per image and under four minutes per scene, the system's cost profile suggests it could be applied to large photo collections as a scene-asset generator, but only if metric depth reliability on arbitrary real-world photographs improves first.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Diorama is a modular, training-free system that takes a single RGB image of an indoor scene and produces a complete 3D scene model: planar architecture, retrieved CAD objects, support relations, and optimized 9-DoF layouts. The pipeline combines pretrained components (OWLv2/SAM for detection, Metric3DV2 for depth and normals, GPT-4o for scene graphs, DuoDuoCLIP for shape retrieval, DINOv2 correspondence matching with GigaPose scale estimation, and a stage-wise layout optimizer). The paper evaluates on SSDB synthetic scenes, ScanNet real images, and internet/text-generated images, reporting comparisons against ACDC, ROCA, DiffCAD, ZSP, and GigaPose. The central claim is that this constitutes the first zero-shot open-world system for holistic, CAD-based scene modeling from monocular RGB without end-to-end training or human annotation.

Significance. If the central claim holds, Diorama offers a practical recipe for turning a single RGB image into a compositional, editable, simulation-ready scene while avoiding costly annotations and per-category training. The paper's strengths are its modular decomposition, its systematic ablations of the optimization stages (Table 5), its explicit cost analysis (Table 1), and its broad set of qualitative demonstrations on internet images and text-to-scene inputs. The main unresolved point is whether the system actually delivers the RGB-to-scene claim under realistic depth estimation: the headline metric is explicitly designed to be invariant to depth error, and the numeric evidence shows the pose module degrades substantially when ground-truth depth is replaced by Metric3DV2 depth. The paper also overstates the significance of its quantitative gains given the small absolute margins and the non-standard ScanNet evaluation protocol.

major comments (3)
  1. [Section 4.1; Tables 4 and 6] The scene-aware Acc metric is defined in Section 4.1 to 'eliminate the effect of inaccurate depth prediction' by measuring relative translation as a direction, yet depth from Metric3DV2 is back-projected to point clouds used for both architecture reconstruction and instance pose lifting (Section 3.1). The paper itself concedes in Section 3.2 that translation and scale are 'significantly influenced by undesired depth estimation and large object occlusion.' The ablation evidence confirms this: replacing ground-truth depth with Metric3DV2 depth lowers 9-DoF alignment Acc from 0.37 to 0.28 (Table 4 vs Table 6) even when object instances are given. Because Table 1's system comparison relies on the relative Acc metric, it does not exercise the system's weakest component, and the ScanNet comparison (Section 4.3) feeds identical mask/depth inputs to both methods, so it isolates neither the depth module nor the full RGB-to-scene claim. To support the abstract claim of holistic modeling from a single RGB observation, please add a scene-level evaluation with estimated depth (and estimated masks if possible), report metric-scale position and scale errors, or add a controlled depth-corruption study that shows graceful degradation.
  2. [Section 4.3; Table 10] The claim of 'comparable performance against DiffCAD' is not established. Table 10 reports results on a different and smaller subset than prior work: 600 images spanning 24 categories here, versus 2,100 images and 6 categories for ROCA/DiffCAD. On that subset, our method attains 0.00 accuracy on bed, bin, and bathtub, and the class-average is 3.33 versus DiffCAD's 2.79-4.34 depending on the number of hypotheses. A fair comparison requires running the baselines on the same images and categories, or clearly presenting this as a separate, harder benchmark rather than as a head-to-head comparison. As written, the numeric evidence is inconclusive and may be reflecting differences in the evaluation set rather than in method quality.
  3. [Abstract; Section 4.2] The abstract and Section 4.2 claim that Diorama 'significantly outperform[s]' prior baselines, but Table 1 shows a scene-aware Acc of 0.08 versus 0.04 for ACDC, with no confidence intervals or significance tests reported. Given the small absolute margin and the fact that the metric deliberately removes depth sensitivity, 'significantly' is unsupported. Please provide per-scene paired statistics (bootstrap or permutation tests), or soften the claim to a directional improvement. This point also applies to the ScanNet discussion, where the reported margins are not tested for significance.
minor comments (5)
  1. [Abstract] The phrase 'the first zero-shot open-world system' is a strong novelty claim; consider wording such as 'to our knowledge' or adding a table that contrasts design choices of prior modular systems (IM2CAD, ACDC, SceneComplete) to make the claim precise.
  2. [Section 4.2; Table 3] The retrieval evaluation in Table 3 uses top-5 retrieval and selects the candidate with minimum L1 Chamfer distance to the ground-truth shape; this is an oracle selection that provides an upper bound, and the text should say so explicitly when interpreting the margins over CLIP and OpenShape.
  3. [Supplement A.3] The supplement openly states that 'the resulting image is not fully defurnished and the inpainted region is blurry' and that 'the former remains a bottleneck of the pipeline.' Because PlainRecon depends on this inpainted image, please quantify the bottleneck, for example by comparing architecture reconstruction with ground-truth defurnished images, so readers can judge how much of Table 2's IoU gain relies on inpainting quality.
  4. [Table 2] PlainRecon's RMSE (1.37 with Metric3DV2 depth) is substantially worse than RaC's (0.645 with DepthAnythingV2), while PlainRecon wins on IoU, PE, EE, and CDb. The text dismisses this as an outlier but does not explain the discrepancy; please provide a per-plane or normalized RMSE breakdown to clarify whether the error is localized to distant or low-texture walls.
  5. [References] Reference [52] contains a typo: 'Pattern Analysis and Machine Inttelligence' should read 'Intelligence'.

Circularity Check

0 steps flagged · score 0.0 of 10

Diorama is a modular assembly of pretrained components; no prediction reduces by construction to a fitted parameter or load-bearing self-citation, so circularity is negligible.

full rationale

The paper derives its central claim from a pipeline of independently pretrained modules (OWLv2, SAM, Metric3DV2, GPT-4o, DINOv2, GigaPose) evaluated against external baselines and public datasets. The system does not fit any parameter to the target scenes and then rename that fit as a prediction; pose, scale, retrieval, and layout outputs are computed by optimization or pretrained models, not by inverting the evaluation metric. The self-citation to DuoDuoCLIP [44] is a component choice, but Table 3 benchmarks it against CLIP and OpenShape on retrieval Chamfer distance, so the choice is empirically supported within the paper rather than accepted on authority. Other self-citations (e.g., [80], [81]) appear only in related work or metric conventions and are not load-bearing for the system claim. The paper's own limitations state that translation and scale are 'significantly influenced by undesired depth estimation and large object occlusion' (Sec. 3.2) and that scene-aware accuracy was introduced to 'eliminate the effect of inaccurate depth prediction' (Sec. 4.1); these passages identify a genuine robustness weakness in the RGB-to-scene claim, but a robustness limitation is not circularity because the predicted outputs are not defined in terms of the test targets. Tables 4 and 6 provide independent evidence of depth sensitivity rather than a constructed equivalence. Consequently, no step in the derivation chain reduces to its own input by definition, and the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim does not derive new constants or physical laws; it assembles pretrained components and hand-tuned optimization terms. No new physical entities are introduced. The main ledger items are domain assumptions about foundation-model generalization, binary support relations, planar architecture and retrieval database coverage, plus hand-chosen hyperparameters.

free parameters (6)
  • Optimization stage weights = 3, 5, 5 (e1, e2, e4)
    Hand-chosen weights for alignment, placement and collision terms in Supplement A.6; no sensitivity analysis reported.
  • Correspondence similarity threshold = 0.7
    Set empirically in Supplement A.5 to keep only strong DINOv2 correspondences.
  • Architecture clustering hyperparameters = km=12, alpha=10 degrees, Nmin=200, KNN k=1
    Chosen for stability of plane clustering in Supplement A.3; no ablation over these values.
  • Alignment accuracy thresholds = 20 degrees, 20 degrees, 0.2 scale; stricter 0.1 cm and 1 degree in ablation
    New scene-aware alignment metric thresholds defined in Section 4.1; arbitrary and not justified by a downstream application.
  • Optimization learning schedule = lr=0.01, momentum=0.9, decay 0.1 every 50 steps, 200 steps
    SGD schedule in Supplement A.6 chosen by hand; no study of sensitivity.
  • Number of multiview renderings per CAD model = 180
    Selected for efficiency and coarse pose selection quality in Supplement B.
assumptions (5)
  • domain assumption Pretrained foundation models (OWLv2, SAM, Metric3DV2, GPT-4o, DINOv2, DuoDuoCLIP, GigaPose) generalize to synthetic and real indoor scenes without fine-tuning.
    The zero-shot claim rests on these models working out of the box; Sections 3.1 and 3.2 use them directly.
  • domain assumption Indoor architecture can be adequately represented by a set of bounded 3D planes.
    PlainRecon represents architecture as planes following prior work; Section 3.1.
  • domain assumption Support relations in a scene are binary and limited to 'placed on' and 'mounted on'.
    The scene graph prompt constrains support relation types; Supplement A.2.
  • domain assumption The retrieval database contains CAD models semantically and geometrically close enough to observed objects.
    The modeling stage retrieves from SSDB, HSSD and Objaverse LVIS; if no close shape exists, pose and layout degrade. Section 3.2.
  • standard math Standard geometric tools (RANSAC, Umeyama, K-means, DBSCAN, Separating Axis Theorem, plane fitting) behave as expected.
    Used without proof in Sections 3.2, A.3 and A.6.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling." pith.science (2026). https://pith.science/paper/2OKANKZZ

@misc{pith2026241119492,
  author       = {Pith},
  title        = {Pith review of: Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2OKANKZZ}},
  note         = {Machine review of arXiv:2411.19492}
}
read the original abstract

Reconstructing structured 3D scenes from RGB images using CAD objects unlocks efficient and compact scene representations that maintain compositionality and interactability. Existing works propose training-heavy methods relying on either expensive yet inaccurate real-world annotations or controllable yet monotonous synthetic data that do not generalize well to unseen objects or domains. We present Diorama, the first zero-shot open-world system that holistically models 3D scenes from single-view RGB observations without requiring end-to-end training or human annotations. We show the feasibility of our approach by decomposing the problem into subtasks and introduce robust, generalizable solutions to each: architecture reconstruction, 3D shape retrieval, object pose estimation, and scene layout optimization. We evaluate our system on both synthetic and real-world data to show we significantly outperform baselines from prior work. We also demonstrate generalization to internet images and the text-to-scene task.

Figures

Figures reproduced from arXiv: 2411.19492 by the authors.

Figure 1
Figure 1. We propose Diorama: a system for zero-shot single-view 3D scene modeling. Our system produces a holistic 3D scene model [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the Diorama pipeline. The input image is processed in the open-world perception component (in orange box) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Our PlainRecon architecture reconstruction approach. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Our zero-shot object pose estimation approach. We [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 6
Figure 6. Figure 6: Comparison examples on SSDB. We capture architec [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Scene modeling examples of Diorama on SSDB images. The third column visualizes the support hierarchy of the generated scene [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Examples of planar architecture reconstruction on SSDB [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Examples on ScanNet images. We use the same mask, [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Examples on real-world internet images. The output scenes capture large and small objects, and represent complex relations [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Examples of applying Diorama in a text-to-scene set [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Illustration of scene graph generation using GPT-4o. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: We show the potential of Diorama in flexible scene editing in terms of different editing types, including object rearrangement, [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Qualitative comparison between different ground-truth [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: More comparison examples on SSDB. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: More Diorama examples on SSDB images. ing view with the minimal cyclical distance [26, 56]. We compute correspondence score as cosine similarity between patch embeddings. Specifically, given a pair of query image Iq and reference rendering image Ir, we construct a “cy…
Figure 17
Figure 17. Figure 17: More examples for real-world internet images. [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: More examples of applying Diorama in a text-to-scene setting. [PITH_FULL_IMAGE:figures/full_fig_p020_18.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Advances in 4D Representation: Geometry, Motion, and Interaction

    cs.CV 2025-10 conditional novelty 4.0 of 10

    A representation-centric survey of 4D generation and reconstruction, organized by geometry, motion, and interaction, with qualitative trade-off comparisons across seven representation families.

Reference graph

Works this paper leans on

100 extracted references · 56 canonical work pages · cited by 1 Pith paper

  1. [1]

    SATR: Zero-shot semantic segmentation of 3D shapes

    Ahmed Abdelreheem, Ivan Skorokhodov, Maks Ovsjanikov, and Peter Wonka. SATR: Zero-shot semantic segmentation of 3D shapes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15166–15179, 2023. 2

    2023

  2. [2]

    SceneCom- plete: Open-world 3D scene completion in complex real world environments for robot manipulation

    Aditya Agarwal, Gaurav Singh, Bipasha Sen, Tom ´as Lozano-P´erez, and Leslie Pack Kaelbling. SceneCom- plete: Open-world 3D scene completion in complex real world environments for robot manipulation. arXiv preprint arXiv:2410.23643, 2024. 2

    arXiv

  3. [3]

    Open-universe indoor scene generation using LLM program synthesis and uncurated object databases

    Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stew- art Morris, Seung Jean Yoo, Aditya Ganeshan, R Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, and Daniel Ritchie. Open-universe indoor scene generation using LLM program synthesis and uncurated object databases. arXiv preprint arXiv:2403.09675, 2024. 2, 3

    arXiv 2024

  4. [4]

    Scan2CAD: Learning CAD model alignment in RGB-D scans

    Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X Chang, and Matthias Nießner. Scan2CAD: Learning CAD model alignment in RGB-D scans. In Pro- ceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 2614–2623, 2019. 2, 8

    2019

  5. [5]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the International Conference on Computer Vi- sion (ICCV), 2021. 7

    2021

  6. [6]

    ShapeNet: An information-rich 3D model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015. 4, 8

    arXiv 2015

  7. [7]

    CLIP2Scene: Towards label-efficient 3D scene un- derstanding by CLIP

    Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wenping Wang. CLIP2Scene: Towards label-efficient 3D scene un- derstanding by CLIP. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7020–7030, 2023. 2

    2023

  8. [8]

    Single-view 3D scene reconstruc- tion with high-fidelity shape and texture

    Yixin Chen, Junfeng Ni, Nan Jiang, Yaowei Zhang, Yixin Zhu, and Siyuan Huang. Single-view 3D scene reconstruc- tion with high-fidelity shape and texture. In 2024 Interna- tional Conference on 3D Vision (3DV) , pages 1456–1467. IEEE, 2024. 2

    2024

Show all 100 references
  1. [9]

    Urdformer: A pipeline for constructing articulated simulation environments from real-world images

    Zoey Chen, Aaron Walsman, Marius Memmel, Kaichun Mo, Alex Fang, Karthikeya Vemuri, Alan Wu, Dieter Fox, and Abhishek Gupta. Urdformer: A pipeline for constructing articulated simulation environments from real-world images. arXiv preprint arXiv:2405.11656, 2024. 2

    2024 arXiv

  2. [10]

    ScanNet: Richly-annotated 3D reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 8

    2017

  3. [11]

    ACDC: Automated creation of digital cousins for robust policy learn- ing

    Tianyuan Dai, Josiah Wong, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. ACDC: Automated creation of digital cousins for robust policy learn- ing. arXiv preprint arXiv:2410.07408, 2024. 2, 5, 6

    2024 arXiv

  4. [12]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  5. [13]

    SceneFun3D: Fine-grained functionality and affordance un- derstanding in 3D scenes

    Alexandros Delitzas, Ayca Takmaz, Federico Tombari, Robert Sumner, Marc Pollefeys, and Francis Engelmann. SceneFun3D: Fine-grained functionality and affordance un- derstanding in 3D scenes. InIEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  6. [14]

    PLA: Language-driven open- vocabulary 3D scene understanding

    Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. PLA: Language-driven open- vocabulary 3D scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7010–7019, 2023. 2

    2023

  7. [15]

    PanoContext-Former: Panoramic total scene understanding with a transformer

    Yuan Dong, Chuan Fang, Liefeng Bo, Zilong Dong, and Ping Tan. PanoContext-Former: Panoramic total scene understanding with a transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28087–28097, 2024. 2

    2024

  8. [16]

    CLIP- Away: Harmonizing focused embeddings for removing ob- jects via diffusion models

    Yi ˘git Ekin, Ahmet Burak Yildirim, Erdem Eren Caglar, Aykut Erdem, Erkut Erdem, and Aysegul Dundar. CLIP- Away: Harmonizing focused embeddings for removing ob- jects via diffusion models. In Advances in Neural Informa- tion Processing Systems, 2024. 15

    2024

  9. [17]

    Prob- ing the 3D awareness of visual foundation models

    Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Ab- hishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas Guibas, Justin Johnson, and Varun Jampani. Prob- ing the 3D awareness of visual foundation models. In Pro- ceedings of the IEEE/CVF Conference on Computer ...

    2024

  10. [18]

    A density-based algorithm for discovering clusters in large spatial databases with noise

    Martin Ester, Hans-Peter Kriegel, J ¨org Sander, and Xiaowei Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Sec- ond International Conference on Knowledge Discovery and Data Mining, 1996. 15

    1996

  11. [19]

    Fischler and Robert C

    Martin A. Fischler and Robert C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 16

    1981

  12. [20]

    Example-based synthesis of 3d object arrangements

    Matthew Fisher, Daniel Ritchie, Manolis Savva, Thomas Funkhouser, and Pat Hanrahan. Example-based synthesis of 3d object arrangements. ACM Transactions on Graphics (TOG), 31(6):1–11, 2012. 5 9

    2012

  13. [21]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 10933–10942,

  14. [22]

    Scene-LLM: Extending language model for 3D visual understanding and reasoning

    Rao Fu, Jingyu Liu, Xilun Chen, Yixin Nie, and Wen- han Xiong. Scene-LLM: Extending language model for 3D visual understanding and reasoning. arXiv preprint arXiv:2403.11401, 2024. 2

    2024 arXiv

  15. [23]

    Any- home: Open-vocabulary generation of structured and tex- tured 3d homes

    Rao Fu, Zehao Wen, Zichen Liu, and Srinath Sridhar. Any- home: Open-vocabulary generation of structured and tex- tured 3d homes. In European Conference on Computer Vi- sion, pages 52–70. Springer, 2025. 2, 3

    2025

  16. [24]

    DiffCAD: Weakly-supervised probabilistic CAD model retrieval and alignment from an RGB image

    Daoyi Gao, David Rozenberszki, Stefan Leutenegger, and Angela Dai. DiffCAD: Weakly-supervised probabilistic CAD model retrieval and alignment from an RGB image. ACM Transactions on Graphics (TOG) , 43(4):1–15, 2024. 2, 3, 6, 8, 21

    2024

  17. [25]

    GraphDreamer: Compositional 3D scene synthesis from scene graphs

    Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Sch ¨olkopf. GraphDreamer: Compositional 3D scene synthesis from scene graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21295–21304, 2024. 3

    2024

  18. [26]

    Zero-shot category-level object pose estimation

    Walter Goodwin, Sagar Vaze, Ioannis Havoutis, and Ingmar Posner. Zero-shot category-level object pose estimation. In European Conference on Computer Vision, pages 516–532. Springer, 2022. 6, 7, 16, 18

    2022

  19. [27]

    ROCA: Robust CAD model retrieval and alignment from a single image

    Can G ¨umeli, Angela Dai, and Matthias Nießner. ROCA: Robust CAD model retrieval and alignment from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2, 3, 8, 21

    2022

  20. [28]

    3D-LLM: In- jecting the 3D world into large language models

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3D-LLM: In- jecting the 3D world into large language models. Advances in Neural Information Processing Systems, 36:20482–20494,

  21. [29]

    Metric3D v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3D v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation. arXiv preprint arXiv:2404.15506, 20...

    2024 arXiv

  22. [30]

    Aladdin: Zero-shot hallucination of stylized 3D assets from abstract scene descriptions

    Ian Huang, Vrishab Krishna, Omoruyi Atekha, and Leonidas Guibas. Aladdin: Zero-shot hallucination of stylized 3D assets from abstract scene descriptions. arXiv preprint arXiv:2306.06212, 2023. 2

    2023 arXiv

  23. [31]

    Holistic 3D scene parsing and re- construction from a single RGB image

    Siyuan Huang, Siyuan Qi, Yixin Zhu, Yinxue Xiao, Yuanlu Xu, and Song-Chun Zhu. Holistic 3D scene parsing and re- construction from a single RGB image. In Proceedings of the European conference on computer vision (ECCV), pages 187–203, 2018. 2

    2018

  24. [32]

    OpenIns3D: Snap and lookup for 3D open-vocabulary instance segmentation

    Zhening Huang, Xiaoyang Wu, Xi Chen, Hengshuang Zhao, Lei Zhu, and Joan Lasenby. OpenIns3D: Snap and lookup for 3D open-vocabulary instance segmentation. arXiv preprint arXiv:2309.00616, 2023. 2

    2023 arXiv

  25. [33]

    CenterSnap: Single-shot multi-object 3D shape reconstruction and categorical 6D pose and size estimation

    Muhammad Zubair Irshad, Thomas Kollar, Michael Laskey, Kevin Stone, and Zsolt Kira. CenterSnap: Single-shot multi-object 3D shape reconstruction and categorical 6D pose and size estimation. In 2022 International Conference on Robotics and Automation (ICRA) , pages 10632–10640....

    2022

  26. [34]

    Hamid Izadinia, Qi Shan, and Steven M Seitz. Im2cad. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5134–5143, 2017. 2

    2017

  27. [35]

    ConceptFusion: Open-set multimodal 3D mapping

    Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Chen, Alaa Maalouf, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, et al. ConceptFusion: Open-set multimodal 3D mapping. arXiv preprint arXiv:2302.07241, 2023. 2

    2023 arXiv

  28. [36]

    SceneVerse: Scaling 3D vision-language learning for grounded scene understanding

    Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. SceneVerse: Scaling 3D vision-language learning for grounded scene understanding. In European Conference on Computer Vision, pages 289–310. Springer, 2025. 2

  29. [37]

    LERF: Language embed- ded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. LERF: Language embed- ded radiance fields. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 19729– 19739, 2023. 2

    2023

  30. [38]

    Habitat synthetic scenes dataset (HSSD-200): An analysis of 3D scene scale and realism tradeoffs for objectgoal naviga- tion

    Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva. Habitat synthetic scenes dataset (HSSD-200): An analysis of 3D scene scale and realism tradeoffs for objectgoal naviga-...

  31. [39]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 3, 14

    2023 arXiv

  32. [40]

    Mask2CAD: 3D shape prediction by learning to seg- ment and retrieve

    Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, and Angela Dai. Mask2CAD: 3D shape prediction by learning to seg- ment and retrieve. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pages 260–277. Springer,

  33. [41]

    Patch2cad: Patchwise embedding learning for in-the- wild shape retrieval from a single image

    Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, and Angela Dai. Patch2cad: Patchwise embedding learning for in-the- wild shape retrieval from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 12589–12599, 2021. 2

    2021

  34. [42]

    Langer, G

    F. Langer, G. Bae, I. Budvytis, and R. Cipolla. SPARC: Sparse render-and-compare for CAD model alignment in a single RGB image. In Proc. British Machine Vision Confer- ence, London, 2022. 2, 3, 21

    2022

  35. [43]

    FastCAD: Real-time CAD re- trieval and alignment from scans and videos

    Florian Langer, Jihong Ju, Georgi Dikov, Gerhard Reitmayr, and Mohsen Ghafoorian. FastCAD: Real-time CAD re- trieval and alignment from scans and videos. arXiv preprint arXiv:2403.15161, 2024. 2

    2024 arXiv

  36. [44]

    Duoduo CLIP: Efficient 3D understanding with multi-view images

    Han-Hung Lee, Yiming Zhang, and Angel X Chang. Duoduo CLIP: Efficient 3D understanding with multi-view images. arXiv preprint arXiv:2406.11579, 2024. 2, 4

    2024 arXiv

  37. [45]

    Evaluating real-world robot manipulation policies in simulation

    Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, 10 Isabel Sieh, Sean Kirmani, et al. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941, 2024. 2

    2024 arXiv

  38. [46]

    InstructScene: Instruction- driven 3D indoor scene synthesis with semantic graph prior

    Chenguo Lin and Yadong Mu. InstructScene: Instruction- driven 3D indoor scene synthesis with semantic graph prior. arXiv preprint arXiv:2402.04717, 2024. 3

    2024 arXiv

  39. [47]

    Towards high-fidelity single-view holistic reconstruction of indoor scenes

    Haolin Liu, Yujian Zheng, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Towards high-fidelity single-view holistic reconstruction of indoor scenes. In European Con- ference on Computer Vision, pages 429–446. Springer, 2022. 2

    2022

  40. [48]

    LASA: Instance reconstruction from real scans using a large-scale aligned shape annotation dataset, 2023, arXiv:2312.12418

    Haolin Liu, Chongjie Ye, Yinyu Nie, Yingfan He, and Xi- aoguang Han. LASA: Instance reconstruction from real scans using a large-scale aligned shape annotation dataset, 2023, arXiv:2312.12418. 2

    2023 arXiv

  41. [49]

    PartSLIP: Low-shot part segmentation for 3D point clouds via pretrained image- language models

    Minghua Liu, Yinhao Zhu, Hong Cai, Shizhong Han, Zhan Ling, Fatih Porikli, and Hao Su. PartSLIP: Low-shot part segmentation for 3D point clouds via pretrained image- language models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages...

    2023

  42. [50]

    OpenShape: Scaling up 3D shape representation towards open-world understanding

    Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xu- anlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. OpenShape: Scaling up 3D shape representation towards open-world understanding. Advances in neural information processing systems, 36, 2024. 2, 7

    2024

  43. [51]

    Open-vocabulary point-cloud object detection without 3D annotation

    Yuheng Lu, Chenfeng Xu, Xiaobao Wei, Xiaodong Xie, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. Open-vocabulary point-cloud object detection without 3D annotation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1190–1199,

  44. [52]

    Vid2CAD: CAD model alignment using multi-view constraints from videos

    Kevis-Kokitsi Maninis, Stefan Popov, Matthias Nießner, and Vittorio Ferrari. Vid2CAD: CAD model alignment using multi-view constraints from videos. IEEE Transactions on Pattern Analysis and Machine Inttelligence, 2022. 2

    2022

  45. [53]

    Cad-estate: Large-scale cad model annota- tion in rgb videos

    Kevis-Kokitsi Maninis, Stefan Popov, Matthias Nießner, and Vittorio Ferrari. Cad-estate: Large-scale cad model annota- tion in rgb videos. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 20189–20199,

  46. [54]

    Scaling open-vocabulary object detection

    Neil Houlsby Matthias Minderer, Alexey Gritsenko. Scaling open-vocabulary object detection. NeurIPS, 2023. 3, 14

    2023

  47. [55]

    Open3DIS: Open-vocabulary 3D instance segmentation with 2D mask guidance

    Phuc Nguyen, Tuan Duc Ngo, Evangelos Kalogerakis, Chuang Gan, Anh Tran, Cuong Pham, and Khoi Nguyen. Open3DIS: Open-vocabulary 3D instance segmentation with 2D mask guidance. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 4018–402...

    2024

  48. [56]

    GigaPose: Fast and Robust Novel Ob- ject Pose Estimation via One Correspondence

    Van Nguyen Nguyen, Thibault Groueix, Mathieu Salzmann, and Vincent Lepetit. GigaPose: Fast and Robust Novel Ob- ject Pose Estimation via One Correspondence. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 4, 6, 7, 16, 18, 20

    2024

  49. [57]

    Total3DUnderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image

    Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. Total3DUnderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2020

  50. [58]

    DINOv2: Learning robust visual features without supervi- sion

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning robust visual features without supervi- sion. arXiv preprint arXiv:2304.07193, 2023. 4, 20

    2023 arXiv

  51. [59]

    ATISS: Autore- gressive transformers for indoor scene synthesis

    Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. ATISS: Autore- gressive transformers for indoor scene synthesis. Advances in Neural Information Processing Systems, 34:12013–12026,

  52. [60]

    OpenScene: 3D scene understanding with open vocabular- ies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. OpenScene: 3D scene understanding with open vocabular- ies. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 815–824, 2023. 2

    2023

  53. [61]

    LangSplat: 3D language gaussian splat- ting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. LangSplat: 3D language gaussian splat- ting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 20051–20060,

  54. [62]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  55. [63]

    Fast and flex- ible indoor scene synthesis via deep convolutional genera- tive models

    Daniel Ritchie, Kai Wang, and Yu-an Lin. Fast and flex- ible indoor scene synthesis via deep convolutional genera- tive models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6182– 6190, 2019. 3

    2019

  56. [64]

    Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE/CVF international conference o...

    2021

  57. [65]

    Estimating generic 3D room structures from 2D annotations

    Denys Rozumnyi, Stefan Popov, Kevis-Kokitsi Maninis, Matthias Nießner, and Vittorio Ferrari. Estimating generic 3D room structures from 2D annotations. In Advances in Neural Information Processing Systems, 2023. 4, 21

    2023

  58. [66]

    Computational geometry

    Michael Ian Shamos. Computational geometry. Yale Uni- versity, 1978. 16

    1978

  59. [67]

    PlaneRecTR: Uni- fied query learning for 3D plane recovery from a single view

    Jingjia Shi, Shuaifeng Zhi, and Kai Xu. PlaneRecTR: Uni- fied query learning for 3D plane recovery from a single view. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9377–9386, 2023. 21

    2023

  60. [68]

    General 3D room layout from a single view by render-and-compare

    Sinisa Stekovic, Shreyas Hampali, Mahdi Rad, Sayan Deb Sarkar, Friedrich Fraundorfer, and Vincent Lepetit. General 3D room layout from a single view by render-and-compare. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 187–203. Springer, 2020. 4, 5, 6, 21

    2020

  61. [69]

    Lempitsky

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, 11 Naejin Kong, Harshith Goka, Kiwoong Park, and Victor S. Lempitsky. Resolution-robust large mask inpainting with Fourier convolutions. In Proceedings of the Winter C...

    2022

  62. [70]

    Habitat 2.0: Training home assistants to rearrange their habitat

    Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wi- jmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, et al. Habitat 2.0: Training home assistants to rearrange their habitat. Advances in neural information processin...

    2021

  63. [71]

    Open- Mask3D: Open-vocabulary 3D instance segmentation

    Ayc ¸a Takmaz, Elisabetta Fedele, Robert W Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. Open- Mask3D: Open-vocabulary 3D instance segmentation. arXiv preprint arXiv:2306.13631, 2023. 2

    2023 arXiv

  64. [72]

    SceneMotifCoder: Example-driven visual program learning for generating 3D object arrangements

    Hou In Ivan Tam, Hou In Derek Pun, Austin T Wang, Angel X Chang, and Manolis Savva. SceneMotifCoder: Example-driven visual program learning for generating 3D object arrangements. arXiv preprint arXiv:2408.02211 ,

    arXiv

  65. [73]

    DiffuScene: Denoising diffu- sion models for generative indoor scene synthesis

    Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias Nießner. DiffuScene: Denoising diffu- sion models for generative indoor scene synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20507–20518, 2024. 3

    2024

  66. [74]

    Least-squares estimation of transforma- tion parameters between two point patterns

    Shinji Umeyama. Least-squares estimation of transforma- tion parameters between two point patterns. IEEE Transac- tions on Pattern Analysis & Machine Intelligence , 13(04): 376–380, 1991. 4

    1991

  67. [75]

    Deep convolutional priors for indoor scene syn- thesis

    Kai Wang, Manolis Savva, Angel X Chang, and Daniel Ritchie. Deep convolutional priors for indoor scene syn- thesis. ACM Transactions on Graphics (TOG), 37(4):1–14,

  68. [76]

    PlanIT: Planning and in- stantiating indoor scenes with relation graph and spatial prior networks

    Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, An- gel X Chang, and Daniel Ritchie. PlanIT: Planning and in- stantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics (TOG), 38(4):1– 15, 2019. 3

    2019

  69. [77]

    Lift3D: Zero-shot lifting of any 2D vi- sion model to 3D

    Peihao Wang, Zhiwen Fan, Zhangyang Wang, Hao Su, Ravi Ramamoorthi, et al. Lift3D: Zero-shot lifting of any 2D vi- sion model to 3D. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 21367–21377, 2024. 2

    2024

  70. [78]

    SceneFormer: Indoor scene generation with transformers

    Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner. SceneFormer: Indoor scene generation with transformers. In 2021 International Conference on 3D Vision (3DV) , pages 106–115. IEEE, 2021. 3

    2021

  71. [79]

    Lego-net: Learning regular rearrangements of ob- jects in rooms

    Qiuhong Anna Wei, Sijie Ding, Jeong Joon Park, Rahul Sajnani, Adrien Poulenard, Srinath Sridhar, and Leonidas Guibas. Lego-net: Learning regular rearrangements of ob- jects in rooms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19...

    2023

  72. [80]

    R3ds: Reality-linked 3d scenes for panoramic scene understanding

    Qirui Wu, Sonia Raychaudhuri, Daniel Ritchie, Manolis Savva, and Angel X Chang. R3ds: Reality-linked 3d scenes for panoramic scene understanding. In Proceedings of the European Conference on Computer Vision (ECCV), 2024. 2

    2024

  73. [81]

    Generalizing single-view 3D shape retrieval to occlu- sions and unseen objects

    Qirui Wu, Daniel Ritchie, Manolis Savva, and Angel X Chang. Generalizing single-view 3D shape retrieval to occlu- sions and unseen objects. In 2024 International Conference on 3D Vision (3DV), pages 893–902. IEEE, 2024. 6

    2024

  74. [82]

    ULIP-2: Towards scal- able multimodal pre-training for 3D understanding

    Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Jun- nan Li, Roberto Mart´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. ULIP-2: Towards scal- able multimodal pre-training for 3D understanding. In Pro- ceedings of the IEEE/CVF Conference on Compute...

    2024

  75. [83]

    Learning to reconstruct 3d non-cuboid room layout from a single rgb image

    Cheng Yang, Jia Zheng, Xili Dai, Rui Tang, Yi Ma, and Xi- aojun Yuan. Learning to reconstruct 3d non-cuboid room layout from a single rgb image. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022. 21

    2022

  76. [84]

    Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4V

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4V. arXiv preprint arXiv:2310.11441, 2023. 3

    2023 arXiv

  77. [85]

    Depth any- thing v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2. arXiv:2406.09414, 2024. 15

    2024 arXiv

  78. [86]

    ImOV3D: Learning open-vocabulary point clouds 3D object detection from only 2D images

    Timing Yang, Yuanliang Ju, and Li Yi. ImOV3D: Learning open-vocabulary point clouds 3D object detection from only 2D images. arXiv preprint arXiv:2410.24001, 2024. 2

    2024 arXiv

  79. [87]

    Holodeck: Language guided gen- eration of 3D embodied AI environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Al- varo Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided gen- eration of 3D embodied AI environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and ...

    2024

  80. [88]

    Multi-view aggregation network for dichoto- mous image segmentation

    Qian Yu, Xiaoqi Zhao, Youwei Pang, Lihe Zhang, and Huchuan Lu. Multi-view aggregation network for dichoto- mous image segmentation. arXiv:2404.07445, 2024. 15

    2024 arXiv

  81. [89]

    Inpaint anything: Segment anything meets image inpainting

    Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. Inpaint anything: Segment anything meets image inpainting. arXiv:2304.06790, 2023. 4, 15

    2023 arXiv

  82. [90]

    Improving 2D feature representations by 3D-aware fine-tuning

    Yuanwen Yue, Anurag Das, Francis Engelmann, Siyu Tang, and Jan Eric Lenssen. Improving 2D feature representations by 3D-aware fine-tuning. In European Conference on Com- puter Vision, pages 57–74. Springer, 2025. 2

    2025

  83. [91]

    DeepPanoCon- text: Panoramic 3D scene understanding with holistic scene context graph and relation-based optimization

    Cheng Zhang, Zhaopeng Cui, Cai Chen, Shuaicheng Liu, Bing Zeng, Hujun Bao, and Yinda Zhang. DeepPanoCon- text: Panoramic 3D scene understanding with holistic scene context graph and relation-based optimization. In Proceed- ings of the IEEE/CVF International Conference on Com- ...

    2021

  84. [92]

    CLIP-FO3D: Learning free open-world 3D scene representations from 2D dense CLIP

    Junbo Zhang, Runpei Dong, and Kaisheng Ma. CLIP-FO3D: Learning free open-world 3D scene representations from 2D dense CLIP. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2048–2059, 2023. 2

    2023

  85. [93]

    Structured3D: A large photo-realistic dataset for structured 3D modeling

    Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou. Structured3D: A large photo-realistic dataset for structured 3D modeling. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 519–535. Springer, 2020. 2

    2020

  86. [94]

    Bilateral refer- 12 ence for high-resolution dichotomous image segmentation

    Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, and Nicu Sebe. Bilateral refer- 12 ence for high-resolution dichotomous image segmentation. CAAI Artificial Intelligence Research, 3:9150038, 2024. 15

    2024

  87. [95]

    Zero-shot scene reconstruction from single images with deep prior as- sembly

    Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Zero-shot scene reconstruction from single images with deep prior as- sembly. arXiv preprint arXiv:2410.15971, 2024. 2

    2024 arXiv

  88. [96]

    Open3D: A modern library for 3D data processing

    Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A modern library for 3D data processing. arXiv:1801.09847,

    arXiv

  89. [97]

    Point- CLIP v2: Prompting CLIP and GPT for powerful 3D open- world learning

    Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Ziyao Zeng, Zipeng Qin, Shanghang Zhang, and Peng Gao. Point- CLIP v2: Prompting CLIP and GPT for powerful 3D open- world learning. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , pages 2639–2650,

  90. [98]

    GRS: Generating robotic simulation tasks from real-world images

    Alex Zook, Fan-Yun Sun, Josef Spjut, Valts Blukis, Stan Birchfield, and Jonathan Tremblay. GRS: Generating robotic simulation tasks from real-world images. arXiv preprint arXiv:2410.15536, 2024. 2

    2024

  91. [99]

    gpt-4o-2024-08-06

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. In Ad- vances in Neural Information Processing Systems, 2023. 15 13 In this supplement, we provide additional details...

    2023

  92. [100]

    a photo of CLASS

    to separate the initial cluster as it is possible to have multiple walls with identical normals in the scene. The al- gorithm terminates when we either run out of seed normals or have less than the threshold number of unclustered points left (Nmin = 200). Finally, we propagate...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.