Pith. sign in

REVIEW 4 major objections 6 minor 114 references

MetaScenes: Towards Automated Replica Creation for Real-world 3D Scans

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that human preference rankings of candidate 3D assets let a learned model turn real-world scans into large interactive training scenes without artist-driven design.

desk verdict A substantial real-to-sim dataset resource with genuinely new ranked-candidate annotation; the evaluation claims need a disclosure-and-rigor pass before acceptance. read the letter →

arxiv 2505.02388 v1 pith:ONCI2RLR submitted 2025-05-05 cs.CV cs.AIcs.LGcs.RO

Huangyue Yu , Baoxiong Jia , Yixin Chen , Yandan Yang , Puhao Li , Rongpeng Su , Jiaxin Li , Qing Li
show 4 more authors
Wei Liang Song-Chun Zhu Tengyu Liu Siyuan Huang
This is my paper · ORCID
classification cs.CVcs.AIcs.LGcs.RO
keywords 3Dscenedatasetreal-to-simassetretrievalmulti-modalalignmentembodiedAIsynthesisvision-and-languagenavigationScanNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MetaScenes claims that a large, simulation-ready 3D scene dataset can be built from real-world scans by replacing every scanned object with a high-quality simulatable asset, and that the replacement process can be automated well enough to remove the usual reliance on artist-designed scenes. The dataset contains 15,366 objects across 831 fine-grained categories in 706 rooms reconstructed from ScanNet, with at least six candidate assets per object. The paper's key move is to have human annotators rank those candidates, turning subjective replacement quality into training labels for Scan2Sim, a multimodal model that selects the best asset from images, text, and point clouds. Two benchmarks, micro-scene synthesis for small-object layouts and cross-domain vision-and-language navigation, are used to show that training on these scenes transfers to unseen rooms and to a real robot. If true, embodied AI could scale training environments directly from everyday scans rather than from manual 3D design.

What carries the argument

The load-bearing mechanism is Scan2Sim, a multi-modal contrastive retrieval model that aligns an object's image and text description with candidate 3D point clouds, trained with a cross-modal matching loss against human ranking annotations. It is supported by a construction pipeline that uses SAM and GPT-4V to caption scanned objects, text-to-3D retrieval and generation to build candidate assets, and a physics-based MCMC optimization that adjusts placements to remove collisions and floating objects. The rankings are what convert subjective replacement quality into a learnable objective, so the model and the benchmarks both depend on them.

What would settle it

Have several independent annotators rank the same candidate asset sets for a sample of objects and measure rank agreement; if agreement is low, or if a Scan2Sim model trained on one set of rankings predicts another annotator's choices no better than a random baseline, then the claim that these rankings teach a reliable automated asset-selection model would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that human preference rankings of replacement assets provide the supervision needed to learn automated replica creation from real-world scans. On the paper's evidence, Scan2Sim trained on these rankings selects the best asset at 28.4% Top-1 accuracy on the MetaScenes test set, outperforming baselines including GPT-4V at 16.5% and ULIP-2 at 13.1%, and the resulting replicas are closer to the original scans than Scan2CAD's, with a Chamfer distance of 0.25 versus 0.35. The same scenes, after physics-based optimization, support a navigation agent that improves held-out-domain success rate by 5.34 percentage points over training on procedurally generated ProcTHOR scenes. The paper presents the dataset itself, the Scan2Sim pipeline, and the two benchmarks as a package: the dataset is the evidence, the ranking annotations are the ground truth, and the benchmarks are the demonstration that the replicas are useful for embodied agents.

Load-bearing premise

The load-bearing premise is that human annotators rank replacement assets consistently and correctly; the paper reports quality checks on only 10% of batches and gives no inter-annotator reliability numbers, so if those rankings are noisy or biased, the retrieval model, the Chamfer-distance comparison, and the benchmark conclusions all inherit that noise.

Editorial extensions

If this is right

  • New real-world scans, such as the ScanNet++ scenes tested in the paper, can be converted into simulatable replicas without per-scene artist work.
  • Training on MetaScenes improves object-goal navigation on unseen scenes and domains compared with training on procedurally generated scenes alone.
  • The Micro-Scene Synthesis benchmark makes small-object layout generation a measurable task, giving manipulation research a data source it previously lacked.
  • The ranking annotations provide a reusable evaluation target for any future automated asset-selection system.
  • The resulting scenes are physically optimized and interactable, so they can be dropped directly into embodied-agent simulators.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If ranking noise is the main bottleneck, then replacing or augmenting human rankings with a learned preference model, or with pairwise comparisons from a larger annotator pool, could push Scan2Sim-style selection well beyond the reported 28.4% Top-1 accuracy.
  • The VLN result that navigation to small items is a weak point suggests the same dataset could be used to probe whether object-goal navigation failures concentrate on small objects, and whether manipulation policies trained in these scenes inherit that benefit.
  • Because the candidate pool is built partly from generative models, the pipeline's ceiling is tied to generator quality; as image-to-3D generation improves, the same annotation pipeline should yield higher-fidelity replicas without redesign.
  • A per-category breakdown of Chamfer distance versus Scan2CAD would clarify whether the reported accuracy gain comes uniformly from all objects or mostly from small items where Scan2CAD has no equivalents.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MetaScenes, a large-scale simulatable 3D scene dataset built by replacing objects in 706 real-world ScanNet scans with 15,366 simulatable assets spanning 831 fine-grained categories, with at least six candidate assets per object (98,423 unique 3D assets in total). The construction pipeline combines room-layout estimation, foundation-model-driven asset curation (text-to-3D, image-to-3D, and retrieval), human ranking and placement annotation, and physics-based optimization. The paper also proposes Scan2Sim, a multimodal alignment model trained on the ranking annotations to automate asset retrieval and pose alignment, and two downstream benchmarks: Micro-Scene Synthesis for small-object layouts and cross-domain vision-and-language navigation (VLN). The central claims are that MetaScenes offers a scalable alternative to artist-driven scene creation, that Scan2Sim outperforms existing baselines on asset selection and pose alignment, and that training on MetaScenes improves agent generalization and sim-to-real transfer.

Significance. If the claims hold, MetaScenes is a substantial resource: it provides a large real-to-sim dataset with per-object candidate pools, human preference rankings, physical attributes, spatial relations, and downstream benchmarks, and it tackles a genuinely important scalability problem in embodied AI. The paper deserves credit for combining dataset construction with a concrete baseline model, two evaluation tasks, and a real-robot deployment, and for attempting quantitative quality analysis against Scan2CAD. However, the load-bearing quality evidence is currently thinner than the claims require. The human ranking annotations are both the training signal for Scan2Sim and the reference for all asset-selection metrics, yet no inter-annotator reliability is reported; the ScanNet++ pose ground truth is produced by the same annotation procedure; and the Chamfer-distance comparisons lack variance and metric details. These are fixable with additional analysis and experiments, but they currently leave the core quality claims under-supported.

major comments (4)
  1. [Sec. 3.2, Supp. A.2, Eq. (2), Table 2] The ranking annotations are the training signal for Scan2Sim (Eq. 2) and the reference for every asset-selection metric in Table 2, but the only reliability check reported is a 10% per-batch QC pass with a 98% threshold. No inter-annotator agreement, rank correlation, or noise floor is reported. If the rankings are noisy or systematically biased, the Top-1 accuracy of 28.4%, the CD/ECD/IoU/color-histogram comparisons, and the conclusion that Scan2Sim outperforms baselines all inherit that noise. Please report inter-annotator agreement (e.g., Kendall's W or Krippendorff's alpha on a subset of objects annotated by multiple annotators) and a sensitivity analysis of Table 2 under label noise, such as training with corrupted rankings or evaluating on a consensus-only subset.
  2. [Sec. 4.1, Table 3] The ScanNet++ pose ground truth 'is annotated following the same procedure in Sec. 3.2', meaning the same annotation interface and placement conventions (center alignment, longest-side scaling, and 30-degree rotation increments) were used. The pose-alignment gains on ScanNet++ (CD 0.21 vs ACDC 0.26) may therefore indicate agreement with the annotators' placement conventions rather than true geometric fidelity to the scans. Please evaluate on independent pose ground truth, for example ICP-refined alignments or manually verified absolute poses with finer rotation resolution, and report absolute pose errors rather than only differences from the annotation convention.
  3. [Sec. 3.3] The Chamfer-distance quality comparison (0.25 vs 0.35) is reported without error bars, normalization, or a precise definition of the metric, including which point clouds are compared, whether the distance is symmetric or one-sided, and what units are used. Additionally, the candidate pool includes image-to-3D reconstructions (TripoSR, InstantMesh, Michelangelo) conditioned on the target object's own image, so low CD may partly reflect appearance overfitting to the target view rather than true replica fidelity. Please specify the CD computation protocol, report per-category statistics and variances, and ablate retrieval-only vs generation-inclusive candidate pools to separate these effects.
  4. [Sec. 4.3, Table 5] The VLN results are reported as single runs without multiple seeds or variance, and the Heldout Scenes differences are small (SR 52.64 vs 51.21 for ProcTHOR, with the combined dataset at 51.36). The Heldout Domains evaluation uses only 10 ScanNet++ scenes. These results are load-bearing for the claim that MetaScenes improves agent generalization, but without variance estimates and significance testing the observed gains may be within noise. Please report means and standard errors over at least three seeds and a paired statistical test (e.g., bootstrap over trajectories or scenes).
minor comments (6)
  1. [Sec. 4.1, Table 2] The modality notation 'I+TØI', 'T→P', etc., is used without definition; please define the arrow notation in the text or table caption.
  2. [Sec. 3.4, Eqs. (1)-(3)] The losses in Eqs. (2) and (3) use vector-valued scores q but do not explicitly show the softmax or candidate dimension; clarify the indexing and whether σ is softmax over the L candidates.
  3. [Sec. 4.3, para. 2] There is a typo: 'ProcPHOR' should be 'ProcTHOR'.
  4. [Sec. 3.3] The phrase 'similarity score' is used for a Chamfer distance where lower is better; please call it a distance or clarify the sign convention.
  5. [Fig. 1, Sec. 3.2] The labels 'A/B/C' in Fig. 1 for scene-level randomization and object-level augmentation are not explained in the caption; please annotate them.
  6. [Fig. A5, Supp. A.1] The asterisk in 'Shape-E*' and 'Michelangelo*' is not defined in the main text; define it as texture optimization in the caption.

Circularity Check

0 steps flagged · score 1.0 of 10

No load-bearing circularity: the central dataset and model claims are self-contained, with only minor self-referential evaluation caveats.

full rationale

The paper's load-bearing claims are the MetaScenes dataset, the Scan2Sim retrieval model, and the downstream benchmarks. Scan2Sim is trained on human ranking annotations (Eq. 2 and Eq. 3) and evaluated on a held-out split of the same annotation procedure; this is standard supervised evaluation, not circular reasoning. The strongest external validation comes from the VLN experiments: training on MetaScenes transfers to Heldout Scenes and to the completely external ScanNet++ Heldout Domain (Tab. 5), and the supplementary compares against ReplicaCAD and real-world AGV deployment, all of which are independent of the asset-ranking annotations. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from prior work, and no ansatz is smuggled in via citation. Two minor caveats keep the score slightly above zero: (i) the paper cites its own SceneVerse work [32] for scene-graph construction and PointNet++ pretraining, but this is a component-level reuse and not load-bearing for the central claim; and (ii) the ScanNet++ pose-alignment ground truth is 'annotated following the same procedure in Sec. 3.2,' making that particular out-of-domain evaluation less independent, and the CD quality comparison in Sec. 3.3 is partly confounded by image-to-3D candidates generated from the target object's own image. These are evaluation-protocol weaknesses rather than reductions-by-construction of the paper's derivation chain, so they do not constitute circularity under the strict standard used here.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central resource is a dataset, so the main unverified inputs are the annotations and model-generated physical properties. No new physical entities or theoretical constructs are introduced, but the quality claims depend on the reliability of human rankings, foundation model outputs, and geometric similarity metrics.

free parameters (1)
  • Per-object physics attributes (mass, friction, bounciness) = e.g., office chair: mass 20 kg, friction 0.5, bounciness 0
    These values are generated by GPT-4V from object label and size prompts (Supp. Tab. A2), not measured or validated, yet they support the claim that scenes are physically realizable.
assumptions (4)
  • domain assumption ScanNet instance segmentation and object labels are accurate enough for layout reconstruction.
    The whole pipeline uses ScanNet object instances as ground truth for placement and evaluation (Sec. 3.1).
  • domain assumption Human annotators give consistent, high-quality rankings of asset replacements.
    Selection and ranking labels are the supervision for Scan2Sim and the quality reference for the dataset; only 10% random batch checking is reported, with no inter-annotator agreement (Supp. A.2).
  • domain assumption Foundation model outputs (SAM masks, GPT-4V captions and physics properties, YOLO small-object detections) are reliable.
    These outputs drive caption generation, small-object discovery, and physical attribute annotation without independent verification (Sec. 3.1, Supp. A.1).
  • domain assumption Chamfer Distance and bounding box IoU capture replacement quality.
    These are the primary quantitative quality metrics in Sec. 3.3 and Table 3, without user studies or functional validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MetaScenes: Towards Automated Replica Creation for Real-world 3D Scans." pith.science (2026). https://pith.science/paper/ONCI2RLR

@misc{pith2026250502388,
  author       = {Pith},
  title        = {Pith review of: MetaScenes: Towards Automated Replica Creation for Real-world 3D Scans},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ONCI2RLR}},
  note         = {Machine review of arXiv:2505.02388}
}
read the original abstract

Embodied AI (EAI) research requires high-quality, diverse 3D scenes to effectively support skill acquisition, sim-to-real transfer, and generalization. Achieving these quality standards, however, necessitates the precise replication of real-world object diversity. Existing datasets demonstrate that this process heavily relies on artist-driven designs, which demand substantial human effort and present significant scalability challenges. To scalably produce realistic and interactive 3D scenes, we first present MetaScenes, a large-scale, simulatable 3D scene dataset constructed from real-world scans, which includes 15366 objects spanning 831 fine-grained categories. Then, we introduce Scan2Sim, a robust multi-modal alignment model, which enables the automated, high-quality replacement of assets, thereby eliminating the reliance on artist-driven designs for scaling 3D scenes. We further propose two benchmarks to evaluate MetaScenes: a detailed scene synthesis task focused on small item layouts for robotic manipulation and a domain transfer task in vision-and-language navigation (VLN) to validate cross-domain transfer. Results confirm MetaScene's potential to enhance EAI by supporting more generalizable agent learning and sim-to-real applications, introducing new possibilities for EAI research. Project website: https://meta-scenes.github.io/.

Figures

Figures reproduced from arXiv: 2505.02388 by the authors.

Figure 1
Figure 1. Overview of METASCENES, a large-scale simulatable 3D scene dataset constructed by replacing objects in real-world 3D scans with realistic and high-quality object assets retrieved or reconstructed from diverse sources. Abstract Embodied AI (EAI) research requires high-quality, di￾verse 3D scenes to effectively support skill acquisition, sim￾to-real transfer, and generalization. Achieving these qual￾ity standards, how… view at source ↗
Figure 2
Figure 2. The construction of METASCENES. METASCENES is composed of three sequential steps: (i) Collection, where we gather diverse 3D asset candidates for each real-world object in the scan; (ii) Annotation, where annotators rank and select the best-matching 3D asset for each object based on visual similarity and geometric fit; and (iii) Optimization, where selected assets undergo post-processing and global optimization to e… view at source ↗
Figure 3
Figure 3. Overview of our optimal asset retrieval model. We provide a multi-modal alignment model to retrieve the best asset from candidates. For each object, we provide a minimum of six asset candi￾dates, resulting in a total of 98423 unique 3D assets in the dataset. These objects covering 831 fine-grained object cate￾gories in 706 replicated scenes spanning various room types. It also includes rich semantic information for … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Automated replica creation. We visualize the optimal asset selection results in METASCENES (left), and a digital replica automatically created via SCAN2SIM on ScanNet++, before (top) and after physics-based optimization (bottom) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Micro-Scene Synthesis results. We visualize the generated results in a) Object-Level with the generated small objects given the large furniture. b) Room-Level by first generating the room layout, and then generating small objects atop the large objects [PITH_FULL_IMAG…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

114 extracted references · 50 canonical work pages

  1. [1]

    Scan2cad: Learning cad model alignment in rgb-d scans

    Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X Chang, and Matthias Nießner. Scan2cad: Learning cad model alignment in rgb-d scans. InConference on Computer Vision and Pattern Recognition (CVPR), 2019. 2, 3, 4, A1

    2019

  2. [2]

    Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. arXiv preprint arXiv:2111.08897, 2021. 2, 3

    arXiv 2021

  3. [3]

    Method for registration of 3-d shapes

    Paul J Besl and Neil D McKay. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, pages 586–606. Spie, 1992. 7

    1992

  4. [4]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017. 3, 4

    arXiv 2017

  5. [5]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 2, 3, 4

    arXiv 2015

  6. [6]

    Holistic++ scene understand- ing: Single-view 3d holistic scene parsing and human pose estimation with human-object interaction and physical com- monsense

    Yixin Chen, Siyuan Huang, Tao Yuan, Siyuan Qi, Yixin Zhu, and Song-Chun Zhu. Holistic++ scene understand- ing: Single-view 3d holistic scene parsing and human pose estimation with human-object interaction and physical com- monsense. In International Conference on Computer Vision (ICCV), 2019. 3

    2019

  7. [7]

    Single-view 3d scene reconstruc- tion with high-fidelity shape and texture

    Yixin Chen, Junfeng Ni, Nan Jiang, Yaowei Zhang, Yixin Zhu, and Siyuan Huang. Single-view 3d scene reconstruc- tion with high-fidelity shape and texture. In International Conference on 3D Vision (3DV), pages 1456–1467. IEEE,

  8. [8]

    Abo: Dataset and benchmarks for real-world 3d object un- derstanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achlesh- war Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object un- derstanding. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 21126–21136, 2022. 3

    2022

Show all 114 references
  1. [9]

    Scannet: Richly- annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly- annotated 3d reconstructions of indoor scenes. InConference on Computer Vision and Pattern Recognition (CVPR), pages 5828–5839, 2017. 2, 3, 4

    2017

  2. [10]

    Acdc: Automated creation of digital cousins for robust policy learn- ing

    Tianyuan Dai, Josiah Wong, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. Acdc: Automated creation of digital cousins for robust policy learn- ing. arXiv preprint arXiv:2410.07408, 2024. 3, 6, 7, A6

    2024 arXiv

  3. [11]

    Embodied question answering

    Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embodied question answering. In Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 3

    2018

  4. [12]

    Robothor: An open simulation-to-real embodied ai platform

    Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, et al. Robothor: An open simulation-to-real embodied ai platform. In Conference on Computer Vision and Pattern Recog...

    2020

  5. [13]

    Procthor: Large-scale embodied ai using procedural generation

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation. In Ad- vances in Neural Information Processing Systems (NeurIPS),

  6. [14]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 131...

    2023

  7. [15]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems (NeurIPS), 36, 2024. 2, 3

    2024

  8. [16]

    V otenet: A deep learning label fusion method for multi-atlas seg- mentation

    Zhipeng Ding, Xu Han, and Marc Niethammer. V otenet: A deep learning label fusion method for multi-atlas seg- mentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI),

  9. [17]

    Selective visual rep- resentations improve convergence and generalization for embodied ai

    Ainaz Eftekhar, Kuo-Hao Zeng, Jiafei Duan, Ali Farhadi, Ani Kembhavi, and Ranjay Krishna. Selective visual rep- resentations improve convergence and generalization for embodied ai. arXiv preprint arXiv:2311.04193, 2023. A7

    2023 arXiv

  10. [18]

    Spoc: Imitating shortest paths in simulation enables effective navigation and manipulation in the real world

    Kiana Ehsani, Tanmay Gupta, Rose Hendrix, Jordi Salvador, Luca Weihs, Kuo-Hao Zeng, Kunal Pratap Singh, Yejin Kim, Winson Han, Alvaro Herrasti, et al. Spoc: Imitating shortest paths in simulation enables effective navigation and manipulation in the real world. In Conference on...

    2024

  11. [19]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. In International Conference on Computer Vision (ICCV), pages 10933–10942, 2021. 2, 4, 8

    2021

  12. [20]

    3d-future: 3d fur- niture shape with texture

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- niture shape with texture. International Journal of Computer Vision (IJCV), 129:3313–3337, 2021. 2, 3

    2021

  13. [21]

    Cat3d: Create any- thing in 3d with multi-view diffusion models

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create any- thing in 3d with multi-view diffusion models. arXiv preprint arXiv:2405.10314, 2024. 3

    2024 arXiv

  14. [22]

    Behavior vision suite: Customizable dataset generation via simulation

    Yunhao Ge, Yihe Tang, Jiashu Xu, Cem Gokmen, Chengshu Li, Wensi Ai, Benjamin Jose Martinez, Arman Aydin, Mona Anvari, Ayush K Chakravarthy, et al. Behavior vision suite: Customizable dataset generation via simulation. In Confer- ence on Computer Vision and Pattern Recognition (CVPR),

  15. [23]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bern- hard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. In Advances in Neural Information Processing Systems (NeurIPS), 2017. 7

    2017

  16. [24]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems (NeurIPS), 2020. 3

    2020

  17. [25]

    Vln bert: A recurrent vision-and- language bert for navigation

    Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez- Opazo, and Stephen Gould. Vln bert: A recurrent vision-and- language bert for navigation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 2, 3

    2021

  18. [26]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 3

    2023 arXiv

  19. [27]

    Scenenn: A scene meshes dataset with annotations

    Binh-Son Hua, Quang-Hieu Pham, Duc Thanh Nguyen, Minh-Khoi Tran, Lap-Fai Yu, and Sai-Kit Yeung. Scenenn: A scene meshes dataset with annotations. In International Conference on 3D Vision (3DV), pages 92–101. Ieee, 2016. 3

    2016

  20. [28]

    An embodied generalist agent in 3d world

    Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent in 3d world. arXiv preprint arXiv:2311.12871, 2023. 3

    2023 arXiv

  21. [29]

    Unveiling the mist over 3d vision-language under- standing: Object-centric evaluation with chain-of-analysis

    Jiangyong Huang, Baoxiong Jia, Yan Wang, Ziyu Zhu, Xiongkun Linghu, Qing Li, Song-Chun Zhu, and Siyuan Huang. Unveiling the mist over 3d vision-language under- standing: Object-centric evaluation with chain-of-analysis. In Conference on Computer Vision and Pattern Recognition ...

    2025

  22. [30]

    Holistic 3d scene parsing and reconstruction from a single rgb image

    Siyuan Huang, Siyuan Qi, Yixin Zhu, Yinxue Xiao, Yuanlu Xu, and Song-Chun Zhu. Holistic 3d scene parsing and reconstruction from a single rgb image. In European Con- ference on Computer Vision (ECCV), 2018. 3

    2018

  23. [31]

    Diffusion- based generation, optimization, and planning in 3d scenes

    Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chun Zhu. Diffusion- based generation, optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16750–16761, 2023. 2

    2023

  24. [32]

    Scen- everse: Scaling 3d vision-language learning for grounded scene understanding

    Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. Scen- everse: Scaling 3d vision-language learning for grounded scene understanding. In European Conference on Computer Vision (ECCV), 2024. 3, 5, A6

    2024

  25. [33]

    Pointgroup: Dual-set point grouping for 3d instance segmentation

    Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi- Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point grouping for 3d instance segmentation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 3

    2020

  26. [34]

    Autonomous character-scene interaction synthesis from text instruction

    Nan Jiang, Zimo He, Zi Wang, Hongjie Li, Yixin Chen, Siyuan Huang, and Yixin Zhu. Autonomous character-scene interaction synthesis from text instruction. In SIGGRAPH Asia 2024 Conference Papers, 2024. 3

    2024

  27. [35]

    Scaling up dynamic human-scene interaction model- ing

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction model- ing. In Conference on Computer Vision and Pattern Recog- nition (CVPR), 2024. 3

    2024

  28. [36]

    Yolo by ultralytics

    Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Yolo by ultralytics. https://github.com/ultralytics/ ultralytics, 2023. A1

    2023

  29. [37]

    Shap-e: Generat- ing conditional 3d implicit functions

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 2, 3, 4

    2023 arXiv

  30. [38]

    Ssd-6d: Making rgb-based 3d de- tection and 6d pose estimation great again

    Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, and Nassir Navab. Ssd-6d: Making rgb-based 3d de- tection and 6d pose estimation great again. In International Conference on Computer Vision (ICCV), 2017. 3

    2017

  31. [39]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  32. [40]

    Habi- tat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation

    Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva. Habi- tat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navig...

    2024

  33. [41]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In International Conference on Computer Vision (ICCV), pages 4015–4026, 2023. 4, A1

    2023

  34. [42]

    Ai2-thor: An interactive 3d environment for visual ai

    Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474,

    arXiv

  35. [43]

    A large-scale hierarchical multi-view rgb-d object dataset

    Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. A large-scale hierarchical multi-view rgb-d object dataset. In International Conference on Robotics and Automation (ICRA), 2011. 2

    2011

  36. [44]

    igibson 2.0: Object-centric simulation for robot learning of everyday household tasks

    Chengshu Li, Fei Xia, Roberto Martín-Martín, Michael Lin- gelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, et al. igibson 2.0: Object-centric simulation for robot learning of everyday household tasks. arXiv preprint arXiv:2108.03272, 2021. 3

    2021 arXiv

  37. [45]

    Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation

    Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on ...

    2023

  38. [46]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 3

    2023 arXiv

  39. [47]

    Ag2manip: Learning novel manipulation skills with agent- agnostic visual and action representations

    Puhao Li, Tengyu Liu, Yuyang Li, Muzhi Han, Haoran Geng, Shu Wang, Yixin Zhu, Song-Chun Zhu, and Siyuan Huang. Ag2manip: Learning novel manipulation skills with agent- agnostic visual and action representations. arXiv preprint arXiv:2404.17521, 2024. 2

    2024 arXiv

  40. [48]

    Controlmanip: Few-shot manipulation fine- tuning via object-centric conditional control

    Puhao Li, Yingying Wu, Wanlin Li, Yuzhe Huang, Zhiyuan Zhang, Yinghan Chen, Song-Chun Zhu, Tengyu Liu, and Siyuan Huang. Controlmanip: Few-shot manipulation fine- tuning via object-centric conditional control. In 7th Robot Learning Workshop: Towards Robots with Human-Level Abi...

    2025

  41. [49]

    Openrooms: An end-to-end open framework for photorealistic indoor scene datasets

    Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gun- davarapu, Jia Shi, et al. Openrooms: An end-to-end open framework for photorealistic indoor scene datasets. arXiv preprint arXiv:2007.12868, 2020. 4

    2007 arXiv

  42. [50]

    Parsing ikea objects: Fine pose estimation

    Joseph J Lim, Hamed Pirsiavash, and Antonio Torralba. Parsing ikea objects: Fine pose estimation. In International Conference on Computer Vision (ICCV), pages 2992–2999,

  43. [51]

    Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior

    Chenguo Lin and Yadong Mu. Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior. In International Conference on Learning Representations (ICLR), 2024. 6

    2024

  44. [52]

    Towards high-fidelity single-view holis- tic reconstruction of indoor scenes

    Haolin Liu, Yujian Zheng, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Towards high-fidelity single-view holis- tic reconstruction of indoor scenes. In European Conference on Computer Vision (ECCV), 2022. 3

    2022

  45. [53]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. In International Confer- ence on Computer Vision (ICCV), 2023. 3

    2023

  46. [54]

    Building interactable replicas of complex articulated objects via gaussian splatting

    Yu Liu, Baoxiong Jia, Ruijie Lu, Junfeng Ni, Song-Chun Zhu, and Siyuan Huang. Building interactable replicas of complex articulated objects via gaussian splatting. In Inter- national Conference on Learning Representations (ICLR),

  47. [55]

    Taco: Taming diffusion for in-the-wild video amodal completion

    Ruijie Lu, Yixin Chen, Yu Liu, Jiaxiang Tang, Junfeng Ni, Diwen Wan, Gang Zeng, and Siyuan Huang. Taco: Taming diffusion for in-the-wild video amodal completion. arXiv preprint arXiv:2503.12049, 2025. 3

    2025 arXiv

  48. [56]

    Movis: En- hancing multi-object novel view synthesis for indoor scenes

    Ruijie Lu, Yixin Chen, Junfeng Ni, Baoxiong Jia, Yu Liu, Diwen Wan, Gang Zeng, and Siyuan Huang. Movis: En- hancing multi-object novel view synthesis for indoor scenes. In Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 3

    2025

  49. [57]

    Openeqa: Embodied question answering in the era of founda- tion models

    Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of founda- tion models. In Conference on Computer Vision and Pattern...

    2024

  50. [58]

    Cad-estate: Large-scale cad model annotation in rgb videos

    Kevis-Kokitsi Maninis, Stefan Popov, Matthias Nießner, and Vittorio Ferrari. Cad-estate: Large-scale cad model annotation in rgb videos. In International Conference on Computer Vision (ICCV), 2023. 3, 4

    2023

  51. [59]

    Multiscan: Scalable rgbd scanning for 3d environments with articulated objects

    Yongsen Mao, Yiming Zhang, Hanxiao Jiang, Angel Chang, and Manolis Savva. Multiscan: Scalable rgbd scanning for 3d environments with articulated objects. In Advances in Neural Information Processing Systems (NeurIPS), 2022. 2, 3

    2022

  52. [60]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 3

    2021

  53. [61]

    Phyrecon: Physically plausible neural scene recon- struction

    Junfeng Ni, Yixin Chen, Bohan Jing, Nan Jiang, Bin Wang, Bo Dai, Puhao Li, Yixin Zhu, Song-Chun Zhu, and Siyuan Huang. Phyrecon: Physically plausible neural scene recon- struction. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 3

    2024

  54. [62]

    Decompositional neural scene reconstruction with generative diffusion prior

    Junfeng Ni, Yu Liu, Ruijie Lu, Zirui Zhou, Song-Chun Zhu, Yixin Chen, and Siyuan Huang. Decompositional neural scene reconstruction with generative diffusion prior. In Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 3

    2025

  55. [63]

    Total3dunderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image

    Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. Total3dunderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image. In Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 3

    2020

  56. [64]

    Open x-embodiment: Robotic learning datasets and rt-x models

    Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, et al. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint arXiv:2310.08864, 2023. 3

    arXiv

  57. [65]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 6, A6

    2023 arXiv

  58. [66]

    Atiss: Autoregres- sive transformers for indoor scene synthesis

    Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregres- sive transformers for indoor scene synthesis. In Advances in Neural Information Processing Systems (NeurIPS), 2021. 2, 3, 6, 7, 8, A6

    2021

  59. [67]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 3

    2022 arXiv

  60. [68]

    Habitat 3.0: A co-habitat for humans, avatars and robots

    Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dal- laire Cote, Tsung-Yen Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, et al. Habitat 3.0: A co-habitat for humans, avatars and robots. arXiv preprint arXiv:2310.13724, 2023. 3

    arXiv

  61. [69]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems (NeurIPS), 2017. 7, A6

    2017

  62. [70]

    Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever

    Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Confere...

    2021

  63. [71]

    Infinigen indoors: Photorealistic indoor scenes using procedural gen- eration

    Alexander Raistrick, Lingjie Mei, Karhan Kayan, David Yan, Yiming Zuo, Beining Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, et al. Infinigen indoors: Photorealistic indoor scenes using procedural gen- eration. In Conference on Computer Vision and Pat...

    2024

  64. [72]

    Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Un- dersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv prepri...

    2021 arXiv

  65. [73]

    Habitat-web: Learning embodied object-search strategies from human demonstrations at scale

    Ram Ramrakhya, Eric Undersander, Dhruv Batra, and Ab- hishek Das. Habitat-web: Learning embodied object-search strategies from human demonstrations at scale. In Confer- ence on Computer Vision and Pattern Recognition (CVPR),

  66. [74]

    Pirlnav: Pretraining with imitation and rl finetuning for objectnav

    Ram Ramrakhya, Dhruv Batra, Erik Wijmans, and Abhishek Das. Pirlnav: Pretraining with imitation and rl finetuning for objectnav. In Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 8

    2023

  67. [75]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3

    2022

  68. [76]

    Wayfair’s 3d model api

    Shrenik Sadalgi. Wayfair’s 3d model api. https : //www.aboutwayfair.com/tech-innovation/ wayfairs-3d-model-api , 2016. [Online; accessed 15-Nov-2023]. 4

    2016

  69. [77]

    Habitat: A Platform for Embodied AI Research

    Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A Platform for Embodied AI Research. In International Conference on Computer Vision (ICCV)...

    2019

  70. [78]

    Mask3d: Mask trans- former for 3d semantic instance segmentation

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d: Mask trans- former for 3d semantic instance segmentation. In Inter- national Conference on Robotics and Automation (ICRA),

  71. [79]

    Alfred: A benchmark for interpreting grounded instructions for everyday tasks

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 2, 3

    2020

  72. [80]

    The replica dataset: A digital replica of indoor spaces

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797,

    1906 arXiv

  73. [81]

    Pix3d: Dataset and methods for single- image 3d shape modeling

    Xingyuan Sun, Jiajun Wu, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B Tenenbaum, and William T Freeman. Pix3d: Dataset and methods for single- image 3d shape modeling. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 2974–2983,

  74. [82]

    Habitat 2.0: Training home assistants to rearrange their habitat

    Andrew Szot, Alex Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir V ondrus, Sameer Dharur, Franziska Meier, Wo- jciech Galuba, Angel Chang, Zsolt Kira, Vladlen Koltun,...

  75. [83]

    Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation. arXiv preprint arXiv:2309.16653,

    arXiv

  76. [84]

    Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior

    Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma, and Dong Chen. Make-it-3d: High-fidelity 3d creation from a single image with diffusion prior. arXiv preprint arXiv:2303.14184, 2023. 3

    2023 arXiv

  77. [85]

    Diffuscene: Denoising diffusion models for gerative indoor scene synthesis

    Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Jus- tus Thies, and Matthias Nießner. Diffuscene: Denoising diffusion models for gerative indoor scene synthesis. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 6, 7, 8, A6

    2024

  78. [86]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024. 2, 5

    2024 arXiv

  79. [87]

    Reconciling reality through simulation: A real-to-sim-to-real approach for robust manipulation

    Marcel Torne, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. Reconciling reality through simulation: A real-to-sim-to-real approach for robust manipulation. arXiv preprint arXiv:2403.03949,

    arXiv

  80. [88]

    Softgroup for 3d instance segmentation on point clouds

    Thang Vu, Kookhoi Kim, Tung M Luu, Thanh Nguyen, and Chang D Yoo. Softgroup for 3d instance segmentation on point clouds. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3

    2022

  81. [89]

    Normalized object coordinate space for category-level 6d object pose and size estimation

    He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 3

    2019

  82. [90]

    Embodiedscan: A holistic multi- modal 3d perception suite towards embodied ai

    Tai Wang, Xiaohan Mao, Chenming Zhu, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, et al. Embodiedscan: A holistic multi- modal 3d perception suite towards embodied ai. In Confer- ence on Computer Vision and Pattern Recognition (CVPR),

  83. [91]

    Masked point-entity contrast for open-vocabulary 3d scene understanding

    Yan Wang, Baoxiong Jia, Ziyu Zhu, and Siyuan Huang. Masked point-entity contrast for open-vocabulary 3d scene understanding. In Conference on Computer Vision and Pat- tern Recognition (CVPR), 2025. 2

    2025

  84. [92]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600–612, 2004. 7

    2004

  85. [93]

    Bridging the imitation gap by adaptive insubordination

    Luca Weihs, Unnat Jain, Iou-Jen Liu, Jordi Salvador, Svet- lana Lazebnik, Aniruddha Kembhavi, and Alex Schwing. Bridging the imitation gap by adaptive insubordination. In Advances in Neural Information Processing Systems (NeurIPS), 2021. 8

    2021

  86. [94]

    R3ds: Reality-linked 3d scenes for panoramic scene understanding

    Qirui Wu, Sonia Raychaudhuri, Daniel Ritchie, Manolis Savva, and Angel X Chang. R3ds: Reality-linked 3d scenes for panoramic scene understanding. arXiv preprint arXiv:2403.12301, 2024. 2, 3, 4, A1

    2024 arXiv

  87. [95]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191,

    arXiv

  88. [96]

    To-scene: A large-scale dataset for understanding 3d table- top scenes

    Mutian Xu, Pei Chen, Haolin Liu, and Xiaoguang Han. To-scene: A large-scale dataset for understanding 3d table- top scenes. In European Conference on Computer Vision (ECCV), 2022. 2

    2022

  89. [97]

    Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation. arXiv preprint arXiv:2403.14621, 2024. 3

    2024 arXiv

  90. [98]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Conference on Computer Vision and Pattern R...

    2023

  91. [99]

    Ulip-2: Towards scalable multimodal pre-training for 3d understanding

    Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Jun- nan Li, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. Ulip-2: Towards scalable multimodal pre-training for 3d understanding. In Confer- ence on Computer Vision and Pattern Recognition...

    2024

  92. [100]

    Offline visual repre- sentation learning for embodied navigation

    Karmesh Yadav, Ram Ramrakhya, Arjun Majumdar, Vincent-Pierre Berges, Sachit Kuhar, Dhruv Batra, Alexei Baevski, and Oleksandr Maksymets. Offline visual repre- sentation learning for embodied navigation. In International Conference on Learning Representations (ICLR), 2023. 8

    2023

  93. [101]

    Physcene: Physically interactable 3d scene synthesis for embodied ai

    Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthesis for embodied ai. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 6, 7, 8, A6

    2024

  94. [102]

    Holodeck: Language guided generation of 3d embodied ai environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Al- varo Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3

    2024

  95. [103]

    The dawn of lmms: Preliminary explorations with gpt-4v (ision)

    Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421, 2023. 4, 7, A1

    2023 arXiv

  96. [104]

    Scannet++: A high-fidelity dataset of 3d indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. In International Conference on Computer Vision (ICCV), pages 12–22, 2023. 2, 3, 6, 7

    2023

  97. [105]

    Point-bert: Pre-training 3d point cloud transformers with masked point modeling

    Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 7, A6

    2022

  98. [106]

    Connecting the dots: Floorplan reconstruction using two-level queries

    Yuanwen Yue, Theodora Kontogianni, Konrad Schindler, and Francis Engelmann. Connecting the dots: Floorplan reconstruction using two-level queries. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 845–854, 2023. 4

    2023

  99. [107]

    Paint3d: Paint anything 3d with lighting-less texture diffusion models

    Xianfang Zeng, Xin Chen, Zhongqi Qi, Wen Liu, Zibo Zhao, Zhibin Wang, Bin Fu, Yong Liu, and Gang Yu. Paint3d: Paint anything 3d with lighting-less texture diffusion models. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 4252–4262, 2024. 5, A1

    2024

  100. [108]

    Echoscene: Indoor scene generation via information echo over scene graph diffusion

    Guangyao Zhai, Evin Pınar Örnek, Dave Zhenyu Chen, Ruo- tong Liao, Yan Di, Nassir Navab, Federico Tombari, and Benjamin Busam. Echoscene: Indoor scene generation via information echo over scene graph diffusion. In European Conference on Computer Vision (ECCV), 2025. 6

    2025

  101. [109]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975–11986, 2023. A7

    2023

  102. [110]

    Holistic 3d scene un- derstanding from a single image with implicit representation

    Cheng Zhang, Zhaopeng Cui, Yinda Zhang, Bing Zeng, Marc Pollefeys, and Shuaicheng Liu. Holistic 3d scene un- derstanding from a single image with implicit representation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 3

    2021

  103. [111]

    Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 2024. 2, 3

    2024

  104. [112]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 7

    2018

  105. [113]

    Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation

    Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. In Advances in Neural Information Processing Systems (NeurIPS),...

    2024

  106. [114]

    EV A02-E-14-plus

    Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. In International Conference on Learning Representations (ICLR), 2024. 5, 7, A4 METASCENES : Towards Automated Replica Creation for Real-wo...

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.