Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Agentic 3D Scene Generation with Spatially Contextualized VLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that injecting a continually evolving spatial context into a vision-language model lets it generate, understand, and edit coherent 3D scenes.

desk verdict Solid systems paper with a plausible mechanism that isn't yet isolated; the missing no-context ablation is the key gap. read the letter →

arxiv 2505.20129 v3 pith:3WXOAV47 submitted 2025-05-26 cs.CV cs.GR

Xinhang Liu , Yu-Wing Tai , Chi-Keung Tang This is my paper · ORCID
classification cs.CVcs.GR
keywords 3Dscenegenerationvision-languagemodelsspatialcontexthypergraphpointcloudagenticpipelineeditingpathplanning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a vision-language model—a model that normally reads text and images—can be turned into an agent that generates, understands, and edits 3D scenes if it is given a structured spatial context that it can read and update as work proceeds. The context has three pieces: a scene portrait that captures the intended look and semantics, a semantically labeled point cloud that supplies object-level geometry, and a scene hypergraph that records spatial relations among objects, including relations that involve more than two objects. On top of this context, the paper builds a four-stage pipeline in which the model generates textured objects, lays out the scene, constructs the surrounding environment with automatic verification, and adjusts poses to satisfy ergonomic constraints. The authors report that this approach produces coherent and stylistically faithful scenes from Chinese poetry, oil paintings, realistic photographs, and unposed image collections, and that the same injected context lets the model perform scene editing and collision-free path planning. If the claim holds, it means general-purpose vision-language models can do structured 3D spatial reasoning without being retrained for 3D.

What carries the argument

The load-bearing mechanism is the three-part spatial context $C=(S, P, G)$: a scene portrait $S$ (text plus an image), a semantically labeled point cloud $P$ (points with color and instance labels), and a scene hypergraph $G$ whose nodes are object instances and whose hyperedges are spatial relations. The hypergraph is what lifts the method beyond pairwise scene graphs: a single hyperedge can encode ternary relations such as symmetry or equidistance, and each relation type is turned into a differentiable loss used during ergonomic adjustment. The point cloud enters the VLM through 2D RGB-plus-instance projections, which is the trick that lets a model trained on images reason about 3D geometry. All three components are updated after each object change, so the context functions as a working memory that keeps the agent's reasoning grounded as the scene evolves.

What would settle it

A decisive test would be to run the pipeline on a multi-view image set with known ground-truth camera poses where the geometric reconstruction is known to fail, such as textureless or highly reflective objects; if the resulting scenes still align and remain semantically coherent, then the geometry stage is not the load-bearing component the paper claims.

Watch

Extended reading notes

Core claim

The central discovery is that spatial reasoning in a vision-language model can be switched on by supplying a continually evolving, geometry-aware context rather than by adding 3D training or a specialized 3D backbone. The context is a triple: a scene portrait, a semantically labeled point cloud, and a scene hypergraph whose hyperedges encode unary, binary, and ternary spatial relations as soft constraints. To make raw geometry legible to a text-and-image model, the point cloud is projected into 2D RGB plus instance maps; the model reads these maps, reasons over the portrait and hypergraph, and then updates the context instance by instance as assets are generated, replaced, or repositioned. The paper demonstrates the resulting agentic pipeline on inputs ranging from classical Chinese poetry to oil paintings to unposed multi-view photo sets, and reports that it outperforms prior scene-generation methods on semantic alignment, aesthetic quality, and functional plausibility, while also enabling downstream editing and path planning.

Load-bearing premise

The pipeline assumes that the geometric foundation model's point cloud and camera poses are accurate enough to act as the scene's ground-truth geometry, and that projecting that point cloud into 2D RGB-plus-instance maps preserves enough spatial information for the vision-language model to reason correctly; if either fails, every downstream stage inherits the error.

Editorial extensions

If this is right

  • A single vision-language model, without 3D-specific training, can go from text or images to a complete, editable 3D environment.
  • The same injected context transfers to downstream spatial tasks, including interactive scene editing and obstacle-aware path planning without explicit obstacle maps.
  • More input views improve the reconstruction, since geometric fidelity and instance overlap on the benchmark improve as the view count grows from 1 to 20.
  • Replacing the hypergraph or the ergonomic adjustment stage degrades layouts, producing collisions, detachment, or floating objects, which supports the claim that relational constraints carry the spatial reasoning.
  • Auto-verification of environment code matters: rendering the scene and having the model critique and revise its own code yields atmospherically and structurally consistent environments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the 2D-projection trick suggests a general recipe: any 3D representation can be fed to a VLM by rendering it into image plus instance channels, which may transfer to robot perception or embodied agents without retraining.
  • The paradigm is likely backbone-agnostic: if the geometric foundation model were swapped for a stronger one, the same spatial context and agent loop could remain in place, so the contribution may not be tied to any single reconstruction model.
  • The scene hypergraph could be extended to learned higher-order relations or temporal relations for dynamic scenes, a direction the paper itself flags as future work; one testable version is letting the VLM edit relations as well as poses during adjustment.
  • The reported collision-free path planning, done without explicit obstacle maps, hints that the spatial context could serve as a lightweight world model for navigation, but validating that would require tests in a physical or simulated embodied setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a framework that equips a vision-language model (GPT-4o) with an evolving spatial context — a scene portrait, a semantically labeled point cloud from Fast3R, and a scene hypergraph — in order to perform agentic 3D scene generation. The pipeline consists of asset generation with geometric restoration, coarse layout planning via ICP alignment, environment setup with VLM-generated Blender code and auto-verification, and hypergraph-based ergonomic adjustment. The authors evaluate on text prompts, single images, and unposed multi-view image sets, and also demonstrate downstream tasks such as scene editing and path planning. The central claim is that injecting spatial context enables VLMs to generate, understand, and edit coherent 3D scenes.

Significance. If the causal role of spatial context were established, this would be a significant contribution to VLM-based 3D scene generation and spatial reasoning. The system is end-to-end, handles a diversity of inputs including Chinese poetry and oil paintings, and integrates several non-trivial components (geometric restoration, auto-verified Blender code, hypergraph-based ergonomic constraints). The paper is also honest about limitations in Appendix A. However, as presented, the experiments do not isolate the effect of spatial context injection, and the evaluation relies heavily on the same model that produces the scenes. The central mechanistic claim therefore remains unproven, even though the engineering contribution is visible.

major comments (3)
  1. [Section 5.2 (Ablation Study), Figs. 6-7] The ablation study never removes the spatial context itself. The ablations vary environment setup and layout/ergonomic adjustment, but every condition still injects the full scene portrait, point cloud, and hypergraph. Since the VLM writes the portrait and hypergraph from the input prompt, the observed scene quality could plausibly come from the VLM's priors plus Meshy and Blender rather than from the geometric scaffold. The central claim of the paper requires a text-only control condition in which the same VLM, asset generator, and Blender toolchain are used but no point cloud, no hypergraph, and no projected point maps are provided. Without such a control, the paper does not establish that spatial context injection is the cause of the reported improvements.
  2. [Section 5, Table 1] The quantitative evaluation has methodological gaps that weaken the comparison. (i) Aesthetic quality (AQ) and functional plausibility (FP) are judged by GPT-4o, the same model that generated the scenes, and the auto-verification loop in Section 4.3 is also self-review; neither provides an independent measure. The user study has only 16 participants and no inter-rater agreement or significance testing. (ii) Table 1 reports LPIPS only for ACDC and Ours, and CLIP/BLIP only for Holodeck/DreamScene/Ours, so no method is compared on all metrics. (iii) No error bars, confidence intervals, or numbers of scenes are given. Please report per-scene scores with uncertainty, add independent/external metrics, and provide a full-metric comparison across all baselines.
  3. [Section 4.1, Eq. (1), Appendix A] The pipeline assumes that Fast3R's point clouds and camera poses are accurate enough to serve as the geometric ground truth for mesh alignment and as the backbone of the spatial context. Appendix A concedes that multi-image performance "heavily relies on the geometric foundation model," but the paper does not quantify this sensitivity. Since the point cloud is both the alignment target and the spatial context source, any depth or pose error propagates through asset generation, layout planning, and ergonomic adjustment. The robustness of the pipeline would be much better supported by evaluating on scenes with known ground-truth geometry or by perturbing estimated poses and measuring downstream CD/IoU degradation.
minor comments (5)
  1. [Section 5.1] SceneCraft is discussed in Related Work as the closest LLM-agent baseline for Blender-based scene generation, but it is not included in any quantitative or qualitative comparison; a direct comparison would strengthen the positioning of the method.
  2. [Section 5, Table 2] The multi-view experiment in Table 2 changes both the number of input views and the point cloud coverage simultaneously, so the improvement in CD/IoU may reflect more complete geometry rather than better spatial context reasoning; this confound should be discussed.
  3. [Section 4.2, Eq. (1)] The objective in Eq. (1) is a one-sided distance from mesh vertices to the point cloud, not a symmetric Chamfer distance; the metric description in Section 5 should clarify this distinction to avoid confusion with the evaluation metric.
  4. [References] The paper cites GPT-4 via the GPT-4 technical report, but the system is described as GPT-4o; please provide the appropriate reference for GPT-4o or clarify the model version used.
  5. [Various sections] There are minor typographical and formatting issues, such as missing spaces in "in theenvironment setupstage" and inconsistent use of "naive" vs. "naïve"; a careful proofreading pass is recommended.

Circularity Check

2 steps flagged · score 4.0 of 10

The spatial context whose injection is credited for the VLM's gains is itself authored by the same VLM, so the causal claim is partly self-definitional; the auto-verification loop and GPT-4o metrics add self-assessment.

  1. self definitional [Section 3.1 (Spatial Context Initialization) and Abstract]
    "Scene portrait. The VLM first constructs a multimodal scene portrait S, a structured, high-level representation of the scene. ... From the list of object instances and their corresponding axis-aligned bounding boxes (AABBs) derived from the point cloud P, the VLM constructs a scene hypergraph G = (V, E). --- Abstract: 'We introduce a new paradigm that enables VLMs to generate, understand, and edit complex 3D environments by injecting a continually evolving spatial context.'"

    The causal variable (spatial context) is produced by the same VLM whose performance it is said to enable. The VLM writes the scene portrait and hypergraph, then reads them back as 'guidance'; the experiments never remove this self-authored context, so gains could come entirely from the VLM's latent priors or from the extra structured reasoning steps. The 'injection' is not an exogenous treatment: the context is a re-representation of the model's own interpretation of the input, so the central claim is underdetermined by the paper's own setup.

  2. other [Section 4.3 (Environment Setup with Auto-Verification) and Section 5 (Metrics)]
    "After producing the initial environment code, the system renders an image of the resulting scene and performs self-evaluation using a chain-of-thought reasoning process to identify inconsistencies between the rendered output and the expected spatial context. ... We evaluate aesthetic quality (AQ) and functional plausibility (FP) through human ratings from a user study with 16 participants and GPT-4o ratings."

    The reference used for verification — the 'expected spatial context' — is the same VLM's self-authored portrait and hypergraph, so the check is consistency with the model's own prior output, not with an external ground truth. The paper also uses GPT-4o ratings for aesthetic quality and functional plausibility, so the same model is generator and judge. This is a self-referential evaluation loop; however, the paper does include external CLIP/BLIP/LPIPS, CD/IoU, and a 16-participant user study, so the loop is partial, not total.

full rationale

This is primarily an experimental systems paper; the core pipeline equations (ICP alignment in Eq. 1, relation-specific losses in Eqs. 2-7) are standard optimization objectives and are not fitted to the evaluation targets. External benchmarks (CLIP, BLIP, LPIPS, Chamfer Distance, IoU) and a user study provide independent grounding, which prevents the paper from collapsing into a purely self-referential exercise. The moderate circularity is in the framing: the spatial context is self-authored by the VLM, and the auto-verification loop plus GPT-4o ratings are self-assessment rather than independent evidence. A text-only ablation (no portrait, no point cloud, no hypergraph) would be needed to establish that the injected context, rather than the VLM's own priors or extra reasoning steps, causes the observed gains. Appendix A's admission that multi-image performance 'heavily relies on the geometric foundation model' is a dependency, not circularity. Under the strict 'reduction by construction' standard, only the self-authored-context issue approaches circularity, hence score 4 rather than higher.

Assumptions & free parameters 5 free parameters · 8 assumptions · 0 invented entities

No new physical or mathematical entities are introduced; the scene portrait and hypergraph are representational structures assembled from existing models. The free parameters are the VLM-determined and unspecified weights, margins, and thresholds in the optimization and restoration stages.

free parameters (5)
  • lambda_re loss weights for hyperedge relations = not specified
    Equation (2) defines the ergonomic adjustment objective as a weighted sum of relation-specific losses, but the paper never reports the weight values; they are chosen by hand or left to intuition.
  • d_min(v) clearance radius = VLM-inferred per object
    Equation (4) requires a per-object minimum clearance distance that the VLM determines from bounding box size or semantic role; this is a free contextual input.
  • alignment axes and symmetry axes (A_r) = VLM-selected per relation
    Equations (5) and (6) rely on projection matrices and axis selectors chosen contextually by the VLM, not derived from data.
  • soft contact margin epsilon = small, unstated
    Equation (3) introduces a soft contact margin that is not quantified.
  • completeness threshold for geometric restoration = not defined
    Section 4.1 says the system evaluates whether a point segment is sufficiently complete without specifying the criterion.
assumptions (8)
  • domain assumption Fast3R produces reliable point maps and camera poses for single and multi-view inputs.
    Used in Sections 3.1 and 4.1 to build the point cloud that anchors all geometry. Appendix A admits failures cause misalignment.
  • domain assumption Grounded-SAM yields accurate instance masks for point labels.
    Section 3.1 uses these masks to merge and label point cloud segments.
  • domain assumption GPT-4o can interpret projected RGB+instance maps and hypergraph text as spatial context.
    Section 3.2 claims this works without native 3D support; no systematic probe of this capacity is provided.
  • domain assumption 2D projection of the point cloud preserves sufficient spatial cues for VLM reasoning.
    Section 3.2 introduces the projection strategy as sufficient for layout and editing.
  • domain assumption Meshy API returns high-quality textured meshes from front-view renderings.
    Section 4.1 and the implementation details rely on this commercial API for asset generation.
  • standard math ICP-style nearest-neighbor alignment converges to the correct similarity transform.
    Equation (1) and the optimization strategy in Section 4.2 assume standard ICP behavior on subsampled points.
  • domain assumption GPT-4o's auto-verification improves its own Blender code.
    Section 4.3 uses self-evaluation as validation; this is not an independent check.
  • domain assumption 3D-FRONT ground truth provides valid reference geometry and semantics for evaluation.
    Table 2 reports Chamfer distance and IoU against 3D-FRONT; the paper does not discuss label alignment issues.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agentic 3D Scene Generation with Spatially Contextualized VLMs." pith.science (2026). https://pith.science/paper/3WXOAV47

@misc{pith2026250520129,
  author       = {Pith},
  title        = {Pith review of: Agentic 3D Scene Generation with Spatially Contextualized VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3WXOAV47}},
  note         = {Machine review of arXiv:2505.20129}
}
read the original abstract

Despite recent advances in multimodal content generation enabled by vision-language models (VLMs), their ability to reason about and generate structured 3D scenes remains largely underexplored. This limitation constrains their utility in spatially grounded tasks such as embodied AI, immersive simulations, and interactive 3D applications. We introduce a new paradigm that enables VLMs to generate, understand, and edit complex 3D environments by injecting a continually evolving spatial context. Constructed from multimodal input, this context consists of three components: a scene portrait that provides a high-level semantic blueprint, a semantically labeled point cloud capturing object-level geometry, and a scene hypergraph that encodes rich spatial relationships, including unary, binary, and higher-order constraints. Together, these components provide the VLM with a structured, geometry-aware working memory that integrates its inherent multimodal reasoning capabilities with structured 3D understanding for effective spatial reasoning. Building on this foundation, we develop an agentic 3D scene generation pipeline in which the VLM iteratively reads from and updates the spatial context. The pipeline features high-quality asset generation with geometric restoration, environment setup with automatic verification, and ergonomic adjustment guided by the scene hypergraph. Experiments show that our framework can handle diverse and challenging inputs, achieving a level of generalization not observed in prior work. Further results demonstrate that injecting spatial context enables VLMs to perform downstream tasks such as interactive scene editing and path planning, suggesting strong potential for spatially intelligent systems in computer graphics, 3D vision, and embodied applications. Project page: https://spatctxvlm.github.io/project_page/.

Figures

Figures reproduced from arXiv: 2505.20129 by the authors.

Figure 1
Figure 1. Spatially contextualized VLMs. We propose a framework that equips VLMs with structured spatial context, enabling them to act as agents for 3D scene generation. Our approach supports diverse inputs—including text prompts, single images, and unstructured, unposed image collections—and produces coherent, semantically aligned 3D environments across a wide range of styles and settings. Abstract Despite recent advances in… view at source ↗
Figure 2
Figure 2. Left: Spatial context. Given multimodal input from the user, we construct a spatial context that is continuously read and updated by the VLM, effectively injecting it with scene-level semantics, geometry, and relational structure. Right: Agentic 3D scene generation. Grounded in this context, the VLM performs a four-stage generation process: asset generation, coarse layout planning, environment setup, and ergonomic a… view at source ↗
Figure 3
Figure 3. Qualitative comparison for text-based 3D scene generation. Our method produces more coherent, stylistically aligned, and visually plausible scenes compared to DreamScene [26] and Holodeck [58]. such as inter-object penetration, detachment, or mis￾alignment with ergonomic expectations. To address these, the VLM performs a joint optimization over ob￾ject poses to refine the overall arrangement and enforce physically a… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison for image-based 3D scene generation. Compared to ACDC [8], our method appears to generate scenes that more consistently reflect the spatial and visual characteristics of the input images. throughout the 3D scene generation pipeline. Prompts used …
Figure 5
Figure 5. Figure 5: Results from multi-view observations. Our method synthesizes consistent scenes from unposed, unstructured image collections. Input w/o Environment Setup Naïve Environment Setup Auto-Verification [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Ablation on environment setup. Without structured setup, scenes lack realistic lighting and environmental elements. Na¨ıve modifiers yield low-fidelity results, while our auto-verified setup produces coherent, atmospheric environments aligned with spatial context. ison…
Figure 8
Figure 8. Figure 8: Scene editing and spatial reasoning. Our method en￾ables downstream spatial tasks such as furniture manipulation and obstacle-aware path planning, by reasoning over the spatial con￾text. ture, but the results often lack realism—waves may appear flat or physically impla…
Figure 9
Figure 9. Figure 9: Additional qualitative results. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI

    cs.RO 2026-07 conditional novelty 6.0 of 10

    EmbodiedGen V2 is a pipeline that converts natural-language tasks into cross-simulator, physics-validated 3D worlds with annotated interaction affordances, supporting downstream robot policy training and sim-to-real transfer.

  2. WorldClaw: Agentic 3D Open-World Generation at Scale

    cs.AI 2026-08 conditional novelty 4.0 of 10

    WorldClaw generates globally coherent, locally detailed, editable 3D worlds from open-ended text using a coarse-to-fine agentic pipeline.

Reference graph

Works this paper leans on

77 extracted references · 67 canonical work pages · cited by 2 Pith papers

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anad- kat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a visual language model for few-shot learn- ing. Advances in Neural Information Processing Systems (NeurIPS) 35 (2022), 23716–23736

    work page 2022

  3. [3]

    Zamir, Martin Fischer, Jitendra Malik, and Silvio Savarese

    Iro Armeni, Zhi-Yang He, JunYoung Gwak, Amir R. Zamir, Martin Fischer, Jitendra Malik, and Silvio Savarese. 2019. 3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera. In IEEE/CVF International Conference on Computer Vision (ICCV)

    work page 2019

  4. [4]

    Miguel Angel Bautista, Pengsheng Guo, Samira Abnar, Wal- ter Talbott, Alexander Toshev, Zhuoyuan Chen, Laurent Dinh, Shuangfei Zhai, Hanlin Goh, Daniel Ulbricht, et al

    work page

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS)33 (2020), 1877– 1901

    work page 2020

  6. [6]

    Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. 2023. Scenedreamer: Unbounded 3d scene generation from 2d im- age collections. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) (2023)

    work page 2023

  7. [7]

    Bob Coyne and Richard Sproat. 2001. WordsEye: An auto- matic text-to-scene conversion system. InProceedings of the 28th annual conference on Computer graphics and interac- tive techniques. 487–496

    work page 2001

  8. [8]

    Tianyuan Dai, Josiah Wong, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. 2024. Automated Creation of Digital Cousins for Robust Policy Learning. In Conference on Robot Learning (CoRL)

    work page 2024

Show all 77 references
  1. [9]

    Terrance DeVries, Miguel Angel Bautista, Nitish Srivastava, Graham W Taylor, and Joshua M Susskind. 2021. Uncon- strained scene generation with locally conditioned radiance fields. In IEEE/CVF International Conference on Computer Vision (ICCV). 14304–14313

    2021

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil 9 Mathur, Alan Schelten, Amy Yang, Angela Fan, et al

  3. [11]

    Dave Epstein, Ben Poole, Ben Mildenhall, Alexei A Efros, and Aleksander Holynski. 2024. Disentangled 3D Scene Generation with Layout Learning. In International Confer- ence on Machine Learning (ICML)

    2024

  4. [12]

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. 2024. Layoutgpt: Compositional visual planning and generation with large language models. Ad- vances in Neural Information Processing Systems (NeurI...

    2024

  5. [13]

    Yifan Feng, Chengwu Yang, Xingliang Hou, Shaoyi Du, Shihui Ying, Zongze Wu, and Yue Gao. 2025. Beyond Graphs: Can Large Language Models Comprehend Hyper- graphs?. In International Conference on Learning Represen- tations (ICLR)

    2025

  6. [14]

    Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel

  7. [15]

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 2021. 3d-front: 3d furnished rooms with layouts and semantics. In IEEE/CVF International Confer- ence on Computer Vision (ICCV). 10933–10942

    2021

  8. [16]

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 2021. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision (IJCV) (2021), 1–25

    2021

  9. [17]

    Rao Fu, Zehao Wen, Zichen Liu, and Srinath Sridhar. 2025. Anyhome: Open-vocabulary generation of structured and textured 3d homes. In European Conference on Computer Vision (ECCV). Springer, 52–70

    2025

  10. [18]

    Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Sch ¨olkopf. 2024. Graphdreamer: Compositional 3d scene synthesis from scene graphs. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) . 21295–21304

    2024

  11. [19]

    Tobias Germer and Martin Schwarz. 2009. Procedural Arrangement of Furniture for Real-Time Walkthroughs. In Computer Graphics Forum, V ol. 28. Wiley Online Library, 2068–2078

    2009

  12. [20]

    Tanmay Gupta and Aniruddha Kembhavi. 2023. Visual pro- gramming: Compositional visual reasoning without training. (2023), 14953–14962

    2023

  13. [21]

    Zekun Hao, Arun Mallya, Serge Belongie, and Ming-Yu Liu. 2021. Gancraft: Unsupervised 3d neural rendering of minecraft worlds. In IEEE/CVF International Conference on Computer Vision (ICCV). 14072–14082

    2021

  14. [22]

    Lukas H ¨ollein, Ang Cao, Andrew Owens, Justin John- son, and Matthias Nießner. 2023. Text2Room: Extract- ing Textured 3D Meshes from 2D Text-to-Image Models. In IEEE/CVF International Conference on Computer Vision (ICCV). 7909–7920

    2023

  15. [23]

    Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi

  16. [24]

    Kari Anne Høier Kjølaas. 2000. Automatic furniture pop- ulation of large architectural models . Ph. D. Dissertation. Massachusetts Institute of Technology

    2000

  17. [25]

    Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. 2017. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474 (2017)

    2017 arXiv

  18. [26]

    Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang, Lik-hang Lee, and Peng Yuan Zhou. 2024. Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling. In European Conference on Computer Vision (ECCV). Springer, 214–230

    2024

  19. [27]

    InInternational Conference on Ma- chine Learning (ICML)

    SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code. InInternational Conference on Ma- chine Learning (ICML)

  20. [28]

    Zhengqi Li, Qianqian Wang, Noah Snavely, and Angjoo Kanazawa. 2022. Infinitenature-zero: Learning perpetual view generation of natural scenes from single images. InEu- ropean Conference on Computer Vision (ECCV) . Springer, 515–534

    2022

  21. [29]

    Long Lian, Boyi Li, Adam Yala, and Trevor Darrell. 2024. LLM-grounded Diffusion: Enhancing Prompt Understand- ing of Text-to-Image Diffusion Models with Large Lan- guage Models. Transactions on Machine Learning Research (2024). Featured Certification

    2024

  22. [30]

    Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, and Angjoo Kanazawa. 2021. Infi- nite nature: Perpetual view generation of natural scenes from a single image. In IEEE/CVF International Conference on Computer Vision (ICCV). 14458–14467

    2021

  23. [31]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi

  24. [32]

    In International Conference on Machine Learning (ICML)

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning (ICML) . PMLR, 19730–19742

  25. [33]

    Wamiq Reyaz Para, Paul Guerrero, Niloy Mitra, and Peter Wonka. 2023. COFS: Controllable furniture layout synthe- sis. In ACM Transactions on Graphics (SIGGRAPH). 1–11

    2023

  26. [34]

    Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. 2021. ATISS: Autoregressive Transformers for Indoor Scene Synthesis. In Advances in Neural Information Processing Systems (NeurIPS). 10

    2021

  27. [35]

    Ryan Po and Gordon Wetzstein. 2024. Compositional 3d scene generation using locally conditioned diffusion. In2024 International Conference on 3D Vision (3DV) . IEEE, 651– 663

    2024

  28. [36]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee

  29. [37]

    In Advances in Neural In- formation Processing Systems (NeurIPS)

    Visual Instruction Tuning. In Advances in Neural In- formation Processing Systems (NeurIPS)

  30. [38]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feed- back. Advances in Neural Information Processing System...

    2022

  31. [39]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. In Advances in Neural Information Processing Systems (NeurIPS)....

    2023

  32. [40]

    Pratyusha Sharma, Tamar Rott Shaham, Manel Baradad, Stephanie Fu, Adrian Rodriguez-Munoz, Shivam Duggal, Phillip Isola, and Antonio Torralba. 2024. A vision check-up for language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 14410–14419

    2024

  33. [41]

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2024. Hugginggpt: Solv- ing ai tasks with chatgpt and its friends in hugging face. Ad- vances in Neural Information Processing Systems (NeurIPS) 36 (2024)

    2024

  34. [42]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. CoRR abs/2103....

    2021 arXiv

  35. [43]

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. 2021. Common Objects in 3D: Large-Scale Learning and Evalua- tion of Real-life 3D Category Reconstruction. In IEEE/CVF International Conference on Computer Vision (ICCV)

    2021

  36. [44]

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. 2024. Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks...

    2024 arXiv

  37. [45]

    Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wi- jmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, et al. 2021. Habitat 2.0: Training home assistants to rear- range their habitat. Advances in Neural Information P...

    2021

  38. [46]

    Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Jus- tus Thies, and Matthias Nießner. 2024. Diffuscene: Denois- ing diffusion models for generative indoor scene synthesis. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 20507–20518

    2024

  39. [47]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalk- wyk, Andrew M Dai, Anja Hauth, Katie Millican, et al

  40. [48]

    Sanjana Srivastava, Chengshu Li, Michael Lingelbach, Roberto Mart´ın-Mart´ın, Fei Xia, Kent Elliott Vainio, Zheng Lian, Cem Gokmen, Shyamal Buch, Karen Liu, et al. 2022. Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments....

    2022

  41. [49]

    Qi Sun, Hang Zhou, Wengang Zhou, Li Li, and Houqiang Li

  42. [50]

    Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner

  43. [51]

    D ´ıdac Sur ´ıs, Sachit Menon, and Carl V ondrick. 2023. Vipergpt: Visual inference via python execution for reason- ing. In IEEE/CVF International Conference on Computer Vi- sion (ICCV). 11888–11898

    2023

  44. [52]

    Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. 2024. Genartist: Multimodal llm as an agent for unified image gen- eration and editing. Advances in Neural Information Pro- cessing Systems (NeurIPS) (2024)

    2024

  45. [53]

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. 2023. Visual chatgpt: Talk- ing, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671 (2023)

    2023 arXiv

  46. [54]

    Tsung-Han Wu, Long Lian, Joseph E Gonzalez, Boyi Li, and Trevor Darrell. 2024. Self-correcting llm-controlled diffu- sion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6327–6336

    2024

  47. [55]

    arXiv preprint arXiv:2312.11805 (2023)

    Gemini: a family of highly capable multimodal mod- els. arXiv preprint arXiv:2312.11805 (2023)

    2023 arXiv

  48. [56]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al

  49. [57]

    arXiv preprint arXiv:2307.09288 (2023)

    Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  50. [58]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandku- mar. 2024. V oyager: An Open-Ended Embodied Agent with Large Language Models. Transactions on Machine Learning Research (2024)

    2024

  51. [59]

    Kaixin Yao, Longwen Zhang, Xinhao Yan, Yan Zeng, Qix- uan Zhang, Lan Xu, Wei Yang, Jiayuan Gu, and Jingyi Yu. 2025. CAST: Component-Aligned 3D Scene Recon- struction from an RGB Image. arXiv:2502.12894 [cs.CV] https://arxiv.org/abs/2502.12894

    2025 arXiv

  52. [60]

    Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, and Charles Her- rmann

    Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T. Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, and Charles Her- rmann. 2024. Wonderjourney: Going from Anywhere to Ev- erywhere. In IEEE/CVF Conference on Computer Vision and Patt...

    2024

  53. [61]

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, Ping Luo, Ziwei Liu, Yali Wang, Limin Wang, and Yu Qiao. 2024. InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation. In Internation...

    2024

  54. [62]

    Guangyao Zhai, Evin Pinar Ornek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. 2023. CommonScenes: Generating Common- sense 3D Indoor Scenes with Scene Graph Diffusion. In Advances in Neural Information Processing Systems (NeurIPS). https://openrev...

    2023

  55. [63]

    Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, and Jing Liao. 2024. Text2nerf: Text-driven 3d scene generation with neural radiance fields. IEEE Transactions on Visualization and Computer Graphics (TVCG) (2024)

    2024

  56. [64]

    Qihang Zhang, Yinghao Xu, Yujun Shen, Bo Dai, Bolei Zhou, and Ceyuan Yang. 2024. BerfScene: Generative Novel View Synthesis with 3D-Aware Diffusion Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  57. [65]

    Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al. 2020. Sapien: A simulated part-based inter- active environment. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11097–11107

    2020

  58. [66]

    Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli

    Jianing Yang, Alexander Sax, Kevin J. Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. 2025. Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2025

  59. [67]

    Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Ste- fano Ermon, and CUI Bin. 2024. Mastering text-to-image 11 diffusion: Recaptioning, planning, and generating with multimodal llms. In International Conference on Machine Learning (ICML)

    2024

  60. [68]

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Kr- ishna, Lingjie Liu, et al. 2024. Holodeck: Language guided generation of 3d embodied ai environments. In IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  61. [71]

    Lap Fai Yu, Sai Kit Yeung, Chi Keung Tang, Demetri Ter- zopoulos, Tony F Chan, and Stanley J Osher. 2011. Make it home: automatic optimization of furniture arrangement. ACM Transactions on Graphics (SIGGRAPH) 30, 4 (2011)

    2011

  62. [75]

    Renrui Zhang, Ziyu Guo, Peng Gao, Rongyao Fang, Bin Zhao, Dong Wang, Yu Qiao, and Hongsheng Li. 2022. Point- M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training. Advances in Neural Information Processing Systems (NeurIPS) (2022)

    2022

  63. [76]

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR)

    2018

  64. [77]

    A minimalist bedroom with an electric guitar hanging on the wall, and a laptop and black desk lamp placed on the table

    Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, and Achuta Kadambi. 2025. Dreamscene360: Uncon- strained text-to-3d scene generation with panoramic gaus- sian splatting. In European Conference on Computer Vision (ECC...

    2025

  65. [2021]

    In International Conference on 3D Vision (3DV)

    Sceneformer: Indoor scene generation with transform- ers. In International Conference on 3D Vision (3DV). IEEE, 106–115

  66. [2022]

    Advances in Neural Information Processing Systems (NeurIPS) 35 (2022), 25102–25116

    Gaudi: A neural architect for immersive 3d scene gen- eration. Advances in Neural Information Processing Systems (NeurIPS) 35 (2022), 25102–25116

    2022

  67. [2023]

    In Advances in Neural Information Processing Systems (NeurIPS), A

    SceneScape: Text-Driven Consistent Scene Genera- tion. In Advances in Neural Information Processing Systems (NeurIPS), A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), V ol. 36. Curran Asso- ciates, Inc., 39897–39914. https://proceedings. neurips.cc/...

    2023

  68. [2024]

    arXiv preprint arXiv:2407.21783 (2024)

    The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  69. [2025]

    In European Conference on Computer Vision (ECCV)

    Forest2seq: Revitalizing order prior for sequential in- door scene synthesis. In European Conference on Computer Vision (ECCV). Springer, 251–268

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.