Pith. sign in

REVIEW 4 major objections 5 minor 76 references

Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that separating semantic verification from physical-feasibility search—graph validation followed by evolutionary-then-gradient optimization—produces state-of-the-art open-vocabulary 3D indoor layouts.

desk verdict A well-specified pipeline that likely improves robustness of VLM-based layout generation, but the reported SOTA claim is not backed by the single-run, same-model-family evaluation. read the letter →

arxiv 2608.03064 v1 pith:BTG2PO2P submitted 2026-08-04 cs.CV

Jialu Huang , Yingxuan You , Fei Wang , Zheng Dang This is my paper · ORCID
classification cs.CV
keywords 3Dindoorlayoutgenerationvision-languagemodelsscenegraphsphysicalplausibilityopen-vocabularyevolutionarysearchgradient-basedrefinementsemanticconstraintverification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that recent VLM-based 3D indoor layout generation fails when it models inter-object relations implicitly or as local pairwise constraints, and when it optimizes poses with gradient descent from a VLM-proposed initialization. The remedy is a pipeline with two dedicated global mechanisms: Global Semantic Verification (GSV), which turns VLM-generated constraints into a directed scene graph and prunes contradictions with rule-based checks, and Global Physical Feasibility Search (GPFS), which evolves a population of layouts and then refines the best with gradient descent. The authors claim this separation of semantic coherence from physical feasibility yields state-of-the-art results on the LayoutVLM benchmark, improving all five reported metrics over LayoutGPT, Holodeck, I-Design, and LayoutVLM, plus better collision and support scores on the larger SceneEval-100. If true, layout generation shifts from initialization-sensitive local fitting to explicit global reasoning, and it no longer needs a reliable VLM-proposed starting configuration.

What carries the argument

The load-bearing object is the directed scene graph G=(V,E) over assets, with edges r_ij=(a_i,a_j,tau_ij,psi_ij) encoding typed constraints such as 'near', 'align_with', 'against_wall', 'place_align', and 'surround'. GSV partitions the graph into functional subgraphs, anchors each on the highest in-degree node, and evaluates a conjunction of rule-based predicates Phi(G_l) that catch directional cycles, distance inconsistencies, wall conflicts, out-of-area violations, missing position/orientation constraints, and common-sense semantic violations; the conflict log is sent back to the VLM, which proposes new edges, and the loop repeats until no conflicts remain. GPFS then searches over poses P_i=(x_i,y_i,z_i,theta_i) with a population-based evolutionary stage—swap-based intra-individual crossover and center-guided Gaussian mutation with pairwise repulsion—followed by gradient descent on the top-elite layouts, using a differentiable oriented-bounding-box collision loss derived from the separating axis theorem and a room-boundary in-bound loss. The key move is the separation: semantic constraints are verified and repaired at the graph level, while physical feasibility is handled by global exploration plus local refinement at the pose level.

What would settle it

Run a deterministic geometric checker (oriented-bounding-box intersection and room-polygon containment) on the released layouts from all methods on the 33-scene LayoutVLM benchmark and compare the measured collision-free and in-bound rates with the reported CF and IB scores; alternatively, have human raters who never see method identities score the same renderings. If the measured rates do not match the reported scores, or if humans do not prefer the GSV+GPFS layouts over the baselines, the central claim is settled in the negative.

Watch

Extended reading notes

Core claim

The central claim, stated in Sec. 4.3, is that the GSV+GPFS pipeline achieves state-of-the-art open-vocabulary 3D indoor layout generation, improving both semantic consistency and physical plausibility. GSV models the scene as a directed graph whose edges are typed relational constraints generated by the VLM, then applies rule-based predicates—cycle consistency, distance consistency, wall compliance, occupancy, completeness, and semantic compatibility—over anchored subgraphs, feeding detected conflicts back to the VLM for iterative graph repair until a conflict-free scaffold G* is reached. GPFS then treats layout realization as a global non-convex search: an evolutionary loop with swap-based crossover and center-guided mutation explores candidate layouts from random initialization, and gradient descent on the top elite layouts refines poses under a differentiable soft separating-axis collision loss and an in-bound loss. On the LayoutVLM benchmark the authors report higher Collision-Free, In-Boundary, Positional Coherency, Rotational Coherency, and Physically-Grounded Semantic Alignment scores than LayoutGPT, Holodeck, I-Design, and LayoutVLM across all room types, and SceneEval-100 results show fewer collisions and better support relations. Ablations attribute semantic gains to graph verification and physical gains to the hybrid search: removing graph checks lowers PSA, removing evolutionary search drops CF to 73.3, and removing gradient refinement collapses CF to 13.3 and PSA to 0.0.

Load-bearing premise

The load-bearing premise is that the GPT-4o visual-language judge used to score rendered layouts is an unbiased and accurate proxy for physical plausibility and semantic fidelity, even though the same model family proposes and refines the layouts; if the judge systematically prefers its own family's outputs, the state-of-the-art claim fails regardless of the reported numbers.

Editorial extensions

If this is right

  • Layout generation no longer depends on a VLM-proposed initial layout: GPFS can start from random populations and still converge to collision-free, in-bound configurations.
  • Rule-based graph verification catches contradictory and missing relational constraints before optimization, which the paper shows is necessary for high semantic alignment scores.
  • The two-stage decomposition transfers to furniture-surface placement: small assets on support planes are constrained with against-edge, point-towards-edge, and center constraints, extending the pipeline beyond floor-level objects.
  • On the LayoutVLM benchmark the method reports higher CF, IB, Pos., Rot., and PSA than LayoutGPT, Holodeck, I-Design, and LayoutVLM in every room type, and SceneEval-100 shows improved collision and support metrics on 100 scenes.
  • Ablations show the components are complementary: removing graph verification degrades semantic scores, removing evolutionary search leaves gradient descent stuck in local minima, and removing gradient refinement leaves unresolved collisions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A controlled variance study—running GPFS from several random seeds and measuring spread in final PSA—would directly test the paper's claim of reduced initialization sensitivity; high variance across seeds would weaken that claim even if average scores stay high.
  • Because GSV's verification predicates are modular and rule-based, the same graph-scaffold idea could be reused in other VLM-driven structured generation tasks, such as robot task planning or procedural room layout, wherever outputs need consistency checking before optimization.
  • The paper's evaluation rests on a GPT-4o judge, so an independent human study or a deterministic physical simulator (gravity, support, and collision checks) would be a stronger test of 'physical plausibility'; disagreement between the two would suggest the reported PSA scores measure VLM preference rather than physical validity.
  • The aggregate-wall-conflict rule in the verification module is effectively a constraint-satisfaction check; a formal analysis of whether the iterative VLM-refinement loop always terminates would clarify the method's guarantees.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies open-vocabulary 3D indoor layout generation from language instructions and unlabeled 3D asset databases. The proposed pipeline has three stages: (1) hierarchical scene proposal and CLIP-based asset retrieval; (2) Global Semantic Verification (GSV), which constructs a directed scene graph from VLM-generated constraints, checks it with hand-authored rules (cycle, distance, wall, occupancy, logic, completeness, semantic predicates), and iteratively asks the VLM to repair detected conflicts; and (3) Global Physical Feasibility Search (GPFS), a population-based evolutionary search with swap crossover and center-guided mutation followed by gradient refinement, using differentiable semantic and physical (Soft-SAT collision and boundary) losses. Experiments compare against LayoutGPT, Holodeck, I-Design, and LayoutVLM on the LayoutVLM 33-scene benchmark and on SceneEval-100, with GPT-4o as judge. The central claim, stated in Sec. 4.3 and the abstract, is state-of-the-art performance in semantic consistency and physical plausibility.

Significance. If validated, the core ideas are valuable: representing relational constraints as a graph and verifying them globally before optimization is a natural fix for the local-consistency failures of LLM/VLM-based scene generation, and the hybrid evolutionary-gradient optimization is a sensible response to the non-convex layout space. The manuscript also provides detailed constraint formulations, a differentiable Soft-SAT collision loss, and a substantial supplementary appendix with exact prompts, which is useful for reproducibility. However, the evaluation currently does not establish the SOTA claim: the only judge on the main benchmark is a VLM from the same model family as the generator, the benchmark has only 33 scenes and appears to be single-run, and Table 1 itself contradicts the 'all metrics and all room types' sentence. These issues are repairable, so the recommended decision is major revision.

major comments (4)
  1. [Sec. 4.3, Table 1] The sentence 'Our method achieves higher scores across all metrics (CF, IB, Pos., Rot., PSA) and all room types' is factually contradicted by Table 1. In Computer Room, Ours(gpt-4o) has Pos./Rot./PSA of 70.0/73.3/66.7 versus LayoutVLM's 85.4/84.5/77.0; in Deli, Ours(gpt-4.1) has IB 66.7 and PSA 60.0 versus LayoutVLM's 88.9 and 74.6, and Ours(gpt-4o) has CF 66.7 versus LayoutVLM's 100.0. The claim should be restricted to the specific variant and room types where it holds, or supported with multi-run statistics showing that the exceptions are noise.
  2. [Sec. 4.1, evaluation protocol] The main benchmark uses GPT-4o as a visual-language judge for both top-down and side-view renderings (Sec. 4.1). The same model family (GPT-4o / GPT-4.1) is also used to generate constraints and refine layouts. No human study or independent geometric verification is provided for the 33-scene benchmark. Therefore the SOTA claim rests on a protocol in which the generator and judge are from the same model family, and a systematic self-preference of the judge cannot be ruled out. A concrete remedy is a human perceptual study on a subset of scenes, or automatic geometric metrics (collision volume, support contact area, boundary penetration depth) computed without the VLM.
  3. [Secs. 4.1, 4.2, 4.4, 5; Tables 1-2] The paper reports a single run per condition without variance or significance testing. Section 5 explicitly concedes that GPT API stochasticity affects object category selection, object quantity, and the subsequent evaluation process. Since Table 1 and Table 2 are single numbers, the reported differences—e.g., the PSA drop from 86.7 to 74.0 when completeness is removed in Table 2, or the margin between Ours(gpt-4.1) and LayoutVLM on several metrics—could be within run-to-run noise. Please report at least 3-5 independent runs per method and condition with mean±std, and state the number of scenes and seeds. In addition, Sec. 4.4 says the ablation is on a 'representative subset' but does not enumerate the specific scene instances or give per-scene breakdowns, which is necessary to assess the component attributions.
  4. [Supplementary Table 1 and Sec. 4.3] The abstract and Sec. 4.3 make an unqualified state-of-the-art claim, but the SceneEval-100 results in Supplementary Table 1 do not show universal improvement: Ours is below LayoutVLM on NAV (93.25 vs 98.75) and ACC (65.73 vs 85.91), and below Holodeck on OOB (2.09 vs 1.12). The claim should be benchmark-specific and metric-specific, or the trade-offs should be discussed explicitly. Presenting SceneEval-100 as showing 'strong improvements in relational consistency and physical plausibility' is only accurate for the subset of metrics where the gains are consistent.
minor comments (5)
  1. [Sec. 3] The phrase 'Gloabl Physical Feasibility Search' in Sec. 3 is a typo for 'Global Physical Feasibility Search'.
  2. [Sec. 4.3 and Fig. 5] The names 'buffer_restaurant' and 'dense_layout' appear in the running text and figure caption, while Table 1 uses 'Buffet Restaurant'; please standardize the naming of scene types.
  3. [Sec. 4.4] The main text refers to 'Fig. E' for visual ablation results, but the supplementary figures are labeled as Fig. D and Fig. E in different places; the cross-references between the main text and the supplementary should be made consistent.
  4. [Table 2] The ablation table reports a single 'Ours' row, but Table 1 includes both Ours(gpt-4o) and Ours(gpt-4.1); please state which backbone is used in the ablation study.
  5. [Reproducibility] No statement about code or data release is provided; for a pipeline that depends on proprietary GPT APIs, a code release with seeds and prompts would materially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivation-level circularity; the SOTA claim is weakened by a same-family GPT evaluator and a Table 1 mismatch, but no prediction reduces to its own inputs by construction.

full rationale

I traced the claimed derivation chain: VLM constraint generation, GSV graph verification, GPFS optimization, and then benchmark evaluation. No fitted constant is renamed as a prediction, and no result is defined in terms of the quantity it is supposed to establish. The CF/IB gains are expected because the physical loss directly minimizes collision and in-bound penalties, and the PSA/Pos/Rot scores are GPT-4o judgments of renderings; neither is an equation-level equivalence with the method's inputs. The cited benchmark protocol and loss formulations come from LayoutVLM and other external groups, so the load-bearing support is not a self-citation chain. The paper's own Sec. 5 limitation statement concedes that GPT API stochasticity 'affects object category selection, object quantity, and the subsequent evaluation process,' and using GPT-4o/GPT-4.1 for generation while using GPT-4o for evaluation is a real validity confound; however, a same-family evaluator is an empirical bias risk, not a derivation that is circular by construction. The absolute SOTA claim in Sec. 4.3 also conflicts with Table 1 (e.g., Deli: Ours(gpt-4.1) IB 66.7 vs LayoutVLM 100.0, and PSA 60.0 vs 74.6), but that is an evidence/claim mismatch rather than circularity. I therefore find no significant circularity under the specified definitions.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central SOTA claim rests on four classes of assumed support: the reliability of the VLM for constraint generation and refinement, the validity of the GPT-4o judge, the completeness of the hand-written graph rules, and standard geometric collision math. The method introduces no new physical entities, but its hyperparameters and rule thresholds are hand-chosen and mostly unreported.

free parameters (3)
  • Evolutionary optimizer hyperparameters = mu=50, top-5 elites, T=30/50 generations, 200 GD steps; swap probability, mutation scale, w_cg, and eta unreported
    Set by hand in Sec. 4.2; no sensitivity analysis is provided, and several values are not reported at all.
  • Loss weights and collision/boundary thresholds = lambda1, lambda2, collision epsilon, in-bound distance, and conflict thresholds epsilon_d/tau unreported
    Defined in supplementary Secs. C and D without numerical values, yet they control the trade-off between semantic and physical terms.
  • CLIP retrieval weights = w1, w2, and top-k unreported
    Supplementary Eq. 16 defines S = w1*C - w2*E but does not give the weights or the number of candidates.
assumptions (4)
  • domain assumption GPT-4o and GPT-4.1 can generate plausible inter-object constraints and revise them from conflict logs
    The entire GSV loop in Sec. 3.1 depends on VLM reliability; no calibration, error analysis, or human check is provided.
  • domain assumption GPT-4o image-based evaluation approximates human judgment of layout plausibility
    Sec. 4.1 uses GPT-4o as the sole evaluator; no human study or correlation analysis is reported.
  • ad hoc to paper The hand-written rule set (cycle, distance, wall, occupancy, logic, completeness, semantic predicates) is sufficient to capture global semantic consistency
    Defined in Sec. 3.1.2 and supplementary D; no completeness argument shows these rules cover all contradictions a scene may contain.
  • standard math The Separating Axis Theorem-based differentiable collision loss is a valid proxy for physical collision
    Supplementary C.2 uses a standard geometric result, citing Boyd et al. [3]; this is standard math and not a point of failure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation." pith.science (2026). https://pith.science/paper/BTG2PO2P

@misc{pith2026260803064,
  author       = {Pith},
  title        = {Pith review of: Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BTG2PO2P}},
  note         = {Machine review of arXiv:2608.03064}
}
read the original abstract

We study open-vocabulary 3D indoor layout generation, which synthesizes diverse and physically plausible scenes from unlabeled 3D assets using free-form language instructions. Recent methods leverage large language models (LLMs) and vision-language models (VLMs) to generate structured scenes from text. However, most model inter-asset relations implicitly or rely on local pairwise constraints and local optimization. These formulations are poorly aligned with the global, highly non-convex layout space, often yielding locally plausible yet globally inconsistent or physically infeasible scenes. We address this problem with a graph-based intermediate representation that separates semantic coherence from physical feasibility, together with a hybrid search-and-refinement strategy. First, Global Semantic Verification (GSV) represents scenes as structured graphs and enforces semantic constraints through rule-based verification. This explicit validation removes contradictory configurations and produces a globally consistent semantic scaffold. Second, Global Physical Feasibility Search (GPFS) combines evolutionary search for global exploration with gradient-based refinement for local exploitation. It reduces dependence on VLM-proposed initialization and improves robustness in non-convex and discontinuous feasible spaces. Together, GSV and GPFS move layout generation beyond local relational modeling and initialization-sensitive optimization toward globally consistent reasoning and search. Experiments show that our method achieves state-of-the-art performance in open-vocabulary 3D indoor layout generation, improving both semantic consistency and physical plausibility.

Figures

Figures reproduced from arXiv: 2608.03064 by the authors.

Figure 1
Figure 1. Existing methods vs. our approach. Prior methods rely on local pairwise consistency and gradient-based refinement, often producing semantically or physically invalid layouts. Our graph-based iterative refinement and optimization yields coherent and physically plausible layouts. 1 Introduction Spatial reasoning and planning refer to understanding, arranging, and manipu￾lating objects in 3D space while respecting phys… view at source ↗
Figure 2
Figure 2. Overview of the proposed pipeline. Given a textual scene description, (1) the framework first constructs a hierarchical scene proposal mapping functional areas and asset set; (2) semantic constraints are then extracted and refined in a graph via Global Semantic Verification (GSV); finally, (3) a physically valid 3D layout is generated using Global Physical Feasibility Search (GPFS). relevant 3D assets A = {ai} N i=1… view at source ↗
Figure 3
Figure 3. The Graph-guided Reflection Process. Through iterative updates to scene-graph edges, the model detects and mitigates physical, semantic, and structural inconsistencies, yielding a coherent and physically plausible layout. where each Φk evaluates a semantic criterion on the subgraph G (0) l , V denotes logical AND. Consequently, Φ(Gl) = 1 if and only if every Φk is satisfied. The K validation predicate Φ = {Φk} K k=1… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison. Compared to baseline methods, our approach pro￾duces layouts with improved semantic and physical consistency. 4.2 Implementation Details We employ GPT-4o and GPT-4.1 as the VLM backbone for semantic reasoning and constraint generation. For asset…
Figure 5
Figure 5. Figure 5: Results. the results demonstrate that the proposed method can generate dense and collision-free indoor layouts 4.3 Benchmark Performance Tab. 1 reports comparisons against LayoutGPT, Holodeck, I-Design, and Lay￾outVLM. Our method achieves higher scores across all metri…
Figure 6
Figure 6. Figure 6: Furniture surface placement. Small asset placement guided by spatial con￾straints. This behavior is consistently observed at both the floor-object and furniture￾object levels (see Figs. 5 and 6), where our framework produces correct relational structures with negligibl…
Figure 7
Figure 7. Figure 7: Ablation study. Comparison of layout results with or without GSV and GPFS module, indicating that GSV ensures semantic coherence and GPFS promotes globally feasible layouts. and relatively high interaction density. Such diversity ensures that the effects of each compon…
Figure 1
Figure 1. Figure 1: SceneEval-100 visualization. relations, demonstrating good generalization beyond the original 33-scene bench￾mark. B VLM-Based Scene Layout Representation This section provides details about VLM-Based Scene Layout Representation, as referenced in Sec. 3 of the main pap…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 67 canonical work pages

  1. [1]

    arXiv preprint arXiv:2403.09675 (2024) 4 16 J

    Aguina-Kang, R., Gumin, M., Han, D.H., Morris, S., Yoo, S.J., Ganeshan, A., Jones, R.K., Wei, Q.A., Fu, K., Ritchie, D.: Open-universe indoor scene genera- tion using llm program synthesis and uncurated object databases. arXiv preprint arXiv:2403.09675 (2024) 4 16 J. Huang et al

    arXiv 2024

  2. [2]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Armeni, I., He, Z.Y., Gwak, J., Zamir, A.R., Fischer, M., Malik, J., Savarese, S.: 3d scene graph: A structure for unified semantics, 3d space, and camera. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 5664–5673 (2019) 4

    work page 2019

  3. [3]

    Cambridge university press (2004) 7

    Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge university press (2004) 7

    work page 2004

  4. [4]

    In: European Conference on Computer Vision

    Çelen,A.,Han,G.,Schindler,K.,VanGool,L.,Armeni,I.,Obukhov,A.,Wang,X.: I-design: Personalized llm interior designer. In: European Conference on Computer Vision. pp. 217–234. Springer (2024) 2, 4, 10

    work page 2024

  5. [5]

    In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)

    Chang, A., Savva, M., Manning, C.D.: Learning spatial knowledge for text to 3d scene generation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). pp. 2028–2038 (2014) 4

    work page 2014

  6. [6]

    arXiv preprint arXiv:1703.00050 (2017) 4

    Chang, A.X., Eric, M., Savva, M., Manning, C.D.: Sceneseer: 3d scene design with natural language. arXiv preprint arXiv:1703.00050 (2017) 4

    arXiv 2017

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Deitke, M., Schwenk, D., Salvador, J., Weihs, L., Michel, O., VanderBilt, E., Schmidt, L., Ehsani, K., Kembhavi, A., Farhadi, A.: Objaverse: A universe of annotated 3d objects. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13142–13153 (2023) 11

    work page 2023

  8. [8]

    In: Proceedings of the Computer Vision and Pattern Recognition Con- ference

    Deng, W., Qi, M., Ma, H.: Global-local tree search in vlms for 3d indoor scene generation. In: Proceedings of the Computer Vision and Pattern Recognition Con- ference. pp. 8975–8984 (2025) 2, 4

    work page 2025

Show all 76 references
  1. [9]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Dhamo, H., Manhardt, F., Navab, N., Tombari, F.: Graph-to-3d: End-to-end gen- eration and manipulation of 3d scenes using scene graphs. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 16352–16361 (2021) 4

    2021

  2. [10]

    Advances in Neural Information Processing Systems36, 18225–18250 (2023) 2, 4, 10

    Feng, W., Zhu, W., Fu, T.j., Jampani, V., Akula, A., He, X., Basu, S., Wang, X.E., Wang, W.Y.: Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems36, 18225–18250 (2023) 2, 4, 10

    2023

  3. [11]

    ACM Transactions on Graphics (TOG)31(6), 1–11 (2012) 4

    Fisher, M., Ritchie, D., Savva, M., Funkhouser, T., Hanrahan, P.: Example-based synthesis of 3d object arrangements. ACM Transactions on Graphics (TOG)31(6), 1–11 (2012) 4

    2012

  4. [12]

    In: ACM SIGGRAPH 2011 papers, pp

    Fisher, M., Savva, M., Hanrahan, P.: Characterizing structural relationships in scenes using graph kernels. In: ACM SIGGRAPH 2011 papers, pp. 1–12. ACM (2011) 4

    2011

  5. [13]

    Inter- national Journal of Computer Vision129(12), 3313–3337 (2021) 11

    Fu, H., Jia, R., Gao, L., et al.: 3d-future: 3d furniture shape with texture. Inter- national Journal of Computer Vision129(12), 3313–3337 (2021) 11

    2021

  6. [14]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Fu, H., Cai, B., Gao, L., Zhang, L.X., Wang, J., Li, C., Zeng, Q., Sun, C., Jia, R., Zhao, B., et al.: 3d-front: 3d furnished rooms with layouts and semantics. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 10933–10942 (2021) 4

    2021

  7. [15]

    ACM Transactions on Graphics (TOG)36(6), 1–13 (2017) 4

    Fu, Q., Chen, X., Wang, X., Wen, S., Zhou, B., Fu, H.: Adaptive synthesis of indoor scenes via activity-associated object relation graphs. ACM Transactions on Graphics (TOG)36(6), 1–13 (2017) 4

    2017

  8. [16]

    In: European Conference on Computer Vision

    Fu, R., Wen, Z., Liu, Z., Sridhar, S.: Anyhome: Open-vocabulary generation of structured and textured 3d homes. In: European Conference on Computer Vision. pp. 52–70. Springer (2024) 2, 4

    2024

  9. [17]

    arXiv preprint arXiv:2405.21066 (2024) 4 Graph-Validated 3D Scene Generation 17

    Hu, S., Arroyo, D.M., Debats, S., Manhardt, F., Carlone, L., Tombari, F.: Mixed diffusion for 3d indoor scene synthesis. arXiv preprint arXiv:2405.21066 (2024) 4 Graph-Validated 3D Scene Generation 17

    2024 arXiv

  10. [18]

    In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition

    Johnson, J., Gupta, A., Fei-Fei, L.: Image generation from scene graphs. In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition. pp. 1219–1228 (2018) 4

    2018

  11. [19]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Khanna, M., Mao, Y., Jiang, H., Haresh, S., Shacklett, B., Batra, D., Clegg, A., Undersander, E., Chang, A.X., Savva, M.: Habitat synthetic scenes dataset (hssd- 200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation. In: Proceedings of the IEEE/CV...

    2024

  12. [20]

    arXiv preprint arXiv:1312.6114 (2013) 4

    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013) 4

    2013 arXiv

  13. [21]

    ACM Transactions on Graphics (TOG)38(2), 1–16 (2019) 4

    Li, M., Patil, A.G., Xu, K., Chaudhuri, S., Khan, O., Shamir, A., Tu, C., Chen, B., Cohen-Or, D., Zhang, H.: Grains: Generative recursive autoencoders for indoor scenes. ACM Transactions on Graphics (TOG)38(2), 1–16 (2019) 4

    2019

  14. [22]

    arXiv preprint arXiv:2402.04717 (2024) 4

    Lin, C., Mu, Y.: Instructscene: Instruction-driven 3d indoor scene synthesis with semantic graph prior. arXiv preprint arXiv:2402.04717 (2024) 4

    2024 arXiv

  15. [23]

    arXiv preprint arXiv:2505.02836 (2025) 2, 4

    Ling, L., Lin, C.H., Lin, T.Y., Ding, Y., Zeng, Y., Sheng, Y., Ge, Y., Liu, M.Y., Bera, A., Li, Z.: Scenethesis: A language and vision agentic framework for 3d scene generation. arXiv preprint arXiv:2505.02836 (2025) 2, 4

    2025 arXiv

  16. [24]

    In: Computer Graphics Forum

    Littlefair, G., Dutt, N.S., Mitra, N.J.: Flairgpt: Repurposing llms for interior de- signs. In: Computer Graphics Forum. p. e70036. Wiley Online Library (2025) 4

    2025

  17. [25]

    ACM Transactions on Graphics (TOG)33(6), 1–12 (2014) 4

    Liu, T., Chaudhuri, S., Kim, V.G., Huang, Q., Mitra, N.J., Funkhouser, T.: Creat- ing consistent scene graphs using a probabilistic grammar. ACM Transactions on Graphics (TOG)33(6), 1–12 (2014) 4

    2014

  18. [26]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Luo, A., Zhang, Z., Wu, J., Tenenbaum, J.B.: End-to-end optimization of scene layout. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3754–3763 (2020) 4

    2020

  19. [27]

    ACM transactions on graphics (TOG)30(4), 1–10 (2011) 4

    Merrell,P.,Schkufza,E.,Li,Z.,Agrawala,M.,Koltun,V.:Interactivefurniturelay- out using interior design guidelines. ACM transactions on graphics (TOG)30(4), 1–10 (2011) 4

    2011

  20. [28]

    arXiv preprint arXiv:2503.16848 (2025) 2

    Pun, H.I.D., Tam, H.I.I., Wang, A.T., Huo, X., Chang, A.X., Savva, M.: Hsm: Hierarchical scene motifs for multi-scale indoor scene generation. arXiv preprint arXiv:2503.16848 (2025) 2

    2025

  21. [29]

    In: European Conference on Computer Vision

    Purkait, P., Zach, C., Reid, I.: Sg-vae: Scene grammar variational autoencoder to generate new indoor scenes. In: European Conference on Computer Vision. pp. 155–171. Springer (2020) 4

    2020

  22. [30]

    Advances in Neural Information Processing Systems 28(2015) 4

    Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. Advances in Neural Information Processing Systems 28(2015) 4

    2015

  23. [31]

    arXiv preprint arXiv:2503.11958 (2025) 4

    Su, C., Fu, Y., Hu, Z., Yang, J., Hanji, P., Wang, S., Zhao, X., Öztireli, C., Zhong, F.: Chord: Generation of collision-free, house-scale, and organized digital twins for 3d indoor scenes with controllable floor plans and optimal layouts. arXiv preprint arXiv:2503.11958 (2025) 4

    2025 arXiv

  24. [32]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Sun, F.Y., Liu, W., Gu, S., Lim, D., Bhat, G., Tombari, F., Li, M., Haber, N., Wu, J.: Layoutvlm: Differentiable optimization of 3d layout via vision-language models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 29469–29478 (2025) 2, 4, 10

    2025

  25. [33]

    In: European Conference on Computer Vision

    Sun, Q., Zhou, H., Zhou, W., Li, L., Li, H.: Forest2seq: Revitalizing order prior for sequential indoor scene synthesis. In: European Conference on Computer Vision. pp. 251–268. Springer (2024) 4 18 J. Huang et al

    2024

  26. [34]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Tang, J., Nie, Y., Markhasin, L., Dai, A., Thies, J., Nießner, M.: Diffuscene: De- noising diffusion models for generative indoor scene synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 20507– 20518 (2024) 4

    2024

  27. [35]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wald, J., Dhamo, H., Navab, N., Tombari, F.: Learning 3d semantic scene graphs from 3d indoor reconstructions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3961–3970 (2020) 4

    2020

  28. [36]

    In: 2021 International Conference on 3D Vision (3DV)

    Wang, X., Yeshwanth, C., Nießner, M.: Sceneformer: Indoor scene generation with transformers. In: 2021 International Conference on 3D Vision (3DV). pp. 106–115. IEEE (2021) 4

    2021

  29. [37]

    In: European Conference on Computer Vision

    Wu, Z., Feng, M., Wang, Y., Xie, H., Dong, W., Miao, B., Mian, A.: External knowledge enhanced 3d scene generation from sketch. In: European Conference on Computer Vision. pp. 286–304. Springer (2024) 4

    2024

  30. [38]

    In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition

    Yang, Y., Jia, B., Zhi, P., Huang, S.: Physcene: Physically interactable 3d scene synthesis for embodied ai. In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. pp. 16262–16272 (2024) 4

    2024

  31. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yang, Y., Sun, F.Y., Weihs, L., VanderBilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al.: Holodeck: Language guided generation of 3d embodied ai environments. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p...

    2024

  32. [40]

    ACM Transactions on Graphics (TOG)31(4), 1–11 (2012) 4

    Yeh, Y.T., Yang, L., Watson, M., Goodman, N.D., Hanrahan, P.: Synthesizing open worlds with constraints using locally annealed reversible jump mcmc. ACM Transactions on Graphics (TOG)31(4), 1–11 (2012) 4

    2012

  33. [41]

    Advances in Neural Information Processing Systems24(2011) 4

    Zhao, Y., Zhu, S.C.: Image parsing with stochastic scene grammar. Advances in Neural Information Processing Systems24(2011) 4

    2011

  34. [42]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Zhou, Y., While, Z., Kalogerakis, E.: Scenegraphnet: Neural message passing for 3d indoor scene augmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 7384–7392 (2019) 4 Global Graph-Validated Optimization for VLM-based 3D Indoor Scene G...

    2019

  35. [43]

    Place the door on one of the room's b ou nd ar y walls , c o n s i d e r i n g the room type . - Do not default to a corner p l a c e m e n t ; choose a lo ca ti on that best su pp or ts the room's f un ct ion and c i r c u l a t i o n - Ensure the door's c o o r d i n a t e s...

  36. [44]

    O p t i o n a l l y place zero or more windows if a p p r o p r i a t e for the room type

  37. [45]

    wall_id

    Follow these rules : - Snap all c o o r d i n a t e s to the nearest 0.25 m grid . - The door and windows center must be at least 0.5 m away from room ve rt ic es . - " wall_id " is the index of the wall in c o u n t e r c l o c k w i s e order : wall 0 = vertex 0 -> vertex 1 ...

  38. [46]

    Start by c o n s i d e r i n g the size of the room

    Start by c o n s i d e r i n g the size of the room . Start by c o n s i d e r i n g the size of the room . With respect to its size and d e s c r i p t i o n what are the po ss ib le objects that can be in the room . Be as th or ou gh as p os sib le

  39. [47]

    You should c ons id er the functional , semantic , and g e o m e t r i c r e l a t i o n s h i p s between the assets

    Ide nt if y the se ma nt ic r e l a t i o n s h i p s between the assets . You should c ons id er the functional , semantic , and g e o m e t r i c r e l a t i o n s h i p s between the assets

  40. [48]

    r o o m _ a n a l y s i s

    Order the areas based on their i m p o r t a n c e to the room description , st ar ti ng with the most i m p o r t a n t area . 3 D C o n v e n t i o n : - The origin is at the lower left of the top view . In the top view , up is + y , right is + x . A d d i t i o n a l rules ...

  41. [49]

    All object choices and counts must comply with it

    Read layout cr it eri a first - it is the primary c o n s t r a i n t . All object choices and counts must comply with it

  42. [50]

    Be as tho ro ug h as po ss ib le

    Based on the room description , Areas description , and layout criteria , add a p p r o p r i a t e floor - sta nd in g objects to each area . Be as tho ro ug h as po ss ib le . Note : Ignore wall - mounted , ceiling - mounted , or ta bl et op objects . Also ignore floor - mou...

  43. [51]

    For each area : specify the objects to be placed in each area and the qu an ti ty of each object

  44. [52]

    For each f u r n i t u r e that is re qu ir ed to be placed on the floor ( e . g . a sofa , a bed , a cabinet , a desk , etc .) : - name : The name of the object ( s ep ar ate d i f f e r e n t c a t e g o r i e s of furniture , e . g . dining_table , office_chair , etc .) . D...

  45. [53]

    areas ": [ {{

    Place at least two objects in each area . Rules : - Each entry in objects r e p r e s e n t s a single type of object . - ** Ensure there is s u f f i c i e n t space for each object to be placed .** - Ignore wall - mounted , ceiling - mounted , or ta bl et op objects . Also I...

  46. [54]

    assets

    o b j e c t s _ i n _ a r e a s : d e s c r i b e s the layout of a room . The room is divided into several f u n c t i o n a l areas . Each area c on tai ns a list of large f u r n i t u r e items ( called " assets ") and a r e c t a n g u l a r b ou nda ry ( called " bounds ...

  47. [55]

    Room d e s c r i p t i o n : { r o o m _ d e s c r i p t i o n }

  48. [56]

    For example , the TV cabinet facing the sofa should have a TV placed on it

    Global render image : Based on the overall layout of the room , d e t e r m i n e what items should be placed on that object . For example , the TV cabinet facing the sofa should have a TV placed on it . ### Your task is as follows :

  49. [57]

    G e n e r a l l y only : tables , bookshelves , TV_stand , cabinet , etc

    From all the listed assets , select those large pieces of f u r n i t u r e s uit ab le for d i s p l a y i n g objects on their su rf ac es . G e n e r a l l y only : tables , bookshelves , TV_stand , cabinet , etc . Do not include other items that do not have storage f u n c...

  50. [58]

    Ignore purely d e c o r a t i v e items , su rf ac es lacking h o r i z o n t a l planes , or small objects . ( e . g . , floor lamps , reading lamps , chairs , s pe ak er s ...)

  51. [59]

    a buffet counter should be eq ui pp ed with plates and cups for placing food

    For each se le ct ed object , write a short , natural - lan gu ag e d e s c r i p t i o n that ex pl ain s what small items can be placed on it , or how it might t y p i c a l l y be used in i nt er io r design . The primary c o n s i d e r a t i o n here should be its in te n...

  52. [60]

    i f _ p l a c e d

    DO NOT contain asset not in o b j e c t s _ i n _ a r e a s . F.6 Small Asset Placement We first cluster mesh triangles to extract all potential horizontal support sur- faces. The resulting side-view and main-view renderings of these surfaces (shown in Fig. B), together with t...

  53. [61]

    count ": the number of items to be placed . Use

    " count ": the number of items to be placed . Use " u n l i m i t e d " only when the item is spread across the full region ( e . g . , a full b o o k s h e l f )

  54. [62]

    z - axis

    "z - axis ": set to true if the item is int en de d to be stacked v e r t i c a l l y . Otherwise , set to false . ** for flat book , z - axis should be set to true .**

  55. [63]

    center

    " center ": specify the a l i g n m e n t of the item within the region in x - y plane .`" center ": true`means the item is placed in the middle of the region , not biased to the left or right

  56. [64]

    d i m e n s i o n s

    " d i m e n s i o n s ": [ length , width , height ] in meters , please infer typical r e a l i s t i c d i m e n s i o n s for the big f u r n i t u r e type and item type . A c c u r a t e l y es ti mat e d i m e n s i o n s based on the region size , the number of objects t...

  57. [65]

    - Only regions with`c l e a r a n c e == 1.0`appear in`obj_list`

    obj _l is t : matched small assets on open su rf ac es ( sta nd in g books exc lu de d ) . - Only regions with`c l e a r a n c e == 1.0`appear in`obj_list`. -`o bj _l is t [ j ][ < key >]`: asset in region j ( e . g .`plate_0`,`cup_1`)

  58. [66]

    -`o p e n _ r e g i o n [ i ]['item']`: label âĘŠ`count`,`z - axis`,`center` -`o p e n _ r e g i o n [ i ]['description']`: natural - la ngu ag e intent for this surface

    o p e n _ r e g i o n ( in the message below ) : s ema nt ic m et ad at a for those same open regions only . -`o p e n _ r e g i o n [ i ]['item']`: label âĘŠ`count`,`z - axis`,`center` -`o p e n _ r e g i o n [ i ]['description']`: natural - la ngu ag e intent for this surface

  59. [67]

    region ( in g e n e r a t e d code ) : use`region [ i ]`when a c o n s t r a i n t needs a region a rg ume nt ( i is str )

  60. [68]

    c o n s t r a i n t s

    global render image : floor - st and in g f u r n i t u r e layout for a d j a c e n c y / edge r e a s o n i n g . Your task : for each open region , output`solver . a d d _ c o n s t r a i n t (...)`code using APIs defined in the system prompt . ```python ob j_ li st = { obj...

  61. [69]

    < wall >

    ** Co nf li ct h an dl in g and se ma nt ic r e f i n e m e n t **: - All [ L o g i c E r r o r ] entries in the log mean the c o n f l i c t i n g edge was already auto - removed . Do NOT re - add those edges u n c h a n g e d . - Among [ L o g i c E r r o r ] ,`o c c u p i e...

  62. [70]

    ** They all need to be solved !**

    Review each [ S e m a n t i c C o m m o n ] , [ C o n s t r a i n t C o m p l e t e n e s s ] entry , and any`a g g r e g a t e _ w a l l _ c o n f l i c t`line . ** They all need to be solved !**

  63. [71]

    D e t e r m i n e if a d d i t i o n a l spatial or r e l a t i o n a l c o n s t r a i n t s are n e c e s s a r y to m ai nta in scene s e m a n t i c s

  64. [72]

    g .`di st an ce (...)`,`a l i g n _ w i t h (...)`)

    Add those c o n s t r a i n t s as direct DSL f un cti on calls ( e . g .`di st an ce (...)`,`a l i g n _ w i t h (...)`)

  65. [73]

    If use fixed point , must define it in your code

    Add all c o n s t r a i n t s needed . If use fixed point , must define it in your code

  66. [74]

    A vs B missing / invalid r el at ion ['distance']

    ** Follow c o n s t r a i n t repair logic **: - If the log says`" A vs B missing / invalid r el at ion ['distance']"`, add a`distance`c o n s t r a i n t : ```DSL di st an ce (A , B , min_distance , m a x _ d i s t a n c e ) ``` - If the log says`" A vs B missing / invalid r ...

  67. [75]

    e a s t _ w a l l

    ** I m p o r t a n t rules **: - Never re - add a c o n s t r a i n t that already exists . - Never add c o n f l i c t i n g c o n s t r a i n t s ( e . g . , two i n c o m p a t i b l e wall sides ) . - Do NOT repeat ex is ti ng c o n s t r a i n t s from c u r r e n t _ c o...

  68. [76]

    > , <** kwargs

    Output format : ```DSL # define new c o n s t r a i n t s only < c o n s t r a i n t _ f u n c t i o n >( < src_key > , < dst_key > , <* args ... > , <** kwargs ... >) ```

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.