REVIEW 6 cited by
Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present a system for generating indoor scenes in response to text prompts. The prompts are not limited to a fixed vocabulary of scene descriptions, and the objects in generated scenes are not restricted to a fixed set of object categories -- we call this setting indoor scene generation. Unlike most prior work on indoor scene generation, our system does not require a large training dataset of existing 3D scenes. Instead, it leverages the world knowledge encoded in pre-trained large language models (LLMs) to synthesize programs in a domain-specific layout language that describe objects and spatial relations between them. Executing such a program produces a specification of a constraint satisfaction problem, which the system solves using a gradient-based optimization scheme to produce object positions and orientations. To produce object geometry, the system retrieves 3D meshes from a database. Unlike prior work which uses databases of category-annotated, mutually-aligned meshes, we develop a pipeline using vision-language models (VLMs) to retrieve meshes from massive databases of un-annotated, inconsistently-aligned meshes. Experimental evaluations show that our system outperforms generative models trained on 3D data for traditional, closed-universe scene generation tasks; it also outperforms a recent LLM-based layout generation method on open-universe scene generation.
Forward citations
Cited by 6 Pith papers
-
TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation
A training-free pipeline generates instance-level, physically interactive 3D tabletop scenes from text or one image, with a differentiable rotation optimizer and top-view spatial alignment for collision-free layouts.
-
HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editing
HOLODECK 2.0 generates and edits stylistically varied 3D scenes from text by combining VLM-based scene analysis and layout planning with generative 3D assets, outperforming the retrieval-based HOLODECK baseline in hum...
-
3D-Generalist: Self-Improving Vision-Language-Action Models for Crafting 3D Worlds
A self-improving vision-language-model policy iteratively crafts 3D environments from text, and renderings of those environments serve as effective synthetic pretraining data for vision models.
-
Video Perception Models for 3D Scene Synthesis
VIPScene synthesizes 3D scenes by generating a video with Cosmos, reconstructing it with Fast3R, extracting objects with Grounded-SAM and MASt3R, and assembling them from Objaverse assets.
-
ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
ReSpace is an autoregressive LLM framework for text-driven 3D indoor scene editing and synthesis, using a structured JSON scene representation and a voxelization-based layout metric.
-
Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques
A survey that maps LLM applications, vulnerabilities, and defenses across eight cybersecurity domains, but with significant citation and rigor problems.
Discussion (0). Sign in to comment.