REVIEW 7 cited by
SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We introduce SceneScript, a method that directly produces full scene models as a sequence of structured language commands using an autoregressive, token-based approach. Our proposed scene representation is inspired by recent successes in transformers & LLMs, and departs from more traditional methods which commonly describe scenes as meshes, voxel grids, point clouds or radiance fields. Our method infers the set of structured language commands directly from encoded visual data using a scene language encoder-decoder architecture. To train SceneScript, we generate and release a large-scale synthetic dataset called Aria Synthetic Environments consisting of 100k high-quality in-door scenes, with photorealistic and ground-truth annotated renders of egocentric scene walkthroughs. Our method gives state-of-the art results in architectural layout estimation, and competitive results in 3D object detection. Lastly, we explore an advantage for SceneScript, which is the ability to readily adapt to new commands via simple additions to the structured language, which we illustrate for tasks such as coarse 3D object part reconstruction.
Forward citations
Cited by 7 Pith papers
-
Vision as Unified Multimodal Generation
A single unified multimodal model matches leading task-specialized vision systems across detection, segmentation, dense geometry, and multi-view 3D by casting all outputs as native text or image generation.
-
DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation
DI-PCG learns to invert procedural 3D generators by diffusing over their parameters, conditioned on DINOv2 image features, producing editable assets in seconds from a photo.
-
GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation
A causal transformer with 3D RoPE generates vector-quantized 3D Gaussian latent grids autoregressively, enabling unconditional synthesis, completion, and open-ended outpainting of indoor scenes.
-
PanSt3R: Multi-view Consistent Panoptic Segmentation
A single network jointly reconstructs 3D scene geometry and predicts multi-view consistent panoptic segmentation from unposed images in one forward pass, without test-time optimization.
-
HOIverse: A Synthetic Scene Graph Dataset With Human Object Interactions
HOIverse is a synthetic indoor scene graph dataset with dense human-object interaction and parametric relation annotations, benchmarked with scene graph generation models.
-
SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos
A neural system that reconstructs dense 3D scenes from monocular RGB video at 20+ FPS by regressing local pointmaps and incrementally registering them into one global model without explicit pose optimization.
-
LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models
LayoutVLM couples VLM-generated pose estimates and spatial relations with differentiable optimization to create physically plausible, instruction-aligned 3D layouts.
Discussion (0). Continue with ORCID to comment.