Pith. sign in

REVIEW 4 major objections 5 minor 108 references

PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenes

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Coarse 3D primitives—cuboids, ellipsoids, and extruded ground polygons—can replace voxel grids as the working representation for generating controllable, editable urban scenes.

desk verdict Genuinely novel primitive-based representation for 3D urban scene generation, but the SOTA claim rests on a confounded comparison and an internal CFG contradiction the authors need to fix. read the letter →

arxiv 2506.19117 v3 pith:VYWAM6NJ submitted 2025-06-23 cs.CV

Christina Ourania Tze , Daniel Dauner , Yiyi Liao , Dzmitry Tsishkou , Andreas Geiger This is my paper · ORCID
classification cs.CV
keywords 3Dsemanticscenegenerationprimitive-basedrepresentationlatentdiffusionurbansynthesiseditingCholeskyparameterizationKITTI-360voxelvsprimitives
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PrITTI argues that compact, semantically meaningful 3D elements—cuboids for buildings and vehicles, ellipsoids for trees, extruded 2D polygons for ground—should replace voxel grids as the working representation for generating urban scenes. The paper builds a two-stage latent diffusion model on this hybrid representation and claims it produces higher-quality, more diverse 3D semantic layouts than voxel-based baselines, while using less memory, generating faster, and supporting direct object-level edits that voxel methods cannot easily offer. The central wager is that this coarse primitive abstraction is not a lossy compromise but an advantage: it gives the generative model a structured, instance-level latent space. If the paper is right, editable semantic urban scene synthesis can be treated as a layout problem on simple parameterized shapes, with photorealism handled downstream.

What carries the argument

The load-bearing mechanism is the joint layout latent space formed by channel-wise concatenating a raster ground latent with an object primitive latent. Ground polygons are ray-cast into bird's-eye-view height maps and occupancy masks and passed through a convolutional encoder; object primitives are processed by a Transformer encoder and scattered onto the same 2D grid via a scatter-mean operation keyed by each object's 2D center. Because ground and object components occupy separate channel groups, the latent is disentangled, so the decoder can split it during reconstruction and the diffusion model can edit one modality while keeping the other fixed. Per-object orientation and size are encoded through the Cholesky decomposition of a positive-definite scatter matrix, a 6D parameterization that avoids the sign-ambiguity problems of quaternions or eigenvectors. On top of this latent, a diffusion transformer (DiT-style backbone) generates unconditionally or conditioned on scene labels, and a RePaint-inspired masked sampling loop performs inpainting and outpainting directly in latent space.

What would settle it

Run the method on urban layouts dominated by structures the primitive set cannot express—bridges, overpasses, tunnels with complex profiles, or dense pole-mounted attachments—and measure reconstruction and generation quality (e.g., mIoU, AP3D, FID). If quality collapses relative to KITTI-360 results, or if manual inspection shows systematically intersecting primitives and buildings cutting into drivable ground, the coarse-primitive abstraction is not sufficient for general urban scene generation.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a hybrid primitive representation unlocks the full capabilities of diffusion transformers for 3D urban scene generation. Objects are encoded as 9-dimensional features (normalized 3D center plus six Cholesky parameters jointly encoding orientation and size), ground classes are rasterized into height and occupancy maps, and a layout variational autoencoder maps both into a single 2D latent grid with separate ground and object channel groups. A diffusion transformer trained on this latent space generates scenes conditioned on controllable labels such as vegetation density, and a RePaint-style manipulation of the latent mask supports inpainting and outpainting without fine-tuning. Compared with voxel methods (SemCity, PDD, XCube on KITTI-360), PrITTI reports the best generation metrics (e.g., FID 71.0–73.9 versus 94.8 for the strongest voxel baseline), 0.58 s mean generation time, and 0.89 GB peak GPU memory, while keeping reconstruction competitive despite being evaluated through an unfavorable voxelization step. The paper states these results as evidence that primitive-based representations offer a more scalable and editable path to semantic urban scene modeling than resolution-bound voxel grids.

Load-bearing premise

The paper's bet is that flat extruded ground and coarse cuboid and ellipsoid boxes capture enough of a real city's structure that scenes built from them remain useful for generation, editing, and downstream simulation, despite losing fine geometry such as overpasses, facades, and small attachments.

Editorial extensions

If this is right

  • Large-scale urban scene synthesis can be decoupled into semantic layout generation and appearance synthesis: the generated primitives can drive a ControlNet-based renderer to produce photorealistic street views without modifying the layout model.
  • Scene editing becomes a parameter update rather than a voxel rewrite: translating, rotating, scaling, or deleting an object is a direct change to one primitive's center and Cholesky parameters, with no need to repair vacated voxel regions.
  • Memory and inference cost no longer scale cubically with desired scene resolution, because the object representation is resolution-independent and only the ground raster has a fixed grid size.
  • A single pre-trained model can grow scenes arbitrarily through iterative outpainting with overlapping windows, preserving road connectivity and semantic coherence across block boundaries.
  • Controllability extends beyond vegetation: the label-conditioning scheme is described as general, and the paper demonstrates it on vehicle density and joint vegetation–vehicle density.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the ground and object latents are disentangled, the framework could likely condition object generation on arbitrary ground semantics (lane graphs, drivable-area masks) without retraining the diffusion model, extending the demonstrated ground-to-object inpainting.
  • The Cholesky-based 6D orientation/size encoding is a portable trick: any bounding-box regression task that struggles with quaternion sign ambiguity could adopt the same parameterization, independent of scene generation.
  • If coarse primitives are accepted as the semantic backbone, then realism becomes a rendering-stage problem; one testable extension is swapping the ControlNet renderer for a Gaussian-splatting or NeRF-based synthesizer to see whether layout edits propagate to appearance.
  • The method's current static-scene and fixed-category limits suggest a natural stress test: adding dynamic objects or unseen classes would require either an open-vocabulary extension or a separate motion model, both of which are unaddressed in this paper.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PrITTI, a two-stage latent diffusion model for 3D semantic urban scene generation. Stage 1 trains a layout VAE with separate ground-raster and object-primitive branches, using a Cholesky-based parameterization for cuboid/ellipsoid object attributes; Stage 2 trains a DiT over the joint latent, conditioned on a vegetation-density label, and uses RePaint-style latent manipulation for inpainting, outpainting, and editing. Experiments on KITTI-360 compare PrITTI with SemCity, PDD, and XCube, reporting lower memory, faster generation, and better FID/Precision/Recall in Table 3. The paper also demonstrates object-level editing, ground-conditioned object generation, large-scale extrapolation, and photo-realistic street-view synthesis via ControlNet, with an additional Argoverse 2 experiment in the supplement.

Significance. If the SOTA generation claim were established, PrITTI would make a useful contribution: it shows that compact vectorized primitives can support controllable 3D urban layout generation with direct editability and lower memory than voxel grids. The two-stage design is clean, the Cholesky parameterization is a sensible alternative to quaternion encodings, and the paper includes helpful supplementary material such as an overfitting/nearest-neighbor analysis, an Argoverse 2 transfer experiment, and broad downstream applications. The central empirical claim, however, is currently weakened by an uncontrolled comparison and by a direct internal contradiction about the inference protocol, so the paper is not yet ready for acceptance.

major comments (4)
  1. [Section 4.2, Table 3] The generation comparison is not controlled for conditioning. PrITTI is conditioned on the vegetation-density label y, with equal sampling over low/medium/high classes, while SemCity, PDD, and XCube are all unconditional. Conditioning gives the model access to target-mode information that the baselines do not receive, so the reported differences in Table 3 cannot be attributed solely to the primitive-based representation. To support the headline claim, the authors should report an unconditional PrITTI variant, or condition the baselines on the same y, or both. A clean comparison is needed before the paper's central 'state-of-the-art generation quality' claim can be accepted.
  2. [Section 4.2 vs. Supplement B.3.4] The main text states that PrITTI is evaluated 'without classifier-free guidance,' while Supplement B.3.4 states that 'Classifier-free guidance is applied with a guidance scale of 4.0.' These statements are mutually inconsistent. Guidance at scale 4.0 typically shifts precision/recall and FID substantially, and no baseline in Table 3 uses guidance. The authors must clarify the exact inference protocol, recompute any affected metrics, and state the guidance scale, if any, directly in the main-text evaluation setup.
  3. [Table 3 and Supplement Fig. 10] The generative metrics in Table 3 are single-point estimates at one evaluation configuration (1K farthest-point-sampled reference samples, k=3) with no error bars, multiple seeds, or sensitivity analysis. Supplement Fig. 10 shows that precision and recall for the proposed method vary substantially with reference-sample count and neighborhood size, with precision dropping roughly from 0.7 at 1K/k=3 to below 0.4 at 10K/k=3. The authors report this sensitivity and then select a favorable configuration; they should either report all configurations shown in Fig. 10, provide confidence intervals, or justify why the single configuration in Table 3 is the primary basis for the SOTA claim.
  4. [Section 4.2, Metrics] All generative quality metrics are computed on 256x256 top-down semantic renders of the 3D scenes, not on the 3D layouts themselves. Given that the paper's contribution is a 3D primitive-based representation, additional 3D-level diagnostics would materially support the '3D scene generation quality' claim: for example, distributions of object counts per category, per-category placement and geometry accuracy, and rates of geometric violations such as primitive intersections or ground-object misalignment. Without such metrics, the FID, IS, Precision, and Recall numbers characterize a 2D BEV projection rather than the proposed 3D representation directly.
minor comments (5)
  1. [Table 1 and Section 4.1] The phrase 'competitive reconstruction' should be quantified more carefully: at the same 256^2x32 voxelization, PrITTI reports mIoU 70.27 against 93.81 for SemCity-1M and 79.47 for XCube, so the 'competitiveness' claim is only meaningful with the caveat that PrITTI is natively primitive-based and voxelization penalizes its measurements.
  2. [Section 3.2 and Supplement A] The fixed per-category primitive count N_c is set to the 95th percentile and additional instances are discarded. The effect of this truncation on dense scenes (e.g., vegetation, fences, or pole-mounted attachments) is not analyzed; a short quantitative or qualitative discussion of truncation sensitivity would be useful.
  3. [Figure 4] The Cholesky-versus-quaternion comparison is performed on synthetic single-object samples with random yaw and scale. A full-scene ablation with multiple objects and varied orientation distributions would be more convincing evidence that the parameterization improves training stability in the actual task.
  4. [Section 4.3] Object-level editing, ground-conditioned generation, and photo-realistic street-view synthesis are evaluated only qualitatively. A simple quantitative measure, such as a user study or per-edit semantic consistency metric, would strengthen these claims.
  5. [Related Work / Experiments] SLEDGE and other layout-generation methods are discussed in Related Work but are not compared empirically. A sentence explaining why a quantitative comparison is infeasible (e.g., different data formats or task definitions) would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; evaluation fairness concerns are correctness risks, not circularity.

full rationale

The paper's derivation chain is self-contained. The central claims—that a two-stage layout VAE plus latent DiT over primitive and raster latents can reconstruct and generate urban layouts—are supported by empirical comparisons against external voxel-based baselines (SemCity, PDD, XCube), not by definitions or self-citations. The primitive representation in Sec. 3.1 is an explicit modeling choice, not a derived result. The Cholesky parameterization in Sec. B.1 is a standard mathematical encoding, and its claimed benefit is tested on a synthetic vehicle-rotation experiment (Fig. 4) rather than assumed. The scene label y is computed from ground-truth vegetation statistics and used as a conditioning input; no fitted parameter is relabeled as a prediction. Self-citations such as SLEDGE [14] and UrbanGIRAFFE [96] appear only as related-work context and carry no load-bearing premise. The main concerns are evaluation fairness—PrITTI is evaluated conditionally while baselines are unconditional, the main text says sampling is done without classifier-free guidance while supplement Sec. B.3.4 states a guidance scale of 4.0, and Fig. 10 shows strong sensitivity of Precision/Recall to evaluation configuration. These are validity and correctness risks, not circularity under the definitional-equivalence tests. No step reduces to its own inputs by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical or theoretical entities are postulated. The free parameters are data-derived counts, thresholds, and training or inference choices. The main load-bearing assumptions are representational adequacy and evaluation adequacy, both of which the paper partially acknowledges in its limitations and metric-sensitivity discussion.

free parameters (6)
  • Per-category primitive counts N_c (95th percentile) = VC 178, VE 159, VS 18, TW 6, H 5, CB 16, CS 77, P 19, TC 17, O 17, VB 2; total 514
    Determined from KITTI-360 dataset statistics to define the fixed-size set prediction capacity of the object encoder and decoder (Supplementary A, Table 4).
  • Object existence probability threshold = 0.3, with stated insensitivity in [0.25, 0.75]
    Applied to decoder outputs to decide which predicted primitives enter the final reconstructed layout (Supplementary B.2.4).
  • Vegetation-density label quantiles = 25th and 75th percentiles of vegetation count and volume
    Defines low, medium, and high conditioning labels used for the class-conditional diffusion model (Supplementary A, Scene Labeling).
  • Classifier-free guidance scale = 4.0 (reported in the supplement; the main text says generation was run without classifier-free guidance)
    Inference-time conditioning strength; the contradiction between the main text and supplement directly affects the generation metrics in Table 3 (Supplementary B.3.4 vs Section 4.2).
  • Outpainting overlap ratio = 50%
    Chosen to balance structural coherence and generative freedom in the sliding-window outpainting scheme (Supplementary B.3.3).
  • KL weight for the layout VAE = 1e-6
    Regularization strength on the joint layout latent in the first-stage training objective (Supplementary B.2.2).
assumptions (5)
  • domain assumption KITTI-360 primitive annotations, cuboids, ellipsoids, and extruded polygons, faithfully represent urban scene semantics.
    All training and evaluation derive from these annotations; if they omit or coarsen structures such as bridges, overpasses, or fine attachments, the generated scenes inherit that incompleteness (Section 3.1, Supplementary A).
  • domain assumption BEV-rendered semantic maps and FID, Precision, Recall, and Inception Score adequately measure 3D scene generation quality.
    Generation metrics are computed on 2D top-down semantic renderings (Section 4.2), so 3D errors such as primitive intersections, floating objects, or ground-object misalignment are not directly penalized; the paper itself shows such artifacts (Supplementary I, Figure 15).
  • domain assumption Retraining voxel baselines on the voxelized KITTI-360 data is a fair test of those baselines.
    SemCity, PDD, and XCube were originally designed for other datasets and setups; retraining them from scratch on PrITTI's voxelized scenes may disadvantage them, and the paper does not provide an independent benchmark on their native datasets (Supplementary D, Table 7).
  • domain assumption The custom train/test split prevents spatial and temporal leakage between training and test poses.
    The split uses equal-distance segments with 15-meter test zones and 5-meter margins, but the proximity in time between neighboring driving frames is not explicitly analyzed, so temporal correlation effects are not fully quantified (Supplementary A).
  • standard math Standard DDPM and DiT machinery is correct and applicable to the learned latent space.
    The method builds on standard generative model theory and implementation from prior literature (Supplementary B.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenes." pith.science (2026). https://pith.science/paper/VYWAM6NJ

@misc{pith2026250619117,
  author       = {Pith},
  title        = {Pith review of: PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VYWAM6NJ}},
  note         = {Machine review of arXiv:2506.19117}
}
read the original abstract

Existing approaches to 3D semantic urban scene generation predominantly rely on voxel-based representations, which are bound by fixed resolution, challenging to edit, and memory-intensive in their dense form. In contrast, we advocate for a primitive-based paradigm where urban scenes are represented using compact, semantically meaningful 3D elements that are easy to manipulate and compose. To this end, we introduce PrITTI, a latent diffusion model that leverages vectorized object primitives and rasterized ground surfaces for generating diverse, controllable, and editable 3D semantic urban scenes. This hybrid representation yields a structured latent space that facilitates object- and ground-level manipulation. Experiments on KITTI-360 show that primitive-based representations unlock the full capabilities of diffusion transformers, achieving state-of-the-art 3D scene generation quality with lower memory requirements, faster inference, and greater editability than voxel-based methods. Beyond generation, PrITTI supports a range of downstream applications, including scene editing, inpainting, outpainting, and photo-realistic street-view synthesis. The source code and more results can be found at https://raniatze.github.io/pritti/.

Figures

Figures reproduced from arXiv: 2506.19117 by the authors.

Figure 1
Figure 1. PrITTI generates (1) high-quality, controllable 3D semantic urban scenes in a compact primitive-based representation using a latent diffusion model. Starting from a generated scene (e.g. middle sample), we demonstrate downstream applications including (2) scene editing, (3) inpainting, (4) outpainting, and (5) photo-realistic street view synthesis. Abstract Existing approaches to 3D semantic urban scene gener￾ation … view at source ↗
Figure 2
Figure 2. Training Overview. An input 3D semantic layout \mathcal {S} comprises object primitives, encoded as feature vectors \mathbf {F} , and extruded ground polygons, rasterized into height maps \mathbf {H} and binary occupancy masks \mathbf {B} (Sec. 3.1). A layout VAE with separate encoder-decoder pairs for objects ( \mathcal {E}_\mathcal {O} /\protect \mathcal {D}_\mathcal {O} ) and ground (\protect \mathcal {E}_\mathca… view at source ↗
Figure 3
Figure 3. Stage 1: Qualitative reconstruction results on the same test scenes shown in each method’s native representation: primitives for PrITTI and voxel grids for SemCity and XCube. Voxel-based methods sometimes yield incomplete geometry and grid-induced distortions, such as vertical clipping at tall primitives. autoencoder (AE) followed by a triplane diffusion model. We adopt its setup and voxelize our primitive scenes in… view at source ↗
Figures from the paper (28 more)
Figure 4
Figure 4. Figure 4: Cholesky vs. quaternion encodings across training sizes. on a binary occupancy grid (foreground vs. background), while mIoU is averaged over all classes excluding the empty one. As PDD lacks a VAE component, we report results for SemCity, XCube, and our voxelized primi…
Figure 5
Figure 5. Figure 5: 3D Semantic Scene Generation. Comparison of 3D semantic layouts generated by PrITTI (Ours, left) and voxel-based baselines (SemCity, PDD, XCube, right). PrITTI enables controllable generation, here conditioned on vegetation density (low, medium, or high), and produces …
Figure 6
Figure 6. Figure 6: Scene-level Editing. (Left) Inpainting and outpainting results. Red outlines indicate the inpainted region (second column) or the original scene area (third column). (Right) An iterative outpainting example where the scene is progressively extended to the right. The re…
Figure 7
Figure 7. Figure 7: Extrusion Process. An illustrative extrusion example showing the composite color map (bottom), height field (middle), and lifted triangle mesh (top). B.2.3. Layout Reconstruction To reconstruct a 3D scene layout, we follow separate proce￾dures for ground surfaces and o…
Figure 8
Figure 8. Figure 8: Latent-space Scene Outpainting. Given a known layout block P (green) with latent z P , the diffusion model predicts the latent z Q for an adjacent block Q (outlined in red) that partially overlaps with P. During each sampling step, the overlapping area (yellow) is re-n…
Figure 9
Figure 9. Figure 9: Low-quality scene generation examples from baseline methods exhibiting fragmented geometry and semantic inconsistencies. Metric SemCity PDD XCube Ours (DiT-B) Peak GPU Memory (GB) 8.30 1.33 4.84 0.89 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Precision and recall comparison between our method (using DiT-B), SemCity, XCube, and PDD under two evaluation protocols: (a) distance-threshold reference-pose sampling and (b) farthest-pose sampling, each tested across varying neighborhood sizes. G. Overfitting Analy…
Figure 11
Figure 11. Figure 11: Nearest-neighbor Analysis. For each generated sample (left), we show its closest training samples (right). Generated scenes differ in layout and object placement, suggesting the model does not memorize training data. for arbitrary semantic categories. When fewer categ…
Figure 12
Figure 12. Figure 12: PrITTI Failure Cases. (a) Fine-grained attachments (e.g. pole-mounted lamps) are often not accurately reconstructed, yielding floating objects. (b) Large empty conditioning regions in outpainting can lead to weak or inconsistent completions. Raster (×10−2 ) Primitive …
Figure 13
Figure 13. Figure 13: PrITTI Scene Generation Results on Argoverse 2 (AV2) Dataset. Generated 3D semantic scenes grouped under three vehicle-density conditions: (a) high, (b) medium, and (c) low. Vehicles appearing outside the drivable surface correspond to parked cars, as AV2 [86] HD maps…
Figure 14
Figure 14. Figure 14: PrITTI Scene Generation Results. Generated 3D semantic scenes grouped under three vegetation-density conditions: (a) high, (b) medium, and (c) low. Across all settings, PrITTI produces diverse and realistic urban layouts with coherent ground surfaces and well-shaped p…
Figure 15
Figure 15. Figure 15: BEV Renderings of PrITTI-generated Scenes. Examples on the left illustrate well-structured and realistic scene generations, while those on the right show poorly synthesized samples with geometric or semantic inconsistencies [PITH_FULL_IMAGE:figures/full_fig_p028_15.png]
Figure 16
Figure 16. Figure 16: PrITTI Vehicle Density-Conditioned Scene Generation. Generated 3D semantic scenes grouped under three vehicle-density conditions: (a) high, (b) medium, and (c) low, demonstrating controllability over primitive categories beyond vegetation [PITH_FULL_IMAGE:figures/ful…
Figure 17
Figure 17. Figure 17: PrITTI Joint Vegetation–Vehicle Density-Conditioned Scene Generation. Generated scenes conditioned on combinations of vegetation and vehicle densities. From the nine possible settings, we show three examples: (a) high vegetation, low vehicles, (b) medium vegetation, m…
Figure 18
Figure 18. Figure 18: SemCity Scene Generation Results. Unconditional 3D semantic scene samples generated by SemCity [38], occasionally exhibiting irregular object shapes, blurred boundaries, and inconsistent semantic structures [PITH_FULL_IMAGE:figures/full_fig_p031_18.png]
Figure 19
Figure 19. Figure 19: BEV Renderings of SemCity-generated Scenes. The examples on the left show relatively well-structured and semantically coherent layouts, while those on the right exhibit fragmented, irregular geometry and inconsistent semantics [PITH_FULL_IMAGE:figures/full_fig_p032_19.png]
Figure 20
Figure 20. Figure 20: PDD Hierarchical Scene Generation Results. Unconditional 3D semantic scenes generated by PDD [44] at successive pyramid levels. While scene details increase across levels, the final stage also introduces noticeable noise and semantic inconsistencies [PITH_FULL_IMAGE:…
Figure 21
Figure 21. Figure 21: Hierarchical BEV Renderings of PDD-generated Scenes. Each column triplet shows results across successive hierarchy levels. Examples on the left exhibit relatively coherent scene structure, whereas those on the right clearly show that the last refinement stage introduc…
Figure 22
Figure 22. Figure 22: XCube Hierarchical Scene Generation Results. Unconditional 3D semantic scenes generated by XCube [64] across successive hierarchy levels. Scene details improve across levels, and the overall structure appears realistic, but irregular object shapes (e.g. vehicles) and …
Figure 23
Figure 23. Figure 23: Hierarchical BEV Renderings of XCube-generated Scenes. Each column pair shows results across successive hierarchy levels. Examples on the left demonstrate high-quality generations, while those on the right exhibit geometric distortions and semantic inconsistencies [P…
Figure 24
Figure 24. Figure 24: PrITTI Inpainting Results. Inpainting applied to (a) top, (b) bottom, (c) right, and (d) left regions of an input scene. PrITTI produces localized edits that blend seamlessly with the surrounding geometry and semantics, yielding consistent and plausible completions […
Figure 25
Figure 25. Figure 25: PrITTI Ground-to-Object Inpainting Results. Leveraging our disentangled joint latent structure, we can inpaint object primitives while keeping the ground layout fixed. The results show that the ground geometry is faithfully preserved from the original scene, while new…
Figure 26
Figure 26. Figure 26: Comparison of scene inpainting and outpainting results on identical test scenes using our method and SemCity [38]. PrITTI (Ours) yields coherent and semantically consistent edits and extensions, while SemCity often produces fragmented or contextually inconsistent stru…
Figure 27
Figure 27. Figure 27: PrITTI Outpainting Results. Outpainting examples produced by our method showing that it can expand a scene beyond its spatial extent while preserving ground layout continuity and generating new objects that integrate naturally with the surrounding context [PITH_FULL_…
Figure 28
Figure 28. Figure 28: PrITTI Large-scale Scene Extrapolation Results. Starting from an initially generated block (outlined in red), our method produces extended layouts that remain globally coherent and semantically consistent, with smooth structural transitions across blocks [PITH_FULL_I…
Figure 29
Figure 29. Figure 29: PDD Large-scale Scene Extrapolation Results. Starting from an initially generated block (outlined in red), PDD progressively extends the scene upward. The extrapolated regions exhibit broken road connectivity, misplaced structures, and inconsistent semantic transition…
Figure 30
Figure 30. Figure 30: PrITTI Object Editing Results. Enabled by our instance-level primitive representation, PrITTI naturally supports a wide range of individual-object manipulations. Columns (b-e) show dropout, rotation, scaling, and translation applied to the same randomly selected vehic…
Figure 31
Figure 31. Figure 31: PrITTI Photo-realistic Street View Synthesis Results. Each pair shows a 2D semantic rendering (left) and the corresponding photo-realistic image (right), generated with ControlNet [104]. Despite the coarse geometry of our primitives, the synthesized scenes exhibit div…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

108 extracted references · 66 canonical work pages

  1. [1]

    https://www.turbosquid.com/3d- models/3d-karton-city-2-model-1196110

    3d karton city model. https://www.turbosquid.com/3d- models/3d-karton-city-2-model-1196110. 20

  2. [2]

    Saor: Single-view articulated object reconstruction

    Mehmet Aygun and Oisin Mac Aodha. Saor: Single-view articulated object reconstruction. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024. 8

    2024

  3. [3]

    Guibas, and Andrea Tagliasacchi

    Sherwin Bahmani, Jeong Joon Park, Despoina Paschalidou, Xingguang Yan, Gordon Wetzstein, Leonidas J. Guibas, and Andrea Tagliasacchi. CC3D: layout-conditioned generation of compositional 3d scenes. InProc. of the IEEE Interna- tional Conf. on Computer Vision (ICCV), 2023. 2

    2023

  4. [4]

    Se- mantickitti: A dataset for semantic scene understanding of lidar sequences

    Jens Behley, Martin Garbade, Andres Milioto, Jan Quen- zel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Se- mantickitti: A dataset for semantic scene understanding of lidar sequences. InProc. of the IEEE International Conf. on Computer Vision (ICCV), 2019. 20

    2019

  5. [5]

    Human image understanding: Recent research and a theory.Computer vision, graphics, and image processing, 1985

    Irving Biederman. Human image understanding: Recent research and a theory.Computer vision, graphics, and image processing, 1985. 2

    1985

  6. [6]

    Visual perception by computer

    Thomas Binford. Visual perception by computer. InProc. IEEE Conf. on Systems and Control, 1975, 1975. 2

    1975

  7. [7]

    Scenefactor: Factored latent 3d diffusion for controllable 3d scene generation.arXiv.org, 2412.01801,

    Alexey Bokhovkin, Quan Meng, Shubham Tulsiani, and Angela Dai. Scenefactor: Factored latent 3d diffusion for controllable 3d scene generation.arXiv.org, 2412.01801,

    arXiv

  8. [8]

    Omni3d: A large benchmark and model for 3d object detection in the wild

    Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3d: A large benchmark and model for 3d object detection in the wild. In Proc. IEEE Conf. on Computer Vision and Pattern Recogni- tion (CVPR), 2023. 6

    2023

Show all 108 references
  1. [9]

    The acronym model-based vision system

    Rodney A Brooks, Russell Creiner, and Thomas O Binford. The acronym model-based vision system. InProc. of the International Joint Conf. on Artificial Intelligence (IJCAI),

  2. [10]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 20...

    2020

  3. [11]

    Wolff, Alex H

    Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric M. Wolff, Alex H. Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml- based planning benchmark for autonomous vehicles. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVP...

    2021

  4. [12]

    End- to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End- to-end object detection with transformers. InProc. of the European Conf. on Computer Vision (ECCV), 2020. 4, 16

    2020

  5. [13]

    Layout2scene: 3d semantic lay- out guided scene generation via geometry and appearance diffusion priors.arXiv.org, 2501.02519, 2025

    Minglin Chen, Longguang Wang, Sheng Ao, Ye Zhang, Kai Xu, and Yulan Guo. Layout2scene: 3d semantic lay- out guided scene generation via geometry and appearance diffusion priors.arXiv.org, 2501.02519, 2025. 2

    2025 arXiv

  6. [14]

    Sledge: Synthesizing driving environments with generative models and rule-based traffic

    Kashyap Chitta, Daniel Dauner, and Andreas Geiger. Sledge: Synthesizing driving environments with generative models and rule-based traffic. InProc. of the European Conf. on Computer Vision (ECCV), 2024. 3

    2024

  7. [15]

    Cholesky

    A. Cholesky. Sur la résolution numérique des systèmes d’équations linéaires.Bulletin Géodésique, 1924. 2, 3, 15

    1924

  8. [16]

    Cvxnet: Learnable convex decomposition

    Boyang Deng, Kyle Genova, Soroosh Yazdani, Sofien Bouaziz, Geoffrey Hinton, and Andrea Tagliasacchi. Cvxnet: Learnable convex decomposition. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020. 2

    2020

  9. [17]

    Citygen: Infinite and controllable 3d city layout generation.arXiv.org, 2312.01508, 2023

    Jie Deng, Wenhao Chai, Jianshu Guo, Qixuan Huang, Wen- hao Hu, Jenq-Neng Hwang, and Gaoang Wang. Citygen: Infinite and controllable 3d city layout generation.arXiv.org, 2312.01508, 2023. 3

    2023 arXiv

  10. [18]

    Graph-to-3d: End-to-end generation and manipulation of 3d scenes using scene graphs

    Helisa Dhamo, Fabian Manhardt, Nassir Navab, and Fed- erico Tombari. Graph-to-3d: End-to-end generation and manipulation of 3d scenes using scene graphs. InProc. of the IEEE International Conf. on Computer Vision (ICCV),

  11. [19]

    Diffusion mod- els beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion mod- els beat gans on image synthesis. InAdvances in Neural Information Processing Systems (NeurIPS), 2021. 6, 22

    2021

  12. [20]

    Scenescape: Text-driven consistent scene generation

    Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel. Scenescape: Text-driven consistent scene generation. InAd- vances in Neural Information Processing Systems (NeurIPS),

  13. [21]

    Learning shape templates with structured implicit functions

    Kyle Genova, Forrester Cole, Daniel Vlasic, Aaron Sarna, William T Freeman, and Thomas Funkhouser. Learning shape templates with structured implicit functions. InProc. of the IEEE International Conf. on Computer Vision (ICCV),

  14. [22]

    Efros, and Martial Hebert

    Abhinav Gupta, Alexei A. Efros, and Martial Hebert. Blocks world revisited: Image understanding using qualitative ge- ometry and mechanics. InProc. of the European Conf. on Computer Vision (ECCV), 2010. 2

    2010

  15. [23]

    King: Generat- ing safety-critical driving scenarios for robust imitation via kinematics gradients

    Niklas Hanselmann, Katrin Renz, Kashyap Chitta, Apra- tim Bhattacharyya, and Andreas Geiger. King: Generat- ing safety-critical driving scenarios for robust imitation via kinematics gradients. InProc. of the European Conf. on Computer Vision (ECCV), 2022. 8

    2022

  16. [24]

    Belongie

    Zekun Hao, Hadar Averbuch-Elor, Noah Snavely, and Serge J. Belongie. Dualsdf: Semantic shape manipulation using a two-level representation. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020. 2

    2020

  17. [25]

    Spaghetti: Editing implicit shapes through part aware generation.ACM Trans

    Amir Hertz, Or Perel, Raja Giryes, Olga Sorkine-Hornung, and Daniel Cohen-Or. Spaghetti: Editing implicit shapes through part aware generation.ACM Trans. on Graphics,

  18. [26]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bern- hard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. InAdvances in Neural Information Processing Systems (NeurIPS), 2017. 6

    2017

  19. [27]

    Classifier-free diffusion guidance.arXiv.org, 2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv.org, 2207.12598, 2022. 20

    2022 arXiv

  20. [28]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. InAdvances in Neural Informa- tion Processing Systems (NeurIPS), 2020. 5, 6, 18

    2020

  21. [29]

    Text2room: Extracting textured 3d meshes from 2d text-to-image models

    Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. InProc. of the IEEE International Conf. on Computer Vision (ICCV), 2023. 2

    2023

  22. [30]

    Scene4u: Hierarchical layered 3d scene reconstruction from single panoramic image for your immerse exploration

    Zilong Huang, Jun He, Junyan Ye, Lihan Jiang, Weijia Li, Yiping Chen, and Ting Han. Scene4u: Hierarchical layered 3d scene reconstruction from single panoramic image for your immerse exploration. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2025. 3

    2025

  23. [31]

    Scenediffuser: Efficient and controllable driving simulation initialization and rollout

    Max Jiang, Yijing Bai, Andre Cornman, Christopher Davis, Xiukun Huang, Hong Jeon, Sakshum Kulshrestha, John Lambert, Shuangyu Li, Xuanyu Zhou, et al. Scenediffuser: Efficient and controllable driving simulation initialization and rollout. InAdvances in Neural Information Proce...

    2024

  24. [32]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. InProc. IEEE Conf. on Computer Vision and Pattern Recog- nition (CVPR), 2019. 6

    2019

  25. [33]

    Alias-free generative adversarial networks

    Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. InAdvances in Neural Information Processing Systems (NeurIPS), 2021. 6

    2021

  26. [34]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding varia- tional bayes.Proc. of the International Conf. on Learning Representations (ICLR), 2014. 2, 17

    2014

  27. [35]

    Salad: Part-level latent diffusion for 3d shape genera- tion and manipulation

    Juil Koo, Seungwoo Yoo, Minh Hieu Nguyen, and Minhyuk Sung. Salad: Part-level latent diffusion for 3d shape genera- tion and manipulation. InProc. of the IEEE International Conf. on Computer Vision (ICCV), 2023. 2

    2023

  28. [36]

    The hungarian method for the assignment problem

    Harold W Kuhn. The hungarian method for the assignment problem. 1955. 4, 16

    1955

  29. [37]

    Improved precision and recall metric for assessing generative models

    Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. InAdvances in Neural Information Processing Systems (NeurIPS), 2019. 6, 22

    2019

  30. [38]

    Semcity: Semantic scene gen- eration with triplane diffusion

    Jumin Lee, Sebin Lee, Changho Jo, Woobin Im, Juhyeong Seon, and Sung-Eui Yoon. Semcity: Semantic scene gen- eration with triplane diffusion. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 5, 7, 20, 21, 24, 31, 39

    2024

  31. [39]

    Grass: Generative recursive autoencoders for shape structures.ACM Trans

    Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. Grass: Generative recursive autoencoders for shape structures.ACM Trans. on Graphics,

  32. [40]

    Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao (Richard) Zhang, and Leonidas J. Guibas. GRASS: generative recursive autoencoders for shape structures.ACM Trans. on Graphics, 36(4), 2017. 2

    2017

  33. [41]

    Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d.IEEE Trans

    Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d.IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 2022. 2, 5, 14, 24

    2022

  34. [42]

    Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior

    Chenguo Lin and Yadong Mu. Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior. InProc. of the International Conf. on Learning Representa- tions (ICLR), 2024. 3

    2024

  35. [43]

    Infinicity: Infinite-scale city synthesis

    Chieh Hubert Lin, Hsin-Ying Lee, Willi Menapace, Menglei Chai, Aliaksandr Siarohin, Ming-Hsuan Yang, and Sergey Tulyakov. Infinicity: Infinite-scale city synthesis. InProc. of the IEEE International Conf. on Computer Vision (ICCV),

  36. [44]

    Pyramid diffusion for fine 3d large scene generation

    Yuheng Liu, Xinke Li, Xueting Li, Lu Qi, Chongshou Li, and Ming-Hsuan Yang. Pyramid diffusion for fine 3d large scene generation. InProc. of the European Conf. on Com- puter Vision (ECCV), 2024. 2, 3, 5, 7, 19, 20, 21, 24, 33

    2024

  37. [45]

    Urban architect: Steerable 3d urban scene generation with layout prior.arXiv.org, 2404.06780, 2024

    Fan Lu, Kwan-Yee Lin, Yan Xu, Hongsheng Li, Guang Chen, and Changjun Jiang. Urban architect: Steerable 3d urban scene generation with layout prior.arXiv.org, 2404.06780, 2024. 2, 8, 22

    2024 arXiv

  38. [46]

    Scenecontrol: Diffusion for controllable traffic scene generation

    Jack Lu, Kelvin Wong, Chris Zhang, Simon Suo, and Raquel Urtasun. Scenecontrol: Diffusion for controllable traffic scene generation. InProc. IEEE International Conf. on Robotics and Automation (ICRA), 2024. 3, 24

    2024

  39. [47]

    Repaint: In- painting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: In- painting using denoising diffusion probabilistic models. In Proc. IEEE Conf. on Computer Vision and Pattern Recogni- tion (CVPR), 2022. 2, 4, 5, 18

    2022

  40. [48]

    Tenenbaum

    Andrew Luo, Zhoutong Zhang, Jiajun Wu, and Joshua B. Tenenbaum. End-to-end optimization of scene layout. In Proc. IEEE Conf. on Computer Vision and Pattern Recogni- tion (CVPR), 2020. 3

    2020

  41. [49]

    Lt3sd: Latent trees for 3d scene diffusion

    Quan Meng, Lei Li, Matthias Nießner, and Angela Dai. Lt3sd: Latent trees for 3d scene diffusion. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR),

  42. [50]

    Hdmapgen: A hierarchical graph gen- erative model of high definition maps

    Lu Mi, Hang Zhao, Charlie Nash, Xiaohan Jin, Jiyang Gao, Chen Sun, Cordelia Schmid, Nir Shavit, Yuning Chai, and Dragomir Anguelov. Hdmapgen: A hierarchical graph gen- erative model of high definition maps. InProceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2021

  43. [51]

    Structurenet: Hierarchi- cal graph networks for 3d shape generation

    Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, and Leonidas Guibas. Structurenet: Hierarchi- cal graph networks for 3d shape generation. InACM Trans. on Graphics, 2019. 2

    2019

  44. [52]

    Structurenet: Hierarchi- cal graph networks for 3d shape generation.ACM Trans

    Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, and Leonidas Guibas. Structurenet: Hierarchi- cal graph networks for 3d shape generation.ACM Trans. on Graphics, 2019. 2

    2019

  45. [53]

    Im2struct: Recovering 3d shape structure from a single RGB image

    Chengjie Niu, Jun Li, and Kai Xu. Im2struct: Recovering 3d shape structure from a single RGB image. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR),

  46. [54]

    Text2immersion: Generative immersive scene with 3d gaussians.arXiv.org, 2312.09242, 2023

    Hao Ouyang, Tiancheng Sun, Stephen Lombardi, and Kathryn Heal. Text2immersion: Generative immersive scene with 3d gaussians.arXiv.org, 2312.09242, 2023. 3

    2023 arXiv

  47. [55]

    Superquadrics revisited: Learning 3d shape parsing beyond cuboids

    Despoina Paschalidou, Ali Osman Ulusoy, and Andreas Geiger. Superquadrics revisited: Learning 3d shape parsing beyond cuboids. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 2

    2019

  48. [56]

    Learning unsupervised hierarchical part decomposition of 3d objects from a single rgb image

    Despoina Paschalidou, Luc van Gool, and Andreas Geiger. Learning unsupervised hierarchical part decomposition of 3d objects from a single rgb image. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020. 2

    2020

  49. [57]

    Atiss: Autoregres- sive transformers for indoor scene synthesis

    Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregres- sive transformers for indoor scene synthesis. InAdvances in Neural Information Processing Systems (NeurIPS), 2021. 3, 6

    2021

  50. [58]

    Neural parts: Learning expressive 3d shape abstractions with invertible neural networks

    Despoina Paschalidou, Angelos Katharopoulos, Andreas Geiger, and Sanja Fidler. Neural parts: Learning expressive 3d shape abstractions with invertible neural networks. In Proc. IEEE Conf. on Computer Vision and Pattern Recogni- tion (CVPR), 2021. 8

    2021

  51. [59]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProc. of the IEEE International Conf. on Computer Vision (ICCV), 2023. 5, 18, 22

    2023

  52. [60]

    Parts: Structured descriptions of shape

    Alex Pentland. Parts: Structured descriptions of shape. In Proc. of the Conf. on Artificial Intelligence (AAAI), 1986. 2

    1986

  53. [61]

    Film: Visual reasoning with a general conditioning layer

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Du- moulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. InProc. of the Conf. on Artificial Intelligence (AAAI), 2018. 5

    2018

  54. [62]

    Compositional 3d scene generation using locally conditioned diffusion

    Ryan Po and Gordon Wetzstein. Compositional 3d scene generation using locally conditioned diffusion. InProc. of the International Conf. on 3D Vision (3DV), 2024. 2

    2024

  55. [63]

    Accelerating 3d deep learning with pytorch3d.arXiv.org, 2007.08501, 2020

    Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d.arXiv.org, 2007.08501, 2020. 6

    2007 arXiv

  56. [64]

    Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies

    Xuanchi Ren, Jiahui Huang, Xiaohui Zeng, Ken Museth, Sanja Fidler, and Francis Williams. Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 5, 7, 20, 21, 24, 35

    2024

  57. [65]

    Gen3c: 3d-informed world-consistent video generation with precise camera con- trol

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world-consistent video generation with precise camera con- trol. InProc. IEEE Conf. on Computer Vision and Patter...

    2025

  58. [66]

    Fast and flexible indoor scene synthesis via deep convolutional generative models

    Daniel Ritchie, Kai Wang, and Yu-An Lin. Fast and flexible indoor scene synthesis via deep convolutional generative models. InProc. IEEE Conf. on Computer Vision and Pat- tern Recognition (CVPR), 2019. 3

    2019

  59. [67]

    PhD thesis, Massachusetts Institute of Technology, 1963

    Lawrence G Roberts.Machine perception of three- dimensional solids. PhD thesis, Massachusetts Institute of Technology, 1963. 2

    1963

  60. [68]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2022. 2, 3, 4, 5, 18

    2022

  61. [69]

    Scenario dreamer: Vec- torized latent diffusion for generating driving simulation environments.arXiv.org, 2503.22496, 2025

    Luke Rowe, Roger Girgis, Anthony Gosselin, Liam Paull, Christopher Pal, and Felix Heide. Scenario dreamer: Vec- torized latent diffusion for generating driving simulation environments.arXiv.org, 2503.22496, 2025. 3

    2025 arXiv

  62. [70]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. InAdvances in Neural Information Pro- cessing Systems (NeurIPS), 2016. 6

    2016

  63. [71]

    A recipe for generating 3d worlds from a single image

    Katja Schwarz, Denis Rozumny, Samuel Rota Bulo, Lorenzo Porzi, and Peter Kontschieder. A recipe for generating 3d worlds from a single image. InProc. of the IEEE Interna- tional Conf. on Computer Vision (ICCV), 2025. 3

    2025

  64. [72]

    Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion

    Jaidev Shriram, Alex Trevithick, Lingjie Liu, and Ravi Ra- mamoorthi. Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion. InProc. of the Interna- tional Conf. on 3D Vision (3DV), 2025. 3

    2025

  65. [73]

    Roomdreamer: Text-driven 3d indoor scene synthesis with coherent geome- try and texture.arXiv.org, 2305.11337, 2023

    Liangchen Song, Liangliang Cao, Hongyu Xu, Kai Kang, Feng Tang, Junsong, Yuan, and Yang Zhao. Roomdreamer: Text-driven 3d indoor scene synthesis with coherent geome- try and texture.arXiv.org, 2305.11337, 2023. 2

    2023 arXiv

  66. [74]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Et- tinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhan...

    2020

  67. [75]

    Drivescenegen: Generating diverse and realistic driving sce- narios from scratch.IEEE Robotics and Automation Letters (RA-L), 2024

    Shuo Sun, Zekai Gu, Tianchen Sun, Jiawei Sun, Chen- gran Yuan, Yuhang Han, Dongen Li, and Marcelo H Ang. Drivescenegen: Generating diverse and realistic driving sce- narios from scratch.IEEE Robotics and Automation Letters (RA-L), 2024. 3

    2024

  68. [76]

    Scenegen: Learning to generate realistic traffic scenes

    Shuhan Tan, Kelvin Wong, Shenlong Wang, Sivabalan Mani- vasagam, Mengye Ren, and Raquel Urtasun. Scenegen: Learning to generate realistic traffic scenes. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR),

  69. [77]

    Diffuscene: Denoising diffusion models for generative indoor scene synthesis

    Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Jus- tus Thies, and Matthias Nießner. Diffuscene: Denoising diffusion models for generative indoor scene synthesis. In Proc. IEEE Conf. on Computer Vision and Pattern Recogni- tion (CVPR), 2024. 3, 6

    2024

  70. [78]

    Learning shape abstractions by assembling volumetric primitives

    Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017. 2

    2017

  71. [79]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems (NeurIPS), pages 5998–6008, 2017. 4

    2017

  72. [80]

    Convex decomposition of indoor scenes

    Vaibhav Vavilala and David Forsyth. Convex decomposition of indoor scenes. InProc. of the IEEE International Conf. on Computer Vision (ICCV), 2023. 2

    2023

  73. [81]

    Blocks2world: Controlling real- istic scenes with editable primitives.arXiv.org, 2307.03847, 2023

    Vaibhav Vavilala, Seemandhar Jain, Rahul Vasanth, Anand Bhattad, and David Forsyth. Blocks2world: Controlling real- istic scenes with editable primitives.arXiv.org, 2307.03847, 2023

    2023 arXiv

  74. [82]

    Improved convex decompo- sition with ensembling and boolean primitives.arXiv.org, 2405.19569, 2024

    Vaibhav Vavilala, Florian Kluger, Seemandhar Jain, Bodo Rosenhahn, and David Forsyth. Improved convex decompo- sition with ensembling and boolean primitives.arXiv.org, 2405.19569, 2024. 2

    2024

  75. [83]

    Diffusers: State-of-the-art diffusion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, Steven Liu, William Berman, Yiyi Xu, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. 5

  76. [84]

    Sceneformer: Indoor scene generation with transformers

    Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In Proc. of the International Conf. on 3D Vision (3DV), 2021. 3

    2021

  77. [85]

    Lego-net: Learning regular rearrangements of ob- jects in rooms

    Qiuhong Anna Wei, Sijie Ding, Jeong Joon Park, Rahul Sajnani, Adrien Poulenard, Srinath Sridhar, and Leonidas Guibas. Lego-net: Learning regular rearrangements of ob- jects in rooms. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2023. 3

    2023

  78. [86]

    Argoverse 2: Next generation datasets for self-driving perception and forecasting

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lam- bert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Rat- nesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Argoverse 2: Next generation datasets for self-driving perception an...

    2021

  79. [87]

    Motionsc: Data set and network for real- time semantic mapping in dynamic environments.IEEE Robotics and Automation Letters (RA-L), 2022

    Joey Wilson, Jingyu Song, Yuewei Fu, Arthur Zhang, An- drew Capodieci, Paramsothy Jayakumar, Kira Barton, and Maani Ghaffari. Motionsc: Data set and network for real- time semantic mapping in dynamic environments.IEEE Robotics and Automation Letters (RA-L), 2022. 20

    2022

  80. [88]

    Blockfusion: Expandable 3d scene generation using latent tri-plane extrapolation

    Zhennan Wu, Yang Li, Han Yan, Taizhang Shang, Weixuan Sun, Senbo Wang, Ruikai Cui, Weizhe Liu, Hiroyuki Sato, Hongdong Li, and Pan Ji. Blockfusion: Expandable 3d scene generation using latent tri-plane extrapolation. In ACM Transactions on Graphics, 2024. 2

    2024

  81. [89]

    Gaussiancity: Generative gaussian splatting for unbounded 3d city generation.arXiv.org, 2406.06526, 2024

    Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, and Ziwei Liu. Gaussiancity: Generative gaussian splatting for unbounded 3d city generation.arXiv.org, 2406.06526, 2024. 3

    2024 arXiv

  82. [90]

    CityDreamer: Compositional generative model of un- bounded 3D cities

    Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, and Ziwei Liu. CityDreamer: Compositional generative model of un- bounded 3D cities. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  83. [91]

    CityDreamer4D: Compositional generative model of un- bounded 4D cities.arXiv.org, 2501.08983, 2025

    Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, and Ziwei Liu. CityDreamer4D: Compositional generative model of un- bounded 4D cities.arXiv.org, 2501.08983, 2025. 3

    2025 arXiv

  84. [92]

    Worldgen: Generate any 3d scene in seconds

    Ziyang Xie. Worldgen: Generate any 3d scene in seconds. https://github.com/ZiYang-xie/WorldGen, 2025. 3

    2025

  85. [93]

    Discoscene: Spatially disentangled generative radiance fields for controllable 3d- aware scene synthesis

    Yinghao Xu, Menglei Chai, Zifan Shi, Sida Peng, Ivan Sko- rokhodov, Aliaksandr Siarohin, Ceyuan Yang, Yujun Shen, Hsin-Ying Lee, Bolei Zhou, et al. Discoscene: Spatially disentangled generative radiance fields for controllable 3d- aware scene synthesis. InProc. IEEE Conf. on C...

    2023

  86. [94]

    Layer- pano3d: Layered 3d panorama for hyper-immersive scene generation.ACM Trans

    Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Yixuan Li, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. Layer- pano3d: Layered 3d panorama for hyper-immersive scene generation.ACM Trans. on Graphics, 2025. 3

    2025

  87. [95]

    Scenecraft: Layout-guided 3d scene generation.Advances in Neural Information Processing Systems (NeurIPS), 2024

    Xiuyu Yang, Yunze Man, Junkun Chen, and Yu-Xiong Wang. Scenecraft: Layout-guided 3d scene generation.Advances in Neural Information Processing Systems (NeurIPS), 2024. 2

    2024

  88. [96]

    Urbangiraffe: Representing urban scenes as compositional generative neural feature fields

    Yuanbo Yang, Yifei Yang, Hanlei Guo, Rong Xiong, Yue Wang, and Yiyi Liao. Urbangiraffe: Representing urban scenes as compositional generative neural feature fields. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2023. 2

    2023

  89. [97]

    Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene genera- tion

    Yuanbo Yang, Jiahao Shao, Xinyang Li, Yujun Shen, An- dreas Geiger, and Yiyi Liao. Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene genera- tion. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2025. 3

    2025

  90. [98]

    Wonderjourney: Going from anywhere to everywhere

    Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, et al. Wonderjourney: Going from anywhere to everywhere. InProc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024. 3

    2024

  91. [99]

    Trajec- torycrafter: Redirecting camera trajectory for monocular videos via diffusion models

    Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajec- torycrafter: Redirecting camera trajectory for monocular videos via diffusion models. InProc. of the IEEE Interna- tional Conf. on Computer Vision (ICCV), 2025

    2025

  92. [100]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv.org, 2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv.org, 2409.02048, 2024. 3

    2024 arXiv

  93. [101]

    Commonscenes: Generating commonsense 3d indoor scenes with scene graphs

    Guangyao Zhai, Evin Pinar Örnek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. Commonscenes: Generating commonsense 3d indoor scenes with scene graphs. InAdvances in Neural Information Pro- cessing Systems (NeurIPS), 2023. 3

    2023

  94. [102]

    Echoscene: Indoor scene generation via information echo over scene graph diffusion

    Guangyao Zhai, Evin Pınar Örnek, Dave Zhenyu Chen, Ruo- tong Liao, Yan Di, Nassir Navab, Federico Tombari, and Benjamin Busam. Echoscene: Indoor scene generation via information echo over scene graph diffusion. InProc. of the European Conf. on Computer Vision (ECCV), 2024. 3

    2024

  95. [103]

    Text2nerf: Text-driven 3d scene generation with neural radiance fields.arXiv.org, 2305.11588, 2023

    Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, and Jing Liao. Text2nerf: Text-driven 3d scene generation with neural radiance fields.arXiv.org, 2305.11588, 2023. 3

    2023 arXiv

  96. [104]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2023. 8, 22, 44

    2023

  97. [105]

    Sseditor: Controllable mask-to-scene generation with diffusion model.arXiv.org, 2411.12290, 2024

    Haowen Zheng and Yanyan Liang. Sseditor: Controllable mask-to-scene generation with diffusion model.arXiv.org, 2411.12290, 2024. 2, 3, 24

    2024 arXiv

  98. [106]

    Decoupled diffusion sparks adap- tive scene generation.arXiv.org, 2504.10485, 2025

    Yunsong Zhou, Naisheng Ye, William Ljungbergh, Tianyu Li, Jiazhi Yang, Zetong Yang, Hongzi Zhu, Christoffer Pe- tersson, and Hongyang Li. Decoupled diffusion sparks adap- tive scene generation.arXiv.org, 2504.10485, 2025. 3

    2025 arXiv

  99. [107]

    3d-prnn: Generating shape primitives with recurrent neural networks

    Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 3d-prnn: Generating shape primitives with recurrent neural networks. InProc. of the IEEE International Conf. on Computer Vision (ICCV), pages 900–909, 2017. 2 PrITTI: Primitive-based Generation of Controllabl...

    2017

  100. [1024]

    An additional linear layer predicts its probability of existence

    For every query, the head outputs a 9D vector en- coding the normalized 3D center location and 6D Cholesky parameters of its corresponding primitive. An additional linear layer predicts its probability of existence. B.2.2. Training The first-stage training loss consists of thr...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.