Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a single diffusion pass can generate an entire 3D scene with multiple, spatially coherent objects from one image.

desk verdict A novel and promising multi-instance diffusion formulation, but the missing coordinate-frame explanation for the frozen VAE makes the central mechanism unverifiable. read the letter →

arxiv 2412.03558 v3 pith:Z3WAOVIW submitted 2024-12-04 cs.CV

Zehuan Huang , Yuan-Chen Guo , Xingqiao An , Yunhan Yang , Yangguang Li , Zi-Xin Zou , Ding Liang , Xihui Liu
show 2 more authors
Yan-Pei Cao Lu Sheng
This is my paper · ORCID
classification cs.CV
keywords multi-instancediffusion3Dscenegenerationsingle-imagetransformerspatialrelationshipsimage-to-3Dattentionlayout
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that generating a full 3D scene from a single image can be done in one diffusion pass, rather than by segmenting the image, generating each object separately, and then optimizing their relative positions. The authors take a pre-trained image-to-3D object generation model and equip it with multi-instance attention, so the latent codes of all objects in the scene are denoised simultaneously and can see each other through the attention layers. Because inter-object interactions are modeled inside the generation process, the resulting instances share a global coordinate frame and preserve the spatial layout of the input. If correct, this would make single-image 3D scene generation faster and less error-prone than multi-stage pipelines, and it would carry the generalization of large object generators over to scenes. On synthetic and real benchmarks the method reports large gains, such as improving volume IoU of bounding boxes on 3D-Front from 0.392 (DiffCAD) to 0.518.

What carries the argument

The load-bearing mechanism is the multi-instance attention layer: a modification of the original object self-attention in which the feature tokens of instance $i$ no longer query only themselves but attend to the concatenated tokens $\{f^j\}_{j=1}^N$ of all instances. This lets the denoising network reason about cross-instance interactions in latent space. The other pieces are: concurrent denoising of $N$ latent codes with a shared noise level $t$ and a rectified-flow loss, and a composite conditioning image $y \in \mathbb{R}^{h \times w \times 7}$ that stacks each object's RGB crop, its mask, and the global scene image before a DINO-initialized ViT encoder. Together they let the model produce per-instance latents $\{z_0^i\}$ that can be decoded and directly combined into a scene.

What would settle it

Generate a scene with two visually identical objects, decode the raw per-instance meshes without any alignment step, and measure the volume IoU of their bounding boxes against the ground-truth layout. If the latent carries no usable position information, the two instances will collapse toward the same location or swap places, which would break the reported spatial-coherence claim.

Watch

Extended reading notes

Core claim

The central claim is that an image-to-3D object diffusion model, pre-trained on large single-object data, can be fine-tuned into a multi-instance diffusion model that generates all objects of a scene at once with correct spatial relationships. The key change is to convert a subset of the transformer self-attention layers into multi-instance attention, so that every token in every instance attends to the tokens of all other instances during denoising. At training time, the model is supervised with scene-level data (3D-Front) to learn inter-object interactions, while single-object data (Objaverse) is mixed in 30% of the time to preserve the pre-trained generalization. Conditioning is a 7-channel composite of the object's RGB image, its mask, and the global scene image, encoded by a ViT and injected through cross-attention. The authors report state-of-the-art results on 3D-Front, BlendSwap, Matterport3D, and ScanNet, and also on stylized images from text-to-image models, claiming that simultaneous multi-instance generation avoids the error accumulation of compositional pipelines.

Load-bearing premise

The whole argument rests on the assumption that a VAE trained on single objects can encode and decode several objects inside one shared 3D coordinate frame, so that the decoded instances sit at the right places in the scene without any post-hoc alignment; the paper does not specify where the positional information lives in the latent.

Editorial extensions

If this is right

  • Scene generation becomes a single feed-forward diffusion pass, cutting runtime to about 40 seconds per scene on an A100, compared with minutes for compositional baselines.
  • Inter-object spatial relationships are baked into the generation process, so the output layout matches the input image better than pipelines that optimize layouts after generating objects one by one.
  • The method inherits the generalization of pre-trained object generators, as shown by results on real-world scans and on stylized images from text-to-image models.
  • Fine-tuning with a small scene dataset is enough if single-object data is used as regularization, because the pre-trained object prior is largely retained.
  • The recipe is described as applicable to pre-trained image-to-3D object generation models generally, not to a single backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the model's latent has no explicit per-object position token, scaling to a large number of objects may degrade; a testable extension is to inject explicit layout or pose tokens into the multi-instance attention, which the paper does not explore.
  • The attention mechanism is order-agnostic, so permuting the instance order at inference should not change the output; if it does, the model has learned order-specific artifacts rather than true cross-instance geometry. This is a cheap consistency check the authors did not report.
  • The same simultaneous-denoising idea could transfer to dynamic scenes or human-object interaction, but the paper acknowledges its current training data only supports simple static interactions.
  • The reported IoU-B gains mix geometry quality and layout accuracy; separating the two, for instance by evaluating with ground-truth object shapes, would clarify how much of the improvement comes from the spatial-coherence mechanism versus the stronger object prior.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MIDI, a method for generating a 3D scene composed of multiple object instances from a single scene image in one diffusion pass. MIDI extends a pre-trained image-to-3D object diffusion model by denoising the latents of all instances simultaneously with a shared denoising network, introducing multi-instance attention layers so that each instance's tokens attend to tokens of all other instances, and conditioning on local object images, masks, and a global scene image. The model is fine-tuned on 3D-Front with mixed training on Objaverse, using a rectified-flow loss extended to multiple instances. The paper reports state-of-the-art quantitative results on 3D-Front and BlendSwap, qualitative results on real-world and stylized images, and an ablation study over the number of multi-instance attention layers, global conditioning, and mixed single-object training.

Significance. If the central claim holds, MIDI is a meaningful advance: it replaces multi-stage object-by-object scene generation with a single simultaneous multi-instance diffusion process, leveraging strong object priors and a novel cross-instance attention mechanism. The paper includes a clear ablation isolating the multi-instance attention contribution, reports runtime, and demonstrates qualitative generalization to real and stylized images. A notable strength is the explicit use of single-object data as regularization to preserve the base model's generalization. However, the method rests on an underspecified assumption about how the frozen object-centric VAE represents and decodes instances in a shared global coordinate frame, and the quantitative evaluation lacks uncertainty estimates and uses an unreleased base model. These issues are load-bearing for the central claim of spatially coherent simultaneous generation and for reproducibility, respectively.

major comments (4)
  1. [Sec. 4.1, Eq. (3); Sec. 10; Supp. Sec. 7] The paper never specifies how the pre-trained single-object VAE encodes and decodes multiple instances in a shared global scene coordinate system, even though this is essential to the claim that denoised latents can be 'decoded and directly combined' into a coherent scene. The loss in Eq. (3) requires ground-truth latents {z_i0} for scene instances, but the source and coordinate frame of these latents are undefined. If each z_i0 is a canonical object-centric latent, it carries shape but no pose, and the 'directly combined' statement in Sec. 4.1 is unsupported without a hidden layout step. If each z_i0 is in global scene coordinates, then the frozen VAE—trained on canonical normalized objects as described in Sec. 3 and Supp. Sec. 7—is out-of-distribution, because point positions at arbitrary scene translations are far from its training distribution. The limitation section (Sec. 10) confirms that generation happens in a normalized global space from -1 to 1, but it does not explain how the decoder maps a global-coordinate latent back to an SDF field at the correct scene position, nor how ground-truth global latents are produced for training. This gap is load-bearing: the central claim of simultaneous spatially coherent generation collapses if the latent representation cannot carry both shape and placement through the frozen VAE. The manuscript must specify the encoding of scene instances during training, the coordinate frame of the latents, and the decoding/placement procedure.
  2. [Table 1; Sec. 5.3] The quantitative comparison in Table 1 reports a single number per metric with no error bars, no variance over multiple runs or seeds, and no statistical significance tests. Given that the claimed improvements are large (e.g., IoU-B 0.518 vs. 0.392 on 3D-Front), the absence of any uncertainty estimate makes it impossible to judge whether the margin is stable. In addition, the real-world evaluation in Sec. 5.3 is qualitative only, using 10 selected scenes, so the paper's claim of 'strong generalization' to real images is not quantitatively supported. The authors should provide error bars or multiple-seed results for the main tables, and, if possible, quantitative metrics on at least a subset of the real-world data with a clear evaluation protocol.
  3. [Sec. 5.1; Sec. 4] The base model is described as 'our own image-to-3D object generation model' and is not released, which makes the experiments non-reproducible for other researchers. Since the paper's central contribution is a paradigm to 'extend pre-trained image-to-3D object generation models,' it would be substantially strengthened by demonstrating the method on at least one publicly available base model (e.g., CLAY, Michelangelo, or Direct3D) in addition to the authors' internal model. Without this, the reader cannot verify whether the reported gains come from the proposed multi-instance attention and training scheme or from properties of the unreleased base model.
  4. [Table 2] The ablation study in Table 2 is reported without variance or repeated runs, and is conducted only on 3D-Front. The conclusion that K=5 is optimal is based on a single configuration. Moreover, the K=0 row reports only scene-level metrics (CD-S and F-Score-S) and leaves object-level metrics as '–', because the model cannot generate separable multi-instances; this is acknowledged in the text, but it means the comparison between K=0 and K=5 is not on the same set of metrics. The authors should either provide a way to measure object-level quality for K=0 or acknowledge more clearly that the K=0 baseline is not directly comparable for those metrics.
minor comments (6)
  1. [Sec. 4.1] In Eq. (3), the expectation is written over {z_i}, y, {epsilon_i}, t, but y is defined only loosely as the composite image; clarify that y includes the global scene image, all local object images, and all masks, and make the dependence explicit in the notation.
  2. [Sec. 5.1] The sentence 'We set the resolution of y to 512' is ambiguous: specify whether this is 512×512 pixels and how the variable-length number of instances (N up to 7) is accommodated in the batching and conditioning.
  3. [Supp. Sec. 7, Eq. (4)] The supplementary material writes 'for t = 1, · · ·, T' for the rectified flow perturbation, while the main text (Sec. 4.3) correctly states t is sampled from 0 to 1. This inconsistency should be corrected, as it creates confusion about the time indexing.
  4. [Sec. 5.2] There is a typo: 'qualitiative' should be 'qualitative'.
  5. [Sec. 1] The sentence 'and then inferences the geometry in a new image' uses the noun 'inferences' as a verb; it should be 'infers the geometry from a new image'.
  6. [Sec. 5.4] The stylized-image evaluation compares MIDI exclusively with REPARO. This is understandable given the input type, but the paper should state explicitly that the comparison is limited and not claim parity with a full benchmark on this data.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MIDI's central results come from a trained multi-instance diffusion model evaluated on held-out data; the unreleased base model and K=5 selection are reproducibility and model-selection concerns, not circular reasoning.

full rationale

MIDI's derivation chain is empirical rather than definitional. The model is trained with the multi-instance rectified-flow loss in Eq. (3), which is a standard extension of the single-object loss in Eq. (5); the ground-truth latents {z_i^0} are dataset encodings, not outputs of the model being evaluated, and the central claims are tested on held-out 3D-Front images, BlendSwap, and real-world scans. The multi-instance attention in Eq. (1) is a proposed architectural mechanism whose contribution is validated by ablations, not asserted by construction. The only self-referential elements are an unreleased 'our own' base model (Sec. 5.1) and the choice K=5 from an ablation table, both of which are reproducibility and model-selection concerns rather than reductions of the prediction to its inputs. The Sec. 10 limitation that generation occurs in a normalized global coordinate space and may under-resolve small objects explicitly concedes a shortcoming but does not make the result equivalent to its inputs. No equation is shown to equal another by construction, and no fitted parameter is renamed as a prediction, so the paper does not exhibit circularity under the stated criteria.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method rests on several training and architecture assumptions that are standard in deep learning, plus one ad hoc assumption about the VAE's ability to represent global positions. The strongest burden is the unexplained coordinate composition of single-object VAE latents.

free parameters (3)
  • K (number of multi-instance attention layers) = 5
    Selected via ablation on the 3D-Front test set (Sec 5.5); the paper reports K=0 yields incoherent layouts and K=21 overfits, so K=5 was chosen by hand.
  • Classifier-free guidance scale = 7.0
    Chosen for inference (Sec 8) to balance fidelity and coherence; no sweep reported.
  • Single-object mixing probability = 0.3
    Used in training to regularize with Objaverse data (Sec 4.3); chosen by hand without sensitivity analysis.
assumptions (4)
  • domain assumption The private pre-trained image-to-3D generator built by the authors provides a strong, generalizable 3D prior.
    Central to the method; the paper says it is 'based on our own image-to-3D object generation model' (Sec 5.1), so the prior is not independently available.
  • ad hoc to paper The base VAE can encode and decode multiple instances in a shared scene coordinate system, allowing latents to be directly combined.
    Required for the central claim (Sec 4.1) but never justified or explained; the base VAE is trained for single objects.
  • domain assumption Multi-instance attention over all instance tokens is sufficient to learn spatial relationships.
    The attention module (Eq. 1) is the only mechanism for cross-instance interaction; the paper relies on it without analyzing what information is propagated.
  • domain assumption Grounded-SAM provides accurate object segmentation at inference.
    Used for inference-time segmentation (Sec 8); errors would propagate to generation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation." pith.science (2026). https://pith.science/paper/Z3WAOVIW

@misc{pith2026241203558,
  author       = {Pith},
  title        = {Pith review of: MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z3WAOVIW}},
  note         = {Machine review of arXiv:2412.03558}
}
read the original abstract

This paper introduces MIDI, a novel paradigm for compositional 3D scene generation from a single image. Unlike existing methods that rely on reconstruction or retrieval techniques or recent approaches that employ multi-stage object-by-object generation, MIDI extends pre-trained image-to-3D object generation models to multi-instance diffusion models, enabling the simultaneous generation of multiple 3D instances with accurate spatial relationships and high generalizability. At its core, MIDI incorporates a novel multi-instance attention mechanism, that effectively captures inter-object interactions and spatial coherence directly within the generation process, without the need for complex multi-step processes. The method utilizes partial object images and global scene context as inputs, directly modeling object completion during 3D generation. During training, we effectively supervise the interactions between 3D instances using a limited amount of scene-level data, while incorporating single-object data for regularization, thereby maintaining the pre-trained generalization ability. MIDI demonstrates state-of-the-art performance in image-to-scene generation, validated through evaluations on synthetic data, real-world scene data, and stylized scene images generated by text-to-image diffusion models.

Figures

Figures reproduced from arXiv: 2412.03558 by the authors.

Figure 1
Figure 1. MIDI generates compositional 3D scenes from a single image by extending pre-trained image-to-3D object generation models [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison between our scene generation pipeline [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Method overview. Based on 3D object generation models, MIDI denoises the latent representations of multiple 3D instances [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Multi-instance attention. We extend the original object [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons on synthetic datasets, including 3D-Front [ [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparisons on real-world data, including Matterport3D [ [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative ablation studies on the number of multi-instance attention layers [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparisons on stylized images that are gen [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Detailed comparison between existing compositional generation methods and our multi-instance diffusion. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Visualization results of textured 3D scene generation [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Failure cases. age as input. The visualization results are shown in [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers

    cs.CV 2025-06 conditional novelty 6.0 of 10

    PartCrafter generates several separable 3D part meshes at once from a single image by fine-tuning a pretrained 3D diffusion transformer with part identity tokens and local-global attention.

Reference graph

Works this paper leans on

86 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Neural rgb-d surface reconstruction

    Dejan Azinovi ´c, Ricardo Martin-Brualla, Dan B Goldman, Matthias Nießner, and Justus Thies. Neural rgb-d surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6290– 6301, 2022. 5, 6

    2022

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  3. [3]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017. 5, 6, 7

    arXiv 2017

  4. [4]

    Single-view 3d scene reconstruc- tion with high-fidelity shape and texture

    Yixin Chen, Junfeng Ni, Nan Jiang, Yaowei Zhang, Yixin Zhu, and Siyuan Huang. Single-view 3d scene reconstruc- tion with high-fidelity shape and texture. In 2024 Interna- tional Conference on 3D Vision (3DV) , pages 1456–1467. IEEE, 2024. 1, 2, 3, 5, 6

    2024

  5. [5]

    Comboverse: Compositional 3d as- sets creation using spatially-aware diffusion guidance

    Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, and Ziwei Liu. Comboverse: Compositional 3d as- sets creation using spatially-aware diffusion guidance. arXiv preprint arXiv:2403.12409, 2024. 2, 3

    arXiv 2024

  6. [6]

    Buol: A bottom-up framework with occupancy-aware lifting for panoptic 3d scene reconstruction from a single image

    Tao Chu, Pan Zhang, Qiong Liu, and Jiaqi Wang. Buol: A bottom-up framework with occupancy-aware lifting for panoptic 3d scene reconstruction from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 4937–4946, 2023. 1, 2, 3

    2023

  7. [7]

    Panoptic 3d scene reconstruction from a single rgb image

    Manuel Dahnert, Ji Hou, Matthias Nießner, and Angela Dai. Panoptic 3d scene reconstruction from a single rgb image. Advances in Neural Information Processing Systems , 34: 8282–8293, 2021. 1, 2, 3, 5, 6

    2021

  8. [8]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 2, 5, 6, 7

    2017

Show all 86 references
  1. [9]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  2. [10]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems, 36, 2024. 3

    2024

  3. [11]

    General- izable 3d scene reconstruction via divide and conquer from a single view

    Andreea Dogaru, Mert ¨Ozer, and Bernhard Egger. General- izable 3d scene reconstruction via divide and conquer from a single view. arXiv preprint arXiv:2404.03421, 2024. 2, 3, 5, 6

    2024 arXiv

  4. [12]

    Tela: Text to layer-wise 3d clothed human generation

    Junting Dong, Qi Fang, Zehuan Huang, Xudong Xu, Jingbo Wang, Sida Peng, and Bo Dai. Tela: Text to layer-wise 3d clothed human generation. InEuropean Conference on Com- puter Vision, pages 19–36. Springer, 2025. 3

    2025

  5. [13]

    Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans

    Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10786–10796, 2021. 3

    2021

  6. [14]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  7. [15]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 10933–10942,

  8. [16]

    3d-future: 3d fur- niture shape with texture

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- niture shape with texture. International Journal of Computer Vision, 129:3313–3337, 2021. 2

    2021

  9. [17]

    Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image

    Daoyi Gao, D ´avid Rozenberszki, Stefan Leutenegger, and Angela Dai. Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image. ACM Transactions on Graphics (TOG) , 43(4):1–15, 2024. 1, 2, 3, 5, 6

    2024

  10. [18]

    Learn- ing 3d object shape and layout without 3d supervision

    Georgia Gkioxari, Nikhila Ravi, and Justin Johnson. Learn- ing 3d object shape and layout without 3d supervision. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 1695–1704, 2022. 1, 2, 3

    2022

  11. [19]

    Roca: Ro- bust cad model retrieval and alignment from a single image

    Can G ¨umeli, Angela Dai, and Matthias Nießner. Roca: Ro- bust cad model retrieval and alignment from a single image. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4022–4031, 2022. 1, 2, 3

    2022

  12. [20]

    threestudio: A unified framework for 3d content generation, 2023

    Yuan-Chen Guo, Ying-Tian Liu, Ruizhi Shao, Christian Laforte, Vikram V oleti, Guan Luo, Chia-Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, et al. threestudio: A unified framework for 3d content generation, 2023. 2

    2023

  13. [21]

    Reparo: Compositional 3d assets generation with differentiable 3d layout alignment

    Haonan Han, Rui Yang, Huan Liao, Jiankai Xing, Zun- nan Xu, Xiaoming Yu, Junwei Zha, Xiu Li, and Wan- hua Li. Reparo: Compositional 3d assets generation with differentiable 3d layout alignment. arXiv preprint arXiv:2405.18525, 2024. 2, 3, 5, 6, 7

    2024 arXiv

  14. [22]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 1

    2022 arXiv

  15. [23]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  16. [24]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2, 3

    2023 arXiv

  17. [25]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 5

    2021 arXiv

  18. [26]

    Mv-adapter: Multi-view consistent image generation made easy

    Zehuan Huang, Yuan-Chen Guo, Haoran Wang, Ran Yi, Lizhuang Ma, Yan-Pei Cao, and Lu Sheng. Mv-adapter: Multi-view consistent image generation made easy. arXiv preprint arXiv:2412.03632, 2024. 3, 1, 2

    2024 arXiv

  19. [27]

    Epidiff: Enhancing multi-view synthesis via localized epipolar-constrained diffusion

    Zehuan Huang, Hao Wen, Junting Dong, Yaohui Wang, Yangguang Li, Xinyuan Chen, Yan-Pei Cao, Ding Liang, Yu Qiao, Bo Dai, et al. Epidiff: Enhancing multi-view synthesis via localized epipolar-constrained diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  20. [28]

    Hamid Izadinia, Qi Shan, and Steven M Seitz. Im2cad. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5134–5143, 2017. 1, 2, 3

    2017

  21. [29]

    Shap-e: Generat- ing conditional 3d implicit functions

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 2, 3

    2023 arXiv

  22. [30]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3

    2013 arXiv

  23. [31]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 3

    2023 arXiv

  24. [32]

    Mask2cad: 3d shape prediction by learning to segment and retrieve

    Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, and Angela Dai. Mask2cad: 3d shape prediction by learning to segment and retrieve. In Computer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Pro- ceedings, Part III 16, pages 260–277. Springer, 2020. 1, 2, 3

    2020

  25. [33]

    Patch2cad: Patchwise embedding learning for in-the- wild shape retrieval from a single image

    Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, and Angela Dai. Patch2cad: Patchwise embedding learning for in-the- wild shape retrieval from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 12589–12599, 2021

    2021

  26. [34]

    Sparc: Sparse render-and-compare for cad model alignment in a single rgb image

    Florian Langer, Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Sparc: Sparse render-and-compare for cad model alignment in a single rgb image. arXiv preprint arXiv:2210.01044, 2022. 1, 2, 3

    2022 arXiv

  27. [35]

    Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner

    Weiyu Li, Jiarui Liu, Rui Chen, Yixun Liang, Xuelin Chen, Ping Tan, and Xiaoxiao Long. Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner. arXiv preprint arXiv:2405.14979, 2024. 2, 3, 1

    2024 arXiv

  28. [36]

    Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models

    Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et al. Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models. arXiv preprint arXiv:2502.06608, 2025. 3

    2025 arXiv

  29. [37]

    Part123: part-aware 3d reconstruction from a single-view image

    Anran Liu, Cheng Lin, Yuan Liu, Xiaoxiao Long, Zhiyang Dou, Hao-Xiang Guo, Ping Luo, and Wenping Wang. Part123: part-aware 3d reconstruction from a single-view image. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024. 3

    2024

  30. [38]

    Towards high-fidelity single-view holistic reconstruction of indoor scenes

    Haolin Liu, Yujian Zheng, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Towards high-fidelity single-view holistic reconstruction of indoor scenes. In European Con- ference on Computer Vision, pages 429–446. Springer, 2022. 1, 2, 3, 5, 6

    2022

  31. [39]

    One-2-3-45++: Fast single im- age to 3d objects with consistent multi-view generation and 3d diffusion

    Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Ji- ayuan Gu, and Hao Su. One-2-3-45++: Fast single im- age to 3d objects with consistent multi-view generation and 3d diffusion. In Proceedings of the IEEE/CVF Conference on ...

    2024

  32. [40]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. Advances in Neural Information Processing Systems , 36, 2024. 2, 3

    2024

  33. [41]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 3

    2023 arXiv

  34. [42]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022. 5, 1

    2022 arXiv

  35. [43]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. arXiv preprint arXiv:2309.03453, 2023. 2, 3

    2023 arXiv

  36. [44]

    Wonder3d: Sin- gle image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  37. [45]

    Marching cubes: A high resolution 3d surface construction algorithm

    William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. InSem- inal graphics: pioneering efforts that shaped the field, pages 347–353. 1998. 4

    1998

  38. [46]

    Lt3sd: Latent trees for 3d scene diffusion

    Quan Meng, Lei Li, Matthias Nießner, and Angela Dai. Lt3sd: Latent trees for 3d scene diffusion. arXiv preprint arXiv:2409.08215, 2024. 3

    2024 arXiv

  39. [47]

    GLIDE: towards photorealis- tic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: towards photorealis- tic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning, I...

    2022

  40. [48]

    Total3dunderstanding: Joint lay- out, object pose and mesh reconstruction for indoor scenes from a single image

    Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. Total3dunderstanding: Joint lay- out, object pose and mesh reconstruction for indoor scenes from a single image. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recog...

    2020

  41. [49]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 4, 5

    2023 arXiv

  42. [50]

    Atiss: Autoregres- sive transformers for indoor scene synthesis

    Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregres- sive transformers for indoor scene synthesis. Advances in Neural Information Processing Systems , 34:12013–12026,

  43. [51]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  44. [52]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 2, 3, 5, 7

    2023 arXiv

  45. [53]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  46. [54]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 3

    2022 arXiv

  47. [55]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks,

  48. [56]

    L3dg: Latent 3d gaussian diffusion

    Barbara Roessle, Norman M ¨uller, Lorenzo Porzi, Samuel Rota Bul `o, Peter Kontschieder, Angela Dai, and Matthias Nießner. L3dg: Latent 3d gaussian diffusion. arXiv preprint arXiv:2410.13530, 2024. 3

    2024 arXiv

  49. [57]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3

    2022

  50. [58]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  51. [59]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 3

    2015

  52. [60]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3

    2010 arXiv

  53. [61]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 3

    2011 arXiv

  54. [62]

    The replica dataset: A digital replica of indoor spaces

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797,

    1906 arXiv

  55. [63]

    Diffuscene: Denoising diffu- sion models for generative indoor scene synthesis

    Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias Nießner. Diffuscene: Denoising diffu- sion models for generative indoor scene synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20507–20518, 2024. 2

    2024

  56. [64]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. In European Conference on Computer Vision, pages 1–18. Springer, 2025. 2, 3

    2025

  57. [65]

    Triposr: Fast 3d object reconstruction from a single image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. arXiv preprint arXiv:2403.02151, 2024

    2024 arXiv

  58. [66]

    Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. In European Conference on Computer...

    2025

  59. [67]

    Neus: Learning neural im- plicit surfaces by volume rendering for multi-view recon- struction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural im- plicit surfaces by volume rendering for multi-view recon- struction. Advances in Neural Information Processing Sys- tems, 34:27171–27183, 2021. 3

    2021

  60. [68]

    Crm: Single image to 3d textured mesh with convolutional reconstruction model

    Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xi- ang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convolutional reconstruction model. arXiv preprint arXiv:2403.05034, 2024. 2, 3

    2024 arXiv

  61. [69]

    Ouroboros3d: Image-to-3d gen- eration via 3d-aware recursive diffusion

    Hao Wen, Zehuan Huang, Yaohui Wang, Xinyuan Chen, Yu Qiao, and Lu Sheng. Ouroboros3d: Image-to-3d gen- eration via 3d-aware recursive diffusion. arXiv preprint arXiv:2406.03184, 2024. 3

    2024 arXiv

  62. [70]

    Unique3d: High-quality and efficient 3d mesh generation from a single image

    Kailu Wu, Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu, Yueqi Duan, and Kaisheng Ma. Unique3d: High-quality and efficient 3d mesh generation from a single image. arXiv preprint arXiv:2405.20343, 2024

    2024 arXiv

  63. [71]

    Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer

    Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer. arXiv preprint arXiv:2405.14832, 2024. 2, 3, 1

    2024 arXiv

  64. [72]

    Blockfusion: Expandable 3d scene gen- eration using latent tri-plane extrapolation

    Zhennan Wu, Yang Li, Han Yan, Taizhang Shang, Weixuan Sun, Senbo Wang, Ruikai Cui, Weizhe Liu, Hiroyuki Sato, Hongdong Li, et al. Blockfusion: Expandable 3d scene gen- eration using latent tri-plane extrapolation. ACM Transac- tions on Graphics (TOG), 43(4):1–17, 2024

    2024

  65. [73]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,

    arXiv

  66. [74]

    Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation. arXiv preprint arXiv:2403.14621, 2024. 3

    2024 arXiv

  67. [75]

    Hifi-123: Towards high-fidelity one image to 3d content gen- eration

    Wangbo Yu, Li Yuan, Yan-Pei Cao, Xiangjun Gao, Xiaoyu Li, Wenbo Hu, Long Quan, Ying Shan, and Yonghong Tian. Hifi-123: Towards high-fidelity one image to 3d content gen- eration. In European Conference on Computer Vision, pages 258–274. Springer, 2024. 2

    2024

  68. [76]

    3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions On Graphics (TOG), 42(4):1–16, 2023

    Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions On Graphics (TOG), 42(4):1–16, 2023. 1

    2023

  69. [77]

    Holistic 3d scene un- derstanding from a single image with implicit representation

    Cheng Zhang, Zhaopeng Cui, Yinda Zhang, Bing Zeng, Marc Pollefeys, and Shuaicheng Liu. Holistic 3d scene un- derstanding from a single image with implicit representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8833–8842, 202...

    2021

  70. [78]

    Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 2, 3, 5, 1

    2024

  71. [79]

    Uni-3d: A universal model for panoptic 3d scene reconstruc- tion

    Xiang Zhang, Zeyuan Chen, Fangyin Wei, and Zhuowen Tu. Uni-3d: A universal model for panoptic 3d scene reconstruc- tion. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 9256–9266, 2023. 1, 2, 3

    2023

  72. [80]

    Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation

    Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in Neural Information Processing Systems, 36, 2024. 2, 3, 5, 1

    2024

  73. [81]

    Zero-shot scene reconstruction from single images with deep prior as- sembly

    Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Zero-shot scene reconstruction from single images with deep prior as- sembly. In Advances in Neural Information Processing Sys- tems (NeurIPS), 2024. 2, 3, 5

    2024

  74. [82]

    Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers

    Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  75. [83]

    Following scalable 3D object generation methods [35, 71, 78, 80], we firstly trains a V AE to com- press 3D geometric representations into a low-dimensional latent space

    Background Base model. Following scalable 3D object generation methods [35, 71, 78, 80], we firstly trains a V AE to com- press 3D geometric representations into a low-dimensional latent space. Specifically, x ∈ RL×6, which represents positions and normals of L points, are map...

  76. [84]

    we trained MIDI to simultaneously generate up to N = 7instances

    Implementation Details Training. we trained MIDI to simultaneously generate up to N = 7instances. We selected this value based on an analysis of the 3D-FRONT dataset [15], where we observed that scenes containing five or fewer objects constitute the majority, while scenes with...

  77. [85]

    compositional generation methods

    Additional Discussions MIDI vs. compositional generation methods. As show in Fig. 9, existing compositional generation methods in- volve a multi-step process, generating 3D objects one by one and then optimizing their spatial relationships. How- ever, this type of methods lack...

  78. [86]

    Limitations We present two typical failure examples of MIDI in Fig. 11. While MIDI generates 3D instances within the global scene coordinate system—specifically, a normalized space rang- ing from −1 to 1—this approach causes smaller objects to occupy a relatively minor portion...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.