Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Text-to-3D without 3D data or test-time optimization

desk verdict Good two-stage recipe, over-claimed novelty: the 'first' claim is unverified against cited work, but the method itself is clear and worth refereeing. read the letter →

arxiv 2412.16717 v1 pith:MSCAXDHR submitted 2024-12-21 cs.CV cs.LG

Souhaib Attaiki , Paul Guerrero , Duygu Ceylan , Niloy J. Mitra , Maks Ovsjanikov This is my paper · ORCID
classification cs.CVcs.LG
keywords text-to-3Dgenerationfeed-forwardtriplanerepresentationdiffusionmodelsGANdistillation2D-onlysupervisionhumanavatarsclassifier-freeguidance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a text-to-3D generator can be trained with nothing but single-view 2D images and still produce high-fidelity, prompt-following 3D humans in a single feed-forward pass. The recipe is to split the job: a GAN learns to generate triplane features of 3D objects from 2D image collections, and a diffusion model then learns to sample those triplanes conditioned on text. This matters because 3D supervision is scarce and test-time optimization is slow; the paper's combination claims both the quality of 2D-supervised GANs and the controllability of diffusion models. The authors report the first such feed-forward text-conditioned 3D generator, and identify the accuracy of automatic captions as the main factor limiting diversity.

What carries the argument

The load-bearing mechanism is 'diffusion in GAN space': the triplane, a set of three orthogonal 2D feature grids that decode through an MLP into a neural field of density and color, serves as the shared substrate. The GAN (AG3D or EG3D) maps noise to triplanes under adversarial supervision from rendered single-view images, and the diffusion model, a StableDiffusion-style UNet with classifier-free guidance, learns to denoise normalized triplane tensors conditioned on text. Diffusion operates on triplanes, not images, so conditioning enters through the modern diffusion formalism while the GAN supplies the 2D-supervised geometry.

What would settle it

Retrain the second stage on the same GAN-generated triplanes but with human-verified captions in place of BLIP captions, then compare CLIP similarity and per-attribute prompt agreement on a held-out prompt set; if neither improves, caption accuracy is not the limiting factor. A simpler check is to hand-label a random sample of rendered triplanes and measure BLIP's per-attribute accuracy directly.

Watch

Extended reading notes

Core claim

The central claim is that a text-conditioned 3D generator can be trained using only single-view 2D data by splitting the problem in two: first an unconditional GAN is trained to produce triplane representations of 3D objects from 2D image collections, then a diffusion model is trained directly on GAN-generated triplanes paired with automatically generated text captions, thereby distilling the GAN's unconditional distribution into a text-conditionable one. The resulting model generates 3D geometry by denoising a random triplane under a prompt, with no per-prompt optimization and no explicit 3D supervision, which the authors position as a first for feed-forward text-to-3D generation.

Load-bearing premise

The load-bearing premise is that the BLIP-based captions attached to GAN-rendered triplanes are accurate enough for the diffusion model to learn faithful text-to-3D associations; the paper's own FFHQ numbers show these labels visibly narrow the learned variety.

Editorial extensions

If this is right

  • Text-to-3D for humans, faces, and cats can be trained from image collections alone, sidestepping the scarcity of 3D datasets.
  • Inference is a single denoising pass with no Score Distillation Sampling, making text-to-3D generation far cheaper per prompt and easier to scale.
  • Distilling GAN samples into a diffusion model removes low-quality outliers from the GAN distribution, improving FID over the stage-1 generator when no upsampler is used.
  • Any category-specific 3D-aware GAN trained with 2D images can in principle be upgraded to text conditioning through this pipeline, as demonstrated with both AG3D and EG3D backbones.
  • Prompt adherence can be tuned at inference time by adjusting the classifier-free guidance scale without retraining the model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The pipeline's ceiling is set by the stage-1 GAN: any geometric error or mode collapse in the GAN is inherited by the diffusion model, so progress in 2D-supervised 3D GANs should transfer directly to this style of generator.
  • Caption quality is the likely scaling bottleneck: the paper's FFHQ result (FID 49.4 vs EG3D's 26.7) is attributed to the limited VQA label set, suggesting that a stronger captioner or prompts sourced from text-to-image models could close much of that gap.
  • A testable extension would replace the GAN with any 2D-supervised generator of a different 3D latent space (NeRF weights, point-cloud latents) to determine whether the triplane substrate is essential or incidental.
  • The synthetic 2D dataset built from SMPL depth and pose renders plus a pretrained text-to-image model offers a cheap recipe for expanding 2D supervision to other categories, provided the captioner stays accurate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GANFusion, a two-stage method for text-conditioned 3D generation from single-view 2D images. In the first stage, a 3D-aware GAN (AG3D or EG3D) is trained on unposed 2D image collections to produce unconditional triplane representations of 3D objects. In the second stage, the trained GAN is used to sample a large number of triplanes, which are rendered and captioned automatically with a BLIP-based VQA pipeline; a StableDiffusion-style denoiser is then trained on the resulting (caption, triplane) pairs. At inference, a text prompt is mapped to a triplane by DDIM sampling with classifier-free guidance, and the result is decoded and rendered with the GAN's renderer and upsampler. The method is evaluated on a synthetic human dataset, as well as FFHQ, AFHQ, and DeepFashion, with FID and CLIP similarity as quantitative metrics. The paper's central claims are that GANFusion is the first feed-forward text-to-3D diffusion generator trained with only single-view 2D supervision and that it combines GAN-level generation quality with text-conditioning ability while avoiding test-time optimization.

Significance. The two-stage design—distilling an unconditional 3D GAN into a text-conditioned diffusion model—is a practical and plausible approach to adding conditioning to 3D GANs without per-prompt optimization. The paper demonstrates architectural generality by using two GAN backbones (AG3D and EG3D) and four datasets, and it includes extensive qualitative comparisons with RenderDiffusion and text-conditioned AG3D variants. If the novelty claim can be substantiated and the evaluation issues are addressed, the method could be a useful building block for text-to-3D generation. However, the current manuscript does not fully support its headline claims: the evidence for 'first' is a single unverified sentence about reference [20], the captioning loop and restricted VQA answer set limit the validity of the prompt-adherence evaluation, and the FID computation is described ambiguously. These issues are load-bearing for the paper's main contributions.

major comments (4)
  1. [Section 2 (Text-to-3D generation), Abstract and Section 1] The central novelty claim that GANFusion is 'the first method that successfully trains a feed-forward text-to-3D diffusion-generator using only single-view 2D data' rests on a single sentence in Related Work stating that reference [20] 'requires test-time optimization for text conditioning.' The manuscript gives no detailed description of [20]'s inference procedure, no architectural comparison, and no experiment to substantiate this characterization. Given that the title of [20] is 'Learning Controllable 3D Diffusion Models from Single-view Images,' the claimed distinction is not self-evident and a reader cannot verify it. Please provide concrete evidence about [20]'s inference protocol, or revise the novelty claim to a comparative statement that is actually established by the paper.
  2. [Section 4.3 and Appendix A] The text-to-3D training data are generated by a closed loop: the GAN produces triplanes, BLIP-VQA captions them from a fixed answer set, and the diffusion model learns to map those captions back to triplanes. This makes the text-conditioning signal self-referential and bounded by both BLIP's labeling accuracy and the predefined answer lists. The paper concedes in Appendix B that the VQA labels restrict variety and are responsible for the FFHQ FID gap (49.4 vs. 26.7 for EG3D), but it does not quantify caption accuracy or evaluate prompt adherence on human-authored prompts that are independent of the procedural template. Because the CLIP-similarity metric in Table 1 is computed on prompts drawn from the same template distribution as the captions, the reported prompt-adherence numbers may overstate the model's true text understanding. Please report BLIP labeling accuracy, add an evaluation with held-out or human-written prompts, and provide a human study or per-attribute accuracy numbers.
  3. [Section 5, Metrics paragraph and Table 1] The FID definition is ambiguous and appears nonstandard: the text says 'treating our generated image dataset as the real distribution,' which inverts the usual reference and generated sets. This is a load-bearing issue because the main quality claim—GANFusion's FID of 68.8 versus RenderDiffusion's 135.7—depends on the FID being computed correctly. Please clarify which distribution is the reference, specify the number of samples and the InceptionV3 feature layer, and report standard deviations over multiple runs or test-set splits. Without such details, the FID comparisons in Table 1 cannot be quantitatively interpreted.
  4. [Section 5, first paragraph; Abstract] The synthetic human dataset is generated by conditioning Stable Diffusion on SMPL-derived depth and pose maps, which are rendered from a deformable 3D human template. While the GAN and diffusion training themselves use only 2D images, the main quantitative evaluation therefore relies on data that indirectly encode a 3D human prior. The claim of using 'only single-view 2D data for supervision' should be qualified to refer to the training objective rather than the data-generation pipeline, and the paper should discuss whether the FFHQ/AFHQ experiments are intended as evidence that the method works without such a 3D prior. The current phrasing is likely to overstate the 2D-only nature of the approach.
minor comments (5)
  1. [Section 5, Metrics paragraph] The name 'Frechet' should be spelled 'Fréchet' throughout the paper.
  2. [Table 1] The '×' entries for unconditional AG3D in the CLIP-similarity row are unexplained; please either report the CLIP similarity of unconditional samples to the test prompts or state explicitly in the caption why the metric is not applicable.
  3. [Eq. (5) and Section 4.3] The noise schedule α_t is not defined; the reference to the Sigmoid schedule [29] is insufficient for reproduction. Please provide the schedule formula or a precise reference, and specify the number of diffusion timesteps T and the DDIM steps used at inference.
  4. [Figures 3, 6, and 9] The baseline names 'AG3DC+text' and 'AG3DC+textPrompt' are used interchangeably across the text and figure captions; please reconcile the naming for consistency.
  5. [Appendix A] The answer lists contain minor typos and inconsistencies (e.g., 'Jean' vs. 'jean', 'pant' vs. 'pants', and inconsistently capitalized footwear entries); these should be cleaned to make the captioning process reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the two-stage GAN-to-diffusion distillation is self-contained, and the BLIP pseudo-labeling loop is an explicit design choice rather than a hidden fit.

full rationale

The derivation chain is: (1) train an unconditional 3D-aware GAN on single-view 2D images with an adversarial loss; (2) sample triplanes from the GAN, caption their renders with BLIP, and train a text-conditioned diffusion model with an L2 denoising loss on (caption, triplane) pairs; (3) at inference, denoise in triplane space and render with the stage-1 renderer/upsampler. Each stage has an independent training objective, and the evaluation metrics (FID, CLIP similarity) are external to those objectives; no parameter is fitted to the test set and no quantity is renamed as a prediction. The BLIP-captioning loop is openly described as 'effectively distilling the unconditional distribution learned by the GAN into a text-conditioned version,' so the fact that the text-conditioned output distribution is confined to the GAN's support is an acknowledged design property, also stated as a limitation in Sec. 6 ('any error in training the GAN will be inherited by the denoising diffusion network'). This is not a circular reduction of the central claim. The self-citations, including RenderDiffusion [3], Plato's Cave [23], 3D-LDM [49], and HoloDiffusion [33], are prior baselines or architectural components and are not used to justify the central novelty. The 'first' claim rests on a literature assertion about [20] ('requires test-time optimization for text conditioning') that is not experimentally verified, but that is a novelty and correctness risk, not a circularity under the definitions of this pass.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper does not introduce new physical entities. Its free parameters are hand-chosen engineering constants (guidance scale, normalization, viewpoint, caption vocabularies). The main load-bearing assumptions are the captioning quality and the suitability of the GAN triplane distribution for direct diffusion learning.

free parameters (5)
  • guidance scale = 7.5
    Used for classifier-free guidance at inference (Sec. 4.3); chosen by hand, standard value from Stable Diffusion.
  • triplane scaling factor = 1/16 with clamp to [-1,1]
    Triplane normalization preprocessing (Sec. 4.3); chosen by hand to remove outliers.
  • viewpoint parameters = elevation 40 degrees, distance 2.34, random azimuth
    Used for dataset generation and evaluation (Sec. 5); chosen by hand.
  • captioning answer sets = 28 top types, 12 bottom types, 11 footwear types, 23 colors
    Hand-defined VQA answer lists (Appendix A); restrict the text-conditioning vocabulary and diversity.
  • dataset sizes = 300k images, 49.5k triplanes, 1k test prompts
    Chosen by the authors; the 49.5k triplane dataset is small for diffusion training and affects quality.
assumptions (5)
  • standard math Volumetric rendering equation (Eq. 2) with pseudo-SDF to density conversion via sigmoid is a valid differentiable renderer for triplane features.
    Adopted from EG3D/AG3D; standard in the field.
  • domain assumption The triplane feature distribution learned by the GAN can be approximated by a diffusion model trained directly in the normalized triplane space, without a separate autoencoder.
    Sec. 4.3 removes the StableDiffusion encoder and denoises triplanes directly; this assumes the feature distribution is amenable to Gaussian diffusion.
  • domain assumption BLIP VQA answers to a fixed question set provide ground-truth text labels for GAN-rendered images.
    Sec. 4.3 and Appendix A rely on these captions as the only text supervision for the diffusion model; the paper acknowledges limited label variety.
  • domain assumption Stable Diffusion 1.5 conditioned on SMPL depth/pose maps yields a diverse and artifact-free 2D dataset for human generation.
    Sec. 5; the paper notes SD does not always follow prompts, justifying a caption-refinement pass.
  • domain assumption Pre-trained CLIP embeddings used for evaluation capture meaningful text-image alignment for the synthetic distribution.
    Sec. 5 Metrics; CLIP similarity is the only text-adherence metric and has known biases.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space." pith.science (2026). https://pith.science/paper/MSCAXDHR

@misc{pith2026241216717,
  author       = {Pith},
  title        = {Pith review of: GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MSCAXDHR}},
  note         = {Machine review of arXiv:2412.16717}
}
read the original abstract

We train a feed-forward text-to-3D diffusion generator for human characters using only single-view 2D data for supervision. Existing 3D generative models cannot yet match the fidelity of image or video generative models. State-of-the-art 3D generators are either trained with explicit 3D supervision and are thus limited by the volume and diversity of existing 3D data. Meanwhile, generators that can be trained with only 2D data as supervision typically produce coarser results, cannot be text-conditioned, or must revert to test-time optimization. We observe that GAN- and diffusion-based generators have complementary qualities: GANs can be trained efficiently with 2D supervision to produce high-quality 3D objects but are hard to condition on text. In contrast, denoising diffusion models can be conditioned efficiently but tend to be hard to train with only 2D supervision. We introduce GANFusion, which starts by generating unconditional triplane features for 3D data using a GAN architecture trained with only single-view 2D data. We then generate random samples from the GAN, caption them, and train a text-conditioned diffusion model that directly learns to sample from the space of good triplane features that can be decoded into 3D objects.

Figures

Figures reproduced from arXiv: 2412.16717 by the authors.

Figure 1
Figure 1. We propose GANFusion, a text-guided feed-forward 3D generator that is trained with only single-view image supervision. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Method Overview. We train a feed-forward text-to-3D diffusion model in two stages. First, we train an unconditional GAN￾based 3D object generator like AG3D [16] with a single-view image dataset. 3D objects are represented as triplanes and rendered with a renderer followed by an upsampler. We generate a large set of triplanes and caption them using BLIP [38]. The resulting (triplane, caption) dataset is then used to … view at source ↗
Figure 3
Figure 3. Qualitative comparison. We provide visual results from our method as well as the baselines. We provide random generations from AG3D [16] which is unconditional. While random samples are of high quality, it is not straightforward to extend AG3D to enable text conditioning even with the addition of CLIP based losses during training. RenderDiffusion [3] can follow the text prompts but lacks in terms of quality. GANFusi… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Qualitative results on the FFHQ dataset [9]. We replace AG3D [16] with EG3D [9] as the generator in our first stage to effectively enable text-conditioning on real-world face data. Unconditional Text Conditioned AG3D (low) AG3D (high) AG3D+text AG3DC+text RenderDiff. G…
Figure 5
Figure 5. Figure 5: Dataset examples. We show a few images and corresponding prompts from our single-view image dataset. Prompts that were initially used to generate the images are often not accurate, we refine them with our BLIP-based captioning approach. “a photo of a man wearing a pink…
Figure 6
Figure 6. Figure 6: Qualitative comparison without upsampling. We provide visual results from our method as well as the baselines. We disable the 2D upsampler for AG3D and our method and provide renderings of the triplane features directly for all methods. and our method, both still signi…
Figure 7
Figure 7. Figure 7: Qualitative results on the AFHQ dataset [9]. We replace AG3D [16] with EG3D [9] as the generator in our first stage to effectively enable text-conditioning on real-world cat data. a full-body photo of a woman wearing a orange tank top, black jean, white sneakers a full…
Figure 8
Figure 8. Figure 8: Qualitative results on the DeepFashion dataset [44]. We use AG3D [16] as the generator in our first stage to effectively enable text-conditioning on realistic 3D human figures [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison. We provide visual results from our method as well as the baselines. Prompt AG3D + text “a photo of a person wearing a red sweater, blue jean, pink socks” Prompt AG3D + text “a full-body photo of a person wearing a blue jacket, gray pant, black b…
Figure 10
Figure 10. Figure 10: Text-conditional AG3D. We attempt to add text conditioning to AG3D [16] by providing text embeddings as additional input both to the generator and the discriminator. However, we find that the training is not stable and does not converge. “a full-body photo of a woman …
Figure 11
Figure 11. Figure 11: Top & Bottom left: Diverse generations produced by our model using the same prompt but different seeds. Bottom right: Images showcasing our model’s ability to generalize to some out-of-distribution prompts not encountered during training [PITH_FULL_IMAGE:figures/full…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Hallo4D uses vision-language models to detect and correct spatial and temporal mistakes in AI-generated 3D and 4D content, improving consistency without retraining the base generators.

Reference graph

Works this paper leans on

75 extracted references · 60 canonical work pages · cited by 1 Pith paper

  1. [20]

    Learning controllable 3d diffusion models from single-view images

    Jiatao Gu, Qingzhe Gao, Shuangfei Zhai, Baoquan Chen, Lingjie Liu, and Josh Susskind. Learning controllable 3d diffusion models from single-view images. arXiv preprint arXiv:2304.06700, 2023. 3

    arXiv 2023

  2. [1]

    Learning representations and generative models for 3d point clouds

    Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. In ICML, 2018. 2

    work page 2018

  3. [2]

    Controlled and conditional text to image generation with diffusion prior, 2023

    Pranav Aggarwal, Hareesh Ravi, Naveen Marri, Sachin Kelkar, Fengbin Chen, Vinh Khuc, Midhun Harikumar, Ri- tiz Tambi, Sudharshan Reddy Kakumanu, Purvak Lapsiya, Alvin Ghouas, Sarah Saber, Malavika Ramprasad, Baldo Faieta, and Ajinkya Kale. Controlled and conditional text to image generation with diffusion prior, 2023. 8

    work page 2023

  4. [3]

    Ren- derdiffusion: Image diffusion for 3d reconstruction, inpaint- ing and generation

    Titas Anciukevi ˇcius, Zexiang Xu, Matthew Fisher, Paul Hen- derson, Hakan Bilen, Niloy J Mitra, and Paul Guerrero. Ren- derdiffusion: Image diffusion for 3d reconstruction, inpaint- ing and generation. In CVPR, pages 12608–12618, 2023. 1, 2, 3, 4, 5, 7, 8

    work page 2023

  5. [4]

    Gaudi: A neural architect for immersive 3d scene generation

    Miguel Angel Bautista, Pengsheng Guo, Samira Abnar, Wal- ter Talbott, Alexander Toshev, Zhuoyuan Chen, Laurent Dinh, Shuangfei Zhai, Hanlin Goh, Daniel Ulbricht, Afshin Dehghan, and Josh Susskind. Gaudi: A neural architect for immersive 3d scene generation. In NeurIPS, 2022. 2

    work page 2022

  6. [5]

    Learning gradient fields for shape generation

    Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariha- ran. Learning gradient fields for shape generation. In ECCV,

    work page

  7. [6]

    HuMMan: Multi-modal 4d human dataset for versatile sensing and modeling

    Zhongang Cai, Daxuan Ren, Ailing Zeng, Zhengyu Lin, Tao Yu, Wenjia Wang, Xiangyu Fan, Yang Gao, Yifan Yu, Liang Pan, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, and Ziwei Liu. HuMMan: Multi-modal 4d human dataset for versatile sensing and modeling. In ECCV, pages 557–577. Springer, 2022. 3

    work page 2022

  8. [7]

    Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models

    Yukang Cao, Yan-Pei Cao, Kai Han, Ying Shan, and Kwan- Yee K Wong. Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models. arXiv preprint arXiv:2304.00916, 2023. 3

    arXiv 2023

Show all 75 references
  1. [8]

    pi-gan: Periodic implicit generative ad- versarial networks for 3d-aware image synthesis

    Eric Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, and Gordon Wetzstein. pi-gan: Periodic implicit generative ad- versarial networks for 3d-aware image synthesis. In CVPR,

  2. [9]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF Conference on Computer V...

    2022

  3. [10]

    Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. arXiv preprint arXiv:2303.13873, 2023. 3

    2023 arXiv

  4. [11]

    gdna: Towards generative detailed neural avatars

    Xu Chen, Tianjian Jiang, Jie Song, Jinlong Yang, Michael J Black, Andreas Geiger, and Otmar Hilliges. gdna: Towards generative detailed neural avatars. In CVPR, 2022. 3

    2022

  5. [12]

    Dna- rendering: A diverse neural actor repository for high-fidelity human-centric rendering

    Wei Cheng, Ruixiang Chen, Wanqi Yin, Siming Fan, Keyu Chen, Honglin He, Huiwen Luo, Zhongang Cai, Jingbo Wang, Yang Gao, Zhengming Yu, Zhengyu Lin, Daxuan Ren, Lei Yang, Ziwei Liu, Chen Change Loy, Chen Qian, Wayne Wu, Dahua Lin, Bo Dai, and Kwan-Yee Lin. Dna- rendering: A div...

    arXiv

  6. [13]

    Diffusion-sdf: Conditional generative modeling of signed distance func- tions

    Gene Chou, Yuval Bahat, and Felix Heide. Diffusion-sdf: Conditional generative modeling of signed distance func- tions. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 2262–2272, 2023. 3

    2023

  7. [14]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In CVPR, pages 13142– 13153, 2023. 2

    2023

  8. [15]

    Gram: Generative radiance manifolds for 3d-aware image generation

    Yu Deng, Jiaolong Yang, Jianfeng Xiang, and Xin Tong. Gram: Generative radiance manifolds for 3d-aware image generation. In CVPR, 2022. 2

    2022

  9. [16]

    AG3D: Learning to gen- erate 3D avatars from 2D image collections

    Zijian Dong, Xu Chen, Jinlong Yang, Michael J.Black, Ot- mar Hilliges, and Andreas Geiger. AG3D: Learning to gen- erate 3D avatars from 2D image collections. In ICCV, 2023. 1, 2, 3, 4, 5, 7, 8, 12, 14, 15

    2023

  10. [17]

    Hyperdiffusion: Generating implicit neural fields with weight-space diffusion

    Ziya Erkoc ¸, Fangchang Ma, Qi Shan, Matthias Nießner, and Angela Dai. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. In ICCV, 2023. 1, 2, 3

    2023

  11. [18]

    Get3d: A generative model of high quality 3d tex- tured shapes learned from images

    Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. Get3d: A generative model of high quality 3d tex- tured shapes learned from images. In NeurIPS, 2022. 2

    2022

  12. [19]

    Textdeformer: Geometry manipu- lation using text guidance

    William Gao, Noam Aigerman, Groueix Thibault, Vladimir Kim, and Rana Hanocka. Textdeformer: Geometry manipu- lation using text guidance. InACM Transactions on Graphics (SIGGRAPH), 2023. 3

    2023

  13. [21]

    Stylenerf: A style-based 3d aware generator for high- resolution image synthesis

    Jiatao Gu, Lingjie Liu, Peng Wang, and Christian Theobalt. Stylenerf: A style-based 3d aware generator for high- resolution image synthesis. In ICLR, 2022. 2

    2022

  14. [22]

    3dgen: Triplane latent diffusion for textured mesh generation

    Anchit Gupta, Wenhan Xiong, Yixin Nie, Ian Jones, and Bar- las O˘guz. 3dgen: Triplane latent diffusion for textured mesh generation. arXiv preprint arXiv:2303.05371, 2023. 3

    2023 arXiv

  15. [23]

    Es- caping plato’s cave: 3d shape from adversarial rendering

    Philipp Henzler, Niloy J Mitra, , and Tobias Ritschel. Es- caping plato’s cave: 3d shape from adversarial rendering. In Computer Vision (ICCV), 2019 IEEE International Confer- ence on Computer Vision, 2019. 2, 5

    2019

  16. [24]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  17. [25]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 1, 4, 6

    2021

  18. [26]

    Eva3d: Compositional 3d human generation from 2d image collections

    Fangzhou Hong, Zhaoxi Chen, Yushi Lan, Liang Pan, and Ziwei Liu. Eva3d: Compositional 3d human generation from 2d image collections. arXiv preprint arXiv:2210.04888 ,

    arXiv

  19. [27]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. ICLR, 2024. 3

    2024

  20. [28]

    TeCH: Text-guided Reconstruction of Lifelike Clothed Humans

    Yangyi Huang, Hongwei Yi, Yuliang Xiu, Tingting Liao, Ji- axiang Tang, Deng Cai, and Justus Thies. TeCH: Text-guided Reconstruction of Lifelike Clothed Humans. InInternational Conference on 3D Vision (3DV), 2024. 3

    2024

  21. [29]

    Fleet, and Ting Chen

    Allan Jabri, David J. Fleet, and Ting Chen. Scalable adaptive computation for iterative generation. In ICML, ICML’23,

  22. [30]

    Avatarcraft: Transforming text into neural human avatars with parameter- ized shape and pose control

    Ruixiang Jiang, Can Wang, Jingbo Zhang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao. Avatarcraft: Transforming text into neural human avatars with parameter- ized shape and pose control. In ICCV, 2023. 3

    2023

  23. [31]

    Scaling up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In CVPR, 2023. 4, 7

    2023

  24. [32]

    Holofusion: Towards photo-realistic 3d gen- erative modeling

    Animesh Karnewar, Niloy J Mitra, Andrea Vedaldi, and David Novotny. Holofusion: Towards photo-realistic 3d gen- erative modeling. In ICCV, 2023. 3

    2023

  25. [33]

    Holodiffusion: Training a 3D diffusion model using 2D images

    Animesh Karnewar, Andrea Vedaldi, David Novotny, and Niloy Mitra. Holodiffusion: Training a 3D diffusion model using 2D images. In CVPR, 2023. 3

    2023

  26. [34]

    Analyzing and improving the image quality of StyleGAN

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of StyleGAN. In CVPR, 2020. 2, 5

    2020

  27. [35]

    Chupa: Carv- ing 3d clothed humans from skinned shape priors using 2d diffusion probabilistic models

    Byungjun Kim, Patrick Kwon, Kwangho Lee, Myunggi Lee, Sookwan Han, Daesik Kim, and Hanbyul Joo. Chupa: Carv- ing 3d clothed humans from skinned shape priors using 2d diffusion probabilistic models. In ICCV, pages 15965– 15976, October 2023. 3

    2023

  28. [36]

    Dreamhuman: Animatable 3d avatars from text.ArXiv, abs/2306.09329, 2023

    Nikos Kolotouros, Thiemo Alldieck, Andrei Zanfir, Ed- uard Gabriel Bazavan, Mihai Fieraru, and Cristian Sminchis- escu. Dreamhuman: Animatable 3d avatars from text.ArXiv, abs/2306.09329, 2023. 3

    2023 arXiv

  29. [37]

    Diffusiongan3d: Boosting text-guided 3d gen- eration and domain adaption by combining 3d gans and dif- fusion priors, 2023

    Biwen Lei, Kai Yu, Mengyang Feng, Miaomiao Cui, and Xu- ansong Xie. Diffusiongan3d: Boosting text-guided 3d gen- eration and domain adaption by combining 3d gans and dif- fusion priors, 2023. 3

    2023

  30. [38]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, pages 12888–12900. PMLR, 2022. 4, 5, 6, 12

    2022

  31. [39]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. https://arxiv.org/abs/2311.06214, 2023. 1

    2023 arXiv

  32. [40]

    SP- GAN:sphere-guided 3d shape generation and manipulation

    Ruihui Li, Xianzhi Li, Ke-Hei Hui, and Chi-Wing Fu. SP- GAN:sphere-guided 3d shape generation and manipulation. SIGGRAPH, 40(4), 2021. 2

    2021

  33. [41]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In CVPR, 2023. 3

    2023

  34. [42]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. NeurIPS, 36, 2024. 3

    2024

  35. [43]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. ICLR, 2024. 3

    2024

  36. [44]

    Deepfashion: Powering robust clothes recog- nition and retrieval with rich annotations

    Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xi- aoou Tang. Deepfashion: Powering robust clothes recog- nition and retrieval with rich annotations. In Proceedings of IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), June 2016. 2, 8, 12, 14

    2016

  37. [45]

    Matthew Loper, Naureen Mahmood, Javier Romero, Ger- ard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1–248:16, Oct. 2015. 2, 3, 6

    2015

  38. [46]

    Att3d: Amortized text-to-3d object synthesis

    Jonathan Lorraine, Kevin Xie, Xiaohui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming- Yu Liu, Sanja Fidler, and James Lucas. Att3d: Amortized text-to-3d object synthesis. In ICCV, 2023. 3

    2023

  39. [47]

    Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, and Michael J. Black. Learn- ing to dress 3d people in generative clothing. In CVPR, June

  40. [48]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 5

    2020

  41. [49]

    3D-LDM: Neural im- plicit 3d shape generation with latent diffusion models.arXiv preprint arXiv:2212.00842, 2022

    Gimin Nam, Mariem Khlifi, Andrew Rodriguez, Alberto Tono, Linqi Zhou, and Paul Guerrero. 3D-LDM: Neural im- plicit 3d shape generation with latent diffusion models.arXiv preprint arXiv:2212.00842, 2022. 3

    2022 arXiv

  42. [50]

    Hologan: Unsupervised learning of 3d representations from natural images

    Thu Nguyen-Phuoc, Chuan Li, Lucas Theis, Christian Richardt, and Yong-Liang Yang. Hologan: Unsupervised learning of 3d representations from natural images. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pages 2037–2040, 2019. 2

    2019

  43. [51]

    Giraffe: Represent- ing scenes as compositional generative neural feature fields

    Michael Niemeyer and Andreas Geiger. Giraffe: Represent- ing scenes as compositional generative neural feature fields. In CVPR, 2021. 2

    2021

  44. [52]

    Au- todecoding latent 3d diffusion models, 2023

    Evangelos Ntavelis, Aliaksandr Siarohin, Kyle Olszewski, Chaoyang Wang, Luc Van Gool, and Sergey Tulyakov. Au- todecoding latent 3d diffusion models, 2023. 3

    2023

  45. [53]

    Spams: Structured implicit parametric models

    Pablo Palafox, Nikolaos Sarafianos, Tony Tung, and Angela Dai. Spams: Structured implicit parametric models. CVPR,

  46. [54]

    Npms: Neural paramet- ric models for 3d deformable shapes

    Pablo Rodr ´ıguez Palafox, Aljavz Bovzivc, Justus Thies, Matthias Nießner, and Angela Dai. Npms: Neural paramet- ric models for 3d deformable shapes. InICCV, pages 12675– 12685, 2021. 3

    2021

  47. [55]

    Deepsdf: Learning con- tinuous signed distance functions for shape representation

    Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning con- tinuous signed distance functions for shape representation. In CVPR, June 2019. 2

    2019

  48. [56]

    Convolutional occupancy networks

    Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolutional occupancy networks. In ECCV, pages 523–540. Springer, 2020. 4, 5

    2020

  49. [57]

    Barron, and Ben Milden- hall

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv,

  50. [58]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and ...

    2021

  51. [59]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, June 2022. 1, 2, 5, 6

    2022

  52. [60]

    StyleGAN-T: Unlocking the power of GANs for fast large-scale text-to-image synthesis

    Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, and Timo Aila. StyleGAN-T: Unlocking the power of GANs for fast large-scale text-to-image synthesis. In ICML, 2023. 4

    2023

  53. [61]

    Laion-5b: An open large-scale dataset for training next generation image-text models, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Lai...

    2022

  54. [62]

    Graf: Generative radiance fields for 3d-aware image synthesis

    Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger. Graf: Generative radiance fields for 3d-aware image synthesis. In NeurIPS, 2020. 2

    2020

  55. [63]

    3d neural field genera- tion using triplane diffusion

    J Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, and Gordon Wetzstein. 3d neural field genera- tion using triplane diffusion. In CVPR, pages 20875–20886,

  56. [64]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2021. 6

    2021

  57. [65]

    Ide-3d: Interactive disentangled edit- ing for high-resolution 3d-aware portrait synthesis

    Jingxiang Sun, Xuan Wang, Yichun Shi, Lizhen Wang, Jue Wang, and Yebin Liu. Ide-3d: Interactive disentangled edit- ing for high-resolution 3d-aware portrait synthesis. ACM Transactions on Graphics (TOG), 41(6):1–10, 2022. 3

    2022

  58. [66]

    Rethinking the inception archi- tecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception archi- tecture for computer vision. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 2818–2826, 2016. 7

    2016

  59. [67]

    Rodin: A genera- tive model for sculpting 3d digital avatars using diffusion

    Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltru ˇsaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, and Baining Guo. Rodin: A genera- tive model for sculpting 3d digital avatars using diffusion. In CVPR, pages 4563–4573, 2022. 1, 3

    2022

  60. [68]

    Free- man, and Joshua B

    Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T. Free- man, and Joshua B. Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In NeurIPS, NIPS’16, page 82–90, 2016. 2

    2016

  61. [69]

    Gram-hd: 3d-consistent image generation at high resolution with generative radiance manifolds

    Jianfeng Xiang, Jiaolong Yang, Yu Deng, and Xin Tong. Gram-hd: 3d-consistent image generation at high resolution with generative radiance manifolds. In ICCV, pages 2195– 2205, October 2023. 2

    2023

  62. [70]

    Style-based point generator with ad- versarial rendering for point cloud completion

    Chulin Xie, Chuxin Wang, Bo Zhang, Hao Yang, Dong Chen, and Fang Wen. Style-based point generator with ad- versarial rendering for point cloud completion. In CVPR, pages 4619–4628, June 2021. 2

    2021

  63. [71]

    Get3dhuman: Lifting stylegan-human into a 3d generative model using pixel-aligned reconstruction priors

    Zhangyang Xiong, Di Kang, Derong Jin, Weikai Chen, Lin- chao Bao, Shuguang Cui, and Xiaoguang Han. Get3dhuman: Lifting stylegan-human into a 3d generative model using pixel-aligned reconstruction priors. In ICCV, pages 9287– 9297, October 2023. 3

    2023

  64. [72]

    Pointflow: 3d point cloud generation with continuous normalizing flows

    Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In ICCV,

  65. [73]

    Lion: Latent point diffusion models for 3d shape generation

    Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation. In NeurIPS,

  66. [74]

    Avatarverse: High-quality & stable 3d avatar creation from text and pose, 2023

    Huichao Zhang, Bowen Chen, Hao Yang, Liao Qu, Xu Wang, Li Chen, Chao Long, Feida Zhu, Kang Du, and Min Zheng. Avatarverse: High-quality & stable 3d avatar creation from text and pose, 2023. 3

    2023

  67. [75]

    a photo of a

    Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, and Tong Sun. Towards language-free training for text-to-image generation. In CVPR, pages 17907–17917, 2022. 4 Supplementary Materials In the following, we provide a detailed...

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.