REVIEW 4 major objections 5 minor 101 references
DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A 7.6M-parameter diffusion model can invert procedural generation from a single image, producing editable 3D assets in seconds.
desk verdict New diffusion-over-parameters idea for inverse PCG, but the claimed parameter accuracy is never directly measured and the ShapeNet eval is filtered—send to review with requests for a parameter-error table. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the pair consisting of a procedural generator and its canonicalized parameter vector. The reversible projection $\phi$ maps generator parameters into $[-1,1]^N$, making mixed continuous and discrete parameters a single diffusion data space, while the Diffusion Transformer denoiser treats that vector as a token sequence and injects DINOv2 image patch features through cross-attention. This machinery carries the argument because the diffusion model amortizes the inverse problem: instead of running the generator thousands of times and scoring each result, one learned model predicts the full posterior and samples it in a few seconds.
What would settle it
Use the unfiltered ShapeNet chair test split, which includes sofa-like and artistic chairs outside the Infinigen chair generator's range, and measure F-Score and Chamfer distance for DI-PCG; if the scores collapse toward baseline levels or a human judge sees the output as a different chair type, the method is not generally inverting the observed geometry but only selecting among expressible variants.
Extended reading notes
Core claim
The central discovery is that a procedural generator's parameter vector is itself a compact, semantically meaningful 3D representation that a diffusion model can learn to sample. Discrete parameters are converted to continuous pieces of the interval $[-1,1]$ and continuous parameters are normalized into $[-1,1]$ through a reversible projection $\phi$, so the whole parameter set becomes a single data space $x \in [-1,1]^N$. DI-PCG trains a Diffusion Transformer to predict noise on $x$ conditioned on DINOv2 patch tokens via cross-attention, then maps sampled parameters back through $\phi^{-1}$ into the generator. The resulting method, with 7.6M parameters and six category-specific models trained on 18,000 synthetic image-parameter pairs each, matches or beats much larger image-to-3D systems on the chair benchmark while keeping outputs clean, editable, and consistently meshed.
Load-bearing premise
The load-bearing premise is that the procedural generator's parameter family can actually express the observed object; if no parameter combination yields the target shape, DI-PCG returns the nearest expressible cousin rather than the true asset.
Editorial extensions
If this is right
- Image-to-3D becomes controllable: sampled parameters can be edited one at a time, such as leg height or arm presence, and re-run through the generator.
- Inference cost drops to seconds with only 7.6M parameters, making inverse PCG practical on a single GPU or in resource-constrained settings.
- Because training data come from the generator itself, no external 3D collection is needed, and any procedural generator can be paired with DI-PCG with nearly no code change.
- Sketch inputs work as well as RGB images, giving designers a simple way to specify shapes.
- Quantitative results suggest that on categories a generator can express, parameter-space diffusion can outperform much larger feed-forward reconstruction models.
Reading between the lines
- The paper's evaluation filters ShapeNet chairs to exclude sofa-like or artistic chairs; an unfiltered benchmark would likely show a sharp performance drop and would better expose the generator-expressiveness ceiling.
- The same conditional diffusion-over-parameters recipe could transfer to other procedural domains with explicit parameters, such as CAD programs, materials, or city layouts, where the usual bottleneck is a scoring function or gradient.
- Combining DI-PCG-style inverse models with automatically generated procedural programs would attack the stated limitation directly, letting the parametric representation itself expand.
- Parameter-space diffusion also gives a natural diversity knob: sampling multiple times from the posterior yields several plausible assets for one image, something single-shot regressors do not offer.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DI-PCG, a diffusion-transformer method for inverse procedural content generation from a single image. The PCG parameters themselves are treated as the denoising target, while DINOv2 image features are injected via cross-attention as conditioning. Training pairs are generated by randomly sampling parameters from six Infinigen generators and rendering the resulting meshes. The authors report that the method needs only 7.6M parameters and about 30 GPU hours per generator, that it outperforms image-to-3D baselines on chair geometry metrics (CD, EMD, F-Score) on both an internal test split and a filtered ShapeNet chair subset, and that it supports sketch conditioning and parameter-level editing. The main claims are efficiency, accuracy of parameter recovery, and generalization to in-the-wild images.
Significance. If the central claim holds, this is a valuable contribution to inverse PCG and image-to-3D generation: a lightweight, generator-agnostic diffusion prior over PCG parameters would enable fast inference, editable outputs, and high-fidelity meshes by construction. The pipeline is self-contained, avoids external 3D data collection, and the architecture is simple and reproducible. The paper explicitly acknowledges the expressiveness limitation of the underlying procedural generators, which is a strength. However, the quantitative evaluation as presented does not yet establish the headline claim of accurate parameter recovery, and the generalization evidence is narrower than the abstract suggests. The significance is therefore conditional on closing this evaluation gap.
major comments (4)
- [Section 3.2 and Table 1] The central claim that DI-PCG 'recovers parameters accurately' (Abstract, Section 1) is not directly evaluated. Table 1 reports only geometric metrics (CD, EMD, F-Score) between the generated and ground-truth meshes, with no comparison between predicted and ground-truth parameters. Because the parameter-to-geometry map of the Infinigen generators is not necessarily injective and some parameters may have only a weak effect on the rendered shape, low CD/EMD and high F-Score can coexist with systematically wrong parameter estimates. This matters because the editing application in Section 4.4 depends on the recovered parameters being semantically correct. Please add a direct parameter-accuracy evaluation on the internal test split, where ground-truth parameters are known (for example, normalized L1 error for continuous parameters and per-choice accuracy for discrete parameters), and discuss the identifiability of the parameter space.
- [Section 4.2] The quantitative evaluation is limited to the chair category, and the ShapeNet generalization test is post-hoc filtered to 218 models after excluding sofa-like or artistic chairs. The claim that DI-PCG is 'generalizing well to in-the-wild images' is therefore supported quantitatively for only one category and a filtered subset; the other five categories are supported only qualitatively. Please either provide quantitative results for the other procedural generators or revise the generalization claim to match the evidence actually presented.
- [Section 4.1, 'Comparison with MCMC'] The MCMC comparison is purely qualitative: Figure 6 shows a single example with iteration counts and wall-clock times, but no measured accuracy or runtime table. Since the paper's efficiency claim is framed relative to MCMC, this comparison is load-bearing. Please provide a quantitative comparison, for example parameter error or geometric metrics as a function of number of iterations and wall-clock time, using the same condition image and scoring metric as the proposed method.
- [Section 4.1, 'Data preparation' and Table 1] The internal test split is rendered by the same procedural generators that are used to create the training pairs. This largely measures the model's ability to interpolate within the training distribution, not its ability to generalize to out-of-distribution conditions. The paper should state this limitation explicitly near Table 1, and if possible quantify robustness to unseen render viewpoints, lighting, or generator variants, since the training render views are limited to azimuths 0/30/60, elevations 30/60, and distances 1.8/2.0.
minor comments (5)
- [Section 3.2, 'Representation'] The canonicalization projection ϕ is described only verbally; please specify exactly how discrete parameters are mapped to continuous intervals and how the inverse projection resolves values near bin boundaries, and discuss whether uniform binning distorts the posterior distribution.
- [Tables 1 and 2] The tables report single numbers with no variance estimates. Please report standard deviations or confidence intervals across multiple runs or random seeds so that the margin over the baselines can be assessed.
- [Section 4.2] The sentence 'It clearly shows that DI-PCG can reliably fit the procedural generator and inversely estimate the parameters with high accuracy' overstates what Table 1 demonstrates, since Table 1 contains no parameter-level metrics.
- [Figure 6] The MCMC timing labels should specify the hardware, the proposal distribution, the scoring function, and the number of chains; without these details the reported wall-clock times are difficult to interpret.
- [Abstract and Section 4.1] The phrase 'generalizing well to in-the-wild images' should be qualified in the abstract, given that the quantitative generalization evidence is restricted to filtered ShapeNet chairs and that in-the-wild results for other categories are qualitative only.
Circularity Check
No significant circularity; the parameter-diffusion pipeline is independently grounded by the external ShapeNet subset and no fitted input is relabeled as a prediction.
full rationale
The paper's derivation chain is: procedural generator parameters p are canonicalized to x; a diffusion transformer is trained with the standard noise-prediction objective conditioned on DINOv2 image features; at inference x is sampled, mapped back to p, and fed into the generator. No step defines the output in terms of the target, and no fitted parameter is renamed as a prediction. The training pairs are self-generated with the Infinigen generators, and the test split of DI-PCG is likewise rendered from those generators; this is an in-distribution evaluation and a generalization limitation, but it is not circularity because the test parameters are held out and the geometry metrics are computed against ground-truth meshes rather than being encoded in the model. The external ShapeNet chair subset (Section 4.2) provides an independent out-of-distribution benchmark, and although it is manually filtered to exclude shapes the generator cannot express, that filtering is an acknowledged expressiveness boundary, not a fitted input. The abstract's claim of 'recovering parameters accurately' is supported only indirectly by geometry metrics (CD/EMD/F-Score) rather than direct parameter error; this is a measurement gap, not a circular reduction. The only self-citation with author overlap is InstantMesh [87], used as a baseline rather than as load-bearing support, and the MCMC comparison is qualitative but does not substitute for the main derivation. No equation in the paper reduces predicted parameters to ground-truth parameters or to the condition features by construction.
Assumptions & free parameters
free parameters (5)
- DiT architecture (12 layers, 6 heads, hidden dim 192) =
7.6M parameters
- Training pairs per generator =
20,000 (18k train / 2k val)
- Render view configuration =
azimuths [0,30,60], elevations [30,60], distances [1.8,2.0]
- Condition encoder selection =
DINOv2 ViT-B/14
- ShapeNet chair test filter =
218 models retained
assumptions (4)
- domain assumption The Infinigen procedural generators define a parameter space that faithfully spans typical instances of each category (chair, table, vase, basket, flower, dandelion).
- domain assumption Single-image DINOv2 features retain enough 3D shape information for parameter inference, given the training render-view distribution.
- ad hoc to paper The canonicalization phi (uniform binning of discrete parameters and normalization to [-1,1]) is invertible and does not distort the parameter posterior.
- domain assumption Rendering with the fixed camera configuration (azimuths 0/30/60, elevations 30/60, distances 1.8/2.0) makes the training image distribution representative of real internet images for these categories.
Cite this review
Pith. "Pith review of DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation." pith.science (2026). https://pith.science/paper/3K4CDGVM
@misc{pith2026241215200,
author = {Pith},
title = {Pith review of: DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation},
year = {2026},
howpublished = {\url{https://pith.science/paper/3K4CDGVM}},
note = {Machine review of arXiv:2412.15200}
}
read the original abstract
Procedural Content Generation (PCG) is powerful in creating high-quality 3D contents, yet controlling it to produce desired shapes is difficult and often requires extensive parameter tuning. Inverse Procedural Content Generation aims to automatically find the best parameters under the input condition. However, existing sampling-based and neural network-based methods still suffer from numerous sample iterations or limited controllability. In this work, we present DI-PCG, a novel and efficient method for Inverse PCG from general image conditions. At its core is a lightweight diffusion transformer model, where PCG parameters are directly treated as the denoising target and the observed images as conditions to control parameter generation. DI-PCG is efficient and effective. With only 7.6M network parameters and 30 GPU hours to train, it demonstrates superior performance in recovering parameters accurately, and generalizing well to in-the-wild images. Quantitative and qualitative experiment results validate the effectiveness of DI-PCG in inverse PCG and image-to-3D generation tasks. DI-PCG offers a promising approach for efficient inverse PCG and represents a valuable exploration step towards a 3D generation path that models how to construct a 3D asset using parametric models.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[1]
Polydiff: Generating 3d polygonal meshes with diffusion models
Antonio Alliegro, Yawar Siddiqui, Tatiana Tommasi, and Matthias Nießner. Polydiff: Generating 3d polygonal meshes with diffusion models. arXiv preprint arXiv:2312.11417 ,
-
[2]
Scene- script: Reconstructing scenes with an autoregressive struc- tured language model
Armen Avetisyan, Christopher Xie, Henry Howard-Jenkins, Tsun-Yi Yang, Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, et al. Scene- script: Reconstructing scenes with an autoregressive struc- tured language model. arXiv preprint arXiv:2403.13064 ,
-
[3]
Guided procedural modeling
Bedrich Bene ˇs, Ondrej ˇSt’ava, Radomir M ˇech, and Gavin Miller. Guided procedural modeling. In Computer graphics forum, pages 325–334. Wiley Online Library, 2011. 3
2011
-
[4]
Video generation models as world simulators
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators
-
[5]
Optimally directed shape gen- eration by shape annealing
J Cagan and WJ Mitchell. Optimally directed shape gen- eration by shape annealing. Environment and Planning B: Planning and Design, 20(1):5–12, 1993. 3
1993
-
[6]
Efficient geometry-aware 3d generative adversarial networks
Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 3
2022
-
[7]
Shapenet: An information-rich 3d model repository
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 8
arXiv 2015
-
[8]
Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023. 4
Show all 101 references
-
[9]
Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 22246–22256, 2023. 3
2023
-
[10]
Sdfusion: Multimodal 3d shape completion, reconstruction, and generation
Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alexan- der G Schwing, and Liang-Yan Gui. Sdfusion: Multimodal 3d shape completion, reconstruction, and generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4456–4465, 2023. 3, 6
2023
-
[11]
3d-r2n2: A unified approach for single and multi-view 3d object reconstruction
Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In Com- puter Vision–ECCV 2016: 14th European Conference, Am- sterdam, The Netherlands, October 11-14, 2016, Proceed- ...
2016
-
[12]
Blender - a 3D modelling and rendering package
Blender Online Community. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018. 2
2018
-
[13]
Texpro: Text- guided pbr texturing with procedural material modeling
Ziqiang Dang, Wenqi Dong, Zesong Yang, Bangbang Yang, Liang Li, Yuewen Ma, and Zhaopeng Cui. Texpro: Text- guided pbr texturing with procedural material modeling. arXiv preprint arXiv:2410.15891, 2024. 2
2024 arXiv
-
[14]
Objaverse: A universe of annotated 3d objects
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...
2023
-
[15]
Inversecsg: Automatic conversion of 3d models to csg trees
Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik. Inversecsg: Automatic conversion of 3d models to csg trees. ACM Transactions on Graphics (TOG), 37(6):1–16, 2018. 3
2018
-
[16]
Com- puter rendering of stochastic models
Alain Fournier, Don Fussell, and Loren Carpenter. Com- puter rendering of stochastic models. Communications of the ACM, 25(6):371–384, 1982. 2
1982
-
[17]
A review of digital terrain modeling
Eric Galin, Eric Gu ´erin, Adrien Peytavie, Guillaume Cor- donnier, Marie-Paule Cani, Bedrich Benes, and James Gain. A review of digital terrain modeling. In Computer Graphics Forum, pages 553–577. Wiley Online Library, 2019. 2
2019
-
[18]
Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image
Daoyi Gao, D ´avid Rozenberszki, Stefan Leutenegger, and Angela Dai. Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image. ACM Transactions on Graphics (TOG), 43(4):1–15, 2024. 3
2024
-
[19]
Single-view 3d reconstruction via inverse procedural mod- eling
Albert Garifullin, Nikolay Maiorov, and Vladimir Frolov. Single-view 3d reconstruction via inverse procedural mod- eling. arXiv preprint arXiv:2310.13373, 2023. 3
2023 arXiv
-
[20]
Reversible jump markov chain monte carlo computation and bayesian model determination
Peter J Green. Reversible jump markov chain monte carlo computation and bayesian model determination. Biometrika, 82(4):711–732, 1995. 2
1995
-
[21]
Inverse procedural modeling of branching structures by in- ferring l-systems
Jianwei Guo, Haiyong Jiang, Bedrich Benes, Oliver Deussen, Xiaopeng Zhang, Dani Lischinski, and Hui Huang. Inverse procedural modeling of branching structures by in- ferring l-systems. ACM Transactions on Graphics (TOG) , 39(5):1–13, 2020. 2, 3
2020
-
[22]
A bayesian inference framework for procedural material pa- rameter estimation
Yu Guo, Milo ˇs Ha ˇsan, Lingqi Yan, and Shuang Zhao. A bayesian inference framework for procedural material pa- rameter estimation. In Computer Graphics Forum , pages 255–266. Wiley Online Library, 2020. 2
2020
-
[23]
3dgen: Triplane latent diffusion for textured mesh generation
Anchit Gupta, Wenhan Xiong, Yixin Nie, Ian Jones, and Bar- las O˘guz. 3dgen: Triplane latent diffusion for textured mesh generation. arXiv preprint arXiv:2303.05371, 2023. 3
2023 arXiv
-
[24]
Monte carlo sampling methods using markov chains and their applications
W Keith Hastings. Monte carlo sampling methods using markov chains and their applications. 1970. 2
1970
-
[25]
Shapegenetics: Using genetic algorithms for proce- dural modeling
Karl Haubenwallner, Hans-Peter Seidel, and Markus Stein- berger. Shapegenetics: Using genetic algorithms for proce- dural modeling. In Computer Graphics Forum, pages 213–
-
[26]
An inverse procedural modeling pipeline for svbrdf maps
Yiwei Hu, Chengan He, Valentin Deschaintre, Julie Dorsey, and Holly Rushmeier. An inverse procedural modeling pipeline for svbrdf maps. ACM Transactions on Graphics (TOG), 41(2):1–17, 2022. 2
2022
-
[27]
Generating procedural materials from text or image prompts
Yiwei Hu, Paul Guerrero, Milos Hasan, Holly Rushmeier, and Valentin Deschaintre. Generating procedural materials from text or image prompts. In ACM SIGGRAPH 2023 Con- ference Proceedings, pages 1–11, 2023. 2 9
2023
-
[28]
Scenecraft: An llm agent for synthesizing 3d scenes as blender code
Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. Scenecraft: An llm agent for synthesizing 3d scenes as blender code. In Forty-first International Conference on Ma- chine Learning, 2024. 2
2024
-
[29]
Shape synthesis from sketches via procedu- ral models and convolutional networks
Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. Shape synthesis from sketches via procedu- ral models and convolutional networks. IEEE transactions on visualization and computer graphics , 23(8):2003–2013,
2003
-
[30]
Blender- alchemy: Editing 3d graphics with vision-language models
Ian Huang, Guandao Yang, and Leonidas Guibas. Blender- alchemy: Editing 3d graphics with vision-language models. arXiv preprint arXiv:2404.17672, 2024. 2
2024 arXiv
-
[31]
Shap-e: Generat- ing conditional 3d implicit functions
Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 6, 7
2023 arXiv
-
[32]
Reparamcad: Zero-shot cad program re-parameterization for interactive manipulation
Milin Kodnongbua, Benjamin T Jones, Maaz Bin Safeer Ah- mad, Vladimir G Kim, and Adriana Schulz. Reparamcad: Zero-shot cad program re-parameterization for interactive manipulation. 2023. 2
2023
-
[33]
Pico: procedural iterative constrained optimizer for geometric modeling
V ojtˇech Krs, Radom´ır Mˇech, Mathieu Gaillard, Nathan Carr, and Bedrich Benes. Pico: procedural iterative constrained optimizer for geometric modeling. IEEE Transactions on Visualization and Computer Graphics , 27(10):3968–3981,
-
[34]
Re-thinking inverse graphics with large language models
Peter Kulits, Haiwen Feng, Weiyang Liu, Victoria Abrevaya, and Michael J Black. Re-thinking inverse graphics with large language models. arXiv preprint arXiv:2404.15228, 2024. 2
2024 arXiv
-
[35]
Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner
Weiyu Li, Jiarui Liu, Rui Chen, Yixun Liang, Xuelin Chen, Ping Tan, and Xiaoxiao Long. Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner. arXiv preprint arXiv:2405.14979, 2024. 5, 6, 7
2024 arXiv
-
[36]
Magic3d: High-resolution text-to-3d content creation
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...
2023
-
[37]
Mathematical models for cellular in- teractions in development i
Aristid Lindenmayer. Mathematical models for cellular in- teractions in development i. filaments with one-sided inputs. Journal of theoretical biology, 18(3):280–299, 1968. 2
1968
-
[38]
Interac- tive visual editing of grammars for procedural architecture
Markus Lipp, Peter Wonka, and Michael Wimmer. Interac- tive visual editing of grammars for procedural architecture. In ACM SIGGRAPH 2008 papers, pages 1–10. 2008. 2
2008
-
[39]
Zero-1-to- 3: Zero-shot one image to 3d object
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 3
2023
-
[40]
Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age
Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. In The Twelfth International Conference on Learning Representations, 2024. 3
2024
-
[41]
Meshdif- fusion: Score-based generative 3d mesh modeling
Zhen Liu, Yao Feng, Michael J Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. Meshdif- fusion: Score-based generative 3d mesh modeling. arXiv preprint arXiv:2303.08133, 2023. 3
2023 arXiv
-
[42]
Wonder3d: Sin- gle image to 3d using cross-domain diffusion
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pa...
2024
-
[43]
Controllable mesh generation through sparse latent point diffusion models
Zhaoyang Lyu, Jinyi Wang, Yuwei An, Ya Zhang, Dahua Lin, and Bo Dai. Controllable mesh generation through sparse latent point diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 271–280, 2023. 3
2023
-
[44]
Visual mod- els of plants interacting with their environment
Radom ´ır Mˇech and Przemyslaw Prusinkiewicz. Visual mod- els of plants interacting with their environment. In Proceed- ings of the 23rd annual conference on Computer graphics and interactive techniques, pages 397–410, 1996. 2
1996
-
[45]
Equa- tion of state calculations by fast computing machines
Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. Equa- tion of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092, 1953. 2, 7
1953
-
[46]
Latent-nerf for shape-guided generation of 3d shapes and textures
Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12663–12673, 2023. 3
2023
-
[47]
Procedural modeling of buildings
Pascal M ¨uller, Peter Wonka, Simon Haegler, Andreas Ulmer, and Luc Van Gool. Procedural modeling of buildings. In ACM SIGGRAPH 2006 Papers, pages 614–623. 2006. 2
2006
-
[48]
Point-e: A system for generat- ing 3d point clouds from complex prompts
Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generat- ing 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751, 2022. 3
2022 arXiv
-
[49]
Interactive sketching of urban procedural models
Gen Nishida, Ignacio Garcia-Dorado, Daniel G Aliaga, Bedrich Benes, and Adrien Bousseau. Interactive sketching of urban procedural models. ACM Transactions on Graphics (TOG), 35(4):1–11, 2016. 2, 3
2016
-
[50]
Dinov2: Learning robust visual features without supervision
Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 5, 7
2023 arXiv
-
[51]
Procedural modeling of cities
Yoav IH Parish and Pascal M ¨uller. Procedural modeling of cities. In Proceedings of the 28th annual conference on Com- puter graphics and interactive techniques , pages 301–308,
-
[52]
Geocode: Interpretable shape programs
Ofek Pearl, Itai Lang, Yuhua Hu, Raymond A Yeh, and Rana Hanocka. Geocode: Interpretable shape programs. arXiv preprint arXiv:2212.11715, 2022. 2, 3
2022 arXiv
-
[53]
Scalable diffusion models with transformers
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,
-
[54]
Dreamfusion: Text-to-3d using 2d diffusion
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. In The Eleventh International Conference on Learning Representa- tions, 2023. 3 10
2023
-
[55]
Graphical applications of l- systems
Przemyslaw Prusinkiewicz. Graphical applications of l- systems. In Proceedings of graphics interface , pages 247– 253, 1986. 2
1986
-
[56]
The algorithmic beauty of plants
Przemyslaw Prusinkiewicz and Aristid Lindenmayer. The algorithmic beauty of plants . Springer Science & Business Media, 2012. 2
2012
-
[57]
Synthetic topiary
Przemyslaw Prusinkiewicz, Mark James, and Radom ´ır Mˇech. Synthetic topiary. In Proceedings of the 21st an- nual conference on Computer graphics and interactive tech- niques, pages 351–358, 1994. 2
1994
-
[58]
Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to- 3d
Lingteng Qiu, Guanying Chen, Xiaodong Gu, Qi Zuo, Mu- tian Xu, Yushuang Wu, Weihao Yuan, Zilong Dong, Liefeng Bo, and Xiaoguang Han. Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to- 3d. In Proceedings of the IEEE/CVF Conference on Com- ...
-
[59]
Infinite photorealistic worlds using procedural generation
Alexander Raistrick, Lahav Lipson, Zeyu Ma, Lingjie Mei, Mingzhe Wang, Yiming Zuo, Karhan Kayan, Hongyu Wen, Beining Han, Yihan Wang, et al. Infinite photorealistic worlds using procedural generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern rec...
2023
-
[60]
In- finigen indoors: Photorealistic indoor scenes using procedu- ral generation
Alexander Raistrick, Lingjie Mei, Karhan Kayan, David Yan, Yiming Zuo, Beining Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, et al. In- finigen indoors: Photorealistic indoor scenes using procedu- ral generation. In Proceedings of the IEEE/CVF Confere...
2024
-
[61]
Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies
Xuanchi Ren, Jiahui Huang, Xiaohui Zeng, Ken Museth, Sanja Fidler, and Francis Williams. Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4209–4219, 2024. 3
2024
-
[62]
Reconstruction of fac ¸ade structures using a formal grammar and rjmcmc
Nora Ripperda and Claus Brenner. Reconstruction of fac ¸ade structures using a formal grammar and rjmcmc. In Joint Pattern Recognition Symposium , pages 750–759. Springer,
-
[63]
Evaluation of structure recognition using labelled facade images
Nora Ripperda and Claus Brenner. Evaluation of structure recognition using labelled facade images. In Joint Pattern Recognition Symposium, pages 532–541. Springer, 2009. 3
2009
-
[64]
Controlling procedural modeling programs with stochastically-ordered sequential monte carlo
Daniel Ritchie, Ben Mildenhall, Noah D Goodman, and Pat Hanrahan. Controlling procedural modeling programs with stochastically-ordered sequential monte carlo. ACM Trans- actions on Graphics (TOG), 34(4):1–11, 2015. 2, 3
2015
-
[65]
Neurally-guided procedural models: Amortized inference for procedural graphics programs using neural net- works
Daniel Ritchie, Anna Thomas, Pat Hanrahan, and Noah Goodman. Neurally-guided procedural models: Amortized inference for procedural graphics programs using neural net- works. Advances in neural information processing systems , 29, 2016. 2, 3
2016
-
[66]
Advanced procedural modeling of architecture
Michael Schwarz and Pascal M ¨uller. Advanced procedural modeling of architecture. ACM Transactions on Graphics (TOG), 34(4):1–12, 2015. 2
2015
-
[67]
Let 2d diffusion model know 3d-consistency for robust text-to-3d generation
Junyoung Seo, Wooseok Jang, Min-Seop Kwak, Hyeonsu Kim, Jaehoon Ko, Junho Kim, Jin-Hwa Kim, Jiyoung Lee, and Seungryong Kim. Let 2d diffusion model know 3d-consistency for robust text-to-3d generation. In The Twelfth International Conference on Learning Representa- tions, 2024. 3
2024
-
[68]
Csgnet: Neural shape parser for constructive solid geometry
Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. Csgnet: Neural shape parser for constructive solid geometry. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 5515–5523, 2018. 3
2018
-
[69]
Deep marching tetrahedra: a hybrid repre- sentation for high-resolution 3d shape synthesis
Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, and Sanja Fidler. Deep marching tetrahedra: a hybrid repre- sentation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems , 34:6087–6101,
-
[70]
Match: Differentiable material graphs for procedural mate- rial capture
Liang Shi, Beichen Li, Milo ˇs Ha ˇsan, Kalyan Sunkavalli, Tamy Boubekeur, Radomir Mech, and Wojciech Matusik. Match: Differentiable material graphs for procedural mate- rial capture. ACM Transactions on Graphics (TOG), 39(6): 1–15, 2020. 2
2020
-
[71]
Zero123++: a single image to consistent multi-view dif- fusion base model
Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model. arXiv preprint arXiv:2310.15110, 2023. 3
-
[72]
Mvdream: Multi-view diffusion for 3d gen- eration
Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. In The Twelfth International Conference on Learn- ing Representations, 2024. 3
2024
-
[73]
Shapewalk: Composi- tional shape editing through language-guided chains
Habib Slim and Mohamed Elhoseiny. Shapewalk: Composi- tional shape editing through language-guided chains. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22574–22583, 2024. 2
2024
-
[74]
Plants, fractals, and formal languages
Alvy Ray Smith. Plants, fractals, and formal languages. ACM SIGGRAPH Computer Graphics, 18(3):1–10, 1984. 2
1984
-
[75]
Inverse procedural modelling of trees
Ondrej Stava, S ¨oren Pirk, Julian Kratt, Baoquan Chen, Radom´ır Mˇech, Oliver Deussen, and Bedrich Benes. Inverse procedural modelling of trees. InComputer Graphics Forum, pages 118–131. Wiley Online Library, 2014. 2, 3
2014
-
[76]
Pytorchgeonodes: En- abling differentiable shape programs for 3d shape recon- struction
Sinisa Stekovic, Stefan Ainetter, Mattia D’Urso, Friedrich Fraundorfer, and Vincent Lepetit. Pytorchgeonodes: En- abling differentiable shape programs for 3d shape recon- struction. arxiv, 2024. 3
2024
-
[77]
Shape grammars and the generative specification of painting and sculpture
George Stiny and James Gips. Shape grammars and the generative specification of painting and sculpture. In IFIP congress (2), pages 125–135. Citeseer, 1971. 2
1971
-
[78]
3d-gpt: Procedural 3d modeling with large language models
Chunyi Sun, Junlin Han, Weijian Deng, Xinlong Wang, Zishan Qin, and Stephen Gould. 3d-gpt: Procedural 3d modeling with large language models. arXiv preprint arXiv:2310.12945, 2023. 2
2023 arXiv
-
[79]
Metropolis procedural modeling
Jerry O Talton, Yu Lou, Steve Lesser, Jared Duke, Radom ´ır Mech, and Vladlen Koltun. Metropolis procedural modeling. ACM Trans. Graph., 30(2):11–1, 2011. 2, 3
2011
-
[80]
Lion: Latent point dif- fusion models for 3d shape generation
Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point dif- fusion models for 3d shape generation. Advances in Neural Information Processing Systems, 35:10021–10039, 2022. 3
2022
-
[81]
Inverse design of urban 11 procedural models
Carlos A Vanegas, Ignacio Garcia-Dorado, Daniel G Aliaga, Bedrich Benes, and Paul Waddell. Inverse design of urban 11 procedural models. ACM Transactions on Graphics (TOG), 31(6):1–11, 2012. 2, 3
2012
-
[82]
Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion
Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. In European Conference on Computer...
2025
-
[83]
Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems , 36, 2024. 3
2024
-
[84]
Instant architecture
Peter Wonka, Michael Wimmer, Franc ¸ois Sillion, and William Ribarsky. Instant architecture. ACM Transactions on Graphics (TOG), 22(3):669–677, 2003. 2
2003
-
[85]
Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer
Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer. arXiv preprint arXiv:2405.14832, 2024. 3, 5
2024 arXiv
-
[86]
Octfusion: Octree- based diffusion models for 3d shape generation
Bojun Xiong, Si-Tong Wei, Xin-Yang Zheng, Yan-Pei Cao, Zhouhui Lian, and Peng-Shuai Wang. Octfusion: Octree- based diffusion models for 3d shape generation. arXiv preprint arXiv:2408.14732, 2024. 3
2024 arXiv
-
[87]
Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models
Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,
-
[88]
L3go: Language agents with chain-of-3d-thoughts for generating unconventional objects
Yutaro Yamada, Khyathi Chandu, Yuchen Lin, Jack Hessel, Ilker Yildirim, and Yejin Choi. L3go: Language agents with chain-of-3d-thoughts for generating unconventional objects. arXiv preprint arXiv:2402.09052, 2024. 2
2024 arXiv
-
[89]
Make it home: automatic optimization of furniture arrangement
Lap Fai Yu, Sai Kit Yeung, Chi Keung Tang, Demetri Terzopoulos, Tony F Chan, and Stanley J Osher. Make it home: automatic optimization of furniture arrangement. ACM Transactions on Graphics (TOG)-Proceedings of ACM SIGGRAPH 2011, v. 30,(4), July 2011, article no. 86, 30(4),
2011
-
[90]
3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023. 3
2023
-
[91]
Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 3, 4, 5
2024
-
[92]
Cityx: Controllable procedural con- tent generation for unbounded 3d cities
Shougao Zhang, Mengqi Zhou, Yuxi Wang, Chuanchen Luo, Rongyu Wang, Yiwei Li, Xucheng Yin, Zhaoxiang Zhang, and Junran Peng. Cityx: Controllable procedural con- tent generation for unbounded 3d cities. arXiv preprint arXiv:2407.17572, 2024. 2
2024 arXiv
-
[93]
Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation
Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in Neural Information Processing Systems, 36, 2024. 3, 6, 7
2024
-
[94]
Scenex: Procedural control- lable large-scale scene generation via large-language models
Mengqi Zhou, Yuxi Wang, Jun Hou, Chuanchen Luo, Zhaox- iang Zhang, and Junran Peng. Scenex: Procedural control- lable large-scale scene generation via large-language models. arXiv preprint arXiv:2403.15698, 2024. 2
2024 arXiv
-
[95]
Deeptree: Modeling trees with situated la- tents
Xiaochen Zhou, Bosheng Li, Bedrich Benes, Songlin Fei, and S ¨oren Pirk. Deeptree: Modeling trees with situated la- tents. arXiv preprint arXiv:2305.05153, 2023. 2, 3
2023 arXiv
-
[96]
Awol: Analysis without synthesis using language
Silvia Zuffi and Michael J Black. Awol: Analysis without synthesis using language. arXiv preprint arXiv:2404.03042,
-
[99]
They contain 48, 19, 12, 14, 9, 15 controllable parameters, respectively
More Implementation Details We use six procedural generators from Infinigen and Infini- gen Indoors, namely chair, table, vase, basket, flower and dandelion generators. They contain 48, 19, 12, 14, 9, 15 controllable parameters, respectively. These are also the in- put token l...
-
[100]
The gen- eration results for the chair, table, and vase categories are shown in Figure 9
More Qualitative Results Here we show more qualitative results of DI-PCG. The gen- eration results for the chair, table, and vase categories are shown in Figure 9. DI-PCG can handle complex shape vari- ations and details, generating high-quality 3D models from input single ima...
-
[101]
Although the adopted generator from Infinigen can cover a wide range of common variations of the corre- sponding category, it still has obvious boundaries
Discussions and Failure Cases As discussed in the main paper, DI-PCG is limited by the generality and granularity of the given procedural genera- tors. Although the adopted generator from Infinigen can cover a wide range of common variations of the corre- sponding category, it...
-
[223]
Wiley Online Library, 2017. 3
2017
-
[2024]
2 12 DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation Supplementary Material
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.