Pith. sign in

REVIEW 3 major objections 4 minor 78 references

VLMaterial: Procedural Material Generation with Large Vision-Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A fine-tuned vision-language model predicts Blender procedural materials as Python programs from single images, outperforming prior generative and retrieval baselines on all three test sets.

desk verdict Solid dataset and fine-tuning contribution, let down by a sloppy 'outperforms' claim that the stress-test correctly catches on real images. read the letter →

arxiv 2501.18623 v2 pith:SVY5RYGJ submitted 2025-01-27 cs.CV cs.GR

Beichen Li , Rundi Wu , Armando Solar-Lezama , Changxi Zheng , Liang Shi , Bernd Bickel , Wojciech Matusik This is my paper · ORCID
classification cs.CVcs.GR
keywords proceduralmaterialsvision-languagemodelsprogramsynthesisinversemodelingdataaugmentationBlendermaterialgenerationnodegraphs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Procedural materials in 3D tools are node graphs that produce shader appearances; they are editable and resolution-independent, but designing one from a photograph is a specialist skill. This paper claims that the task can be reframed as image-conditioned program synthesis: transpile Blender material graphs into Python, fine-tune a large vision-language model on image–program pairs, and let the model write the program. Because only about 1.6K clean artist-made materials were available, the authors built an open dataset and expanded it to more than 550K pairs using LLM-driven structural crossover plus random parameter perturbation. They report that the fine-tuned model beats conditional MatFormer, BlenderAlchemy, nearest-neighbor retrieval, and GPT-4o-mini on Blender, Substance, and real-photo test sets, and that professional artists rate its outputs as usable starting points. If the claim holds, inverse design of editable materials becomes an accessible, open capability rather than a proprietary expert workflow.

What carries the argument

The central mechanism is transpiling Blender procedural material node graphs into standard Python programs, making material generation a code-generation task. The model is LLaVA-NeXT, a CLIP vision encoder paired with a LLaMA 3 8B decoder and an MLP projector, fine-tuned with LoRA on image–program pairs. To overcome data scarcity, the paper adds a structural augmentation stage in which GPT-4o-mini performs genetic crossover between pairs of artist-created programs, yielding 50.4K new graph structures, followed by a parameter perturbation stage that multiplies the set to over 550K materials. Finally, an MCMC-based local search refines node parameters against a VGG-style loss, providing post-optimization without requiring a differentiable Blender graph.

What would settle it

Re-run the paper's evaluation using human raters to label whether each generated render is a meaningful material texture, replacing the 12KB JPEG rule as the validity criterion; if the style-loss ordering or program-correctness numbers shift materially, the claimed advantage is an artifact of the heuristic.

Watch

Extended reading notes

Core claim

The paper's central claim is that a fine-tuned VLM can generate valid, editable Blender procedural materials from a single input image, and that this outperforms prior approaches on both synthetic and real inputs. Under the N=50, K=20 sampling protocol, the method reaches style loss 0.019 on Blender, 0.026 on Substance, and 0.025 on real images, with program correctness above 0.87 on all three sets; the numbers improve further when an MCMC local parameter search refines node parameters after generation. The authors attribute the gains to fine-tuning on a sufficiently diverse dataset, arguing that zero-shot commercial VLMs produce mostly invalid programs and that earlier custom transformers trained on proprietary Substance data do not transfer. The result is a direct corollary of the paper's design: procedural materials can be treated as ordinary code, which is exactly the kind of output a VLM is built to produce.

Load-bearing premise

The 12KB JPEG-size threshold, chosen by inspecting the authors' own renders, is used both to filter training data and to define what counts as a valid generated program, so the reported correctness and win margins depend on this hand-set cutoff.

Editorial extensions

If this is right

  • A user can go from a single photograph to a renderable, editable material graph, something that previously required either proprietary training data or substantial manual expertise.
  • The approach transfers beyond its training distribution: it works on Substance-rendered images and real smartphone photos, where a transformer trained on Substance materials loses ground.
  • The open release of the dataset and code means single-image material generation can be reproduced, compared, and extended by other groups without access to proprietary material repositories.
  • Because the generated programs remain within Blender's node system, post-optimization can adjust colors and structures while preserving editability, with reported style loss dropping further after MCMC refinement.
  • Fine-tuning dominates prompt engineering for this domain: program correctness rises from 0.29 (GPT-4o-mini zero-shot) to 0.91 (fine-tuned VLM) on Blender test materials.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • By extension, any procedural system with a Python API and an expressive node graph—CAD histories, geometry nodes, or other engines' shader graphs—could be attacked with the same fine-tuning-plus-augmentation recipe.
  • A natural next test is to replace the 12KB JPEG validity heuristic with human-rated labels and re-run the comparison; the paper's margins may depend on this threshold, since it defines both training filtration and evaluation success.
  • We would expect the strongest practical gains where editability matters most: measuring artist time to reach a target look, rather than pixel-level style loss, would test whether program structure is the real asset.
  • The observed correctness dip for very short and very long programs suggests an inference-time token mask or length-regularized sampling could extend the method's range without additional training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes VLMaterial, a method that fine-tunes a large vision-language model (LLaVA-NeXT with a LLaMA 3 decoder) to generate Blender procedural material programs, expressed as Python code, from a single input image. The authors contribute a new open-source Blender procedural material dataset, use GPT-4o-mini to perform program-level structure augmentation via crossover and also augment node parameters, and then fine-tune the VLM on roughly 550K image-program pairs. They evaluate on Blender, Substance, and real-image test sets against GPT-4o-mini zero-shot prompting, Conditional MatFormer, BlenderAlchemy, and nearest-neighbor retrieval, and report style loss, sliced Wasserstein distance, CLIP similarity, and program correctness. They also present a user study and ablations. The central claim is that the method outperforms previous methods on both synthetic and real-world examples and produces editable, usable procedural materials.

Significance. If validated, the work is significant because it demonstrates a practical, open-source route to inverse procedural material generation: rather than training a custom graph generator, it fine-tunes a general-purpose VLM to emit executable Python programs for Blender materials. The release of the dataset, code, and model is a concrete contribution, as prior Substance-based datasets could not be shared. The method also gives a clear baseline for future work on VLM-driven visual program synthesis. The paper includes several strengths: held-out evaluation on three datasets, comparison with multiple baselines including a proprietary trained model, a user study with professional artists, ablations isolating structure and parameter augmentation, and an openly available implementation. However, the headline performance claim is not fully supported by the reported numbers on the real-image test set, and the evaluation protocol reuses a hand-tuned JPEG-size threshold in a way that affects the central comparison. These issues are fixable but require a revised presentation and additional robustness analysis.

major comments (3)
  1. [Section 5.2, Table 1] The claim in Section 5.2 that 'Our method outperforms the baselines on all three test datasets' is contradicted by the reported style loss on the real-image test set: Ours is 0.025 while Nearest Neighbor is 0.021, and lower style loss is better. The same pattern appears in Table 3 under the constrained budget (Ours 0.030 vs. Nearest Neighbor 0.021). The abstract's statement that the method 'outperforms previous methods on both synthetic and real-world examples' is therefore too broad without qualification by metric. Please either correct the claim, report the comparison separately per metric, or explain why the style-loss deficit is outweighed by the other metrics and the qualitative/user-study results.
  2. [Section 4.1 and Section 5.1] The 12KB JPEG-size threshold is introduced in Section 4.1 as a heuristic to discard 'meaningless' materials from the training set, chosen after inspecting the authors' own data, and is then reused in Section 5.1 as the definition of a valid generated program. This threshold drives the program-correctness metric and the best-of-K selection of outputs. Because the threshold was not validated against human judgment and no sensitivity analysis is provided, it is unclear how the central comparison would change under reasonable variations of this cutoff. Please report results across a range of threshold values, or provide an independent validation of the validity criterion.
  3. [Section 5.1 and Tables 1-3] The test sets are small (44 Blender, 64 Substance, 64 real images) and all quantitative metrics are reported as point estimates without error bars, confidence intervals, or significance tests. Many margins in Table 1 are small (e.g., Substance style loss 0.026 vs. 0.027 for nearest neighbor), so the claim of systematic improvement is not statistically supported as written. Please add per-image standard errors, bootstrap confidence intervals, or paired significance tests for the main comparisons, and discuss which differences are robust.
minor comments (4)
  1. [Title page] The title appears with a spacing artifact as 'VLM ATERIAL : P ROCEDURAL MATERIAL GENERA - TION...' in the compiled header; please correct the typesetting.
  2. [Appendix B] The heading 'Quantative comparison' should read 'Quantitative comparison'.
  3. [Section 5.1] The nearest-neighbor baseline retrieves materials using the same style loss that is later used as an evaluation metric and as the selection criterion in the best-of-K protocol. This overlap should be acknowledged when interpreting the style-loss comparisons, since it may favor the retrieval baseline on that particular metric.
  4. [Appendix C] The user study is small (16 participants, 12 images) and the preference scores are reported without confidence intervals; please state this limitation explicitly in the main text or appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is evaluated on held-out test images against independent baselines, and the 12KB validity threshold and GPT-4o-mini augmentation are operational choices, not self-derived predictions.

full rationale

The paper's derivation chain is not circular. The central claim is that a fine-tuned VLM can generate Blender procedural material programs from images, and this is tested on held-out Blender, Substance, and real-image sets using several independent baselines (GPT-4o-mini, Conditional MatFormer, BlenderAlchemy, and nearest-neighbor retrieval). The training data is augmented with GPT-4o-mini-generated programs, and GPT-4o-mini is also used as a zero-shot baseline; this is a potential training-data influence or style bias, but it is not a reduction of the evaluated result to the baseline's outputs, because the fine-tuned model is tested on held-out inputs and the baseline is not optimized or used as the test-time predictor. The 12KB JPEG threshold is used both to filter training materials and to define generated-program validity; this is a consistent operational definition rather than a fitted parameter renamed as a prediction, and the main quantitative comparisons are reported before post-optimization using held-out images. Self-citations appear for prior inverse procedural material work and post-optimization methodology, but none is load-bearing in the sense of importing an unverified uniqueness theorem or smuggling in the target result. The strongest issue in the paper is a correctness/consistency concern, not circularity: Table 1 shows the nearest-neighbor baseline achieving style loss 0.021 on real images versus Ours at 0.025, which contradicts the unqualified statement in Section 5.2 that 'Our method outperforms the baselines on all three test datasets.' This is an internal inconsistency in the reported comparison, but it does not make the derivation circular. The paper's core evaluation is self-contained against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical: it depends on the expressive coverage of the curated and augmented Blender dataset, on the perceptual metrics used for evaluation, and on a hand-set validity threshold. No new physical or mathematical entities are introduced; the free parameters are engineering choices that shape the data and evaluation.

free parameters (5)
  • JPEG size threshold = 12 KB
    Used to discard low-complexity materials from training and to define validity of generated programs; set by sorting the collected dataset by rendered-image JPEG size (Section 4.1).
  • Graph size limit = 30 nodes
    Materials with more than 30 nodes are dropped during curation; LLM prompts also forbid more than 30 nodes, constraining the expressiveness of generated programs (Sections 4.1 and A.1).
  • Program token limit = 2048 LLaMA-3 tokens
    Programs longer than this are filtered out of the dataset and augmented samples are trimmed to fit (Sections 4.1 and A.1).
  • Parameter augmentation ranges and probabilities = continuous +/- 25% (min +/- 0.05), HSV hue [0,1], discrete sample prob 0.25, default-reset prob 0.2
    Hand-tuned sampling strategy for creating 10x parameter variations; these choices determine the diversity of augmented appearances (Appendix A.1).
  • MCMC post-optimization settings = Niters=200, p_acc=0.05, 10% sampled params, +/- 20% intervals
    Hand-set local-search budget used to refine predicted materials on the test image (Algorithm 1, Appendix A.3).
assumptions (4)
  • domain assumption Blender 3.3's Python API and node types are a sufficiently expressive and stable representation for matching real material appearances.
    The method only produces programs in this fixed API; the failure cases in Figure 7 show where expressiveness runs out.
  • domain assumption Style loss, SWD, and CLIP similarity computed between flat renders and the input image are adequate proxies for perceptual material match.
    All quantitative conclusions and the best-of-K selection use these metrics; they may not transfer to arbitrary geometry, lighting, or surface scale.
  • ad hoc to paper The augmented dataset produced by GPT-4o-mini crossover is representative of the test distribution and does not leak held-out material content.
    Structure augmentation assumes LLM-mixed programs remain valid, diverse materials; no leakage check between crossover outputs and held-out test materials is described.
  • ad hoc to paper The 12KB JPEG-size heuristic correctly separates meaningful from meaningless material appearances.
    It is a data-dependent threshold that controls which materials enter training and which generated programs count as valid; human validation is not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VLMaterial: Procedural Material Generation with Large Vision-Language Models." pith.science (2026). https://pith.science/paper/SVY5RYGJ

@misc{pith2026250118623,
  author       = {Pith},
  title        = {Pith review of: VLMaterial: Procedural Material Generation with Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SVY5RYGJ}},
  note         = {Machine review of arXiv:2501.18623}
}
read the original abstract

Procedural materials, represented as functional node graphs, are ubiquitous in computer graphics for photorealistic material appearance design. They allow users to perform intuitive and precise editing to achieve desired visual appearances. However, creating a procedural material given an input image requires professional knowledge and significant effort. In this work, we leverage the ability to convert procedural materials into standard Python programs and fine-tune a large pre-trained vision-language model (VLM) to generate such programs from input images. To enable effective fine-tuning, we also contribute an open-source procedural material dataset and propose to perform program-level augmentation by prompting another pre-trained large language model (LLM). Through extensive evaluation, we show that our method outperforms previous methods on both synthetic and real-world examples.

Figures

Figures reproduced from arXiv: 2501.18623 by the authors.

Figure 1
Figure 1. Given single input images (left), our model generates corresponding procedural materials [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Workflow overview. Given an input image, we use our fine-tuned VLM to predict a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. A qualitative comparison of our method with several baselines. The input images are [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Visual examples for ablation study. grams with sufficient capacity to represent the input image, the post-optimization algorithm refines the overall color and spatial structures, achieving noticeably better alignment with the input. 6 DISCUSSION AND CONCLUSION [PITH_F…
Figure 5
Figure 5. Figure 5: More qualitative comparisons with baselines. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: More examples for ablation study. Input Ours Input Ours Input Ours [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Failure cases. The first two examples (Column 1-4) come from the [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: The average style loss and program correctness of VLM-generated materials with varied [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 42 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Adobe Substance 3D Designer, 2024

    Adobe. Adobe Substance 3D Designer, 2024. URL https://www.adobe.com/products/substance3d

    work page 2024

  3. [3]

    Blender - a 3D modelling and rendering package

    Blender. Blender - a 3D modelling and rendering package. Blender Foundation, 2024 a . URL http://www.blender.org

    work page 2024

  4. [4]

    Blender Python API

    Blender. Blender Python API. Blender Foundation, 2024 b . URL https://docs.blender.org/api/current

    work page 2024

  5. [5]

    Sliced and radon wasserstein barycenters of measures

    Nicolas Bonneel, Julien Rabin, Gabriel Peyr \'e , and Hanspeter Pfister. Sliced and radon wasserstein barycenters of measures. Journal of Mathematical Imaging and Vision, 51: 0 22--45, 2015

    work page 2015

  6. [6]

    Deepsvg: A hierarchical generative network for vector graphics animation

    Alexandre Carlier, Martin Danelljan, Alexandre Alahi, and Radu Timofte. Deepsvg: A hierarchical generative network for vector graphics animation. Advances in Neural Information Processing Systems, 33: 0 16351--16361, 2020

    work page 2020

  7. [7]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023

    arXiv 2023

  8. [8]

    Single-image svbrdf capture with a rendering-aware deep network

    Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and Adrien Bousseau. Single-image svbrdf capture with a rendering-aware deep network. ACM Transactions on Graphics (ToG), 37 0 (4): 0 1--15, 2018

    work page 2018

Show all 78 references
  1. [9]

    The markov chain monte carlo revolution

    Persi Diaconis. The markov chain monte carlo revolution. Bulletin of the American Mathematical Society, 46: 0 179--205, 2009. URL https://doi.org/10.1090/S0273-0979-08-01238-X

    2009 doi

  2. [10]

    Inversecsg: Automatic conversion of 3d models to csg trees

    Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik. Inversecsg: Automatic conversion of 3d models to csg trees. ACM Transactions on Graphics (TOG), 37 0 (6): 0 1--16, 2018

    2018

  3. [11]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  4. [12]

    Learning to infer graphics programs from hand-drawn images

    Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, and Josh Tenenbaum. Learning to infer graphics programs from hand-drawn images. Advances in neural information processing systems, 31, 2018

    2018

  5. [13]

    Write, execute, assess: Program synthesis with a repl

    Kevin Ellis, Maxwell Nye, Yewen Pu, Felix Sosa, Josh Tenenbaum, and Armando Solar-Lezama. Write, execute, assess: Program synthesis with a repl. Advances in Neural Information Processing Systems, 32, 2019

    2019

  6. [14]

    Dreamcoder: growing generalizable, interpretable knowledge with wake--sleep bayesian program learning

    Kevin Ellis, Lionel Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lore Anaya Pozo, Luke Hewitt, Armando Solar-Lezama, and Joshua B Tenenbaum. Dreamcoder: growing generalizable, interpretable knowledge with wake--sleep bayesian program learning. Philosophical Transactions o...

    2023

  7. [15]

    Unreal Engine

    Epic. Unreal Engine. Epic Games, 2024. URL https://www.unrealengine.com

    2024

  8. [16]

    Deep inverse rendering for high-resolution svbrdf estimation from an arbitrary number of images

    Duan Gao, Xiao Li, Yue Dong, Pieter Peers, Kun Xu, and Xin Tong. Deep inverse rendering for high-resolution svbrdf estimation from an arbitrary number of images. ACM Trans. Graph., 38 0 (4): 0 134--1, 2019

    2019

  9. [17]

    Texture synthesis using convolutional neural networks

    Leon Gatys, Alexander S Ecker, and Matthias Bethge. Texture synthesis using convolutional neural networks. Advances in neural information processing systems, 28, 2015

    2015

  10. [18]

    A neural algorithm of artistic style

    Leon A Gatys. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015

    2015 arXiv

  11. [19]

    Practical markov chain monte carlo

    Charles J Geyer. Practical markov chain monte carlo. Statistical science, pp.\ 473--483, 1992

    1992

  12. [20]

    Matformer: A generative model for procedural materials

    Paul Guerrero, Milo s Ha s an, Kalyan Sunkavalli, Radom \' r M e ch, Tamy Boubekeur, and Niloy J Mitra. Matformer: A generative model for procedural materials. arXiv preprint arXiv:2207.01044, 2022

    2022 arXiv

  13. [21]

    Highlight-aware two-stream network for single-image svbrdf acquisition

    Jie Guo, Shuichang Lai, Chengzhi Tao, Yuelong Cai, Lei Wang, Yanwen Guo, and Ling-Qi Yan. Highlight-aware two-stream network for single-image svbrdf acquisition. 2021

    2021

  14. [22]

    Materialgan: Reflectance capture using a generative svbrdf model

    Yu Guo, Cameron Smith, Milo s Ha s an, Kalyan Sunkavalli, and Shuang Zhao. Materialgan: Reflectance capture using a generative svbrdf model. arXiv preprint arXiv:2010.00114, 2020

    2010 arXiv

  15. [23]

    A sliced wasserstein loss for neural texture synthesis

    Eric Heitz, Kenneth Vanhoey, Thomas Chambon, and Laurent Belcour. A sliced wasserstein loss for neural texture synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 9412--9420, 2021

    2021

  16. [24]

    Generative modelling of brdf textures from flash images

    Philipp Henzler, Valentin Deschaintre, Niloy J Mitra, and Tobias Ritschel. Generative modelling of brdf textures from flash images. arXiv preprint arXiv:2102.11861, 2021

    2021 arXiv

  17. [25]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  18. [26]

    A novel framework for inverse procedural texture modeling

    Yiwei Hu, Julie Dorsey, and Holly Rushmeier. A novel framework for inverse procedural texture modeling. ACM Transactions on Graphics (ToG), 38 0 (6): 0 1--14, 2019

    2019

  19. [27]

    Node graph optimization using differentiable proxies

    Yiwei Hu, Paul Guerrero, Milos Hasan, Holly Rushmeier, and Valentin Deschaintre. Node graph optimization using differentiable proxies. In ACM SIGGRAPH 2022 conference proceedings, pp.\ 1--9, 2022 a

    2022

  20. [28]

    An inverse procedural modeling pipeline for svbrdf maps

    Yiwei Hu, Chengan He, Valentin Deschaintre, Julie Dorsey, and Holly Rushmeier. An inverse procedural modeling pipeline for svbrdf maps. ACM Transactions on Graphics (TOG), 41 0 (2): 0 1--17, 2022 b

    2022

  21. [29]

    Generating procedural materials from text or image prompts

    Yiwei Hu, Paul Guerrero, Milos Hasan, Holly Rushmeier, and Valentin Deschaintre. Generating procedural materials from text or image prompts. In ACM SIGGRAPH 2023 Conference Proceedings, pp.\ 1--11, 2023

    2023

  22. [30]

    Blenderalchemy: Editing 3d graphics with vision-language models

    Ian Huang, Guandao Yang, and Leonidas Guibas. Blenderalchemy: Editing 3d graphics with vision-language models. arXiv preprint arXiv:2404.17672, 2024

    2024 arXiv

  23. [31]

    Inferring and executing programs for visual reasoning

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Judy Hoffman, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Inferring and executing programs for visual reasoning. In Proceedings of the IEEE international conference on computer vision, pp.\ 2989--2998, 2017

    2017

  24. [32]

    Shapeassembly: Learning to generate programs for 3d shape structure synthesis

    R Kenny Jones, Theresa Barton, Xianghao Xu, Kai Wang, Ellen Jiang, Paul Guerrero, Niloy J Mitra, and Daniel Ritchie. Shapeassembly: Learning to generate programs for 3d shape structure synthesis. ACM Transactions on Graphics (TOG), 39 0 (6): 0 1--20, 2020

    2020

  25. [33]

    Ucsg-net-unsupervised discovering of constructive solid geometry tree

    Kacper Kania, Maciej Zieba, and Tomasz Kajdanowicz. Ucsg-net-unsupervised discovering of constructive solid geometry tree. Advances in neural information processing systems, 33: 0 8776--8786, 2020

    2020

  26. [34]

    End-to-end procedural material capture with proxy-free mixed-integer optimization

    Beichen Li, Liang Shi, and Wojciech Matusik. End-to-end procedural material capture with proxy-free mixed-integer optimization. ACM Transactions on Graphics (TOG), 42 0 (4): 0 1--15, 2023

    2023

  27. [35]

    Procedural material generation with reinforcement learning

    Beichen Li, Yiwei Hu, Paul Guerrero, Milos Hasan, Liang Shi, Valentin Deschaintre, and Wojciech Matusik. Procedural material generation with reinforcement learning. ACM Trans. Graph., 43 0 (6), November 2024 a . ISSN 0730-0301. doi:10.1145/3687979. URL https://doi.org/10.1145/3687979

    2024 doi

  28. [36]

    Llava-next: Stronger llms supercharge multimodal capabilities in the wild, May 2024 b

    Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. Llava-next: Stronger llms supercharge multimodal capabilities in the wild, May 2024 b . URL https://llava-vl.github.io/blog/2024-05-10-llava-next-stronger-llms/

    2024

  29. [37]

    Free2cad: Parsing freehand drawings into cad commands

    Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J Mitra. Free2cad: Parsing freehand drawings into cad commands. ACM Transactions on Graphics (TOG), 41 0 (4): 0 1--16, 2022

    2022

  30. [38]

    Modeling surface appearance from a single photograph using self-augmented convolutional neural networks

    Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. Modeling surface appearance from a single photograph using self-augmented convolutional neural networks. ACM Transactions on Graphics (ToG), 36 0 (4): 0 1--11, 2017

    2017

  31. [39]

    Materials for masses: Svbrdf acquisition with a single mobile phone image

    Zhengqin Li, Kalyan Sunkavalli, and Manmohan Chandraker. Materials for masses: Svbrdf acquisition with a single mobile phone image. In Proceedings of the European conference on computer vision (ECCV), pp.\ 72--87, 2018

    2018

  32. [40]

    Improved baselines with visual instruction tuning, 2023 a

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023 a

    2023

  33. [41]

    Visual instruction tuning, 2023 b

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023 b

    2023

  34. [42]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  35. [43]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  36. [44]

    Materia: Single image high-resolution material capture in the wild

    Rosalie Martin, Arthur Roullier, Romain Rouffet, Adrien Kaiser, and Tamy Boubekeur. Materia: Single image high-resolution material capture in the wild. In Computer Graphics Forum, volume 41, pp.\ 163--177. Wiley Online Library, 2022

    2022

  37. [45]

    Is self-repair a silver bullet for code generation? In The Twelfth International Conference on Learning Representations, 2023

    Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. Is self-repair a silver bullet for code generation? In The Twelfth International Conference on Learning Representations, 2023

    2023

  38. [46]

    GPT-4o, 2024 a

    OpenAI. GPT-4o, 2024 a . URL https://openai.com/index/hello-gpt-4o/

    2024

  39. [47]

    GPT-4o-mini, 2024 b

    OpenAI. GPT-4o-mini, 2024 b . URL https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence

    2024

  40. [48]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

    2021

  41. [49]

    Infinite photorealistic worlds using procedural generation

    Alexander Raistrick, Lahav Lipson, Zeyu Ma, Lingjie Mei, Mingzhe Wang, Yiming Zuo, Karhan Kayan, Hongyu Wen, Beining Han, Yihan Wang, et al. Infinite photorealistic worlds using procedural generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern rec...

    2023

  42. [50]

    Zero: Memory optimizations toward training trillion parameter models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp.\ 1--16. IEEE, 2020

    2020

  43. [51]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.\ 35...

    2020

  44. [52]

    Im2vec: Synthesizing vector graphics without vector supervision

    Pradyumna Reddy, Michael Gharbi, Michal Lukac, and Niloy J Mitra. Im2vec: Synthesizing vector graphics without vector supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 7342--7351, 2021

    2021

  45. [53]

    Starvector: Generating scalable vector graphics code from images

    Juan A Rodriguez, Shubham Agarwal, Issam H Laradji, Pau Rodriguez, David Vazquez, Christopher Pal, and Marco Pedersoli. Starvector: Generating scalable vector graphics code from images. arXiv preprint arXiv:2312.11556, 2023

    2023 arXiv

  46. [54]

    Mathematical discoveries from program search with large language models

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 625 0 (7...

    2024

  47. [55]

    Stochastic superoptimization

    Eric Schkufza, Rahul Sharma, and Alex Aiken. Stochastic superoptimization. ACM SIGARCH Computer Architecture News, 41 0 (1): 0 305--316, 2013

    2013

  48. [56]

    Csgnet: Neural shape parser for constructive solid geometry

    Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. Csgnet: Neural shape parser for constructive solid geometry. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 5515--5523, 2018

    2018

  49. [57]

    Match: Differentiable material graphs for procedural material capture

    Liang Shi, Beichen Li, Milo s Ha s an, Kalyan Sunkavalli, Tamy Boubekeur, Radomir Mech, and Wojciech Matusik. Match: Differentiable material graphs for procedural material capture. ACM Transactions on Graphics (TOG), 39 0 (6): 0 1--15, 2020

    2020

  50. [58]

    3d-gpt: Procedural 3d modeling with large language models

    Chunyi Sun, Junlin Han, Weijian Deng, Xinlong Wang, Zishan Qin, and Stephen Gould. 3d-gpt: Procedural 3d modeling with large language models. arXiv preprint arXiv:2310.12945, 2023

    2023 arXiv

  51. [59]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  52. [60]

    Learning to infer and execute 3d shape programs

    Yonglong Tian, Andrew Luo, Xingyuan Sun, Kevin Ellis, William T Freeman, Joshua B Tenenbaum, and Jiajun Wu. Learning to infer and execute 3d shape programs. arXiv preprint arXiv:1901.02875, 2019

    1901 arXiv

  53. [61]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  54. [62]

    Point2cyl: Reverse engineering 3d objects from point clouds to extrusion cylinders

    Mikaela Angelina Uy, Yen-Yu Chang, Minhyuk Sung, Purvi Goel, Joseph G Lambourne, Tolga Birdal, and Leonidas J Guibas. Point2cyl: Reverse engineering 3d objects from point clouds to extrusion cylinders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2022

  55. [63]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Info...

    2017

  56. [64]

    Surfacenet: Adversarial svbrdf estimation from a single image

    Giuseppe Vecchio, Simone Palazzo, and Concetto Spampinato. Surfacenet: Adversarial svbrdf estimation from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 12840--12848, 2021

    2021

  57. [65]

    Controlmat: A controlled generative approach to material capture

    Giuseppe Vecchio, Rosalie Martin, Arthur Roullier, Adrien Kaiser, Romain Rouffet, Valentin Deschaintre, and Tamy Boubekeur. Controlmat: A controlled generative approach to material capture. ACM Transactions on Graphics, 43 0 (5): 0 1--17, 2024 a

    2024

  58. [66]

    Matfuse: controllable material generation with diffusion models

    Giuseppe Vecchio, Renato Sortino, Simone Palazzo, and Concetto Spampinato. Matfuse: controllable material generation with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 4429--4438, 2024 b

    2024

  59. [67]

    Optimal transport: old and new, volume 338

    C \'e dric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009

    2009

  60. [68]

    Deepcad: A deep generative network for computer-aided design models

    Rundi Wu, Chang Xiao, and Changxi Zheng. Deepcad: A deep generative network for computer-aided design models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 6772--6782, 2021

    2021

  61. [69]

    Hierarchical neural coding for controllable cad model generation

    Xiang Xu, Pradeep Kumar Jayaraman, Joseph G Lambourne, Karl DD Willis, and Yasutaka Furukawa. Hierarchical neural coding for controllable cad model generation. arXiv preprint arXiv:2307.00149, 2023

    2023 arXiv

  62. [70]

    Inferring cad modeling sequences using zone graphs

    Xianghao Xu, Wenzhe Peng, Chin-Yi Cheng, Karl DD Willis, and Daniel Ritchie. Inferring cad modeling sequences using zone graphs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 6062--6070, 2021

    2021

  63. [71]

    L3go: Language agents with chain-of-3d-thoughts for generating unconventional objects

    Yutaro Yamada, Khyathi Chandu, Yuchen Lin, Jack Hessel, Ilker Yildirim, and Yejin Choi. L3go: Language agents with chain-of-3d-thoughts for generating unconventional objects. arXiv preprint arXiv:2402.09052, 2024

    2024 arXiv

  64. [72]

    Holodeck: Language guided generation of 3d embodied ai environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  65. [73]

    Adversarial single-image svbrdf estimation with hybrid training

    Xilong Zhou and Nima Khademi Kalantari. Adversarial single-image svbrdf estimation with hybrid training. In Computer Graphics Forum, volume 40, pp.\ 315--325. Wiley Online Library, 2021

    2021

  66. [74]

    Tilegen: Tileable, controllable material generation and capture

    Xilong Zhou, Milos Hasan, Valentin Deschaintre, Paul Guerrero, Kalyan Sunkavalli, and Nima Khademi Kalantari. Tilegen: Tileable, controllable material generation and capture. In SIGGRAPH Asia 2022 conference papers, pp.\ 1--9, 2022

    2022

  67. [75]

    Photomat: A material generator learned from single flash photos

    Xilong Zhou, Milos Hasan, Valentin Deschaintre, Paul Guerrero, Yannick Hold-Geoffroy, Kalyan Sunkavalli, and Nima Khademi Kalantari. Photomat: A material generator learned from single flash photos. In ACM SIGGRAPH 2023 Conference Proceedings, pp.\ 1--11, 2023

    2023

  68. [76]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  69. [77]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  70. [78]

    A" or "B

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.