MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models

Biao Zhang; John Femiani; Michael Birsak; Peter Wonka

REVIEW 5 major objections 5 minor 21 references

MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MatCLIP learns a material descriptor that is stable across shapes and lighting, and uses it to assign PBR materials to 3D objects from RGB images, including images generated by latent diffusion models, with top-1 accuracy of 76.69%, over…

desk verdict A practical CLIP-based material matcher whose headline lighting-invariance claim is undercut by a same-lighting evaluation; the method is solid, the claim needs retesting. read the letter →

arxiv 2501.15981 v2 pith:KYJRN7JV submitted 2025-01-27 cs.CV cs.GRcs.LG

Michael Birsak , John Femiani , Biao Zhang , Peter Wonka This is my paper · ORCID

classification cs.CVcs.GRcs.LG

keywords PBRmaterialsmaterialassignmentAlpha-CLIPCLIPembeddinglatentdiffusionmodelsshapeinvariancelighting3Dtexturing

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes MatCLIP, a method that learns a descriptor for physically based rendering (PBR) materials that stays stable when the material is viewed on different shapes and under different studio lights. The descriptor is built by encoding 42 renderings of each material—6 primitive shapes under 7 HDR environment maps—and passing the resulting CLIP embeddings through a transformer; a second encoder reads masked parts of an RGB image. A contrastive loss aligns the two embeddings. The authors report 76.69% top-1 accuracy on the MatSynth material database, more than 15 percentage points above prior methods PhotoShape and MatAtlas, and use Stable Diffusion or Flux images to assign plausible materials to parts of 3D chairs. If the claim holds, it offers a reproducible, open-data route from a single image or a diffusion-model output to physically based textures on 3D shapes.

What carries the argument

The load-bearing object is the material descriptor: for each material, 42 renderings are produced (6 primitive shapes—cone, cylinder, plane, sphere, Suzanne, torus—under 7 studio HDR environment maps, with a fixed UV mapping and a frontal 100 mm camera). Each rendering is cropped to its largest inscribed rectangle of pure texture, embedded with a CLIP image encoder, and the 42 embeddings form a sequence that a transformer (built from Alpha-CLIP's text-encoder transformer) compresses into a single material vector. A parallel part encoder, based on Alpha-CLIP's image encoder with alpha-channel masking, embeds a masked part of the input image; contrastive learning aligns the two. The fixed UV mapping and studio lighting are deliberate simplifications that keep the sampled appearance space small enough to train.

What would settle it

Render 100 materials held out from training on a real chair mesh under 50 non-studio HDRIs (mixed indoor/outdoor, colored lights) with randomized UV layouts, and measure MatCLIP's top-1 retrieval accuracy against the ground-truth material. The paper's unseen-lighting result is 35.31%; if the same protocol produces accuracy below 20%—near the untrained baselines—the claimed shape- and lighting-insensitivity is refuted.

Watch

Extended reading notes

Core claim

MatCLIP's central claim is that a PBR material can be represented as a set of appearance tokens—CLIP image embeddings of the material rendered on several primitive shapes under several studio lighting environments—and that a transformer over those tokens yields an embedding that is insensitive to shape and lighting yet still lives in CLIP space. Combined with an Alpha-CLIP-based part encoder that reads a masked region of an input image, contrastive training aligns material embeddings with part embeddings so that the nearest material in the database matches the part's appearance. The paper reports that this setup achieves 76.69% top-1 and 89.02% top-5 classification accuracy on MatSynth over the full material database, outperforming PhotoShape (best 60.86%) and MatAtlas (best 18.77%) by a wide margin, and that it generalizes to unseen shape categories (54.74% top-1) and unseen lighting (35.31% top-1). It also shows subjective preference for materials assigned via MatCLIP over procedural assignment in 77.37% of paired comparisons.

Load-bearing premise

The whole approach rests on the assumption that the 42 fixed renderings per material—six primitive shapes, seven studio lighting maps, one UV layout, one camera angle—span the appearance variation found in real photographs and in diffusion-model outputs.

Editorial extensions

If this is right

Material assignment for large 3D shape collections can be automated from a single RGB image or a latent-diffusion-generated image, producing coherent results across the parts of an object.
Because the descriptor lives in CLIP space, it can be combined with text-based guidance and zero-shot applications without retraining the core model.
Evaluation on non-chair shape categories of 3DCoMPaT++ shows the descriptor transfers beyond the chair shapes used in training, with 54.74% top-1 accuracy on unseen shapes.
Releasing the code and data makes the baseline reproducible and lets others extend the material database, shapes, or lighting sets while keeping the same training pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the same descriptor could be used in inverse-rendering or material-editing loops where relighting consistency matters; the paper does not test whether nearest-neighbor matches stay stable under continuous light changes.
The fixed UV-mapping assumption may break for real-world scanned meshes with arbitrary UV layouts; a testable extension would be to randomize UV layouts during training and measure the accuracy gain on such meshes.
The paper's own ablation suggests that diversity is the main driver of robustness, so training on a broader set of HDR maps and shapes should push unseen-lighting accuracy well above the reported 35.31%.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, two things to know. The paper is a solid, practical extension of CLIP for PBR material assignment: it encodes each material as a set of 42 renderings across 6 shapes and 7 HDRs, feeds those through a CLIP encoder plus a transformer, and aligns them with an Alpha-CLIP part encoder using contrastive loss. That combination is genuinely new relative to MatAtlas, VLF, and PSDR-Room. The other thing: the headline “light- and shape-insensitive” is not yet supported. The main 76.69% number comes from a closed material set where query renderings are lit by the same seven HDRs used for the material database. When the authors actually hold out lighting (Table 1, Unseen Lighting), top-1 drops to 35.31%. That pattern is consistent with the model learning lighting-specific cues rather than a truly invariant material signature.

What the paper does well: it relies on public datasets, discloses its baseline modifications (MatAtlas without ChatGPT class restriction, PhotoShape with a newer backbone), and the ablation shows that diversity in shapes and environments is what drives accuracy. The user study gives some evidence the method produces plausible assignments in practice. The writing is clear about the closed-set design and the drop on unseen materials, though the abstract overstates things.

Soft spots, in order of importance. First, the lighting confound above — the central contribution is described as lighting-insensitive, but the main experiment never tests that. Second, no error bars or seeds; differences of 15 points are likely real, but we can't judge variance. Third, the unseen-materials evaluation is substance-only on ~400 manual annotations, and the authors themselves note the gap is narrower and the annotation uncertain; it's not a strong generalization test. Minor: Table 2's full model validation accuracy (60.54) differs from the main test result (76.69), presumably a different split, but it's not explained.

Who's this for? Researchers working on material assignment, texture transfer, or 3D content pipelines. It deserves a serious referee, but with a clear request: repeat runs with seeds, unmodified baselines, and ideally training on more HDRs to back the invariance claim, or a rewritten abstract that conditions the claim on studio lighting.

Referee Report

5 major / 5 minor

Summary. The paper proposes MatCLIP, a method that extends Alpha-CLIP to learn material descriptors from 42 renderings per PBR material (6 primitive shapes under 7 studio HDR maps) and aligns these descriptors with masked part images from 3DCoMPaT++ using a contrastive loss. The authors report a top-1 material-classification accuracy of 76.69% on a closed material set, compare against PhotoShape, OpenCLIP, and MatAtlas baselines, and additionally present qualitative material-assignment results for Stable Diffusion and Flux outputs, together with an MTurk user study. The central claim is that the learned descriptors are shape- and lighting-insensitive and enable plausible PBR material assignment to 3D shapes from RGB images.

Significance. If the central claim were fully supported, MatCLIP would be a practically useful tool for assigning high-resolution PBR materials to 3D shape collections, and the reliance on public datasets plus the planned code/data release is a genuine strength. The user study and qualitative results with multiple generative models are also valuable. However, the paper's headline claim of lighting insensitivity is not supported by the main experimental protocol, because the main evaluation uses the same seven HDR maps for material renderings and query images, while the only held-out-lighting experiment shows a 41-percentage-point drop. The evaluation is also conducted on a closed material set, so the claimed generalization to unseen material distributions is only qualitatively demonstrated. These gaps are load-bearing because the abstract and introduction frame the method as practical for arbitrary photographs and latent-diffusion outputs.

major comments (5)

[Section 3.1, Table 1, Section 4.3] The headline accuracy is measured under a same-lighting condition. Section 3.1 states that the chair part images are rendered under the same set of HDR environment maps used in the material database, and Section 4.3's unseen-lighting experiment renders queries under 769 Poly Haven maps and reports a top-1 accuracy of 35.31%, compared with 76.69% in the main evaluation. The abstract's claim of a 'lighting-insensitive' descriptor is therefore not supported by the main evaluation. The paper should either make the unseen-lighting result the basis for the robustness claim or explicitly condition the abstract and conclusion on the seven training HDR maps.
[Section 4 and Table 1] The quantitative evaluation is a closed-material-set evaluation. Section 4 acknowledges that all materials are included in both training and test sets, so the main, unseen-shapes, and unseen-lighting columns in Table 1 measure retrieval among materials already seen in training. The 'Unseen Materials' column is a substance-only manual annotation evaluation, and the table's own footnote states that its values are not directly comparable. Consequently, the abstract's and Section 5's statements about generalization to unseen materials and unseen material distributions are not quantitatively supported.
[Table 2 and Section 4.5] The ablation study reports a full-model validation accuracy of 60.54% in Table 2, while Section 4.1 and Table 1 report a main-evaluation accuracy of 76.69% for the same configuration. The paper does not explain whether these numbers come from different training durations, splits, or evaluation protocols. Since Section 4.5 says the ablation was monitored over the first 9,000 training steps, the table is hard to interpret, and the discrepancy should be resolved before the ablation is used to support the importance of shape and lighting diversity.
[Section 4.1] The MatAtlas comparison is weakened by the evaluation protocol. The paper states that MatAtlas was evaluated without the ChatGPT-suggested class restriction, and that this adjustment reduced MatAtlas's performance. Reporting only this deliberately handicapped variant makes the 'outperforming MatAtlas by over 15 percentage points' claim in the abstract less persuasive. The authors should also report MatAtlas under its intended protocol, or explicitly state why the restricted protocol is not a fair baseline for the comparison.
[Section 4] No error bars, confidence intervals, or random seeds are reported for any of the quantitative results. Given the large drop in the unseen-lighting condition (76.69% to 35.31%) and the multiple dataset-filtering steps described in Sections 4.2 and 4.3, the paper should report variability across seeds or data subsets to establish that the reported differences, especially the claimed margin over baselines, are stable.

minor comments (5)

[Abstract and Section 4.1] The abstract reports 76.6% top-1 accuracy while Section 4.1 reports 76.69%; the precision should be made consistent.
[Table 2] The caption of Table 2 uses 'Val. Acc.' without defining the validation split; please specify what data the values are computed on.
[Section 4.4] The manual-annotation evaluation reports about 400 annotations but no inter-annotator agreement; since the paper acknowledges the task is ambiguous, adding an agreement measure would make the unseen-materials numbers more interpretable.
[Section 3.1 and Figure 2] The text refers to 'Figure 2 (left)' and 'Fig. 2 (left)' for different aspects of the pipeline; labeling the subfigures explicitly (e.g., Figure 2a, Figure 2b) would improve readability.
[Section 5 and Figure 4] The limitations section discusses failure cases where Stable Diffusion or Flux adds geometry, but Figure 4 shows only two examples; a small quantitative account of how often such failures occur would be useful.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity found; the derivation is an empirical contrastive-learning pipeline. The main caveat is that the headline accuracy is measured under the same 7 HDR maps used in training, while the only held-out-lighting test drops sharply, weakening the 'lighting-insensitive' claim but not making it circular.

full rationale

MatCLIP's derivation chain is self-contained: it trains a part encoder and a material encoder with a contrastive loss on rendered images, then ranks materials by cosine similarity. No equation, theorem, or descriptor is defined in terms of the target result, and no fitted parameter is renamed as a prediction. The one substantive caveat is the lighting setup. Section 3.1 fixes 7 Poly Haven HDR maps for the material database, and the Training Data paragraph states that part images are rendered 'under the same set of HDR environment maps used in the material database.' The headline 76.69% top-1 accuracy in Table 1 is therefore a same-lighting retrieval result; the only held-out-lighting test in Section 4.3 drops to 35.31% top-1. This weakens the abstract claim of lighting insensitivity, but the paper reports the held-out number honestly rather than substituting it with the training-condition number, so this is a generalization/validity limitation, not circularity. The 3DCoMPaT++ citations [Li et al. 2022; Slim et al. 2023] are co-authored by P. Wonka, but the dataset is public and used as an external benchmark, not as an unverified premise supporting the central argument. Section 5 also candidly flags that generalization to unseen material distributions remains an open challenge. No self-citation chain or uniqueness import is load-bearing, and the CLIP/Alpha-CLIP foundations are external. Overall, no circular derivation is exhibited; at most there is a minor co-authored benchmark citation and an overbroad claim drawn from a same-lighting evaluation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

All free parameters are choices in the rendering and training pipeline rather than physical constants; they directly determine the material descriptor, so the main empirical claim depends on them. No invented physical entities, mediators, dimensions, or conserved quantities are introduced. The listed axioms are empirical preconditions about render coverage, UV invariance, transfer learning, LDM proxy quality, and annotation reliability.

free parameters (6)

n_env = 7
Section 3.1: number of HDR environment maps chosen by hand from Poly Haven; all material renderings and training images use these maps, so the learned descriptor's lighting invariance is defined by this count.
n_shapes = 6
Section 3.1: base shapes chosen by hand (cone, cylinder, plane, sphere, Suzanne, torus); 42 renderings per material is 6 shapes times 7 maps, so the descriptor space is shaped by these primitives.
focal_length = 100 mm
Section 3.1: camera focal length chosen to mimic product photography; all renderings use this setting, so the model may be calibrated to this camera rather than being truly shape-invariant.
batch_size = 256
Section 4: contrastive training batch size inherited from Alpha-CLIP training; it affects the contrastive alignment quality.
learning_rate = 1.0e-4
Section 4: chosen learning rate for training both encoders with the contrastive loss.
training_steps = not stated (48 hours; ablations at 9,000 steps)
Final model is trained about 48 hours on 4 A100s; Table 2 reports validation at 9,000 steps, so the ablation accuracies are not directly comparable to the main results.

assumptions (5)

domain assumption Fixed UV mapping with physical scale for all material renderings is sufficient; the model does not need to be invariant to UV mapping.
Section 3.1 states consistent UV-mappings ensure physical scale and avoid needing robustness to varying UV-mappings. Real 3D assets often have arbitrary UV layouts, so this is an empirical precondition for the descriptor's invariance claim.
domain assumption Six primitive shapes plus seven studio HDR maps adequately sample the material appearance space.
Sections 3.1 and 3.2 define the material descriptor as the set of 42 renderings. If this set misses important lighting or geometry conditions, the learned invariance will be incomplete.
domain assumption Alpha-CLIP's pretrained encoders provide sufficiently rich features and can be fine-tuned with contrastive loss.
Sections 3.3 and 4 build directly on Alpha-CLIP rather than deriving a new backbone; this is a standard transfer-learning assumption.
domain assumption Stable Diffusion and Flux outputs conditioned on depth, edges, and text are plausible proxy target images for material assignment.
Section 3.4 uses LDM outputs as target images because true material assignments are unknown. Downstream accuracy depends on these generated images containing material information rather than geometry artifacts, which the authors themselves note in Section 5.
domain assumption Manual annotations of 400 LDM output parts are reliable enough to serve as ground truth for unseen materials.
Section 4.4 describes a GUI annotation procedure with about 400 samples. The paper concedes this task is difficult and subjective, so this ground truth is weaker than dataset labels.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models." pith.science (2026). https://pith.science/paper/KYJRN7JV

@misc{pith2026250115981,
  author       = {Pith},
  title        = {Pith review of: MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KYJRN7JV}},
  note         = {Machine review of arXiv:2501.15981}
}

read the original abstract

Assigning realistic materials to 3D models remains a significant challenge in computer graphics. We propose MatCLIP, a novel method that extracts shape- and lighting-insensitive descriptors of Physically Based Rendering (PBR) materials to assign plausible textures to 3D objects based on images, such as the output of Latent Diffusion Models (LDMs) or photographs. Matching PBR materials to static images is challenging because the PBR representation captures the dynamic appearance of materials under varying viewing angles, shapes, and lighting conditions. By extending an Alpha-CLIP-based model on material renderings across diverse shapes and lighting, and encoding multiple viewing conditions for PBR materials, our approach generates descriptors that bridge the domains of PBR representations with photographs or renderings, including LDM outputs. This enables consistent material assignments without requiring explicit knowledge of material relationships between different parts of an object. MatCLIP achieves a top-1 classification accuracy of 76.6%, outperforming state-of-the-art methods such as PhotoShape and MatAtlas by over 15 percentage points on publicly available datasets. Our method can be used to construct material assignments for 3D shape datasets such as ShapeNet, 3DCoMPaT++, and Objaverse. All code and data will be released.

Figures

Figures reproduced from arXiv: 2501.15981 by the authors.

**Figure 2.** (left) Our method learns robust material descriptors through diverse renderings across geometric shapes and lighting [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Examples (in each row from left to right) of 3D mod [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: MatCLIP struggles when Stable Diffusion (left side) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Matching materials from Stable Diffusion output images to materials of our material database using our MatCLIP [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Material matching results using MatCLIP on chair images generated by Flux [Labs 2023]. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: Material matching results using MatCLIP on images of various shape categories from 3DCoMPaT++ [Li et al [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 5 canonical work pages

[2]

arXiv:2404.02899 [cs.CV] Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner

MatAt- las: Text-driven Consistent Geometry Texturing and Material Assignment. arXiv:2404.02899 [cs.CV] Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner

arXiv
[4]

arXiv preprint arXiv:2402.13251 (2024)

FlashTex: Fast Relightable Mesh Texturing with LightControlNet. arXiv preprint arXiv:2402.13251 (2024). Valentin Deschaintre, Julia Guerrero-Viu, Diego Gutierrez, Tamy Boubekeur, and Belen Masia

arXiv 2024
[5]

ACM Trans

The Visual Language of Fabrics. ACM Trans. Graph. 42, 4, Article 50 (jul 2023), 15 pages. https://doi.org/10.1145/3592391 Patrick Esser, Sumith Kulal, A. Blattmann, Rahim Entezari, Jonas Muller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dock- horn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach

doi:10.1145/3592391 2023
[6]

arXiv preprint arXiv:2403.03206 (2024)

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. arXiv preprint arXiv:2403.03206 (2024). https://arxiv.org/abs/2403.03206 Michael Fischer, Iliyan Georgiev, Thibault Groueix, Vladimir G Kim, Tobias Ritschel, and Valentin Deschaintre

arXiv 2024
[7]

arXiv preprint arXiv:2411.19322 (2024)

SAMa: Material-aware 3D Selection and Segmen- tation. arXiv preprint arXiv:2411.19322 (2024). Yu Guo, Cameron Smith, Miloš Hašan, Kalyan Sunkavalli, and Shuang Zhao

arXiv 2024
[9]

arXiv preprint arXiv:2409.12431 (2024)

Flexitex: Enhancing texture generation with visual guidance. arXiv preprint arXiv:2409.12431 (2024). Black Forest Labs

arXiv 2024
[10]

In 2024 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR)

TexOct: Generating Textures of 3D Models with Octree-based Diffusion. In 2024 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) . 4284–4293. https: //doi.org/10.1109/CVPR52733.2024.00410 Xiaohe Ma, Valentin Deschaintre, Miloš Hašan, Fujun Luan, Kun Zhou, Hongzhi Wu, and Yiwei Hu

arXiv 2024
[11]

MaterialPicker: Multi-Modal DiT-Based Material Generation

MaterialPicker: Multi-Modal Material Generation with Diffusion Transformers. arXiv:2412.03225 [cs.CV] https://arxiv.org/abs/2412.03225 Keunhong Park, Konstantinos Rematas, Ali Farhadi, and Steven M. Seitz

work page Pith review arXiv

Show all 21 references

[13]

InSpecial Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings (SIGGRAPH ’23)

TEXTure: Text-Guided Texturing of 3D Shapes. InSpecial Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings (SIGGRAPH ’23). ACM. https://doi.org/10.1145/3588432.3591503 Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser,...
[15]

arXiv preprint arXiv:2310.18511 (2023)

3DCoMPaT++: An improved Large-scale 3D Vision Dataset for Compositional Recognition. arXiv preprint arXiv:2310.18511 (2023). Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang

2023 arXiv
[16]

arXiv:2312.03818 [cs.CV] Giuseppe Vecchio and Valentin Deschaintre

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want. arXiv:2312.03818 [cs.CV] Giuseppe Vecchio and Valentin Deschaintre

arXiv
[17]

arXiv:2309.01700 [cs.CV] Giuseppe Vecchio, Simone Palazzo, and Concetto Spampinato

ControlMat: A Controlled Gen- erative Approach to Material Capture. arXiv:2309.01700 [cs.CV] Giuseppe Vecchio, Simone Palazzo, and Concetto Spampinato

arXiv
[18]

In2021 IEEE/CVF International Conference on Computer Vision (ICCV)

SurfaceNet: Adversarial SVBRDF Estimation from a Single Image. In2021 IEEE/CVF International Conference on Computer Vision (ICCV) . IEEE Computer Society, Los Alamitos, CA, USA, 12820–12828. https://doi.org/10.1109/ICCV48922.2021.01260 Giuseppe Vecchio, Renato Sortino, Simone ...
[19]

ACM Transactions on Graphics (TOG) 43, 6 (2024), 1–14

StyleTex: Style Image-Guided Texture Generation for 3D Models. ACM Transactions on Graphics (TOG) 43, 6 (2024), 1–14. Kai Yan, Fujun Luan, Miloš Hašan, Thibault Groueix, Valentin Deschaintre, and Shuang Zhao

2024
[20]

In SIGGRAPH Asia 2023 Conference Papers (<conf-loc>, <city>Sydney</city>, <state>NSW</state>, <country>Australia</country>, </conf-loc>) (SA ’23)

PSDR-Room: Single Photo to Scene using Differentiable Render- ing. In SIGGRAPH Asia 2023 Conference Papers (<conf-loc>, <city>Sydney</city>, <state>NSW</state>, <country>Australia</country>, </conf-loc>) (SA ’23). As- sociation for Computing Machinery, New York, NY, USA, Artic...

2023
[21]

MaPa: Text-driven Photorealistic Material Painting for 3D Shapes. arXiv:2404.17569 [cs.CV] MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models Stable Diffusion Ours 3DCoMPaT++ Stable Diffusion Ours 3DCoMPaT++ Figure 5: Matching materials from Stable Diffusi...

2022 arXiv
[2018]

ACM Trans

Pho- toShape: Photorealistic Materials for Large-Scale Shape Collections. ACM Trans. Graph. 37, 6, Article 192 (Nov. 2018). Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen ...

2018
[2020]

ACM Trans

MaterialGAN: reflectance capture using a generative SVBRDF model. ACM Trans. Graph. 39, 6, Article 254 (nov 2020), 13 pages. https://doi.org/10.1145/3414685. 3417779 Dong Huo, Zixin Guo, Xinxin Zuo, Zhihao Shi, Juwei Lu, Peng Dai, Songcen Xu, Li Cheng, and Yee-Hong Yang

2020 doi
[2021]

High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752 [cs.CV] Habib Slim, Xiang Li, Yuchen Li, Mahmoud Ahmed, Mohamed Ayman, Ujjwal Upad- hyay, Ahmed Abdelreheem, Arpit Prajapati, Suhail Pothigara, Peter Wonka, and Mohamed Elhoseiny

arXiv
[2023]

In 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

Text2Tex: Text-driven Texture Synthesis via Diffusion Models. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . IEEE. https: //doi.org/10.1109/iccv51070.2023.01701 Kangle Deng, Timothy Omernick, Alexander Weiss, Deva Ramanan, Jun-Yan Zhu, Tinghui Zhou, and ...

2023
[2024]

arXiv preprint arXiv:2407.02430 (2024)

Meta 3d texturegen: Fast and consistent texture generation for 3d objects. arXiv preprint arXiv:2407.02430 (2024). Duygu Ceylan, Valentin Deschaintre, Thibault Groueix, Rosalie Martin, Chun-Hao Huang, Romain Rouffet, Vladimir Kim, and Gaëtan Lassagne

2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [2]

arXiv:2404.02899 [cs.CV] Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner

MatAt- las: Text-driven Consistent Geometry Texturing and Material Assignment. arXiv:2404.02899 [cs.CV] Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner

arXiv

[2] [4]

arXiv preprint arXiv:2402.13251 (2024)

FlashTex: Fast Relightable Mesh Texturing with LightControlNet. arXiv preprint arXiv:2402.13251 (2024). Valentin Deschaintre, Julia Guerrero-Viu, Diego Gutierrez, Tamy Boubekeur, and Belen Masia

arXiv 2024

[3] [5]

ACM Trans

The Visual Language of Fabrics. ACM Trans. Graph. 42, 4, Article 50 (jul 2023), 15 pages. https://doi.org/10.1145/3592391 Patrick Esser, Sumith Kulal, A. Blattmann, Rahim Entezari, Jonas Muller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dock- horn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach

doi:10.1145/3592391 2023

[4] [6]

arXiv preprint arXiv:2403.03206 (2024)

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. arXiv preprint arXiv:2403.03206 (2024). https://arxiv.org/abs/2403.03206 Michael Fischer, Iliyan Georgiev, Thibault Groueix, Vladimir G Kim, Tobias Ritschel, and Valentin Deschaintre

arXiv 2024

[5] [7]

arXiv preprint arXiv:2411.19322 (2024)

SAMa: Material-aware 3D Selection and Segmen- tation. arXiv preprint arXiv:2411.19322 (2024). Yu Guo, Cameron Smith, Miloš Hašan, Kalyan Sunkavalli, and Shuang Zhao

arXiv 2024

[6] [9]

arXiv preprint arXiv:2409.12431 (2024)

Flexitex: Enhancing texture generation with visual guidance. arXiv preprint arXiv:2409.12431 (2024). Black Forest Labs

arXiv 2024

[7] [10]

In 2024 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR)

TexOct: Generating Textures of 3D Models with Octree-based Diffusion. In 2024 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) . 4284–4293. https: //doi.org/10.1109/CVPR52733.2024.00410 Xiaohe Ma, Valentin Deschaintre, Miloš Hašan, Fujun Luan, Kun Zhou, Hongzhi Wu, and Yiwei Hu

arXiv 2024

[8] [11]

MaterialPicker: Multi-Modal DiT-Based Material Generation

MaterialPicker: Multi-Modal Material Generation with Diffusion Transformers. arXiv:2412.03225 [cs.CV] https://arxiv.org/abs/2412.03225 Keunhong Park, Konstantinos Rematas, Ali Farhadi, and Steven M. Seitz

work page Pith review arXiv