Pith. sign in

REVIEW 4 major objections 4 minor 47 references

FMMC: Harnessing the Power of Foundation Models for Accurate Material Classification

T0 review · 4 major / 4 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Synthetic material images plus vision-language priors let a dual-stream model classify real materials far more accurately than zero-shot foundation models.

desk verdict Solid engineering recipe that turns diffusion + Grounded-SAM + DINOv2/CLIP into clear SOTA gains on material classification; the auto-label fidelity claim is the softest link but does not sink the external results. read the letter →

arxiv 2603.17390 v2 pith:FFHDF7EL submitted 2026-03-18 cs.CV

Qingran Lin , Fengwei Yang , Chaolun Zhu This is my paper · ORCID
classification cs.CV
keywords materialclassificationsyntheticdatagenerationvision-languagemodelsauto-labelingDINOv2CLIPcross-modalfusionfoundation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Material classification is held back by scarce, unbalanced annotations, so even large vision-language models still misidentify surfaces such as plastic versus metal. This paper shows that a carefully generated and auto-labeled synthetic dataset of 21 material classes can close that gap. Diffusion models produce material-centric images from object-material prompts; a segmentation model then isolates the object region and assigns the prompt’s material label, yielding clean training patches. A dual-stream classifier freezes a vision foundation model for texture features and a language model for GPT-generated material descriptors, then trains only a lightweight fusion head. The resulting system reaches 89 % accuracy on the classic FMD benchmark and 92 % on a real-world Google-image set, substantially beating both zero-shot CLIP/GPT-4v and a prior specialized material classifier. The work therefore claims that synthetic data plus modest cross-modal fine-tuning is enough to turn general foundation models into reliable material recognizers.

What carries the argument

Semantic-grounded auto-labeling: object-material-adjective prompts drive diffusion image generation; Grounded-SAM then segments the named object and maps the material label solely onto that mask, producing reliable region-level supervision without manual annotation.

What would settle it

Train the identical dual-stream model on a version of the synthetic set whose labels have been deliberately corrupted by 10–15 % (or whose images are replaced by pure background patches) and measure whether accuracy on FMD and Google-test collapses below the MatSim baseline.

Watch

Extended reading notes

Core claim

A dual-stream network that fuses frozen DINOv2 patch features with CLIP embeddings of GPT-4v material descriptions, trained only on the authors’ synthetically generated and auto-labeled 21-class dataset, achieves 0.89 accuracy on FMD, 0.64 on DMS-test and 0.92 on Google-test—large absolute gains over zero-shot CLIP, GPT-4v and the MatSim baseline.

Load-bearing premise

The material labels transferred from text prompts via object segmentation are accurate enough, and the resulting synthetic images match the visual statistics of real materials well enough, for frozen foundation features to transfer.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper addresses material classification under scarce annotated data by (1) synthesizing a balanced 21-class dataset via diffusion models (SDv2.1) with hierarchical LLM prompts, then auto-labeling target regions via Grounded-SAM object masks derived from the same prompts, and (2) training a dual-stream classifier that max-pools masked DINOv2 patch features, concatenates them with frozen CLIP embeddings of GPT-4v material descriptors, and maps the fusion through a lightweight MLP (foundations largely frozen). Experiments on FMD (10-class), a 21-class DMS-test subset, and a self-collected 21-class Google-test set report 0.89 / 0.64 / 0.92 accuracy, substantially above zero-shot CLIP, GPT-4v and MatSim; ablations isolate contributions of each stream, backbone choice, dataset scale and semantic masking, while PCA and cross-dataset tables argue that the synthetic distribution bridges real-material feature statistics.

Significance. If the reported transfer holds, the work supplies a practical, reproducible route to high-accuracy material recognition that bypasses expensive manual annotation—directly useful for graphics pipelines (procedural material retrieval, 3D texturing) and scene understanding. The semantic-grounding auto-labeling idea and the dual-stream prior fusion are cleanly engineered and well-ablated; the promised release of code plus the 20 k+ synthetic set further raises the contribution’s utility. The gains over strong zero-shot VLMs and a recent specialized baseline (MatSim) are large enough to matter for downstream applications.

major comments (4)
  1. [Section 3.1] Section 3.1 asserts “98 % label accuracy on human-verified samples” for the Grounded-SAM + prompt-mapping pipeline, yet supplies neither sample size N, sampling protocol, inter-annotator agreement, nor error typology (background leakage, multi-material objects, prompt–image misalignment). Because every downstream number (Tables 1–3, ablations) rests on the fidelity of these auto-labels, the claim is load-bearing and currently unsubstantiated; a quantified verification study (or at least a publicly released labeled subset with error analysis) is required.
  2. [Abstract / §3.3 / Table 5] Abstract and Introduction claim a “joint fine-tuning method that optimizes a pre-trained vision foundation model alongside VLM-derived priors” and “finetune the head of DINOv2 together with the MLP,” while Section 3.3 and the training protocol state that DINOv2 and CLIP are frozen and only the MLP is trained. Table 5 further contrasts “head” vs “full” regimes. The manuscript must resolve this inconsistency and clearly state which parameters are updated; the current wording overstates the adaptation of the foundation models.
  3. [Table 2 / DMS-test] Table 2 reports only 0.64 accuracy on the 21-class DMS-test set (still the most realistic multi-material benchmark). While better than the ~0.4 baselines, the absolute number remains modest; the paper needs a failure-mode analysis (class-confusion matrix, qualitative examples of systematic errors such as ceramic/glass or plastic/rubber) and discussion of whether residual domain gap or label noise is responsible. Without it the claim of “significant improvements on multiple datasets” is unevenly supported.
  4. [§4.1 / Table 2] The Google-test set (Table 2, 0.92 accuracy) is self-curated from Google Images with no public release details, selection criteria, or inter-annotator protocol supplied. Given that it drives the strongest absolute numbers and the cross-dataset argument of Table 3, selection bias cannot be ruled out; either release the set with the promised code or replace/augment it with an independent public collection.
minor comments (4)
  1. [Figure 5] Figure 5 PCA ellipses are informative but axes are fixed to [−40,40] without stating the explained-variance ratio of the first two components; adding that percentage would strengthen the domain-gap claim.
  2. [Tables 1–5] No error bars, standard deviations across seeds, or statistical significance tests appear for any table; even a single multi-seed run would increase confidence.
  3. [Supplementary] Supplementary Table 6/7 class-wise numbers are useful; consider moving a compact confusion summary into the main text for DMS-test.
  4. [Figure 1 / Related Work] Minor wording: “polypropylene natural stainless-steel au…” in Figure 1 caption appears truncated; several references to “Swim-T” should be “Swin-T”.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: synthetic auto-labels and frozen VLM priors train an MLP that is evaluated on independent real-image benchmarks (FMD, DMS-test, Google-test).

full rationale

The paper's central claims are empirical classification accuracies obtained by training a lightweight MLP head on a synthetically generated 21-class dataset (diffusion images + Grounded-SAM object masks that inherit the material token from the generation prompt) and then testing on three external real-image collections. The language stream consists of fixed, offline GPT-4v class descriptors encoded once by CLIP; they are never fitted to any test label. DINOv2 remains frozen. No equation equates a reported accuracy to a fitted parameter, no uniqueness theorem is imported from the authors' prior work, and no self-citation is load-bearing for the numerical results. The only self-collected set (Google-test) is used solely for evaluation and does not affect the FMD or DMS-test numbers. Consequently the derivation chain contains no self-definitional, fitted-as-prediction, or self-citation circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The work is an empirical computer-vision system paper. Its load-bearing premises are domain assumptions about the fidelity of existing foundation models rather than free parameters fitted to the target metric or newly postulated physical entities. Hyper-parameters (learning rate, pooling choice, number of generated images) are conventional and do not drive the central claim.

free parameters (2)
  • AdamW learning rate = 5e-5
    Set to 5e-5 by the authors; only the MLP is optimized. Standard hyper-parameter, not fitted to the final test metrics.
  • synthetic dataset scale = ~20k images
    Approximately 20 k images generated from 2 448 prompts (5 images each for the 10-class FMD subset). Scale is chosen by the authors; ablation shows diminishing returns beyond 2×.
assumptions (4)
  • domain assumption Stable Diffusion v2.1, when prompted with hierarchical object-material text, produces images whose primary object exhibits the named material appearance.
    Invoked throughout Section 3.1; model selection is qualitative only.
  • domain assumption Grounded SAM / Grounding DINO correctly segments the object named in the prompt, allowing the material label to be transferred with high precision (claimed 98 %).
    Core of the auto-labeling pipeline (Section 3.1, Figure 2).
  • domain assumption Frozen DINOv2 patch features, after max-pooling over a binary mask, retain material-discriminative texture and reflectance information.
    Vision stream definition (Section 3.2–3.3).
  • domain assumption Short natural-language material descriptors produced by GPT-4v and encoded by CLIP supply complementary semantic priors useful for disambiguation.
    Language stream (Section 3.2–3.3) and ablation Table 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FMMC: Harnessing the Power of Foundation Models for Accurate Material Classification." pith.science (2026). https://pith.science/paper/FFHDF7EL

@misc{pith2026260317390,
  author       = {Pith},
  title        = {Pith review of: FMMC: Harnessing the Power of Foundation Models for Accurate Material Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FFHDF7EL}},
  note         = {Machine review of arXiv:2603.17390}
}
read the original abstract

Material classification has emerged as a critical task in computer vision and graphics, supporting the assignment of accurate material properties to a wide range of digital and real-world applications. While traditionally framed as an image classification task, this domain faces significant challenges due to the scarcity of annotated data, limiting the accuracy and generalizability of trained models. Recent advances in vision-language foundation models (VLMs) offer promising avenues to address these issues, yet existing solutions leveraging these models still exhibit unsatisfying results in material recognition tasks. In this work, we propose a novel framework that effectively harnesses foundation models to overcome data limitations and enhance classification accuracy. Our method integrates two key innovations: (a) a robust image generation and auto-labeling pipeline that creates a diverse and high-quality training dataset with material-centric images, and automatically assigns labels by fusing object semantics and material attributes in text prompts; (b) a prior incorporation strategy to distill information from VLMs, combined with a joint fine-tuning method that optimizes a pre-trained vision foundation model alongside VLM-derived priors, preserving broad generalizability while adapting to material-specific features. Extensive experiments demonstrate significant improvements on multiple datasets. We show that our synthetic dataset effectively captures the characteristics of real world materials, and the integration of priors from vision-language models significantly enhances the final performance. The source code and dataset will be released.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 13 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [2]

    Opensurfaces: A richly annotated catalog of surface appear- ance.ACM TOG, 32(4):1–17, 2013

    Sean Bell, Paul Upchurch, Noah Snavely, and Kavita Bala. Opensurfaces: A richly annotated catalog of surface appear- ance.ACM TOG, 32(4):1–17, 2013. 2, 3

    2013

  3. [3]

    Material recognition in the wild with the materials in context database

    Sean Bell, Paul Upchurch, Noah Snavely, and Kavita Bala. Material recognition in the wild with the materials in context database. InCVPR, pages 3479–3487, 2015. 2

    2015

  4. [4]

    Rgb road scene material segmentation

    Sudong Cai, Ryosuke Wakaki, Shohei Nobuhara, and Ko Nishino. Rgb road scene material segmentation. InACCV, pages 3051–3067, 2022. 3

    2022

  5. [5]

    Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023. 4

    arXiv 2023

  6. [6]

    Zest: Zero-shot material trans- fer from a single image

    Ta-Ying Cheng, Prafull Sharma, Andrew Markham, Niki Trigoni, and Varun Jampani. Zest: Zero-shot material trans- fer from a single image. InEuropean Conference on Com- puter Vision, pages 370–386. Springer, 2025. 1

    2025

  7. [7]

    Deep filter banks for texture recognition and segmentation

    Mircea Cimpoi, Subhransu Maji, and Andrea Vedaldi. Deep filter banks for texture recognition and segmentation. In CVPR, pages 3828–3836, 2015. 3

    2015

  8. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 3, 8

    arXiv 2010

Show all 47 references
  1. [9]

    One-shot recognition of any material anywhere using contrastive learning with physics-based ren- dering

    Manuel S Drehwald, Sagi Eppel, Jolina Li, Han Hao, and Alan Aspuru-Guzik. One-shot recognition of any material anywhere using contrastive learning with physics-based ren- dering. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 23524–23533, 20...

    2023

  2. [10]

    Diversify your vision datasets with automatic diffusion-based augmentation.Ad- vances in neural information processing systems, 36:79024– 79034, 2023

    Lisa Dunlap, Alyssa Umino, Han Zhang, Jiezhi Yang, Joseph E Gonzalez, and Trevor Darrell. Diversify your vision datasets with automatic diffusion-based augmentation.Ad- vances in neural information processing systems, 36:79024– 79034, 2023. 3

    2023

  3. [11]

    Make-it-real: Unleashing large multimodal model’s ability for painting 3d objects with real- istic materials.arXiv preprint arXiv:2404.16829, 2024

    Ye Fang, Zeyi Sun, Tong Wu, Jiaqi Wang, Ziwei Liu, Gordon Wetzstein, and Dahua Lin. Make-it-real: Unleashing large multimodal model’s ability for painting 3d objects with real- istic materials.arXiv preprint arXiv:2404.16829, 2024. 1, 3, 6

    2024 arXiv

  4. [12]

    Matformer: A generative model for procedural materials.arXiv preprint arXiv:2207.01044, 2022

    Paul Guerrero, Milo ˇs Ha ˇsan, Kalyan Sunkavalli, Radom ´ır Mˇech, Tamy Boubekeur, and Niloy J Mitra. Matformer: A generative model for procedural materials.arXiv preprint arXiv:2207.01044, 2022. 1

    2022 arXiv

  5. [13]

    Materialgan: Reflectance capture using a generative svbrdf model.arXiv preprint arXiv:2010.00114,

    Yu Guo, Cameron Smith, Milo ˇs Haˇsan, Kalyan Sunkavalli, and Shuang Zhao. Materialgan: Reflectance capture using a generative svbrdf model.arXiv preprint arXiv:2010.00114,

    2010 arXiv

  6. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, pages 770–778, 2016. 8

    2016

  7. [15]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 3

    2022

  8. [16]

    Controlling material appearance by examples

    Yiwei Hu, Milo ˇs Ha ˇsan, Paul Guerrero, Holly Rushmeier, and Valentin Deschaintre. Controlling material appearance by examples. InComputer graphics forum, pages 117–128. Wiley Online Library, 2022. 1

    2022

  9. [17]

    Generating procedural materials from text or image prompts

    Yiwei Hu, Paul Guerrero, Milos Hasan, Holly Rushmeier, and Valentin Deschaintre. Generating procedural materials from text or image prompts. InACM SIGGRAPH 2023 Con- ference Proceedings, pages 1–11, 2023. 1

    2023

  10. [18]

    Ma- terial anything: Generating materials for any 3d object via diffusion.arXiv preprint arXiv:2411.15138, 2024

    Xin Huang, Tengfei Wang, Ziwei Liu, and Qing Wang. Ma- terial anything: Generating materials for any 3d object via diffusion.arXiv preprint arXiv:2411.15138, 2024. 3

    2024 arXiv

  11. [19]

    Playground v2

    Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Lin- miao Xu, and Suhail Doshi. Playground v2. 5: Three in- sights towards enhancing aesthetic quality in text-to-image generation.arXiv preprint arXiv:2402.17245, 2024. 4

    2024 arXiv

  12. [20]

    Materialseg3d: Segmenting dense materi- als from 2d priors for 3d assets

    Zeyu Li, Ruitong Gan, Chuanchen Luo, Yuxi Wang, Jiaheng Liu, Ziwei Zhu, Qing Li, Xucheng Yin, Man Zhang, Zhaox- iang Zhang, et al. Materialseg3d: Segmenting dense materi- als from 2d priors for 3d assets. InProceedings of the 32nd ACM International Conference on Multimedia, pa...

    2024

  13. [21]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023. 2, 4

    2023 arXiv

  14. [22]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 3

    2021

  15. [23]

    Material palette: Extraction of materials from a single image

    Ivan Lopes, Fabio Pizzati, and Raoul de Charette. Material palette: Extraction of materials from a single image. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4379–4388, 2024. 1

    2024

  16. [24]

    Glass segmentation using intensity and spectral polarization cues

    Haiyang Mei, Bo Dong, Wen Dong, Jiaxi Yang, Seung- Hwan Baek, Felix Heide, Pieter Peers, Xiaopeng Wei, and Xin Yang. Glass segmentation using intensity and spectral polarization cues. InCVPR, pages 12622–12631, 2022. 3

    2022

  17. [25]

    A dataset of multi-illumination images in the wild

    Lukas Murmann, Michael Gharbi, Miika Aittala, and Fredo Durand. A dataset of multi-illumination images in the wild. InCVPR, pages 4080–4089, 2019. 3

    2019

  18. [26]

    Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic segmentation.Advances in Neural Information Processing Systems, 36, 2024

    Quang Nguyen, Truong Vu, Anh Tran, and Khoi Nguyen. Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic segmentation.Advances in Neural Information Processing Systems, 36, 2024. 3 9

    2024

  19. [27]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2, 3, 5

    2023 arXiv

  20. [28]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 4

    2023 arXiv

  21. [29]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  22. [30]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 2

    2021

  23. [31]

    Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159,

    arXiv

  24. [32]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 4

    2022

  25. [33]

    Lavanya Sharan, Ruth Rosenholtz, and Edward H. Adelson. Accuracy and speed of material categorization in real-world images.J. Vis., 14(10), 2014. 2, 3, 6, 7, 8

    2014

  26. [34]

    Alchemist: Parametric control of material proper- ties with diffusion models

    Prafull Sharma, Varun Jampani, Yuanzhen Li, Xuhui Jia, Dmitry Lagun, Fredo Durand, Bill Freeman, and Mark Matthews. Alchemist: Parametric control of material proper- ties with diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  27. [35]

    High-resolution representations for labeling pixels and regions.arXiv preprint arXiv:1904.04514, 2019

    Ke Sun, Yang Zhao, Borui Jiang, Tianheng Cheng, Bin Xiao, Dong Liu, Yadong Mu, Xinggang Wang, Wenyu Liu, and Jingdong Wang. High-resolution representations for labeling pixels and regions.arXiv preprint arXiv:1904.04514, 2019. 3

    1904 arXiv

  28. [36]

    Satsynth: Augmenting image-mask pairs through diffusion models for aerial semantic segmentation

    Aysim Toker, Marvin Eisenberger, Daniel Cremers, and Laura Leal-Taix´e. Satsynth: Augmenting image-mask pairs through diffusion models for aerial semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27695–27705, 2024. 3

    2024

  29. [37]

    A dense material segmenta- tion dataset for indoor and outdoor scene parsing

    Paul Upchurch and Ransen Niu. A dense material segmenta- tion dataset for indoor and outdoor scene parsing. InECCV, pages 450–466, 2022. 2, 3, 6, 7

    2022

  30. [38]

    A 4d light-field dataset and cnn architectures for material recogni- tion

    Ting-Chun Wang, Jun-Yan Zhu, Ebi Hiroaki, Manmohan Chandraker, Alexei A Efros, and Ravi Ramamoorthi. A 4d light-field dataset and cnn architectures for material recogni- tion. InECCV, pages 121–138, 2016. 3

    2016

  31. [39]

    Con- vnext v2: Co-designing and scaling convnets with masked autoencoders

    Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Con- vnext v2: Co-designing and scaling convnets with masked autoencoders. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16133– 1614...

    2023

  32. [40]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 3

    2024

  33. [41]

    Mapa: Text-driven photorealistic mate- rial painting for 3d shapes

    Shangzhan Zhang, Sida Peng, Tao Xu, Yuanbo Yang, Tian- run Chen, Nan Xue, Yujun Shen, Hujun Bao, Ruizhen Hu, and Xiaowei Zhou. Mapa: Text-driven photorealistic mate- rial painting for 3d shapes. InACM SIGGRAPH 2024 Con- ference Papers, pages 1–12, 2024. 1, 3, 6

    2024

  34. [42]

    Ti- legen: Tileable, controllable material generation and cap- ture

    Xilong Zhou, Milos Hasan, Valentin Deschaintre, Paul Guer- rero, Kalyan Sunkavalli, and Nima Khademi Kalantari. Ti- legen: Tileable, controllable material generation and cap- ture. InSIGGRAPH Asia 2022 conference papers, pages 1–9, 2022. 1

    2022

  35. [43]

    Photomat: A material generator learned from single flash photos

    Xilong Zhou, Milos Hasan, Valentin Deschaintre, Paul Guerrero, Yannick Hold-Geoffroy, Kalyan Sunkavalli, and Nima Khademi Kalantari. Photomat: A material generator learned from single flash photos. InACM SIGGRAPH 2023 Conference Proceedings, pages 1–11, 2023. 1 10 Harnessing t...

    2023

  36. [44]

    reports the per-class classification accuracy on the DMS-test dataset

    Class-wise Accuracy on DMS-test Table 6. reports the per-class classification accuracy on the DMS-test dataset. While Table 2 in the main text only provides the averaged results, this table further breaks down the performance into individual classes.Figure 7 illustrates one re...

  37. [45]

    Table 7 presents the class-wise classification accuracy on the Google-test dataset, comple- menting the averaged results in the main text

    Class-wise Accuracy on Google-test In addition to the averaged results on the Google-test dataset reported in Table 2 of the main text, we provide a detailed per-class breakdown here. Table 7 presents the class-wise classification accuracy on the Google-test dataset, comple- m...

  38. [46]

    While DMS exhibits significant imbal- ance across classes, our generative dataset provides a more uniform distribution, enabling better supervision across rare categories

    Class image statistics We reported detailed sample statistics for all 21 categories in our own dataset and compared them with the DMS dataset in Table 8 . While DMS exhibits significant imbal- ance across classes, our generative dataset provides a more uniform distribution, en...

  39. [47]

    Samples from our generated dataset Our generated dataset provides diverse material samples across 21 categories, each accompanied by extracted se- mantic patches that highlight the fine-grained local cues present in the Figure 9 and Figure 10. These patches capture critical te...

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.