REVIEW 5 major objections 4 minor 32 references
3DCoMPaT200: Language-Grounded Compositional Understanding of Parts and Materials of 3D Shapes
T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read 3DCoMPaT200 supplies 19,051 shapes across 200 categories with 1,031 fine-grained parts and 293 materials, and its benchmark shows retrieval improving as more parts are described.
desk verdict A substantial 3D part-material dataset with a useful retrieval benchmark, but the evaluation protocol needs cleanup before the numbers can be fully trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the compositional annotation hierarchy: each shape's fine-grained part labels are mapped to 118 coarse-grained parts, each part is linked to one of 293 fine-grained materials grouped into 13 coarse materials, and these pairings are recombined into up to 1,000 styled compositions per shape. For the retrieval benchmark, the fine-grained materials are rendered and described in color by a vision-language model, then fused with part names into templated captions such as 'The telephone is made of a dark gray metal base, a purple plastic dialing button, and a bright blue plastic headset.' Text and colored point-cloud embeddings from contrastive models (ULIP, Uni3D, OpenShape) are then matched, and retrieval is scored at 1, 3, and 6 parts per caption.
What would settle it
Check the dataset by re-annotating 200 randomly selected shapes with two independent annotators and by running geometric duplicate detection between training and test splits. If duplicates appear across splits or the annotators agree poorly on fine-grained parts, then the reported benchmark improvements could reflect memorization rather than compositional understanding.
Extended reading notes
Core claim
The paper's discovery is that a 3D dataset an order of magnitude larger in object vocabulary than earlier part datasets can be densely annotated at part and material level while remaining usable by current point-cloud models. Specifically, on the new Compositional Shape Retrieval benchmark, all three tested contrastive text-3D models retrieve shapes more accurately when captions specify more parts: Uni3D's R@1 rises from 26.74% with one part to 59.4% with six parts, ULIP's from 23.04% to 54.8%, and OpenShape's from 24.53% to 56.62%. The paper interprets this monotone improvement as showing that the compositional dataset is what lets models learn to combine part and material information into a global shape representation.
Load-bearing premise
The entire dataset's value rests on the correctness and consistency of the human annotations for parts, materials, and the fine-to-coarse mappings; if those contain systematic errors or if duplicate shapes appear across splits, every benchmark number in the paper would shift.
Editorial extensions
If this is right
- Benchmarks for part classification, material segmentation, and Grounded Compositional Recognition can now be run over 200 categories instead of 16 to 42, which tests generalization more honestly.
- Retrieval from compositional text becomes a measurable task: all tested models improve as parts are added, so the metric can track compositional reasoning rather than object-category matching.
- The 1,000 styled compositions per shape supply a large text-shape training signal that may transfer to text-to-3D and image-to-3D generation of objects with specific part materials.
- Because material segmentation reaches roughly 98 percent mIoU, the dataset lets research focus on the harder part and part-material composition problems while treating material recognition as nearly solved.
Reading between the lines
- The paper's monotone retrieval-curve result leaves open whether the gain comes from compositional supervision or simply from longer text; an editorially suggested ablation would use captions with matched token counts but scrambled part-material structure and test retrieval.
- Since the authors note possible duplicate shapes and do not report inter-annotator agreement, an extension would be to run geometric duplicate detection between train and test splits and re-annotate a random subset; the benchmark numbers would be more conclusive if those checks pass.
- The dataset's stated uses include generation, so a natural next step is to condition a text-to-3D generator on part-material captions and measure whether generated shapes exhibit the specified part colors and materials; this would test whether the compositional signal is usable beyond retrieval.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces 3DCoMPaT200, a large-scale dataset for compositional part-material understanding of 3D shapes. It extends 3DCoMPaT from 42 to 200 object categories, includes 19,051 shapes, 1,031 fine-grained and 118 coarse-grained part labels, 293 fine-grained and 13 coarse-grained material classes, and provides rendered images, depth maps, segmentation maps, and compositional captions. The authors benchmark shape classification, part and material segmentation, grounded compositional recognition (GCR), and introduce a compositional part-shape retrieval task evaluated with ULIP, Uni3D, and OpenShape. The main empirical claim is that retrieval accuracy increases as captions include more parts, which the authors interpret as evidence of the dataset's value for compositional 3D understanding.
Significance. If the dataset and benchmarks are reliable, this is a substantial resource contribution: it is a large expansion of category coverage over previous part-material datasets, provides hierarchical part annotations and material labels, releases code and data, and offers a new evaluation task for compositional text-to-shape retrieval. The benchmarks cover several mainstream point-cloud models, and the dataset documentation is generally thorough. However, the central benchmark claims currently depend on unresolved inconsistencies in the evaluation protocol and on unquantified duplicate-leakage risk; these issues need to be resolved before the dataset's performance numbers can be taken at face value.
major comments (5)
- [§4, Tables 3 and 4] The experimental protocol is internally inconsistent. Section 4 states that all results are 'done on one set of random assignment of material-part suitable pairs' and that 'all results are reported on a testing set of one composition and models were trained on one composition as well.' However, Tables 3 and 4 are captioned 'evaluated on 10 Compositions,' and Section 4.4 says 'we train models using 10 compositions per shape.' The benchmark numbers in Tables 3 through 7 cannot be interpreted until the authors specify exactly how many compositions were used for training, validation, and testing in each task, and whether the reported numbers are averages over 10 compositions or results on a single composition.
- [§3.2 and Limitations] The Limitations section acknowledges 'the possibility of duplicate shapes appearing in the data,' but no duplicate-removal, duplicate-detection, or cross-split leakage analysis is reported. Since Tables 2 through 7 report classification, segmentation, and retrieval numbers on the released split, even partial duplicate leakage between train and test can inflate results; the monotonically increasing R@1 values in Table 7 are especially sensitive to this. The authors should describe a concrete deduplication procedure (e.g., geometric hashing or perceptual hashing on rendered views), report the number of exact and near-duplicates found within and across splits, and, if any are found, rerun the benchmarks on a deduplicated split and state whether the compositional trend persists.
- [§3.2, Data Collection] Annotation quality is described only qualitatively as following 'a stringent, multi-stage review protocol,' with no inter-annotator agreement statistics. Because fine-grained part labels and the fine-to-coarse part mappings are used as ground truth in every benchmark in Tables 3 through 5, the paper should report agreement measures on a held-out annotation sample (for example, part-label IoU or material-assignment agreement). Without such measures, the fine-grained benchmark numbers are difficult to interpret as accuracy against a reliable ground truth.
- [§4.2, Table 4] The material-segmentation protocol is under-specified. The text says material segmentation classifies materials into 'one of 13 coarse-grained classes,' but Table 4 also reports fine-grained results with accuracy and mIoU columns. It should be clarified whether the fine-grained material experiments train on the 293 fine-grained material classes, what label set is used for each column, and whether the same models and training procedures are used for both granularities.
- [§4.4, Table 7] The compositional retrieval benchmark is under-specified. In addition to the one-versus-ten composition discrepancy, it is unclear (i) whether retrieval is performed over all 200 object categories or only classes seen during training, (ii) how many captions per shape and per part count are used, (iii) whether the 1-, 3-, and 6-part captions are generated from the same set of compositions, and (iv) how ties are broken. The central claim that retrieval improves with compositional complexity would also be more convincing if the authors reported per-composition variance or standard errors, since the captions are generated from only 10 compositions per shape.
minor comments (4)
- [Contributions, bullet 1] The phrase '5 orders of magnitude increase' is not consistent with the expansion from 42 to 200 object categories; the abstract's 'approximately 5 times larger' is the mathematically correct description (200/42 is about 4.8). Please correct the contributions bullet.
- [§3.1, §3.2, Appendix A.2 and F.1] The number of released compositions is stated inconsistently: Section 3.2 and Appendix F.1 mention up to 1000 compositions per shape, while Section 4.4 and Appendix A.2 refer to 10 compositions and a 'Shards.zip' archive of '10 different compositions.' Please clarify which compositions are released as rendered data and which are available as metadata-only.
- [§2.2 heading] The heading 'Grounded Compositional Grounding' appears to be a typo; it should likely read 'Grounded Compositional Recognition.'
- [Table 1] The table formatting is difficult to read: several entries use placeholder symbols (for example, question-mark or circle glyphs) and some cells are ambiguous (e.g., '19k 19M'). Please add a clear legend and ensure all symbols render correctly.
Circularity Check
No circularity: 3DCoMPaT200 is a dataset and benchmark paper whose claims are empirical measurements, not derivations that reduce to their own inputs.
full rationale
This paper is a dataset and benchmark report rather than a derivation of predictions from first principles. The central quantities—shape classification accuracy, segmentation mIoU, GCR metrics, and text-to-3D retrieval R@1/R@5—are empirical results obtained by training models on the released training split and evaluating on a held-out test split. None of these numbers is fitted to a subset and then reported as a prediction of a closely related quantity; they are standard held-out evaluations. The claim that retrieval performance improves with more parts in the caption is an observed trend of the evaluated models (ULIP, Uni3D, OpenShape), not a consequence of the dataset's construction by definition. The self-citations to 3DCoMPaT [8,9] are used to describe the lineage and pipeline of the dataset, not to import an unverified uniqueness theorem or to smuggle in an ansatz; the prior work is an antecedent dataset, and extending it is a normal dataset contribution. The appendix's note about possible duplicate shapes is an honest data-quality limitation, but it does not make the benchmark numbers circular—it is a validity caveat, not a derivation that reduces to its inputs. No equation, fitted parameter, or load-bearing self-citation can be exhibited that turns the paper's claims into tautologies. Therefore the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption Human annotations of parts and materials across 200 categories are accurate and consistent.
- domain assumption The template captions generated from metadata and GPT-4V colors faithfully describe the shape for retrieval.
- ad hoc to paper The 10 selected compositions per shape are representative of the dataset distribution.
Cite this review
Pith. "Pith review of 3DCoMPaT200: Language-Grounded Compositional Understanding of Parts and Materials of 3D Shapes." pith.science (2026). https://pith.science/paper/N6U6AVNK
@misc{pith2026250106785,
author = {Pith},
title = {Pith review of: 3DCoMPaT200: Language-Grounded Compositional Understanding of Parts and Materials of 3D Shapes},
year = {2026},
howpublished = {\url{https://pith.science/paper/N6U6AVNK}},
note = {Machine review of arXiv:2501.06785}
}
abstract
Understanding objects in 3D at the part level is essential for humans and robots to navigate and interact with the environment. Current datasets for part-level 3D object understanding encompass a limited range of categories. For instance, the ShapeNet-Part and PartNet datasets only include 16, and 24 object categories respectively. The 3DCoMPaT dataset, specifically designed for compositional understanding of parts and materials, contains only 42 object categories. To foster richer and fine-grained part-level 3D understanding, we introduce 3DCoMPaT200, a large-scale dataset tailored for compositional understanding of object parts and materials, with 200 object categories with $\approx$5 times larger object vocabulary compared to 3DCoMPaT and $\approx$ 4 times larger part categories. Concretely, 3DCoMPaT200 significantly expands upon 3DCoMPaT, featuring 1,031 fine-grained part categories and 293 distinct material classes for compositional application to 3D object parts. Additionally, to address the complexities of compositional 3D modeling, we propose a novel task of Compositional Part Shape Retrieval using ULIP to provide a strong 3D foundational model for 3D Compositional Understanding. This method evaluates the model shape retrieval performance given one, three, or six parts described in text format. These results show that the model's performance improves with an increasing number of style compositions, highlighting the critical role of the compositional dataset. Such results underscore the dataset's effectiveness in enhancing models' capability to understand complex 3D shapes from a compositional perspective. Code and Data can be found at http://github.com/3DCoMPaT200/3DCoMPaT200
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An Information-Rich 3D Model Repository, December 2015. arXiv:1512.03012 [cs]
arXiv 2015
-
[2]
3D ShapeNets: A Deep Representation for V olumetric Shapes, April 2015
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3D ShapeNets: A Deep Representation for V olumetric Shapes, April 2015. arXiv:1406.5670 [cs]
arXiv 2015
-
[3]
3D-FUTURE: 3D Furniture shape with TextURE, September 2020
Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3D-FUTURE: 3D Furniture shape with TextURE, September 2020. arXiv:2009.09633 [cs]
arXiv 2020
-
[4]
Objaverse: A Universe of Annotated 3D Objects, December 2022
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A Universe of Annotated 3D Objects, December 2022. arXiv:2212.08051 [cs] version: 1
arXiv 2022
-
[5]
Objaverse-XL: A Universe of 10M+ 3D Objects
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, Eli VanderBilt, Aniruddha Kembhavi, Carl V ondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objaverse-XL: A Universe of 10M+ 3D Objects
-
[6]
Li Yi, Vladimir G. Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3D shape collections. ACM Transactions on Graphics, 35(6):1–12, November 2016
work page 2016
-
[7]
Chang, Li Yi, Subarna Tripathi, Leonidas J
Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding, December 2018. arXiv:1812.02713 [cs]
arXiv 2018
-
[8]
3D CoMPaT: Composition of Materials on Parts of 3D Things
Yuchen Li, Ujjwal Upadhyay, Habib Slim, Ahmed Abdelreheem, Arpit Prajapati, Suhail Pothigara, Peter Wonka, and Mohamed Elhoseiny. 3D CoMPaT: Composition of Materials on Parts of 3D Things. In Computer Vision – ECCV 2022, pages 110–127. Springer Science, 2022. Series Title: Lecture Notes in Computer Science
work page 2022
Show all 32 references
-
[9]
3DCoMPaT++: An improved large-scale 3d vision dataset for compositional recognition
Habib Slim, Xiang Li, Mahmoud Ahmed Yuchen Li, Mohamed Ayman, Ujjwal Upadhyay Ahmed Abdelreheem, Suhail Pothigara Arpit Prajapati, Peter Wonka, and Mohamed Elhoseiny. 3DCoMPaT++: An improved large-scale 3d vision dataset for compositional recognition. In arXiv, 2023
2023
-
[10]
Ulip-2: Towards scalable multimodal pre-training for 3d understanding
Le Xue, Ning Yu, Shu Zhang, Junnan Li, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip-2: Towards scalable multimodal pre-training for 3d understanding. arXiv preprint arXiv:2305.08275, 2023
2023 arXiv
-
[11]
Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding
Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF Conference on C...
2023
-
[12]
ObjectNet3D: A Large Scale Database for 3D Object Recognition
Yu Xiang, Wonhui Kim, Wei Chen, Jingwei Ji, Christopher Choy, Hao Su, Roozbeh Mottaghi, Leonidas Guibas, and Silvio Savarese. ObjectNet3D: A Large Scale Database for 3D Object Recognition. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECC...
2016
-
[13]
Yago Vicente, Thomas Dideriksen, Himanshu Arora, Matthieu Guillaumin, and Jitendra Malik
Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F. Yago Vicente, Thomas Dideriksen, Himanshu Arora, Matthieu Guillaumin, and Jitendra Malik. ABO: Dataset and Benchmarks for Real-World 3D Object Understanding, June 2022. ar...
2022 arXiv
-
[14]
Kim, Siddhartha Chaudhuri, and Kavita Bala
Hubert Lin, Melinos Averkiou, Evangelos Kalogerakis, Balazs Kovacs, Siddhant Ranade, Vladimir G. Kim, Siddhartha Chaudhuri, and Kavita Bala. Learning Material-Aware Local Descriptors for 3D Shapes. In 2018 International Conference on 3D Vision (3DV) , pages 150–159, September ...
2018 arXiv
-
[15]
OmniObject3D: Large-V ocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation, April 2023
Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu. OmniObject3D: Large-V ocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation, April 2023. arXiv:2301.07525 [cs]
2023 arXiv
-
[16]
Grounded Situation Recognition, March 2020
Sarah Pratt, Mark Yatskar, Luca Weihs, Ali Farhadi, and Aniruddha Kembhavi. Grounded Situation Recognition, March 2020. arXiv:2003.12058 [cs]. 11
2020 arXiv
-
[17]
Situation Recognition: Visual Semantic Role Labeling for Image Understanding
Mark Yatskar, Luke Zettlemoyer, and Ali Farhadi. Situation Recognition: Visual Semantic Role Labeling for Image Understanding. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5534–5542, Las Vegas, NV , USA, June 2016. IEEE
2016
-
[18]
Bidirectional Projection Network for Cross Dimension Scene Understanding, March 2021
Wenbo Hu, Hengshuang Zhao, Li Jiang, Jiaya Jia, and Tien-Tsin Wong. Bidirectional Projection Network for Cross Dimension Scene Understanding, March 2021. arXiv:2103.14326 [cs]
2021 arXiv
-
[19]
Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023
2023
-
[20]
Openshape: Scaling up 3d shape representation towards open-world understanding, 2023
Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understanding, 2023
2023
-
[21]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...
2021
-
[22]
Uni3d: Exploring unified 3d representation at scale
Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. In International Conference on Learning Representations (ICLR), 2024
2024
-
[23]
Han-Hung Lee, Yiming Zhang, and Angel X. Chang. Duoduo clip: Efficient 3d understanding with multi-view images, 2024
2024
-
[24]
Pointnet++: Deep hierarchical feature learning on point sets in a metric space
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems , 30, 2017
2017
-
[25]
Martin, and Shi-Min Hu
Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R. Martin, and Shi-Min Hu. PCT: Point cloud transformer. Computational Visual Media, 7(2):187–199, June 2021. arXiv:2012.09688 [cs]
2021 arXiv
-
[26]
Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis, July 2021
Tiange Xiang, Chaoyi Zhang, Yang Song, Jianhui Yu, and Weidong Cai. Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis, July 2021. arXiv:2105.01288 [cs]
2021 arXiv
-
[27]
Point-bert: Pre-training 3d point cloud transformers with masked point modeling, 2022
Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling, 2022
2022
-
[28]
Point transformer v2: Grouped vec- tor attention and partition-based pooling
Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Hengshuang Zhao. Point transformer v2: Grouped vec- tor attention and partition-based pooling. Advances in Neural Information Processing Systems , 35:33330– 33342, 2022
2022
-
[29]
Point transformer v3: Simpler, faster, stronger
Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler, faster, stronger. In CVPR, 2024
2024
-
[30]
Openclip, July 2021
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. If you use this software, please cite it as below
2021
-
[31]
Gpt-4 technical report, 2023
OpenAI. Gpt-4 technical report, 2023. 12 A 3DCoMPaT200 Documentation and Intended Uses A.1 Overview 3DCoMPaT200 consists of 19,051 3D shapes with 1,031 segmented fine-grained parts, 118 coarse- grained parts, 293 fine-grained materials, and 13 coarse-grained materials with a 1...
2023
-
[32]
The colors are then merged with the metadata in the format of The {shape_name} is made of {color} {material_name} {part_name} as shown in figure 7 For each level of parts in the Compositional Shape Retrieval Benchmark, we create captions with up to 1, 3, and 6 parts per shape ...
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.