REVIEW 4 major objections 6 minor 31 references
Generalizable Articulated Object Perception with Superpoints
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A superpoint-based pipeline with SAM-guided queries lifts articulated-object part segmentation to 77.9% AP50 on seen categories and 39.3% on unseen categories.
desk verdict A solid, well-ablated architectural extension for articulated part segmentation, but the key SAM-guided query mechanism is under-analyzed and the results lack reproducibility details. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery has three stacked parts. First, part-aware superpoints: each point is softly assigned to its nearest six superpoints through an MLP that combines coordinate and feature similarities, and the assignment is refined with a self-supervised consistency loss so superpoint boundaries align with articulation parts. Second, SAM-guided queries: the 2D foundation model segments the image into pixel regions, each region's center is back-projected into the point cloud, and the superpoint containing that point becomes a query. Third, a six-layer query-based transformer decoder with cross-attention and many-to-one matching, trained with classification, BCE, Dice, and IoU-score losses, turns those queries into part masks.
What would settle it
Measure, on the unseen GAPartNet categories, the fraction of SAM-region centers whose back-projected 3D point falls in a superpoint whose ground-truth part label disagrees with the region's majority label, or whose superpoint overlaps multiple parts; if that fraction is large while AP50 stays high, or if replacing SAM-guided centers with random or rule-based centers leaves performance unchanged, then the SAM-query mechanism is not the source of the claimed generalization.
Extended reading notes
Core claim
The central claim is that part segmentation in point clouds becomes both more accurate and more transferable when (i) superpoints are learned to respect part boundaries rather than fixed by hand-crafted grouping, and (ii) the queries that seed a transformer decoder come from back-projecting 2D SAM region centers into 3D instead of being parameterized or projected from points. In support, the paper reports AP50 of 77.9% on seen categories and 39.3% on unseen categories, including an 11.1-point gain on the small slider-button class and best results on all nine unseen part classes. Ablations trace the gains to all three components: superpoint representation over raw points, SAM-guided query selection over parameterized and point-to-center queries, and the part-aware refinement of superpoint boundaries.
Load-bearing premise
The load-bearing assumption is that the center of each SAM-segmented 2D region, when back-projected into the 3D point cloud, lands inside the correct articulation part and selects a superpoint that does not straddle multiple parts; the paper does not measure how often this localization fails on unseen categories.
Editorial extensions
If this is right
- A robot using GAPS can segment the movable parts of a previously unseen articulated-object category without retraining, since the unseen-category AP50 of 39.3% is the best reported on GAPartNet for every part class.
- The superpoint representation is the largest single driver of transfer in the ablations: removing it drops seen AP50 from 77.9% to 49.9% and unseen AP50 from 39.3% to 21.5%.
- SAM-guided query selection matters most for generalization: it beats parameterized queries (20.2% unseen AP50) and point-to-center projection queries (31.1%) by wide margins.
- The method's biggest seen-category improvement is on the small slider-button part (11.1 points), indicating that the design helps resolve small movable parts that prior methods miss.
Reading between the lines
- If the reported numbers hold, the SAM-guided query idea could be tested as a general way to inject 2D semantic anchors into other point-cloud tasks, such as 6-DoF pose estimation or affordance prediction, without full 3D supervision.
- The GAPartNet gains are measured on rendered simulation data; a natural next experiment is real-world RGB-D scans, where SAM's 2D regions and the back-projection step may degrade, so the 11.6-point unseen gain is an upper bound rather than a guarantee.
- Because the ablations show superpoints matter most, an open question is whether the part-aware superpoint refinement transfers to scene-level or multi-object point clouds, not just single objects.
- One testable extension is to feed the segmented superpoint queries into an articulation-model estimator, converting part masks into joint-axis predictions for planning a manipulation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript proposes GAPS, a 3D part-segmentation method for articulated objects. It first builds superpoints with a learnable part-aware soft association map, then uses centers of SAM-segmented 2D regions to select 3D superpoints as queries for a six-layer transformer decoder with many-to-one matching. Experiments on GAPartNet report AP50 of 77.9% on seen and 39.3% on unseen categories, surpassing PointGroup, SoftGroup, AutoGPart, GAPartNet, and SPFormer; ablations attribute gains to the superpoint representation, position-based queries, and SAM-guided query selection.
Significance. The empirical claim, if reproduced, is of value to the articulated-object perception community: the method reports a 4.1-point absolute gain over the previous state of the art on unseen categories and a 3.3-point gain on seen categories on a public benchmark, with per-part gains on all unseen part classes. The use of standard baselines and a public dataset is a strength, as is the inclusion of sequential ablations. The main weakness is evidentiary: the central attribution of the unseen-category gain to SAM-guided query selection rests on an unmeasured projection step, and the absence of error bars leaves the magnitude of several reported differences unknown. The contribution relative to SPNet/SPFormer is incremental but distinct.
major comments (4)
- [Section III-B and Table II] The paper's core mechanism is never measured. The 'Proj. query' to 'Ours' ablation is the only evidence that SAM-guided query selection causes the 8.2-point unseen gain, but the manuscript does not report how often a SAM region center back-projects to a superpoint whose ground-truth part matches the region's part, nor how often a SAM region spans multiple parts. Please add a direct evaluation of query-selection accuracy (e.g., part-consistency hit rate) and an ablation replacing SAM centers with randomly sampled superpoint queries matched in count; without this, the gain could be due to extra queries or position embeddings rather than semantic localization.
- [Section IV-A] The seen/unseen category split is described only as '17 seen and 10 unseen categories, ensuring all 9 part classes are represented in both.' The actual category lists are not provided. Because the entire cross-category claim depends on this split, please enumerate the 17/10 categories and report per-category AP; this also lets readers check for accidental category overlap or easy/hard splits.
- [Section III-B] The back-projection procedure from SAM pixel-region centers to 3D superpoint queries is underspecified. The text and Fig. 1 say 'Map to 3d position' but do not state how the 2D center is converted to a 3D coordinate (camera intrinsics and depth association), how the corresponding superpoint is selected (nearest center, all points, etc.), or how occlusions and multiple SAM regions per part are handled. Please provide the exact algorithm and hyperparameters, since this is the method's main novelty.
- [Tables I and II] All results are single-run, with no standard deviation or number of seeds. Several claimed advantages are small (e.g., seen Sd.Dw 66.4 vs GAP 64.1; seen Hg.Hl 87.5 vs GAP 90.1), so it is unclear whether the differences are significant. Please report mean and standard deviation over at least three training runs, or otherwise justify the absence of variance reporting.
minor comments (6)
- [Section III-B] After Eq. (5), 'corss-attention' should be 'cross-attention'.
- [Section III-A, Eq. (4)] The expression for the pseudo labels, written as a 'mod' operation, should be defined precisely; since the point labels are one-hot vectors, the operation is presumably a mode/voting step, and the notation should be clarified.
- [Section III-B] The text is ambiguous about whether the decoder queries are only position embeddings from SAM-selected superpoints or the concatenation of superpoint features and position embeddings; please clarify the exact query construction.
- [Abstract and Section IV-B] The improvements '4.4%' and '11.6%' are relative percentages (the absolute AP50 gains are +3.3 and +4.1). Please state this explicitly to avoid misinterpretation.
- [Section IV-A] Please specify the rendering/sensor setup, including the number of views per object and whether the same RGB-D frames are used for all methods in Table I.
- [Table II caption] The rows 'Para. query' and 'Proj. query' are defined in the text, but the caption itself would benefit from one-sentence definitions, as the table is often read standalone.
Circularity Check
No circularity: GAPS is an empirically evaluated supervised segmentation method whose claims are independent test-set measurements.
full rationale
The paper makes no formal derivation whose output is equivalent to its input by construction. Its central claims are experimental AP50 numbers measured on the GAPartNet test split, including held-out unseen categories, which are external to the method's own design choices. The superpoint supervision loss in Eq. (4) uses ground-truth part labels during training, which is standard supervised learning rather than a fitted parameter renamed as a prediction. The SAM-guided query selection described in Section III-B is a heuristic that maps 2D region centers to 3D superpoints; it is an input to the pipeline, not a quantity derived from the segmentation outputs, so its correctness is an empirical risk rather than a circular step. The ablation study in Table II compares alternative query schemes on the same held-out data, and the reported gains are measured rather than deduced from the method's assumptions. The self-citations appearing in the references (e.g., [1], [3], [15], [18]) are background or related-work items and are not load-bearing for the paper's main argument. No quoted equation or cited prior result is invoked to forbid alternatives or to define the target quantity in terms of the prediction itself. Therefore, no significant circularity is present, and the derivation chain is self-contained with respect to the empirical evaluation.
Assumptions & free parameters
free parameters (3)
- Loss weights =
lambda_cls=1.5, lambda_bce=1.25, lambda_dice=1.0, lambda_score=1.0
- Nearest superpoint count =
6
- Decoder depth =
6 layers
assumptions (3)
- domain assumption Points within the same superpoint belong to the same object part
- domain assumption SAM provides reliable 2D region centers for part localization on both seen and unseen categories
- domain assumption The 17 seen and 10 unseen category split is representative and covers all nine part classes
Cite this review
Pith. "Pith review of Generalizable Articulated Object Perception with Superpoints." pith.science (2026). https://pith.science/paper/3FX6M55V
@misc{pith2026241216656,
author = {Pith},
title = {Pith review of: Generalizable Articulated Object Perception with Superpoints},
year = {2026},
howpublished = {\url{https://pith.science/paper/3FX6M55V}},
note = {Machine review of arXiv:2412.16656}
}
abstract
Manipulating articulated objects with robotic arms is challenging due to the complex kinematic structure, which requires precise part segmentation for efficient manipulation. In this work, we introduce a novel superpoint-based perception method designed to improve part segmentation in 3D point clouds of articulated objects. We propose a learnable, part-aware superpoint generation technique that efficiently groups points based on their geometric and semantic similarities, resulting in clearer part boundaries. Furthermore, by leveraging the segmentation capabilities of the 2D foundation model SAM, we identify the centers of pixel regions and select corresponding superpoints as candidate query points. Integrating a query-based transformer decoder further enhances our method's ability to achieve precise part segmentation. Experimental results on the GAPartNet dataset show that our method outperforms existing state-of-the-art approaches in cross-category part segmentation, achieving AP50 scores of 77.9% for seen categories (4.4% improvement) and $39.3\%$ for unseen categories (11.6% improvement), with superior results in 5 out of 9 part categories for seen objects and outperforming all previous methods across all part categories for unseen objects.
Figures
Reference graph
Works this paper leans on
-
[1]
L. Zhang, Z. Han, Y . Zhong, Q. Yu, X. Wu et al., “V ocapter: V oting- based pose tracking for category-level articulated object via inter-frame priors,” in ACM Multimedia 2024 , 2024
work page 2024
-
[2]
Kpa- tracker: Towards robust and real-time category-level articulated object 6d pose tracking,
L. Liu, A. Huang, Q. Wu, D. Guo, X. Yang, and M. Wang, “Kpa- tracker: Towards robust and real-time category-level articulated object 6d pose tracking,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 4, 2024, pp. 3684–3692
work page 2024
-
[3]
Manipose: A comprehensive benchmark for pose-aware object manipulation in robotics,
Q. Yu, C. Hao, J. Wang, W. Liu, L. Liu, Y . Mu, Y . You, H. Yan, and C. Lu, “Manipose: A comprehensive benchmark for pose-aware object manipulation in robotics,” arXiv preprint arXiv:2403.13365 , 2024
arXiv 2024
-
[4]
Gapartnet: Cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts,
H. Geng, H. Xu, C. Zhao, C. Xu, L. Yi, S. Huang, and H. Wang, “Gapartnet: Cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7081–7091
2023
-
[5]
End- to-end affordance learning for robotic manipulation,
Y . Geng, B. An, H. Geng, Y . Chen, Y . Yang, and H. Dong, “End- to-end affordance learning for robotic manipulation,” arXiv preprint arXiv:2209.12941, 2022
arXiv 2022
-
[6]
Where2act: From pixels to actions for articulated 3d objects,
K. Mo, L. J. Guibas, M. Mukadam, A. Gupta, and S. Tulsiani, “Where2act: From pixels to actions for articulated 3d objects,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 6813–6823
2021
-
[7]
Category-level articulated object 9d pose estimation via reinforcement learning,
L. Liu, J. Du, H. Wu, X. Yang, Z. Liu, R. Hong, and M. Wang, “Category-level articulated object 9d pose estimation via reinforcement learning,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 728–736
work page 2023
-
[8]
Vat-mart: Learning visual action trajectory proposals for manipulating 3d articulated objects,
R. Wu, Y . Zhao, K. Mo, Z. Guo, Y . Wang, T. Wu, Q. Fan, X. Chen, L. Guibas, and H. Dong, “Vat-mart: Learning visual action trajectory proposals for manipulating 3d articulated objects,” arXiv preprint arXiv:2106.14440, 2021
arXiv 2021
Show all 31 references
-
[9]
Where2explore: Few- shot affordance learning for unseen novel categories of articulated objects,
C. Ning, R. Wu, H. Lu, K. Mo, and H. Dong, “Where2explore: Few- shot affordance learning for unseen novel categories of articulated objects,” Advances in Neural Information Processing Systems , vol. 36, 2024
2024
-
[10]
Survey on modeling of articulated objects,
J. Liu, M. Savva, and A. Mahdavi-Amiri, “Survey on modeling of articulated objects,” arXiv preprint arXiv:2403.14937 , 2024
2024 arXiv
-
[11]
Captra: Category-level pose tracking for rigid and articulated objects from point clouds,
Y . Weng, H. Wang, Q. Zhou, Y . Qin, Y . Duan, Q. Fan, B. Chen, H. Su, and L. J. Guibas, “Captra: Category-level pose tracking for rigid and articulated objects from point clouds,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 13 209–13 218
2021
-
[12]
Screwnet: Category- independent articulation model estimation from depth images using screw theory,
A. Jain, R. Lioutikov, C. Chuck, and S. Niekum, “Screwnet: Category- independent articulation model estimation from depth images using screw theory,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 13 670–13 677
2021
-
[13]
Flowbot++: Learning generalized articulated objects manipulation via articulation projection,
H. Zhang, B. Eisner, and D. Held, “Flowbot++: Learning generalized articulated objects manipulation via articulation projection,” arXiv preprint arXiv:2306.12893, 2023
2023 arXiv
-
[14]
Ditto: Building digital twins of articulated objects from interaction,
Z. Jiang, C.-C. Hsu, and Y . Zhu, “Ditto: Building digital twins of articulated objects from interaction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 5616–5626
2022
-
[15]
Gamma: Generalizable articulation modeling and manipulation for articulated objects,
Q. Yu, J. Wang, W. Liu, C. Hao, L. Liu, L. Shao, W. Wang, and C. Lu, “Gamma: Generalizable articulation modeling and manipulation for articulated objects,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 13 670–13 677
2024
-
[16]
Pointnet++: Deep hierarchical feature learning on point sets in a metric space,
C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” 2017
2017
-
[17]
Category-level articulated object pose estimation,
X. Li, H. Wang, L. Yi, L. J. Guibas, A. L. Abbott, and S. Song, “Category-level articulated object pose estimation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 3706–3715
2020
-
[18]
Rethinking 3d convolution in ℓp-norm space,
L. Zhang, Y . Zhong, J. Wang, Z. Min, L. Liu et al., “Rethinking 3d convolution in ℓp-norm space,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024
2024
-
[19]
Efficient 3d semantic segmentation with superpoint transformer,
D. Robert, H. Raguet, and L. Landrieu, “Efficient 3d semantic segmentation with superpoint transformer,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 17 195–17 204
2023
-
[20]
Superpoint network for point cloud oversegmentation,
L. Hui, J. Yuan, M. Cheng, J. Xie, X. Zhang, and J. Yang, “Superpoint network for point cloud oversegmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 5510–5519
2021
-
[21]
Oneformer3d: One transformer for unified point cloud segmentation,
M. Kolodiazhnyi, A. V orontsova, A. Konushin, and D. Rukhovich, “Oneformer3d: One transformer for unified point cloud segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 943–20 953
2024
-
[22]
Segment anything,
A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026
2023
-
[23]
Point transformer v2: Grouped vector attention and partition-based pooling,
X. Wu, Y . Lao, L. Jiang, X. Liu, and H. Zhao, “Point transformer v2: Grouped vector attention and partition-based pooling,” Advances in Neural Information Processing Systems , vol. 35, pp. 33 330–33 342, 2022
2022
-
[24]
Weakly supervised segmentation-aided classification of urban scenes from 3d lidar point clouds,
S. Guinard and L. Landrieu, “Weakly supervised segmentation-aided classification of urban scenes from 3d lidar point clouds,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences , vol. 42, pp. 151–157, 2017
2017
-
[25]
Superpoint transformer for 3d scene instance segmentation,
J. Sun, C. Qing, J. Tan, and X. Xu, “Superpoint transformer for 3d scene instance segmentation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 2, 2023, pp. 2393–2401
2023
-
[26]
Query refinement transformer for 3d instance segmentation,
J. Lu, J. Deng, C. Wang, J. He, and T. Zhang, “Query refinement transformer for 3d instance segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 18 516–18 526
2023
-
[27]
Sapien: A simulated part-based interactive environment,
F. Xiang, Y . Qin, K. Mo, Y . Xia, H. Zhu, F. Liu, M. Liu, H. Jiang, Y . Yuan, H. Wanget al., “Sapien: A simulated part-based interactive environment,” 2020
2020
-
[28]
Scannet: Richly-annotated 3d reconstructions of indoor scenes,
A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 5828–5839
2017
-
[29]
Pointgroup: Dual- set point grouping for 3d instance segmentation,
L. Jiang, H. Zhao, S. Shi, S. Liu, C.-W. Fu, and J. Jia, “Pointgroup: Dual- set point grouping for 3d instance segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and Pattern recognition , 2020, pp. 4867–4876
2020
-
[30]
Softgroup for 3d instance segmentation on point clouds,
T. Vu, K. Kim, T. M. Luu, T. Nguyen, and C. D. Yoo, “Softgroup for 3d instance segmentation on point clouds,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 2708–2717
2022
-
[31]
Autogpart: Intermediate supervision search for generalizable 3d part segmentation,
X. Liu, X. Xu, A. Rao, C. Gan, and L. Yi, “Autogpart: Intermediate supervision search for generalizable 3d part segmentation,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11 624–11 634
2022
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.