REVIEW 4 major objections 7 minor 1 cited by
Personalized Interpretability -- Interactive Alignment of Prototypical Parts Networks
T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Interactive splitting lets users fix a neural network's mixed-up concepts.
desk verdict A genuinely novel split-by-duplication mechanism, but the headline accuracy claim is only tested for automatic splits, not the user-driven use case. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a three-step split: first, the kernel and classification weights of an inconsistent prototype are duplicated into a new channel; second, the prototype's most-activated patches are partitioned into two concept sets S1 and S2 plus a reference set Sr; third, only the two kernels are fine-tuned, using an activation loss to push patches of S1 to the original and S2 to the copy, and a deactivation loss to suppress both on Sr. Because the two channels share a softmax, boosting one automatically suppresses the other. Inconsistency detection is a separate mechanism: a cosine-similarity graph over patch features, maximal cliques (groups of mutually similar features) found by the Bron-Kerbosch algorithm, and a threshold that favors prototypes whose two largest cliques are large and dissimilar.
What would settle it
Take a prototype whose most-activated patches contain three equally frequent visual concepts (for example, head, wing, and tail), run the binary split, and check whether the two resulting prototypes each contain a single concept: if either still mixes two or more concepts, or if user-heuristic agreement on such prototypes falls to chance, the central claim fails.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that concept inconsistency in prototypical-parts networks is fixable interactively: duplicating the prototype kernel, freezing everything else, and fine-tuning the two copies against user-labeled patches reliably separates a mixed concept into two clear ones. The paper presents this as a step toward personalized interpretability, where users are not stuck with whatever concepts the model learned but can reshape explanations to their preferences. It also claims that the heuristic-driven selection of prototypes by maximal cliques in a patch-feature similarity graph aligns with human judgment, with user-heuristic agreement of 92% and 78% on CUB and CARS, and that after splitting ten prototypes per model the accuracy remains at the level of the PIP-Net backbone.
Load-bearing premise
The method assumes that every fixable inconsistency can be captured by a split into exactly two groups: the image regions a prototype responds to must fall into two distinct visual concepts, and enough regions must belong to each group for a person or the automatic detector to label them.
Editorial extensions
If this is right
- After splitting the ten most inconsistent prototypes, CUB purity (the share of a prototype's top patches belonging to its dominant concept) rises from 0.84 to 0.90 while test accuracy stays at 84.3%, matching the PIP-Net ConvNext-Tiny baseline.
- Users rate 88.3% of split prototypes on CUB, 84.2% on CARS, and 80.4% on PETS as more consistent than before, so the interactive correction transfers beyond synthetic data.
- The automatic heuristic reproduces user concept choices well on CUB (92%) and CARS (78%), so the manual labeling burden can be reduced in those regimes.
- FunnyBirds experiments show pattern purity increases most after the first splits, while accuracy declines gradually as more prototypes are split, bounding how many corrections a model can absorb.
Reading between the lines
- If binary splitting is the only operation, the approach likely generalizes to iterative rounds — split, re-check, split again — which the paper names as future work; a natural test is whether three-way inconsistent prototypes become consistent after two sequential binary splits.
- The same kernel-duplication trick could be applied to other prototype-based or concept-bottleneck architectures, not just PIP-Net, since it only requires a softmax over prototype channels and a sparse classification head.
- The 50-54% user-heuristic agreement on PETS suggests that when prototypes contain more than two concepts or concepts are subjective, the automatic selector may need a non-binary clustering step; a testable extension is measuring whether a three-cluster selector restores agreement.
- A possible consequence the paper does not draw out: personalized splitting makes the explanation space depend on the individual user's choices, so reproducibility of explanations may need to be tracked per user session.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. YoursProtoP extends PIP-Net by letting a user (or an automatic heuristic) identify an 'inconsistent' prototype that mixes two visual concepts, duplicate its kernel and classification weights, and fine-tune the two kernels on labeled patch sets so that each specializes to one concept. An automatic selection strategy builds a similarity graph over a prototype's most-activated patches, finds maximal cliques via Bron-Kerbosch, and flags prototypes with two large disjoint cliques as inconsistent. Experiments on FunnyBirds (using segmentation masks) and on CUB/CARS/PETS (with a Prolific-based user study) report improved pattern purity and user-perceived consistency after splitting, with accuracy roughly maintained for automatic splits.
Significance. If it holds, the work offers a practical way to reduce concept inconsistency in case-based interpretable models, a recognized obstacle to user trust. The FunnyBirds evaluation is a strength because it uses external segmentation masks as objective ground truth; the paper also includes a real user study with 54 participants and states that code is released. However, the central claim about accuracy preservation in the interactive setting is not directly tested, and several hyperparameters are tuned on the evaluation datasets, so the evidence is currently weaker than the abstract suggests.
major comments (4)
- [Abstract; Sec. 4.2; Table 2] The headline claim 'without compromising the accuracy of the model' is only verified for automatic heuristic-driven splits (Table 1 and Fig. 5). In the user study, splits are driven by user labels, but Table 2 reports only consistency ratings, agreement, and effort; no test accuracy is reported for user-driven splits. Since PETS shows U-vs-H agreement of only 54% and U-vs-U agreement of 50%, user labels can diverge substantially from the heuristic, and there is no evidence that such splits preserve accuracy. This gap directly affects the paper's central contribution and should be closed with an accuracy measurement in the interactive setting.
- [Sec. 4; Fig. 4; Supplementary Algorithm 1] The values of Q (minimal clique size), the similarity threshold delta*, and the convergence criteria (99.9% per-concept accuracy or loss below 0.02) are selected based on experiments on the same natural datasets (CUB, CARS, PETS) that are later used for evaluation. Figure 4 and Algorithm 1 in the supplementary make this explicit. This constitutes tuning on the test data and may inflate the reported purity and accuracy results; an evaluation on held-out datasets or a clear validation/test split is needed before the reported gains can be taken as out-of-sample evidence.
- [Sec. 5.2; Table 1; Sec. 5.3; Table 2] The natural-dataset evidence for improved consistency is partly circular. The purity improvement on CUB is computed on the 10 most activated patches of the very prototypes that were selected and split, and the user-study consistency ratings are collected from the same users immediately after they performed the binary labeling, with no control condition. As a result, the reported improvements may reflect selection and confirmation effects rather than genuine concept alignment. A blinded or independent evaluation (e.g., separate annotators judging original vs. split prototypes without knowing the source) would substantially strengthen the claim.
- [Abstract; Sec. 5.1; Fig. 5] The claim that splitting can be done 'without compromising the accuracy of the model' is contradicted by Fig. 5, which shows accuracy decreasing monotonically as the number of split prototypes increases on FunnyBirds (from roughly 1.00 to 0.95 after 10 splits). The paper itself states that accuracy decreases. The claim should be qualified, for example to 'with only a small accuracy drop' or by specifying an acceptable tolerance, and the trade-off between number of splits and accuracy should be discussed explicitly.
minor comments (7)
- [Sec. 4] The text says 'we set alpha = 2 to prioritize the deactivation loss in Equation (2)' but the loss is defined in Equation (3); the reference should be corrected.
- [Sec. 5.2; Table 1] The caption and body text are ambiguous about whether the purity values 0.84 and 0.90 in Table 1 are averages over all prototypes or only over the 10 split prototypes; Section 5.2 says 'these channels,' which suggests the latter. Please clarify.
- [Supplementary Algorithm 1] There is a typo in the pseudocode ('Find = cliques' should be 'Find cliques'), and the notation for the two cliques C^1_d and C^2_d is not defined before it is used in the score function.
- [Sec. 4; Sec. 6] There are several typos: 'patientience' should be 'patience' (Sec. 4), 'heruistics' should be 'heuristics' (Sec. 4.2), and 'confunders' should be 'confounders' (Sec. 6).
- [Sec. 5.3] The explanation that PETS low agreement stems from prototypes containing more than two major concepts is speculative; the user study could be extended with a 'more than two concepts' option to test this directly.
- [Supplementary Code] The paper states that code is released under the MIT License but does not provide a repository URL; please include one in the main text or supplementary material.
- [Fig. 4] The figure legend and axes are unclear: the caption mentions both 'No. Prototypes' and 'Avg. Dissimilarity' but does not explain which curve/marker corresponds to which quantity or how the two y-axes (if present) are scaled.
Circularity Check
Moderate self-referential evaluation: FunnyBirds purity gains mirror the training stopping rule, and heuristic thresholds are tuned on the same natural datasets used for the headline results; the interactive accuracy claim is unmeasured.
-
fitted input called prediction
[Section 4 (Experimental Setup, convergence criteria) and Section 5.1 / Figure 5]
"Based on experimental results of split prototypical parts, we set up the convergence criteria during training as either achieving 99.9% per-concept accuracy or reducing the loss below 0.02 ... This value is computed only for the selected and split prototypical parts."
On FunnyBirds, the concept sets S1/S2 used in the splitting loss (Eq. 3) are derived from segmentation masks, i.e., from the same ground-truth patterns that pattern purity PP counts. The procedure is run until per-concept accuracy reaches 99.9%, so the model is forced to route S1 patches to one new prototype and S2 patches to the other. The reported PP increase is then computed only on those selected/split prototypes. Thus the top panel of Fig. 5 is essentially the training objective rewritten as a purity metric: any run that hits the stopping criterion will, by construction, show high PP on the concept sets it was optimized on. The accuracy panel is not forced, which limits the circularity.
-
fitted input called prediction
[Section 4, Figure 4 and Q selection paragraph]
"To determine the optimal value for Q, we have performed an analysis on all of the natural datasets. As can be seen on Figure 4, the dissimilarity between cliques increases dramatically as it reaches a minimum size of Q = 2."
The heuristic threshold Q (and the similarity threshold delta in Algorithm 1) is selected by inspecting the same CUB/CARS/PETS models and datasets later used for the headline Table 1 and Table 2 numbers. The 'most inconsistent' prototypes are defined as those maximizing the dissimilarity score that Q controls, and the purity/user-consistency results are reported on exactly those chosen prototypes. So the detector is fit in-sample to the evaluation distribution; the reported improvements are not an independent out-of-sample prediction, though user agreement on PETS (50-54%) shows the selection is not trivially forced.
full rationale
The paper's central mechanism (duplicate kernel, split loss, frozen backbone) is not circular: Eq. 3 is a genuine supervised objective, and FunnyBirds provides an external segmentation-based benchmark, so the method has independent content. No load-bearing self-citation was found; [7] is a third-party baseline and author self-citations in Related Work are not used to justify the central claim. The two self-referential aspects are (i) the FunnyBirds purity gain is computed on the same prototypes whose split was driven by the ground-truth masks and stopped at 99.9% per-concept accuracy, making the PP curve a restatement of the stopping rule; and (ii) the Q/delta hyperparameters are chosen by analyzing all natural datasets and then the same datasets are used for the reported results, so the detector is fit in-sample. Additionally, the abstract's 'without compromising accuracy' is verified only for heuristic splits (Table 1); the user study (Table 2) measures perceived consistency and agreement but not test accuracy after user-driven splits, an evidence gap rather than circularity. Overall, a moderate self-referential evaluation burden, not a derivation that reduces to its inputs.
Assumptions & free parameters
free parameters (5)
- alpha (alpha) =
2
- kappa (kappa) =
0.1
- Q (minimum clique size) =
2
- similarity threshold delta* =
not reported
- number of split prototypes per model =
10
assumptions (3)
- domain assumption Inconsistency of a prototype is defined as containing two disjoint sets of patches S1,S2 each of size at least Q, corresponding to two distinct visual concepts.
- domain assumption The automatically found maximal cliques in feature space correspond to human-interpretable visual concepts.
- domain assumption Copying a prototype kernel and its classification weights, then fine-tuning only the original and copied kernels while freezing everything else, preserves the model's accuracy and the meaning of all other prototypes.
Cite this review
Pith. "Pith review of Personalized Interpretability -- Interactive Alignment of Prototypical Parts Networks." pith.science (2026). https://pith.science/paper/RFX7ZPCS
@misc{pith2026250605533,
author = {Pith},
title = {Pith review of: Personalized Interpretability -- Interactive Alignment of Prototypical Parts Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/RFX7ZPCS}},
note = {Machine review of arXiv:2506.05533}
}
read the original abstract
Concept-based interpretable neural networks have gained significant attention due to their intuitive and easy-to-understand explanations based on case-based reasoning, such as "this bird looks like those sparrows". However, a major limitation is that these explanations may not always be comprehensible to users due to concept inconsistency, where multiple visual features are inappropriately mixed (e.g., a bird's head and wings treated as a single concept). This inconsistency breaks the alignment between model reasoning and human understanding. Furthermore, users have specific preferences for how concepts should look, yet current approaches provide no mechanism for incorporating their feedback. To address these issues, we introduce YoursProtoP, a novel interactive strategy that enables the personalization of prototypical parts - the visual concepts used by the model - according to user needs. By incorporating user supervision, YoursProtoP adapts and splits concepts used for both prediction and explanation to better match the user's preferences and understanding. Through experiments on both the synthetic FunnyBirds dataset and a real-world scenario using the CUB, CARS, and PETS datasets in a comprehensive user study, we demonstrate the effectiveness of YoursProtoP in achieving concept consistency without compromising the accuracy of the model.
Figures
Figures from the paper (8 more)
Forward citations
Cited by 1 Pith paper
-
Position: Explainability Research Must Prioritize Foundations over Ad-hoc Methods
Explainable AI research should prioritize definitions, properties, evaluations, and actionability over new ad-hoc methods, on evidence from 617 papers and 34 practitioners.
Reference graph
Works this paper leans on
-
[1]
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206–215, 2019
2019
-
[2]
why should i trust you?
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016
2016
-
[3]
On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015
work page 2015
-
[4]
Grad-cam: Visual explanations from deep networks via gradient- based localization
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017
work page 2017
-
[5]
Sanity checks for saliency maps
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Advances in neural information processing systems, vol- ume 31, 2018
work page 2018
-
[6]
Concept bottleneck models
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pages 5338–5348. PMLR, 2020
2020
-
[7]
Pip-net: Patch-based intuitive prototypes for interpretable image classification
Meike Nauta, Jörg Schlötterer, Maurice Van Keulen, and Christin Seifert. Pip-net: Patch-based intuitive prototypes for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2744–2753, 2023
work page 2023
-
[8]
This looks like that: deep learning for interpretable image recognition
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019
2019
Show all 30 references
-
[9]
Hive: Evaluating the human interpretability of visual explanations
Sunnie SY Kim, Nicole Meister, Vikram V Ramaswamy, Ruth Fong, and Olga Russakovsky. Hive: Evaluating the human interpretability of visual explanations. In European Conference on Computer Vision, pages 280–298. Springer, 2022
2022
-
[10]
This looks like those: Illumi- nating prototypical concepts using multiple visualizations
Chiyu Ma, Brandon Zhao, Chaofan Chen, and Cynthia Rudin. This looks like those: Illumi- nating prototypical concepts using multiple visualizations. Advances in Neural Information Processing Systems, 36:39212–39235, 2023
2023
-
[11]
Lucidppn: Un- ambiguous prototypical parts network for user-centric interpretable computer vision
Mateusz Pach, Dawid Rymarczyk, Koryna Lewandowska, Jacek Tabor, et al. Lucidppn: Un- ambiguous prototypical parts network for user-centric interpretable computer vision. ICLR, 2025
2025
-
[12]
Interactive disentanglement: Learning concepts by interacting with their prototype representations
Wolfgang Stammer, Marius Memmel, Patrick Schramowski, and Kristian Kersting. Interactive disentanglement: Learning concepts by interacting with their prototype representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10317–10...
2022
-
[13]
Concept-level debugging of part-prototype networks
Andrea Bontempelli, Stefano Teso, Katya Tentori, Fausto Giunchiglia, and Andrea Passerini. Concept-level debugging of part-prototype networks. arXiv preprint arXiv:2205.15769, 2022
2022 arXiv
-
[14]
Constructing concept-based models to mitigate spu- rious correlations with minimal human effort
Jeeyung Kim, Ze Wang, and Qiang Qiu. Constructing concept-based models to mitigate spu- rious correlations with minimal human effort. In European Conference on Computer Vision, pages 137–153. Springer, 2024
2024
-
[15]
Slim: Spuriousness mitigation with minimal human annotations
Xiwei Xuan, Ziquan Deng, Hsuan-Tien Lin, and Kwan-Liu Ma. Slim: Spuriousness mitigation with minimal human annotations. In European Conference on Computer Vision, pages 215–
-
[16]
Funnybirds: A synthetic vision dataset for a part-based analysis of explainable ai methods
Robin Hesse, Simone Schaub-Meyer, and Stefan Roth. Funnybirds: A synthetic vision dataset for a part-based analysis of explainable ai methods. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3981–3991, 2023. 11
2023
-
[17]
Caltech-ucsd birds 200
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-ucsd birds 200. 2010
2010
-
[18]
3d object representations for fine- grained categorization
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pages 554–561, 2013
2013
-
[19]
Cats and dogs
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and C V Jawahar. Cats and dogs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3498–3505, 2012
2012
-
[20]
ProtoPShare: Pro- totypical Parts Sharing for Similarity Discovery in Interpretable Image Classification
Dawid Rymarczyk, Łukasz Struski, Jacek Tabor, and Bartosz Zieli ´nski. ProtoPShare: Pro- totypical Parts Sharing for Similarity Discovery in Interpretable Image Classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, page 1420–1430, 2021
2021
-
[21]
Interpretable image classification with differentiable prototypes assign- ment
Dawid Rymarczyk, Łukasz Struski, Michał Górszczak, Koryna Lewandowska, Jacek Tabor, and Bartosz Zieli´nski. Interpretable image classification with differentiable prototypes assign- ment. In European Conference on Computer Vision, pages 351–368. Springer, 2022
2022
-
[22]
Principles of ex- planatory debugging to personalize interactive machine learning
Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. Principles of ex- planatory debugging to personalize interactive machine learning. In IUI, 2015
2015
-
[23]
Explanatory interactive machine learning
Stefano Teso and Kristian Kersting. Explanatory interactive machine learning. In AIES, 2019
2019
-
[24]
Making deep neural networks right for the right scientific reasons by interacting with their explanations.Nat
Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne-Katrin Mahlein, and Kristian Kersting. Making deep neural networks right for the right scientific reasons by interacting with their explanations.Nat. Mac...
2020
-
[25]
A case-based interpretable deep learning model for clas- sification of mass lesions in digital mammography
Alina Jade Barnett, Fides Regina Schwartz, Chaofan Tao, Chaofan Chen, Yinhao Ren, Joseph Y Lo, and Cynthia Rudin. A case-based interpretable deep learning model for clas- sification of mass lesions in digital mammography. Nat. Mach. Intell., 2021
2021
-
[26]
Help Me Help the AI
Sunnie S. Y . Kim, Elizabeth Anne Watkins, Olga Russakovsky, Ruth Fong, and Andrés Monroy-Hernández. "Help Me Help the AI": Understanding How Explainability Can Sup- port Human-AI Interaction. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pag...
2023 doi
-
[27]
README.md: Contains detailed setup instructions, dependency information, and a step- by-step guide to reproduce our experimental results
-
[28]
Method Documentation: Comprehensive documentation of the YoursProtoP method, including the prototype splitting mechanism and heuristics for detecting inconsistent pro- totypes
-
[29]
Configuration Files: All hyperparameters and configuration settings used in our experi- ments
-
[30]
The code is released under the MIT License, allowing for both academic and commercial use with proper attribution
Preprocessing Scripts: Code for data preparation and preprocessing for each dataset. The code is released under the MIT License, allowing for both academic and commercial use with proper attribution. No personal data from user studies is included in our released assets. E User...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.