Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Concept-Based Mechanistic Interpretability Using Structured Knowledge Graphs

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces BAGEL, a model-agnostic framework that maps how semantic concepts surface and interact across a network's layers and compares them against dataset biases in an interactive knowledge graph.

desk verdict BAGEL is a useful bias-auditing tool, but the mechanistic-interpretability framing overreaches and the main recall metric is partly circular on the CLIP-labeled datasets. read the letter →

arxiv 2507.05810 v1 pith:PHHOHQGU submitted 2025-07-08 cs.LG cs.AIcs.CV

Sofiia Chorna , Kateryna Tarelkina , Eloïse Berthier , Gianni Franchi This is my paper
classification cs.LGcs.AIcs.CV
keywords BAGELconcept-basedinterpretabilitymechanisticknowledgegraphsdatasetbiasspuriouscorrelationsCLIPzero-shotlabelingmodel-agnosticanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BAGEL is a model-agnostic framework that tries to move concept-based interpretability from single-prediction explanations to a global, layer-by-layer account of how a vision model uses high-level semantic concepts. It estimates, for each layer and class, how often a concept is present in internal activations, and compares those numbers against the concept-class frequencies in the training dataset. The comparison is rendered as an interactive knowledge graph whose edge colors mark concepts the dataset and model agree on, biases the model failed to learn, and biases the model invented on its own. If the framework works as claimed, it gives practitioners a direct way to see which spurious correlations a model has absorbed and where in the network they live.

What carries the argument

The load-bearing object is the per-layer concept probability $p_{\text{model}}^{(\ell)}(c_k|y_i)$ (Eq. 7), built by applying global average pooling to each convolutional layer to get a vector $\phi^\ell(x)$ (Eq. 4), training a logistic regression $g_k^{(\ell)}$ to predict concept presence from that vector (Eq. 6), and averaging its outputs over images of a class. This is compared with the empirical dataset bias $p_{\text{dataset}}(c_k|y_i)=N_{y_i,c_k}/N_{y_i}$ (Eq. 5), where concept labels come from CLIP zero-shot classification. The comparison is summarized by weighted F1 and Jensen-Shannon divergence, and visualized in a directed labeled graph $G=(V,E,W)$ whose edge colors encode overlap, dataset-only, and model-only biases.

What would settle it

Run BAGEL on a dataset where CLIP's concept labels are known to be wrong (for example, stylized or textured images that defeat its text-image matching) and check whether the red 'model-specific bias' edges concentrate on those mislabeled concepts; if they do, the claimed bias signal is partly an artifact of the labeler rather than the model's internal circuitry.

Watch

Extended reading notes

Core claim

The central claim is that a network's internal representations can be read as a layer-resolved probability distribution over human-interpretable concepts, and that comparing this distribution to the dataset's empirical concept-class distribution exposes the mechanisms behind biased decisions. For each class $y_i$, concept $c_k$, and layer $\ell$, BAGEL computes $p_{\text{model}}^{(\ell)}(c_k|y_i)$ by training logistic-regression concept classifiers on global-average-pooled activations, and contrasts it with $p_{\text{dataset}}(c_k|y_i)$ estimated from CLIP zero-shot labels. Weighted F1 and Jensen-Shannon divergence quantify alignment; the knowledge graph colors each class-concept edge green (both agree), blue (dataset bias the model does not carry), or red (model-specific bias absent from the data). The paper reports that BAGEL recovers top dataset-biased concepts with recall comparable to or better than TCAV and SAE across five datasets and eight architectures.

Load-bearing premise

The entire pipeline assumes that CLIP's zero-shot concept labels correctly say which concepts appear in an image, because those same labels set the dataset-bias statistics and train the concept classifiers.

Editorial extensions

If this is right

  • A user can adjust the threshold $\tau$, navigate through layers, and visually separate dataset-driven biases from model-invented ones for any class-concept pair.
  • Because the pipeline only needs forward hooks to read activations, it applies to convolutional and fully connected networks without retraining.
  • Layer-wise concept probabilities show a consistent trend across architectures: low-level concepts such as colors and textures peak in early layers, while object parts become prominent in deeper layers.
  • On the reported benchmarks, BAGEL's F1-based recall reaches 0.8 for best-block performance on most datasets, and its JS-based variant matches or exceeds TCAV on MonuMAI and KitFox vs RedFox.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: replace the CLIP concept labeler with human annotations or an independent detector and rerun the pipeline; if the red/blue edge patterns shift, part of what BAGEL labels as model bias is actually labeler bias.
  • A testable extension: use red edges that persist across many layers to prioritize data collection, because those concepts are ones the model relies on even though the dataset does not support them.
  • A practical extension: track each concept's layer-resolved probabilities over training checkpoints to see when a spurious correlation becomes linearly decodable, which could inform early stopping or targeted debiasing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces BAGEL, a post-hoc concept-based interpretability framework that quantifies dataset-level concept-class bias using CLIP zero-shot labels, trains per-layer logistic-regression probes on global-average-pooled activations to estimate concept presence, compares dataset and model probabilities with a binarized F1 score and JS divergence, and visualizes the results as an interactive knowledge graph. It is evaluated on five datasets and eight architectures, with recall-based comparison against TCAV and SAE. The authors claim the method bridges concept-based XAI and mechanistic interpretability by 'revealing latent circuits and information flow' that underlie model decisions.

Significance. If validated, a model-agnostic, scalable global bias-analysis tool with per-layer concept tracking and interactive visualization would be a useful complement to local XAI and neuron-level dissection. The paper has concrete strengths: it is model-agnostic, uses a unified GAP representation across layers, includes an interactive demo, and contains an honest caveat in Section 5.2 that concept detection does not confirm causal reliance. However, the central evaluation is circular, the probe quality is never measured, and the mechanistic claims go beyond what linear probes on pooled activations can establish, so the contribution as currently stated is not established.

major comments (5)
  1. [Section 3.2/3.3, Eq. (5)-(7), Table 1] The evaluation is circular: the ground-truth top biased concepts are computed from p_dataset(c_k|y_i), which is obtained from CLIP zero-shot labels, and the same labels serve as training targets y_k in Eq. (6) for the logistic regressions. High recall therefore measures BAGEL's consistency with CLIP rather than its ability to recover biases actually used by the DNN. The authors should validate CLIP concept detection on the human-annotated datasets (Derm7pt, MonuMAI) and either use independent ground truth or explicitly analyze the impact of CLIP label noise on the reported recall.
  2. [Section 3.3, Table 2] No evidence is provided that the per-layer logistic probes are accurate; the paper never reports probe accuracy, AUC, or confidence intervals. On Derm7pt and Cats/Dogs, the BAGEL F1 'Best Block' is exactly 0.80 for every architecture in Table 2, which is suspiciously constant and suggests metric saturation or an artifact of the top-k/binarization procedure. The authors should report per-block probe performance with error bars over multiple seeds and statistical significance tests.
  3. [Section 5.2, Supp. A, Table 2] The evaluation reports only the best-performing block for each method, which cherry-picks the layer and undermines the paper's central 'across layers' claim. If BAGEL is meant to show how concepts emerge and propagate through the network, the recall metric must be reported per block for all blocks, not only the maximum; otherwise the comparison cannot distinguish a method that works at a single layer from one that tracks concepts consistently throughout the network.
  4. [Abstract, Section 1, Section 5.2] The claims of revealing 'latent circuits and information flow' and explaining 'how and why' the model makes decisions are not supported by the method. BAGEL fits independent logistic regressions on global-average-pooled activations; this yields correlational, single-layer concept probabilities with no interaction terms, no connectivity, and no causal intervention. The paper itself concedes in Section 5.2 that concept detection does not confirm causal reliance. Unless causal validation is added, the mechanistic-interpretability framing should be removed or explicitly downgraded to correlational bias analysis.
  5. [Supp. A] The TCAV and SAE baselines are adapted to the task, but the adaptations are underspecified: no concept sets, number of CAVs, SAE architecture, latent count, or layer-selection details are given. Without a shared, precisely specified protocol, the recall comparison in Tables 1 and 2 cannot be reproduced or interpreted fairly.
minor comments (5)
  1. [Eq. (7)] The notation p_model^(ell)(c_k|y_i) averages image-level probabilities pk(x) over class y_i; please clarify that this is an empirical estimate of P(c_k|y_i) and not a model conditional probability in the strict sense.
  2. [Section 3.5] The edge width formula uses 'n_j' and 'c_i' in place of the earlier 'c_k' and 'y_i'; align the notation with the rest of Section 3.
  3. [Figure 3 caption] The caption mentions ResNet50 while the experimental setup lists ResNet18 but not ResNet50; verify which architecture was actually used.
  4. [References] Reference [26] is a blog post; replace it with a peer-reviewed sparse-autoencoder reference, and merge the duplicated CLIP entries [44] and [45].
  5. [Tables 1-2] The main-text table reports only aggregated recall without error bars; include per-seed variance or confidence intervals so the reader can assess stability.

Circularity Check

1 steps flagged · score 6.0 of 10

The recall evaluation is partially circular: the CLIP labels defining the dataset-bias ground truth are the same labels used to train BAGEL's concept probes.

  1. fitted input called prediction [Section 5.2 / Table 1; ground truth defined in Section 3.2 Eq. (5), probes trained in Section 3.3 Eq. (6).]
    "The recall is calculated by comparing the top 5 concepts biased towards one class in the dataset, as described in Section 3.2, with the top-k biased concepts detected by the techniques, measuring the proportion of true positive concepts correctly identified."

    The ground-truth "top 5 concepts biased towards one class" is computed from p_dataset(c_k|y_i) in Eq. (5), which is the empirical frequency of CLIP zero-shot labels y_k. BAGEL's concept classifiers g^(k)_ell in Eq. (6) are trained on exactly the same CLIP labels y_k, and Eq. (7) aggregates their outputs into p_model^(ell)(c_k|y_i). The recall therefore compares one function of the CLIP labels (averaged over classes) against another function of the same labels (a trained probe's class-average output). Any systematic CLIP misclassification is shared by both sides, so a high recall can reflect self-consistency of the probe with its own training labels rather than evidence that the DNN's decisions rely on the concept.

full rationale

The core BAGEL pipeline is not internally circular: per-layer concept probabilities are obtained from independently trained logistic regressions applied to DNN activations, and the comparison against dataset-bias statistics is a legitimate design choice for probing linear decodability. The paper also contains only minor self-citations that are not load-bearing, and it does not invoke any uniqueness theorem or ansatz smuggled in from the authors' prior work. However, the quantitative evaluation in Table 1 is partially circular in a specific, quotable way. The reference set for recall is the top-biased concepts derived from p_dataset(c_k|y_i), which is built from CLIP zero-shot labels. BAGEL's concept classifiers are trained on those same CLIP labels, so the metric measures, at least in part, how well a linear probe can reconstruct its own training labels when aggregated by class. Any systematic error in CLIP's concept detection is present on both sides of the comparison, inflating recall without providing evidence about the DNN's actual decision mechanism. This does not invalidate the framework as a tool for measuring whether CLIP-defined concepts are linearly decodable from internal representations, but it does mean the headline claim of "revealing latent circuits and information flow that underlie model decision-making" goes beyond what the evaluation can support. The partial circularity is confined to the evaluation metric rather than the entire derivation, so the score is moderate rather than extreme.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central comparison depends on CLIP labels being correct, linear probes being sufficient, and the hand-chosen concept vocabulary being representative. No code or data is released, and the threshold and top-k are user- or author-set.

free parameters (4)
  • Threshold tau = 0.5 (default, user-adjustable)
    Binarizes dataset and model concept probabilities for F1 and edge inclusion (Section 3.4, 3.5). Not derived, hand-set.
  • Logistic regression regularization C = typically 0.1
    Tuned by grid search over {0.01, 0.1, 1.0, 10.0} with 5-fold CV (Section 4). A fitted hyperparameter per layer and concept.
  • Concept set per dataset = hand-defined lists, e.g. 10 concepts per class for Husky/Wolf
    Supplementary Table 4. Vocabulary is fixed by authors, despite Section 2.1 claiming dynamic derivation.
  • Top-k in recall metric = unspecified
    Section 5.2 compares top-k model concepts to top-5 dataset concepts; k is not reported, which changes recall values.
assumptions (4)
  • domain assumption CLIP zero-shot classification reliably detects concept presence in images
    Section 3.2, Eq. 5: p_dataset uses CLIP labels as ground truth; no validation against human annotation.
  • domain assumption Concept presence is linearly decodable from global-average-pooled activations
    Section 3.3, Eq. 6-7: logistic regression on GAP features; probe accuracy not reported.
  • domain assumption Dataset-level concept-class frequency is an appropriate reference for model bias
    Sections 3.2-3.4 compare model probabilities to p_dataset as the bias baseline.
  • domain assumption Global average pooling preserves enough information to detect localized concepts
    Section 3.1: GAP discards spatial position; the authors acknowledge this but use it for uniformity across layers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Concept-Based Mechanistic Interpretability Using Structured Knowledge Graphs." pith.science (2026). https://pith.science/paper/PHHOHQGU

@misc{pith2026250705810,
  author       = {Pith},
  title        = {Pith review of: Concept-Based Mechanistic Interpretability Using Structured Knowledge Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PHHOHQGU}},
  note         = {Machine review of arXiv:2507.05810}
}
read the original abstract

While concept-based interpretability methods have traditionally focused on local explanations of neural network predictions, we propose a novel framework and interactive tool that extends these methods into the domain of mechanistic interpretability. Our approach enables a global dissection of model behavior by analyzing how high-level semantic attributes (referred to as concepts) emerge, interact, and propagate through internal model components. Unlike prior work that isolates individual neurons or predictions, our framework systematically quantifies how semantic concepts are represented across layers, revealing latent circuits and information flow that underlie model decision-making. A key innovation is our visualization platform that we named BAGEL (for Bias Analysis with a Graph for global Explanation Layers), which presents these insights in a structured knowledge graph, allowing users to explore concept-class relationships, identify spurious correlations, and enhance model trustworthiness. Our framework is model-agnostic, scalable, and contributes to a deeper understanding of how deep learning models generalize (or fail to) in the presence of dataset biases. The demonstration is available at https://knowledge-graph-ui-4a7cb5.gitlab.io/.

Figures

Figures reproduced from arXiv: 2507.05810 by the authors.

Figure 1
Figure 1. BAGEL. A conceptual network analysis approach [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed BAGEL framework. The distribution of the concepts in the embeddings across DNN layers is compared to the original concept distribution in the dataset. A knowledge graph visualizes the alignment between these distributions to enhance the exploration and understanding of biases. Models. The following models were used in the com￾parison: AlexNet [34], ResNet18 [22], DenseNet121 and DenseNet169 … view at source ↗
Figure 4
Figure 4. Example of concept detection dynamics across the layers. Probabilities show if the concept is present in VGG’s residual layer features for a class Cat upon Cats/Dogs dataset. In addition, we investigate the transition of the con￾cept probabilities within the layers embeddings. Fig￾ure 4 shows the evolution of these probabilities across the layers of the VGG model trained on the Cats/Dogs dataset. Concept probabiliti… view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Examples of found biases using BAGEL. • Or￾ange nodes are classes, colored nodes are concepts, edge col￾ors show bias sources: green (overlapping), blue (dataset￾specific), red (model-specific). a) ResNet50 (layer 3) on Redfox vs. Kitfox; b) DenseNet121 (denseblock4) o…
Figure 5
Figure 5. Figure 5: Dynamics of concept representation across layers. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Geometric Unification of Concept Learning with Concept Cones

    cs.AI 2025-12 conditional novelty 4.0 of 10

    CBMs and SAEs both learn nonnegative linear concept cones; a new containment metric suite scores SAE dictionaries against CBM concepts.

Reference graph

Works this paper leans on

57 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    Sanity checks for saliency maps

    Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. InAdvances in Neural Information Processing Systems (NeurIPS), pages 9505–9515, 2018. 3, 8

    work page 2018

  2. [2]

    Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai.Information fu- sion, 58:82–115, 2020

    Alejandro Barredo Arrieta, Natalia D ´ıaz-Rodr´ıguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garc ´ıa, Sergio Gil-L ´opez, Daniel Molina, Richard Benjamins, et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai.Information fu- sion, 58:82–115, 2020. 1

    work page 2020

  3. [3]

    Andrew Bai, Chih-Kuan Yeh, Pradeep Ravikumar, Neil Y . C. Lin, and Cho-Jui Hsieh. Concept Gradient: Concept-based Interpretation Without Linear Assump- tion, 2024. arXiv:2208.14966. 2

    work page Pith review arXiv 2024

  4. [4]

    Network dissection: Quantifying in- terpretability of deep visual representations

    David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying in- terpretability of deep visual representations. InComputer Vision and Pattern Recognition, 2017. 3, 7

    work page 2017

  5. [5]

    Under- standing the role of individual units in a deep neural net- work.Proceedings of the National Academy of Sciences, 117(48):30071–30078, 2020

    David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. Under- standing the role of individual units in a deep neural net- work.Proceedings of the National Academy of Sciences, 117(48):30071–30078, 2020. 3, 7

    work page 2020

  6. [6]

    Greybox XAI: A Neural-Symbolic learning framework to produce inter- pretable predictions for image classification.Knowledge- Based Systems, 258:109947, 2022

    Adrien Bennetot, Gianni Franchi, Javier Del Ser, Raja Chatila, and Natalia D ´ıaz-Rodr´ıguez. Greybox XAI: A Neural-Symbolic learning framework to produce inter- pretable predictions for image classification.Knowledge- Based Systems, 258:109947, 2022. 2

    work page 2022

  7. [7]

    Support-vector networks.Machine Learning, 20(3):273–297, 1995

    Corinna Cortes and Vladimir Vapnik. Support-vector networks.Machine Learning, 20(3):273–297, 1995. 1

    work page 1995

  8. [8]

    Imagenet: A large-scale hierarchical im- age database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database. InCVPR09, 2009. 6

    work page 2009

Show all 57 references
  1. [9]

    Bert: Pre-training of deep bidirec- tional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. InPro- ceedings of NAACL-HLT, 2019. 1

    2019

  2. [10]

    Natalia D ´ıaz-Rodr´ıguez, Alberto Lamas, Jules Sanchez, Gianni Franchi, Ivan Donadello, Siham Tabik, David Filliat, Policarpo Cruz, Rosana Montes, and Francisco Herrera. Explainable neural-symbolic learning (x-nesyl) methodology to fuse deep learning representations with expe...

    2022

  3. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  4. [12]

    Explainable ai (xai): Core ideas, techniques, and solutions.ACM Computing Surveys, 55(9):1–33, 2023

    Rudresh Dwivedi, Devam Dave, Het Naik, Smiti Sing- hal, Rana Omer, Pankesh Patel, Bin Qian, Zhenyu Wen, Tejal Shah, Graham Morgan, et al. Explainable ai (xai): Core ideas, techniques, and solutions.ACM Computing Surveys, 55(9):1–33, 2023. 1

    2023

  5. [13]

    Visualizing higher-layer features of a deep network.University of Montreal, 1341(3):1, 2009

    Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of a deep network.University of Montreal, 1341(3):1, 2009. 3

    2009

  6. [14]

    A holistic approach to unify- ing automatic concept extraction and concept importance estimation.Advances in Neural Information Processing Systems, 36:54805–54818, 2023

    Thomas Fel, Victor Boutin, Louis B ´ethune, R ´emi Cad`ene, Mazda Moayeri, L ´eo And´eol, Mathieu Chalvi- dal, and Thomas Serre. A holistic approach to unify- ing automatic concept extraction and concept importance estimation.Advances in Neural Information Processing Systems, ...

    2023

  7. [15]

    Craft: Concept recursive activation factorization for explainability

    Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, R ´emi Cad `ene, and Thomas Serre. Craft: Concept recursive activation factorization for explainability. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2023

  8. [16]

    Archetypal sae: Adaptive and stable dictionary learning for concept extraction in large vision models

    Thomas Fel, Ekdeep Singh Lubana, Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal sae: Adaptive and stable dictionary learning for concept extraction in large vision models. arXiv preprint...

    2025 arXiv

  9. [17]

    Interpretable explana- tions of black boxes by meaningful perturbation

    Ruth C Fong and Andrea Vedaldi. Interpretable explana- tions of black boxes by meaningful perturbation. InPro- ceedings of the IEEE International Conference on Com- puter Vision (ICCV), pages 3429–3437, 2017. 3

    2017

  10. [18]

    Regression towards mediocrity in hered- itary stature.The Journal of the Anthropological Institute of Great Britain and Ireland, 15:246–263, 1886

    Francis Galton. Regression towards mediocrity in hered- itary stature.The Journal of the Anthropological Institute of Great Britain and Ireland, 15:246–263, 1886. 1

  11. [19]

    In- terpretation of neural networks is fragile

    Amirata Ghorbani, Abubakar Abid, and James Zou. In- terpretation of neural networks is fragile. InProceedings of the AAAI conference on artificial intelligence, pages 3681–3688, 2019. 2

    2019

  12. [20]

    Towards automatic concept-based explana- tions.Advances in neural information processing sys- tems, 32, 2019

    Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept-based explana- tions.Advances in neural information processing sys- tems, 32, 2019. 2

    2019

  13. [21]

    Addressing leakage in concept bottleneck models.Ad- vances in Neural Information Processing Systems, 35: 23386–23397, 2022

    Marton Havasi, Sonali Parbhoo, and Finale Doshi-Velez. Addressing leakage in concept bottleneck models.Ad- vances in Neural Information Processing Systems, 35: 23386–23397, 2022. 2

    2022

  14. [22]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. 6

    2016

  15. [23]

    Nat- ural language descriptions of deep visual features

    Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. Nat- ural language descriptions of deep visual features. In International Conference on Learning Representations,

  16. [24]

    Rashid, Anisa Rula, Lukas Schmelzeisen, Juan Sequeda, Steffen Staab, and Antoine Zimmermann

    Aidan Hogan, Eva Blomqvist, Michael Cochez, Clau- dia D’amato, Gerard De Melo, Claudio Gutierrez, Sab- rina Kirrane, Jos ´e Emilio Labra Gayo, Roberto Nav- igli, Sebastian Neumaier, Axel-Cyrille Ngonga Ngomo, Axel Polleres, Sabbir M. Rashid, Anisa Rula, Lukas Schmelzeisen, Jua...

    2021

  17. [25]

    Densely connected convolutional networks

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4700–4708, 2017. 6

    2017

  18. [26]

    An intuitive explanation of sparse autoencoders for llm interpretability

    Adam Karvonen. An intuitive explanation of sparse autoencoders for llm interpretability. Blog post, 2024.https : / / adamkarvonen . github . io / machine _ learning / 2024 / 06 / 11 / sae - intuitions.html. 8

    2024

  19. [27]

    Seven-point checklist and skin lesion classification using multitask multimodal neural nets.IEEE Journal of Biomedical and Health In- formatics, 23(2):538–546, 2019

    Jeremy Kawahara, Sara Daneshvar, Giuseppe Argen- ziano, and Ghassan Hamarneh. Seven-point checklist and skin lesion classification using multitask multimodal neural nets.IEEE Journal of Biomedical and Health In- formatics, 23(2):538–546, 2019. 6

    2019

  20. [28]

    CLIP-QDA: An Explainable Concept Bottleneck Model.Transactions on Machine Learning Research, 2023

    R ´emi Kazmierczak, Elo ¨ıse Berthier, Goran Frehse, and Gianni Franchi. CLIP-QDA: An Explainable Concept Bottleneck Model.Transactions on Machine Learning Research, 2023. 2, 6

    2023

  21. [29]

    Explainability for vision foundation models: A survey.arXiv preprint arXiv:2501.12203,

    R ´emi Kazmierczak, Elo ¨ıse Berthier, Goran Frehse, and Gianni Franchi. Explainability for vision foundation models: A survey.arXiv preprint arXiv:2501.12203,

    arXiv

  22. [30]

    Undoing the dam- age of dataset bias

    Aditya Khosla, Tinghui Zhou, Tomasz Malisiewicz, Alexei A Efros, and Antonio Torralba. Undoing the dam- age of dataset bias. InProceedings of the European Con- ference on Computer Vision (ECCV), pages 158–171. Springer, 2012. 3

    2012

  23. [31]

    In- terpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCA V), 2018

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. In- terpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCA V), 2018. arXiv:1711.11279. 2

    2018 arXiv

  24. [32]

    Inter- pretability beyond feature attribution: Quantitative test- ing with concept activation vectors (tcav), 2018

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Inter- pretability beyond feature attribution: Quantitative test- ing with concept activation vectors (tcav), 2018. 7, 8

    2018

  25. [33]

    Concept bottleneck models

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. InProceedings of the 37th International Conference on Machine Learning, pages 5338–5348. PMLR, 2020. 2

    2020

  26. [34]

    Imagenet classification with deep convolutional neu- ral networks.Communications of the ACM, 60(6):84–90,

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hin- ton. Imagenet classification with deep convolutional neu- ral networks.Communications of the ACM, 60(6):84–90,

  27. [35]

    Monumai: Dataset, deep learning pipeline and citizen science based app for monumental heritage taxonomy and classification.Neurocomputing, 420:266– 280, 2021

    Alberto Lamas, Siham Tabik, Policarpo Cruz, Rosana Montes, ´Alvaro Mart´ınez-Sevilla, Teresa Cruz, and Fran- cisco Herrera. Monumai: Dataset, deep learning pipeline and citizen science based app for monumental heritage taxonomy and classification.Neurocomputing, 420:266– 280, 2021. 6

    2021

  28. [36]

    Promises and pitfalls of black-box concept learning models.arXiv preprint arXiv:2106.13314, 2021

    Anita Mahinpei, Justin Clark, Isaac Lage, Finale Doshi- Velez, and Weiwei Pan. Promises and pitfalls of black-box concept learning models.arXiv preprint arXiv:2106.13314, 2021. 2

    2021 arXiv

  29. [37]

    Progress measures for grokking via mechanistic interpretability

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. InThe Eleventh Interna- tional Conference on Learning Representations. 1

  30. [38]

    Clip-dissect: Automatic description of neuron representations in deep vision networks

    Tuomas Oikarinen and Tsui-Wei Weng. Clip-dissect: Automatic description of neuron representations in deep vision networks. InThe Eleventh International Confer- ence on Learning Representations. 3

  31. [39]

    Nguyen, and Tsui-Wei Weng

    Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui-Wei Weng. Label-Free Concept Bottleneck Models,

  32. [40]

    Feature visualization.Distill, 2(11):e7, 2017

    Chris Olah, Alexander Mordvintsev, and Ludwig Schu- bert. Feature visualization.Distill, 2(11):e7, 2017. 3

    2017

  33. [41]

    Zoom in: An introduction to circuits.Distill, 2020

    Chris Olah, Nick Cammarata, Ludwig Schubert, and et al. Zoom in: An introduction to circuits.Distill, 2020. 1

    2020

  34. [42]

    Discover: making vision networks interpretable via competition and dissection.Advances in Neural Information Process- ing Systems, 36:27063–27078, 2023

    Konstantinos Panousis and Sotirios Chatzis. Discover: making vision networks interpretable via competition and dissection.Advances in Neural Information Process- ing Systems, 36:27063–27078, 2023. 3

    2023

  35. [43]

    J. R. Quinlan. Induction of decision trees.Machine Learning, 1(1):81–106, 1986. 1

    1986

  36. [44]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pam Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInterna- tional Con...

    2021

  37. [45]

    Learning Transferable Vi- sual Models From Natural Language Supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Vi- sual Models From Natural Language Supervision, 2021. arXiv:2103.00020. 2

    2021 arXiv

  38. [46]

    Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities.Knowledge-based systems, 263: 110273, 2023

    Waddah Saeed and Christian Omlin. Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities.Knowledge-based systems, 263: 110273, 2023. 1

    2023

  39. [47]

    Very deep con- volutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep con- volutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 6

    2014 arXiv

  40. [48]

    Deep inside convolutional networks: Visualising image classification models and saliency maps.arXiv preprint arXiv:1312.6034, 2013

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisser- man. Deep inside convolutional networks: Visualising image classification models and saliency maps.arXiv preprint arXiv:1312.6034, 2013. 3

    2013 arXiv

  41. [49]

    Don’t judge an object by its context: Learning to overcome contextual bias

    Krishna Kumar Singh, Huan Yu, Gokhan Sumbul, Zux- uan Jiang, and Mubarak Shah. Don’t judge an object by its context: Learning to overcome contextual bias. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 11070–11078,

  42. [50]

    Vlg- cbm: Training concept bottleneck models with vision- language guidance.Advances in Neural Information Pro- cessing Systems, 37:79057–79094, 2024

    Divyansh Srivastava, Ge Yan, and Lily Weng. Vlg- cbm: Training concept bottleneck models with vision- language guidance.Advances in Neural Information Pro- cessing Systems, 37:79057–79094, 2024. 2

    2024

  43. [51]

    Going deeper with convolutions

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Ser- manet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1–...

    2015

  44. [52]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. InPro- ceedings of the International Conference on Machine Learning (ICML), pages 6105–6114, 2019. 6

    2019

  45. [53]

    Unbiased look at dataset bias

    Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1521–1528, 2011. 3

    2011

  46. [54]

    Knowledge graphs for empirical concept retrieval, 2024

    Lenka T ˇetkov´a, Teresa Karen Scheidt, Maria Man- drup Fogh, Ellen Marie Gaunby Jørgensen, Finn ˚Arup Nielsen, and Lars Kai Hansen. Knowledge graphs for empirical concept retrieval, 2024. arXiv:2404.07008. 2

    2024 arXiv

  47. [55]

    Aggregated residual transformations for deep neural networks

    Saining Xie, Ross Girshick, Piotr Doll ´ar, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1492–1500, 2017. 6

    2017

  48. [56]

    Language in a bottle: Language model guided concept bottlenecks for interpretable image classification

    Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. InPro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Patter...

    2023

  49. [57]

    Visualizing and understanding convolutional networks

    Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pages 818–833. Springer, 2014. 3 Table of Contents - Supplementary M...

    2014

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.