Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

A Geometric Unification of Concept Learning with Concept Cones

T0 review · 5 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Both supervised concept bottleneck models and unsupervised sparse autoencoders learn the same geometric object — a concept cone formed by nonnegative combinations of learned directions — and differ only in how they choose it.

desk verdict A correct but mostly definitional unification, paired with a useful metric suite; the empirical 'sweet spot' claim is not supported because the coverage metric is inflated by dictionary span. read the letter →

arxiv 2512.07355 v2 pith:KU2VHDGK submitted 2025-12-08 cs.AI cs.CVcs.LG

Alexandre Rocchi , Thomas Fel , Gianni Franchi This is my paper
classification cs.AIcs.CVcs.LG
keywords conceptconesbottleneckmodelssparseautoencoderslinearrepresentationhypothesisdictionarylearninginterpretabilitynonnegativecodingconecontainment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to prove that two apparently separate traditions in interpretability — concept bottleneck models, which use human labels to prescribe concepts, and sparse autoencoders, which discover patterns unsupervised — are actually solving the same geometric problem. In both cases the learned object is a 'concept cone': the set of vectors you can form by taking nonnegative weighted sums of a small dictionary of directions in activation space. Supervision and sparsity are just two different regularizers for choosing that cone. The authors then propose metrics that ask whether the cone discovered by an autoencoder contains or approximates the human-defined cone of a bottleneck model, and use them to show that intermediate sparsity and dictionary size give the best alignment. A reader should care because this gives unsupervised concept discovery a reference point it currently lacks: a way to measure progress toward human-plausible structure rather than only reconstruction error.

What carries the argument

The concept cone, defined as the conic hull of the rows of a concept dictionary (for CBMs, the bottleneck weights; for SAEs, the decoder atoms): C_D = {v : v = αᵀD, α ≥ 0}. Its load-bearing role is to convert 'does the SAE understand the same concepts as the CBM?' into a convex-geometric question of whether one cone contains another, with containment testable by a sparse nonnegative reconstruction solved as a Lasso regression. Nonnegativity is the hinge: sigmoid outputs in CBMs and ReLU or TopK codes in SAEs both force codes into the nonnegative orthant, which is why both paradigms land on cones rather than subspaces.

What would settle it

Construct or find a representation where a human-verified concept direction is known to require negative coefficients — for example, the concept 'not striped' or a difference between two positive directions — so it lies outside the nonnegative cone spanned by any valid SAE dictionary. If the proposed coverage and R² metrics still report near-perfect containment for an SAE trained on this representation, then nonnegative cone containment is not carrying the concept-recoverability claim.

Watch

Extended reading notes

Core claim

The central claim, formalized in Observation 1, is that every method in the two families — regardless of whether it uses orthogonality constraints, nonnegativity, archetypal analysis, or a sparsity projection on the unsupervised side, or joint, sequential, label-free, or post-hoc variants on the supervised side — produces nonnegative concept codes and therefore implicitly selects a convex cone in activation space. A concept vector learned by one method is recoverable from another method's dictionary exactly when it lies inside that dictionary's cone; so agreement between methods is literally cone overlap. The paper's operational move is to treat the CBM cone as a human-curated anchor and to

Load-bearing premise

That concepts are linearly decodable directions in activation space whose composition is a nonnegative sum; if a network encodes a meaningful concept as a difference of directions, or encodes it through a nonlinear subspace, the concept cone and its containment metrics measure an artifact of the linear projection rather than actual semantic content.

Editorial extensions

If this is right

  • SAE quality can be benchmarked against human-curated concept cones, giving a measurable 'desirability signal' in addition to reconstruction loss and sparsity.
  • Choosing SAE hyperparameters (sparsity level, dictionary size, expansion factor) can be guided by containment metrics; the paper reports sweet spots at roughly 0.01–0.05% sparsity and around 3× expansion.
  • The framework predicts that methods with better cone containment also produce more human-plausible decompositions, which is directly testable on other backbones and datasets.
  • Layer depth matters: post-pooling features in later ResNet blocks contain the CBM cone better than early layers, suggesting where to attach concept extractors.
  • Randomly initialized SAEs fail the alignment metrics (except coverage, which is inflated), so the metrics distinguish learned structure from apparent interpretability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The same containment test could serve as a training objective — one could fine-tune an SAE to maximize coverage of a CBM-defined cone, blending supervision into unsupervised discovery in a way the paper explores only as evaluation.
  • Editorial inference: The framework suggests a new take on non-identifiability: instead of searching for one true factorization, accept a family of cones and rank them by inclusion and alignment to human anchors; this could carry over to language-model SAEs, where human concept vocabularies are harder to define.
  • Editorial inference: Because the metrics operate on activations and dictionaries only, they could be applied to compare two unsupervised dictionaries to each other, offering a principled way to measure SAE stability across seeds and training runs.
  • A testable extension: applying the containment metrics to models with known ground-truth generative factors would directly test whether the nonnegative-cone assumption holds or whether concepts sometimes require subtraction of directions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a geometric unification of Concept Bottleneck Models (CBMs) and Sparse Autoencoders (SAEs). It observes that both paradigms learn dictionaries of concept directions with nonnegative codes, so their representable subspaces are convex cones (Definition 2, Observation 1). It then introduces a set of metrics — reconstruction residual, coverage, geometric correlation, activation correlation, R2, entropy, and F1 — to measure whether the SAE cone contains or approximates the CBM cone, and reports experiments across backbones and SAE variants. The paper claims a 'sweet spot' in sparsity and expansion factor where geometric and semantic alignment are balanced. The formal observation is correct by construction; the empirical/operational claims, however, depend on an uncalibrated coverage metric and on qualitative readings of tables that largely show monotone trends.

Significance. The geometric perspective is elegant and potentially useful as an organizing framework for comparing supervised and unsupervised concept extraction. The paper is transparent about the plausible-versus-faithful status of CBM concepts (footnote 1) and about the fact that coverage is inflated for random SAEs (Table 1). These are strengths. However, as a theorem, Observation 1 is definitional: nonnegative codes imply the reconstruction lies in the conic hull. The practical value of the paper therefore rests on the metrics and the empirical study. That part is currently not reliable: the coverage metric is demonstrably inflated by dictionary span, the 'sweet spot' claims are not supported by the reported tables, and a key hyperparameter (the Lasso λ in Eq. 5) is unreported. With recalibration against random/spanning baselines, a formal definition of the tradeoff, and a revised set of conclusions, the framework could become a useful evaluation protocol for SAE interpretability.

major comments (5)
  1. [Section 3, Eq. (7), Table 1] The coverage metric is not a valid containment measure as used. With the factor 2 in the denominator of Eq. (7), exact reconstruction of every CBM direction yields Cov = 0.5, not 'close to one'; values above 0.5 are produced by Lasso overfitting in an overcomplete dictionary. Table 1 confirms the problem: untrained SAEs obtain higher coverage than trained SAEs (R50-CUB 0.662 vs 0.241; ViT-IN 0.908 vs 0.012; DINO-IN 0.745 vs 0.028). The text concedes coverage is 'artificially inflated in the absence of learned structure,' yet Section 3 and Tables A.4–A.6 use coverage as primary evidence for the sweet spot and for BatchTopK/Archetypal superiority. Coverage must be calibrated against random/spanning baselines, or the metric redefined, before any containment claim can be drawn.
  2. [Section 3, 'A Sparsity Sweet Spot' and 'Expansion Factor Sweet Spot'] The claimed sweet spots are not supported by the reported tables. In Table A.4, coverage, R2, and F1 increase monotonically as target_L0 increases (e.g., R50-CUB coverage goes 0.029 → 0.735); in Table A.6, coverage increases monotonically with expansion factor (0.051 → 0.898 for R50-CUB) and no ×3 condition is tested. The statement that a sweet spot emerges 'at approximately 3× expansion' is therefore not an empirical finding. If the intended claim is a tradeoff between coverage and ρgeom/ρact, it needs a formal definition — e.g., a Pareto frontier, a scalar objective, or a statistical comparison — rather than a qualitative reading of the tables.
  3. [Section 3, Experimental Settings, Eq. (5)] The Lasso penalty λ in Eq. (5) is a free parameter that directly controls reconstruction norm and sparsity, yet its value is never reported. Because coverage, δi, and active-set size si all depend on λ, comparisons across SAE variants and hyperparameter settings are not reproducible and may reflect different λ rather than different geometric alignment. The paper should report λ values (or a cross-validation procedure) and include a sensitivity analysis.
  4. [Section 2, Eq. (2) and 'Concept Cones' paragraph] The claim that 'across all these formulations, the learned codes Z are nonnegative or zero' is false for the ICE method listed in Eq. (2), which constrains DᵀD = I but does not constrain Z ≥ 0; SVD-based codes are signed. Thus Observation 1 does not extend to all methods in Eq. (2) as stated. The unification is correct for CBMs, SAEs, CRAFT, and other explicitly nonnegative methods, but the universality claim needs qualification or the constraint set needs revision.
  5. [Section 3, 'Sanity Check Experiment' vs. later analyses] The sanity check in Table 1 is an internal contradiction for the paper's main argument. The text correctly notes that coverage 'tends to be artificially inflated in the absence of learned structure,' but the later analyses treat higher coverage as evidence of better semantic alignment (Tables A.4–A.6 and the claims about BatchTopK and Archetypal in Table 2). If coverage is known to be inflated by random span, it cannot be used as a measure of plausibility without a baseline correction. The paper should either fix the metric or restrict its conclusions to the metrics that pass the sanity check (e.g., ρgeom, ρact, R2).
minor comments (6)
  1. [Section 2, Definition 1 and Appendix B] Definition 1 calls H a 'measurable right-inverse' of f∘G, but the condition H∘f∘G ≈ Id_C makes H a left-inverse. Also, the main text uses 'Observation 1', while §2.1 and Appendix B refer to 'Proposition 1' and 'Observation 2' for the same statement. Harmonize the numbering and terminology.
  2. [Table A.4] Several entries are unreadable due to missing spacing, e.g., '0.6171.006.86' for CUB–R50 ×4. Please reformat the tables so each column is clearly separated.
  3. [Section 3, 'A Sparsity Sweet Spot'] The phrase 'intermediate sparsity regimes (≈ 0.01–0.05% of sparsity)' is confusing: the tables report sparsity as the fraction of inactive units (e.g., 0.995), not as a percentage of active units. Specify whether the cited range refers to target_L0 or to active fraction.
  4. [Section 2, Notation] Typo: 'so the the number of independent attribute' should be 'so the number of independent attributes.'
  5. [Figure 3] Figure 3 is referenced in Section 3 but no image appears in the submitted text. Ensure the radar plots are included and their axes are legible.
  6. [Table 1] The heading 'CUB / CUB' should be clarified; I assume the first entry is the test dataset and the second is the SAE training dataset. State this explicitly in the caption.

Circularity Check

1 steps flagged · score 6.0 of 10

The empirical 'sweet spot' is partially a spanning artifact: coverage is defined as reconstruction of CBM directions from the SAE cone, so larger/denser cones mechanically score higher.

  1. fitted input called prediction [Section 3 'Expansion Factor Sweet Spot' (metric defined in Eq. 7, Section 2.1; sanity check Table 1)]
    "larger latent spaces systematically improve coverage, R2, and sample-level F1 ... A sweet spot emerges at approximately 3× expansion Beyond this threshold, further expansion yields diminishing returns: coverage saturates while directional alignment continues to diffuse."

    Cov (Eq. 7) is the fraction of CBM direction energy reconstructed by Lasso from the SAE dictionary (Eq. 5). Any dictionary whose conic hull spans more of the activation space trivially increases Cov; no semantic alignment is required. The paper's own sanity check shows untrained SAEs have coverage 0.662–0.908 versus 0.012–0.275 for trained SAEs, and the text admits coverage is 'artificially inflated in the absence of learned structure.' Yet the sparsity and expansion sweeps use this same Cov to conclude that larger/denser SAE cones 'better reconstruct the CBM concept space' and define the reported 'sweet spot' — a conclusion that follows directly from the definition of Cov rather than from independent semantic evidence.

full rationale

The core unifying observation (Observation 1) is essentially definitional: the concept cone is defined as the conic hull of nonnegative code reconstructions, so both CBMs and SAEs trivially 'instantiate' it. I do not treat this as circular by itself because it is presented as an observation rather than as a testable prediction. The loading-bearing empirical payoff, however, rests on the coverage metric (Eq. 7), which measures how much of the CBM cone is spanned by the SAE dictionary. The paper's own sanity check demonstrates that coverage tracks cone volume rather than semantics: random SAEs score higher than trained SAEs, and the text explicitly says coverage is 'artificially inflated in the absence of learned structure.' The expansion and sparsity 'sweet spot' findings are therefore partially forced by construction: increasing dictionary size or code density is definitionally guaranteed to raise coverage. Some independent content remains in the tradeoff with ρgeom and in the other metrics, so the circularity is partial rather than total. Score 6: one or more empirical 'predictions' reduce by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central unification is definitional: once codes are nonnegative, all reconstructions lie in the conic hull. The empirical contribution depends on several chosen hyperparameters, post-hoc concept filtering, and the unvalidated assumption that human CBM concepts are the right geometric reference.

free parameters (5)
  • Lasso penalty lambda (Eq. 5) = not reported
    Controls the sparsity of SAE-to-CBM reconstruction; all containment and sparsity metrics depend on it, yet no value or selection procedure is given.
  • Coverage denominator factor 2 (Eq. 7) = 2
    Arbitrary normalization constant in the coverage definition; affects absolute coverage values and cross-experiment comparisons.
  • Concept filtering thresholds = 168/312 CUB; 3545/5305 ImageNet
    Post-hoc removal of uncertain or rare concepts before CBM training changes the reference cone and therefore all alignment scores.
  • TopK sparsity target = default 0.005; varied 0.0012-0.1
    Chosen hyperparameter; the claimed 'sweet spot' is located on this grid but no optimum criterion is defined.
  • SAE expansion factor = default x2; varied x1-x8
    Chosen hyperparameter controlling dictionary size and fitting capacity.
assumptions (5)
  • domain assumption Concepts correspond to linear directions in activation space and are recovered by nonnegative combinations (Linear Representation Hypothesis).
    Invoked in Section 2 to justify working in a linear chart of A; if false, concept cones do not capture semantic structure.
  • domain assumption A data-generating process exists with independent latent factors, with G C1 and injective (Definition 1).
    Justifies the definition of concepts as coordinates of a right-inverse H; not verified for real backbones or datasets.
  • domain assumption CBM human-annotated concepts are a valid reference for SAE evaluation.
    The paper admits CBM concepts are 'plausible by construction' but not necessarily faithful; treating them as the reference geometry is a normative choice.
  • ad hoc to paper Containment hypothesis: the SAE cone should contain or sparsely approximate the CBM cone.
    Introduced to motivate Eq. 5; no theoretical or empirical justification independent of the proposed metrics.
  • standard math SAE codes and CBM codes are nonnegative (ReLU and sigmoid, respectively).
    Architectural facts used in Observation 1; true for the considered models.
invented entities (1)
  • Concept cone C_D
    purpose: Formal object claimed to unify supervised and unsupervised concept learning.
    A definition (conic hull of dictionary rows), not a falsifiable entity; it makes no external prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Geometric Unification of Concept Learning with Concept Cones." pith.science (2026). https://pith.science/paper/KU2VHDGK

@misc{pith2026251207355,
  author       = {Pith},
  title        = {Pith review of: A Geometric Unification of Concept Learning with Concept Cones},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KU2VHDGK}},
  note         = {Machine review of arXiv:2512.07355}
}
read the original abstract

Two traditions of interpretability have evolved side by side but seldom spoken to each other: Concept Bottleneck Models (CBMs), which prescribe what a concept should be, and Sparse Autoencoders (SAEs), which discover what concepts emerge. While CBMs use supervision to align activations with human-labeled concepts, SAEs rely on sparse coding to uncover emergent ones. We show that both paradigms instantiate the same geometric structure: each learns a set of linear directions in activation space whose nonnegative combinations form a concept cone. Supervised and unsupervised methods thus differ not in kind but in how they select this cone. Building on this view, we propose an operational bridge between the two paradigms. CBMs provide human-defined reference geometries, while SAEs can be evaluated by how well their learned cones approximate or contain those of CBMs. This containment framework yields quantitative metrics linking inductive biases -- such as SAE type, sparsity, or expansion ratio -- to emergence of plausible\footnote{We adopt the terminology of \citet{jacovi2020towards}, who distinguish between faithful explanations (accurately reflecting model computations) and plausible explanations (aligning with human intuition and domain knowledge). CBM concepts are plausible by construction -- selected or annotated by humans -- though not necessarily faithful to the true latent factors that organise the data manifold.} concepts. Using these metrics, we uncover a ``sweet spot'' in both sparsity and expansion factor that maximizes both geometric and semantic alignment with CBM concepts. Overall, our work unifies supervised and unsupervised concept discovery through a shared geometric framework, providing principled metrics to measure SAE progress and assess how well discovered concept align with plausible human concepts.

Figures

Figures reproduced from arXiv: 2512.07355 by the authors.

Figure 1
Figure 1. Illustration of the concept learning spaces. Each image x ∈ X comes from hidden factors c ∈ C through a generative process G : C→X . A neural network f : X →A maps inputs to activations a = f(x). Concept extraction aims to find a function H : A → C that recovers these hidden factors. Here, C is the concept space with interpretable axes (e.g., rabbit, tree), X is the input space of images, and A is the activation spa… view at source ↗
Figure 2
Figure 2. CBMs as anchors for unsupervised concept discovery. Both supervised and unsupervised methods learn linear directions in activation space A, but they do so under different objectives. Unsupervised approaches such as SAEs decompose activations A into a dictionary D and sparse codes Z, uncovering emergent concept directions without human supervision. CBMs, in contrast, learn a set of concept directions Wc aligned with … view at source ↗
Figure 3
Figure 3. Radar plot illustrating the influence of the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Geometric View for Understanding Concept Learning and Neuron Interpretation in Sparse Autoencoders

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Formalizes concept learning in sparse autoencoders as set alignment between human-defined and model-induced concepts, distinguishing detection, separation, and approximation with geometric conditions for neuron repres...

Reference graph

Works this paper leans on

110 extracted references · 3 linked inside Pith · cited by 1 Pith paper

  1. [45]

    Projecting assumptions: The duality be- tween sparse autoencoders and concept geometry.Advances in Neural Information Processing Systems (NeurIPS), 2025

    Sai Sumedh R Hindupur, Ekdeep Singh Lubana, Thomas Fel, and Demba Ba. Projecting assumptions: The duality be- tween sparse autoencoders and concept geometry.Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  2. [1]

    Can we recon- cile safety objectives with machine learning performances? ERTS, 2022

    Lucian Alecu, Hugues Bonnin, Thomas Fel, Laurent Gardes, Sébastien Gerchinovitz, Ludovic Ponsolle, Franck Mamalet, Éric Jenn, Vincent Mussot, Cyril Cappi, et al. Can we recon- cile safety objectives with machine learning performances? ERTS, 2022

    2022

  3. [2]

    Cross-modal conceptualization in bottleneck models.Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

    Danis Alukaev, Semen Kiselev, Ilya Pershin, Bulat Ibrag- imov, Vladimir Ivanov, Alexey Kornaev, and Ivan Titov. Cross-modal conceptualization in bottleneck models.Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

    2023

  4. [3]

    Linear algebraic structure of word senses, with applications to polysemy.Transactions of the Associa- tion for Computational Linguistics, 2018

    Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy.Transactions of the Associa- tion for Computational Linguistics, 2018

    2018

  5. [4]

    On pixel-wise explanations for non-linear classi- fier decisions by layer-wise relevance propagation.Public Library of Science (PloS One), 2015

    Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classi- fier decisions by layer-wise relevance propagation.Public Library of Science (PloS One), 2015

    2015

  6. [5]

    Back to the fea- tures: Dino as a foundation for video world models.ArXiv e-print, 2025

    Federico Baldassarre, Marc Szafraniec, Basile Terver, Vasil Khalidov, Francisco Massa, Yann LeCun, Patrick Labatut, Maximilian Seitzer, and Piotr Bojanowski. Back to the fea- tures: Dino as a foundation for video world models.ArXiv e-print, 2025

    2025

  7. [6]

    Learning in high dimension always amounts to extrapolation

    Randall Balestriero, Jerome Pesenti, and Yann LeCun. Learning in high dimension always amounts to extrapolation. ArXiv e-print, 2021

    2021

  8. [7]

    Network dissection: Quantifying inter- pretability of deep visual representations.Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2017

    David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying inter- pretability of deep visual representations.Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2017

    2017

Show all 110 references
  1. [8]

    Interpreting clip with sparse linear concept embeddings (splice).ArXiv e-print, 2024

    Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P Cal- mon, and Himabindu Lakkaraju. Interpreting clip with sparse linear concept embeddings (splice).ArXiv e-print, 2024

    2024

  2. [9]

    Towards monosemanticity: Decomposing language models with dic- tionary learning.Transformer Circuits Thread, 2023

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen,...

    2023

  3. [10]

    Batchtopk sparse autoencoders.ArXiv e-print, 2024

    Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders.ArXiv e-print, 2024

    2024

  4. [11]

    Interactive concept bottleneck models.Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2023

    Kushal Chauhan, Rishabh Tiwari, Jan Freyberg, Pradeep Shenoy, and Krishnamurthy Dvijotham. Interactive concept bottleneck models.Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2023

    2023

  5. [12]

    Concept-based mechanistic interpretabil- ity using structured knowledge graphs.arXiv preprint arXiv:2507.05810, 2025

    Sofiia Chorna, Kateryna Tarelkina, Eloise Berthier, and Gianni Franchi. Concept-based mechanistic interpretabil- ity using structured knowledge graphs.arXiv preprint arXiv:2507.05810, 2025

    2025 arXiv

  6. [13]

    Neural population geometry: An approach for understanding biological and artificial neural networks

    SueYeon Chung. Neural population geometry: An approach for understanding biological and artificial neural networks. Current opinion in neurobiology, 2021

    2021

  7. [14]

    Separability and geometry of object manifolds in deep neural networks.Nature communications, 2020

    Uri Cohen, SueYeon Chung, Daniel D Lee, and Haim Som- polinsky. Separability and geometry of object manifolds in deep neural networks.Nature communications, 2020

    2020

  8. [15]

    What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods.Advances in Neural Information Processing Sys- tems (NeurIPS), 2021

    Julien Colin, Thomas Fel, Rémi Cadène, and Thomas Serre. What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods.Advances in Neural Information Processing Sys- tems (NeurIPS), 2021

    2021

  9. [16]

    From flat to hierarchical: Ex- tracting sparse representations with matching pursuit.Ad- vances in Neural Information Processing Systems (NeurIPS), 2025

    Valérie Costa, Thomas Fel, Ekdeep Singh Lubana, Bahareh Tolooshams, and Demba Ba. From flat to hierarchical: Ex- tracting sparse representations with matching pursuit.Ad- vances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  10. [17]

    Sparse autoencoders find highly interpretable features in language models.ArXiv e-print, 2023

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models.ArXiv e-print, 2023

    2023

  11. [18]

    Vision transformers need registers.ArXiv e-print, 2023

    Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers.ArXiv e-print, 2023

    2023

  12. [19]

    If con- cept bottlenecks are the question, are foundation models the answer?arXiv preprint arXiv:2504.19774, 2025

    Nicola Debole, Pietro Barbiero, Francesco Giannini, Andrea Passerini, Stefano Teso, and Emanuele Marconato. If con- cept bottlenecks are the question, are foundation models the answer?arXiv preprint arXiv:2504.19774, 2025

    2025 arXiv

  13. [20]

    Scaling vision transformers to 22 billion parameters.Proceedings of the International Conference on Machine Learning (ICML), 2023

    Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdul- mohsin, et al. Scaling vision transformers to 22 billion parameters.Proceedings of the International Conferen...

    2023

  14. [21]

    Natalia Díaz-Rodríguez, Alberto Lamas, Jules Sanchez, Gi- anni Franchi, Ivan Donadello, Siham Tabik, David Filliat, Policarpo Cruz, Rosana Montes, and Francisco Herrera. Ex- plainable neural-symbolic learning (x-nesyl) methodology to fuse deep learning representations with exp...

    2022

  15. [22]

    Towards a rigorous science of interpretable machine learning.ArXiv e-print, 2017

    Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning.ArXiv e-print, 2017

    2017

  16. [23]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.Proceedings of the In- ternational Conference on Learning Representations (ICLR), 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale.Proceedings of th...

    2020

  17. [24]

    Sparse and redundant representations: from theory to applications in signal and image processing

    Michael Elad. Sparse and redundant representations: from theory to applications in signal and image processing. Springer International Publishing, 2010

    2010

  18. [25]

    Toy models of superposition.Transformer Circuits Thread, 2022

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield- Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superpos...

    2022

  19. [26]

    PhD thesis, Université de Toulouse, 2024

    Thomas Fel.Sparks of explainability: recent advancements in explaining large vision models. PhD thesis, Université de Toulouse, 2024

    2024

  20. [27]

    Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis.Advances in Neural Information Pro- cessing Systems (NeurIPS), 2021

    Thomas Fel, Remi Cadene, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, and Thomas Serre. Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis.Advances in Neural Information Pro- cessing Systems (NeurIPS), 2021

    2021

  21. [28]

    A holistic approach to unifying automatic concept extraction and concept importance estimation.Advances in Neural Information Processing Systems (NeurIPS), 2023

    Thomas Fel, Victor Boutin, Mazda Moayeri, Remi Cadene, Louis Bethune, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation.Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  22. [29]

    Craft: Concept recursive activation factoriza- tion for explainability.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. Craft: Concept recursive activation factoriza- tion for explainability.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  23. [30]

    Thomas Fel, Ekdeep Singh Lubana, Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. Archety- pal sae: Adaptive and stable dictionary learning for concept extraction in large vision models.Proceedings o...

    2025

  24. [31]

    Fong and Andrea Vedaldi

    Ruth C. Fong and Andrea Vedaldi. Interpretable explana- tions of black boxes by meaningful perturbation.Proceed- ings of the IEEE International Conference on Computer Vision (ICCV), 2017

    2017

  25. [32]

    Scaling and evaluating sparse autoencoders

    Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. Proceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  26. [33]

    Inter- pretation of neural networks is fragile.Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017

    Amirata Ghorbani, Abubakar Abid, and James Zou. Inter- pretation of neural networks is fragile.Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017

    2017

  27. [34]

    Towards automatic concept-based explanations.Ad- vances in Neural Information Processing Systems (NeurIPS), 2019

    Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept-based explanations.Ad- vances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  28. [35]

    Springer International Publishing, 2020

    Nicolas Gillis.Nonnegative matrix factorization. Springer International Publishing, 2020

    2020

  29. [36]

    Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal

    Leilani H. Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explaining explana- tions: An overview of interpretability of machine learning. Proceedings of the IEEE International Conference on data science and advanced analytics (DSAA), 2018

    2018

  30. [37]

    The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision

    Liv Gorton. The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision. ArXiv e-print, 2024

    2024

  31. [38]

    Concept discovery and dataset exploration with singular value decomposition.Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    Mara Graziani, An-phi Nguyen, Laura O’Mahony, Henning Müller, and Vincent Andrearczyk. Concept discovery and dataset exploration with singular value decomposition.Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  32. [39]

    A survey of deep learning techniques for autonomous driving.Journal of field robotics, 37(3): 362–386, 2020

    Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu. A survey of deep learning techniques for autonomous driving.Journal of field robotics, 37(3): 362–386, 2020

    2020

  33. [40]

    Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms.ArXiv e-print, 2024

    Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms.ArXiv e-print, 2024

    2024

  34. [41]

    Peter Hase and Mohit Bansal. Evaluating explainable ai: Which algorithmic explanations help users predict model behavior?Proceedings of the Annual Meeting of the Associ- ation for Computational Linguistics (ACL), 2020

    2020

  35. [42]

    Deep residual learning for image recognition.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  36. [43]

    Masked autoencoders are scal- able vision learners.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scal- able vision learners.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  37. [44]

    Sparse autoencoders can interpret randomly ini- tialized transformers.ArXiv e-print, 2025

    Thomas Heap, Tim Lawson, Lucy Farnik, and Laurence Aitchison. Sparse autoencoders can interpret randomly ini- tialized transformers.ArXiv e-print, 2025

    2025

  38. [46]

    Evaluations and methods for explanation through robustness analysis.Proceedings of the International Con- ference on Learning Representations (ICLR), 2021

    Cheng-Yu Hsieh, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Ravikumar, Seungyeon Kim, Sanjiv Kumar, and Cho-Jui Hsieh. Evaluations and methods for explanation through robustness analysis.Proceedings of the International Con- ference on Learning Representations (ICLR), 2021

    2021

  39. [47]

    Alon Jacovi and Yoav Goldberg. Towards faithfully inter- pretable nlp systems: How should we define and evaluate faithfulness?Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2020

    2020

  40. [48]

    Automatic concept extraction for concept bottleneck-based video classification.ArXiv e-print, 2022

    Jeya Vikranth Jeyakumar, Luke Dickens, Luis Garcia, Yu-Hsi Cheng, Diego Ramirez Echavarria, Joseph Noor, Alessandra Russo, Lance Kaplan, Erik Blasch, and Mani Srivastava. Automatic concept extraction for concept bottleneck-based video classification.ArXiv e-print, 2022

    2022

  41. [49]

    Steering clip’s vision transformer with sparse autoencoders.ArXiv e-print, 2025

    Sonia Joseph, Praneet Suresh, Ethan Goldfarb, Lorenz Hufe, Yossi Gandelsman, Robert Graham, Danilo Bzdok, Wojciech Samek, and Blake Aaron Richards. Steering clip’s vision transformer with sparse autoencoders.ArXiv e-print, 2025

    2025

  42. [50]

    Prisma: An open source toolkit for mechanistic interpretability in vision and video.ArXiv e-print, 2025

    Sonia Joseph, Praneet Suresh, Lorenz Hufe, Edward Stevin- son, Robert Graham, Yash Vadi, Danilo Bzdok, Sebastian La- puschkin, Lee Sharkey, and Blake Aaron Richards. Prisma: An open source toolkit for mechanistic interpretability in vision and video.ArXiv e-print, 2025. 10

    2025

  43. [51]

    Highly accurate protein structure prediction with alphafold

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvu- nakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583–589, 2021

    2021

  44. [52]

    Clip-qda: An explainable concept bottleneck model.The Journal of Transactions on Machine Learning Research (TMLR), 2023

    Rémi Kazmierczak, Eloïse Berthier, Goran Frehse, and Gi- anni Franchi. Clip-qda: An explainable concept bottleneck model.The Journal of Transactions on Machine Learning Research (TMLR), 2023

    2023

  45. [53]

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability be- yond feature attribution: Quantitative testing with concept activation vectors (tcav).Proceedings of the International Conference on Machine Learning (ICML), 2018

    2018

  46. [54]

    Probabilistic concept bottleneck models

    Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic concept bottleneck models. Proceedings of the International Conference on Machine Learning (ICML), 2023

    2023

  47. [55]

    Sunnie S. Y . Kim, Nicole Meister, Vikram V . Ramaswamy, Ruth Fong, and Olga Russakovsky. HIVE: Evaluating the hu- man interpretability of visual explanations.Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2022

    2022

  48. [56]

    Towards nonlinear disentanglement in natural data with tem- poral sparse coding.ArXiv e-print, 2020

    David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhani- nov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with tem- poral sparse coding.ArXiv e-print, 2020

    2020

  49. [57]

    Identifying interpretable visual features in artificial and biological neural systems.ArXiv e-print, 2023

    David Klindt, Sophia Sanborn, Francisco Acosta, Frederic Poitevin, and Nina Miolane. Identifying interpretable visual features in artificial and biological neural systems.ArXiv e-print, 2023

    2023

  50. [58]

    From superposition to sparse codes: interpretable representations in neural networks

    David Klindt, Charles O’Neill, Patrik Reizinger, Harald Maurer, and Nina Miolane. From superposition to sparse codes: interpretable representations in neural networks. ArXiv e-print, 2025

    2025

  51. [59]

    Concept bottleneck models.Proceedings of the International Conference on Machine Learning (ICML), 2020

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models.Proceedings of the International Conference on Machine Learning (ICML), 2020

    2020

  52. [60]

    Understand- ing video transformers via universal concept discovery.Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    Matthew Kowal, Achal Dave, Rares Ambrus, Adrien Gaidon, Konstantinos G Derpanis, and Pavel Tokmakov. Understand- ing video transformers via universal concept discovery.Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  53. [61]

    Matthew Kowal, Richard P Wildes, and Konstantinos G Derpanis. Visual concept connectome (vcc): Open world concept discovery and their interlayer connections in deep models.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  54. [62]

    Deep learning.Nature, 2015

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.Nature, 2015

    2015

  55. [63]

    Challenging common assumptions in the unsu- pervised learning of disentangled representations.Proceed- ings of the International Conference on Machine Learning (ICML), 2019

    Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Scholkopf, and Olivier Bachem. Challenging common assumptions in the unsu- pervised learning of disentangled representations.Proceed- ings of the International Conference on Machine Learnin...

    2019

  56. [64]

    K-sparse autoencoders

    Alireza Makhzani and Brendan Frey. K-sparse autoencoders. Proceedings of the International Conference on Learning Representations (ICLR), 2014

    2014

  57. [65]

    Do concept bottleneck models learn as intended?arXiv preprint arXiv:2105.04289, 2021

    Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. Do concept bottleneck models learn as intended?arXiv preprint arXiv:2105.04289, 2021

    2021 arXiv

  58. [66]

    Text-to-concept (and back) via cross-model alignment

    Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and Soheil Feizi. Text-to-concept (and back) via cross-model alignment. Proceedings of the International Conference on Machine Learning (ICML), 2023

    2023

  59. [67]

    Gradient strikes back: How filtering out high frequencies improves explanations.Pro- ceedings of the International Conference on Learning Rep- resentations (ICLR), 2024

    Sabine Muzellec, Leo Andeol, Thomas Fel, Rufin Van- Rullen, and Thomas Serre. Gradient strikes back: How filtering out high frequencies improves explanations.Pro- ceedings of the International Conference on Learning Rep- resentations (ICLR), 2024

    2024

  60. [68]

    The effectiveness of feature attribution methods and its correla- tion with automatic evaluation scores.Advances in Neural Information Processing Systems (NeurIPS), 2021

    Giang Nguyen, Daeyoung Kim, and Anh Nguyen. The effectiveness of feature attribution methods and its correla- tion with automatic evaluation scores.Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  61. [69]

    Making sense of dependence: Efficient black-box explanations us- ing dependence measure.Advances in Neural Information Processing Systems (NeurIPS), 2022

    Paul Novello, Thomas Fel, and David Vigouroux. Making sense of dependence: Efficient black-box explanations us- ing dependence measure.Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  62. [70]

    Clip-dissect: Auto- matic description of neuron representations in deep vision networks.Proceedings of the International Conference on Learning Representations (ICLR), 2023

    Tuomas Oikarinen and Tsui-Wei Weng. Clip-dissect: Auto- matic description of neuron representations in deep vision networks.Proceedings of the International Conference on Learning Representations (ICLR), 2023

    2023

  63. [71]

    Label-free concept bottleneck models.Proceed- ings of the International Conference on Learning Represen- tations (ICLR), 2023

    Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui- Wei Weng. Label-free concept bottleneck models.Proceed- ings of the International Conference on Learning Represen- tations (ICLR), 2023

    2023

  64. [72]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. ArXiv e-print, 2023

    2023

  65. [73]

    Discover: mak- ing vision networks interpretable via competition and dissec- tion.Advances in Neural Information Processing Systems (NeurIPS), 2023

    Konstantinos Panousis and Sotirios Chatzis. Discover: mak- ing vision networks interpretable via competition and dissec- tion.Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  66. [74]

    Interpreting the linear structure of vision-language model embedding spaces.Proceedings of the Conference on Language Modeling (COLM), 2025

    Isabel Papadimitriou, Huangyuan Su, Thomas Fel, Sham Kakade, and Stephanie Gil. Interpreting the linear structure of vision-language model embedding spaces.Proceedings of the Conference on Language Modeling (COLM), 2025

    2025

  67. [75]

    A concept-based explainability framework for large multimodal models.ArXiv e-print, 2024

    Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Alasdair Newson, and Matthieu Cord. A concept-based explainability framework for large multimodal models.ArXiv e-print, 2024

    2024

  68. [76]

    The linear rep- resentation hypothesis and the geometry of large language models.Proceedings of the International Conference on Machine Learning (ICML), 2024

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear rep- resentation hypothesis and the geometry of large language models.Proceedings of the International Conference on Machine Learning (ICML), 2024

    2024

  69. [77]

    Pytorch: An imperative style, high-performance deep learning library.Ad- 11 vances in Neural Information Processing Systems (NeurIPS), 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library.Ad- 11 vances in Neural Information Processing Systems ...

    2019

  70. [78]

    Sparse autoencoders trained on the same data learn different features.ArXiv e-print, 2025

    Goncalo Paulo and Nora Belrose. Sparse autoencoders trained on the same data learn different features.ArXiv e-print, 2025

    2025

  71. [79]

    Concept-based explainable artificial intelligence: A survey.ArXiv e-print, 2023

    Eleonora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. Concept-based explainable artificial intelligence: A survey.ArXiv e-print, 2023

    2023

  72. [80]

    Jumping ahead: Improving reconstruction fi- delity with jumprelu sparse autoencoders.ArXiv e-print, 2024

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Son- nerat, Arthur Conmy, Vikrant Varma, Janos Kramar, and Neel Nanda. Jumping ahead: Improving reconstruction fi- delity with jumprelu sparse autoencoders.ArXiv e-print, 2024

    2024

  73. [81]

    Discover-then-name: Task-agnostic concept bottle- necks via automated concept discovery.Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2024

    Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-name: Task-agnostic concept bottle- necks via automated concept discovery.Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2024

    2024

  74. [82]

    Karsten Roth, Jae Myung Kim, Andrew Koepke, Oriol Vinyals, Cordelia Schmid, and Zeynep Akata. Waffling around for performance: Visual classification with random words and broad concepts.Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  75. [83]

    Imagenet large scale visual recognition challenge.International journal of computer vision, 115(3):211–252, 2015

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge.International journal of computer vision, 115(3):211–252, 2015

    2015

  76. [84]

    Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities.Knowledge-Based Systems, 2023

    Waddah Saeed and Christian Omlin. Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities.Knowledge-Based Systems, 2023

    2023

  77. [85]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Ba- tra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Ba- tra. Grad-cam: Visual explanations from deep networks via gradient-based localization.Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017

    2017

  78. [86]

    Deep learning: The good, the bad, and the ugly.Annual review of vision science, 2019

    Thomas Serre. Deep learning: The good, the bad, and the ugly.Annual review of vision science, 2019

    2019

  79. [87]

    Implicit multimodal alignment: On the generalization of frozen llms to multi- modal inputs.ArXiv e-print, 2024

    Mustafa Shukor and Matthieu Cord. Implicit multimodal alignment: On the generalization of frozen llms to multi- modal inputs.ArXiv e-print, 2024

    2024

  80. [88]

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image clas- sification models and saliency maps.Proceedings of the In- ternational Conference on Learning Representations (ICLR), 2013

    2013

  81. [89]

    When explanations lie: Why many modified bp attributions fail

    Leon Sixt, Maximilian Granz, and Tim Landgraf. When explanations lie: Why many modified bp attributions fail. Proceedings of the International Conference on Machine Learning (ICML), 2020

    2020

  82. [90]

    Smoothgrad: removing noise by adding noise.Proceedings of the International Conference on Machine Learning (ICML), 2017

    Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise.Proceedings of the International Conference on Machine Learning (ICML), 2017

    2017

  83. [91]

    Striving for simplicity: The all convolutional net.Workshop Proceedings of the Inter- national Conference on Learning Representations (ICLR), 2014

    Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net.Workshop Proceedings of the Inter- national Conference on Learning Representations (ICLR), 2014

    2014

  84. [92]

    Vlg-cbm: Training concept bottleneck models with vision-language guidance.Advances in Neural Information Processing Sys- tems, 37:79057–79094, 2024

    Divyansh Srivastava, Ge Yan, and Lily Weng. Vlg-cbm: Training concept bottleneck models with vision-language guidance.Advances in Neural Information Processing Sys- tems, 37:79057–79094, 2024

    2024

  85. [93]

    Axiomatic attribution for deep networks.Proceedings of the Interna- tional Conference on Machine Learning (ICML), 2017

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks.Proceedings of the Interna- tional Conference on Machine Learning (ICML), 2017

    2017

  86. [94]

    Explain via any concept: Concept bottleneck model with open vocabulary concepts.Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2024

    Andong Tan, Fengtao Zhou, and Hao Chen. Explain via any concept: Concept bottleneck model with open vocabulary concepts.Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2024

    2024

  87. [95]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lind- sey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunning- ham, Nicholas L Turner, Callum McDougall, Monte Mac- Diarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, J...

    2024

  88. [96]

    Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society Series B: Statistical Methodology, 1996

    Robert Tibshirani. Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society Series B: Statistical Methodology, 1996

    1996

  89. [97]

    Applications of machine learning in drug discovery and development

    Jessica Vamathevan, Dominic Clark, Paul Czodrowski, Ian Dunham, Edgardo Ferran, George Lee, Bin Li, Anant Mad- abhushi, Parantu Shah, Michaela Spitzer, et al. Applications of machine learning in drug discovery and development. Nature reviews Drug discovery, 18(6):463–477, 2019

    2019

  90. [98]

    Stochastic concept bottleneck models.Ad- vances in Neural Information Processing Systems (NeurIPS), 2024

    Moritz Vandenhirtz, Sonia Laguna, Ricards Marcinkevics, and Julia V ogt. Stochastic concept bottleneck models.Ad- vances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  91. [99]

    Multi-dimensional concept discovery (mcd): A unifying framework with completeness guarantees.The Journal of Transactions on Machine Learning Research (TMLR), 2023

    Johanna Vielhaben, Stefan Blücher, and Nils Strodthoff. Multi-dimensional concept discovery (mcd): A unifying framework with completeness guarantees.The Journal of Transactions on Machine Learning Research (TMLR), 2023

    2023

  92. [100]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011

    2011

  93. [101]

    Unsuper- vised learning of artistic styles with archetypal style anal- ysis.Advances in Neural Information Processing Systems (NeurIPS), 2018

    Daan Wynen, Cordelia Schmid, and Julien Mairal. Unsuper- vised learning of artistic styles with archetypal style anal- ysis.Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  94. [102]

    Post-hoc concept bottleneck models.Proceedings of the International Conference on Learning Representations (ICLR), 2023

    Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models.Proceedings of the International Conference on Learning Representations (ICLR), 2023

    2023

  95. [103]

    Visualizing and under- standing convolutional networks.Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2014

    Matthew D Zeiler and Rob Fergus. Visualizing and under- standing convolutional networks.Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2014

    2014

  96. [104]

    Scaling vision transformers.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  97. [105]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. 12 Proceedings of the IEEE International Conference on Com- puter Vision (ICCV), 2023

    2023

  98. [106]

    red feathers,

    Ruihan Zhang, Prashan Madumal, Tim Miller, Krista A Ehinger, and Benjamin IP Rubinstein. Invertible concept- based explanations for cnn models with non-negative con- cept activation vectors.Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2021. 13 A Geomet...

    2021

  99. [107]

    Start with a vocabulary V of concept words (e.g., 20k common English words)

  100. [108]

    Embed each word using CLIP text encoder: ti = CLIPtext(vi)

  101. [109]

    , t|V| })∈R c×d

    Apply PCA to reduce dimensionality: DCLIP = PCA({t1, . . . , t|V| })∈R c×d

  102. [110]

    carve out

    For an image, compute concept representation: ˆci = a⊤ i D⊤ CLIP Quadratic Discriminant Analysis.Instead of learning a linear classifier Wy, CLIP-QDA learns class-specific Gaus- sian distributions: p(ˆc|y) =N(µ y,Σ y)(27) ˆyi = arg max y [logp( ˆci |y) + logp(y)](28) 2 This re...

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.