Enhancing Performance of Explainable AI Models with Constrained Concept Refinement

Geyu Liang; Salar Fattahi; Senne Michielssen

REVIEW 3 major objections 4 minor 70 references

Enhancing Performance of Explainable AI Models with Constrained Concept Refinement

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Constrained Concept Refinement provably removes the accuracy cost of interpretable-by-design models while keeping embeddings close to their human-meaningful starting points.

desk verdict A genuine new constrained concept-refinement method with a first convergence guarantee, but the headline theorem has an unstated square-dimension assumption and the proof does not cover the deployed classifier. read the letter →

arxiv 2502.06775 v2 pith:H3M6B5WE submitted 2025-02-10 cs.LG

Geyu Liang , Senne Michielssen , Salar Fattahi This is my paper · ORCID

classification cs.LG MSC 68T0762J07

keywords constrainedconceptrefinementinterpretable-by-designbottleneckmodelsorthogonalmatchingpursuitsparsecodingdictionarylearningCLIPimageclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Interpretable-by-design models predict through human-meaningful concept embeddings, but those embeddings are often imperfect, and the paper shows that even small deviations from the true concept directions cause a proportional loss in prediction accuracy. The paper's remedy, Constrained Concept Refinement (CCR), optimizes the concept embeddings within a small radius around their initial values, so interpretability is preserved by construction while accuracy is recovered. Under a sparse generative model the paper proves that projected gradient descent on a differentiable surrogate of the IP-OMP estimator converges linearly to the ground-truth concept matrix, driving the squared loss to zero. On image benchmarks CCR outperforms CLIP-IP-OMP and label-free CBM while running roughly ten times faster on ImageNet.

What carries the argument

The load-bearing object is the surrogate estimator $\tilde{f}_D(x,r) = \sum_{i \in S} \langle d_i, x\rangle r_i$ with $S$ the indices of the top-$k$ largest values of $|\langle d_i, x\rangle|$, which Lemma 3.1 shows is exactly the IP-OMP estimator when $D = D^*$ is column-orthogonal. This surrogate makes the loss differentiable almost surely along the trajectory and turns concept refinement into projected gradient descent on the query-feature matrix $D$ within the ball $\|\Delta D\|_{1,2} \le \rho$. The proof tracks an auxiliary vector $\hat{d}_i = d_i^{(0)} + (\beta_i - x^\top d_i^{(0)}/\|x\|_2^2)\,x$ for each column, which aligns the gradient direction with the error $d_i - d_i^*$ and makes the projection step contract the error. For images, the same mechanism is realized with CLIP embeddings, a concept-dispersion step that enforces near-orthogonality, hard thresholding $HT_\lambda$, and per-iteration normalization and projection.

What would settle it

Generate data from the paper's model with a full-rank orthonormal $D^*$, perturb each column by an independent vector of norm $\epsilon$, and run CCR and vanilla IP-OMP with identical sparsity $k$. If the squared-loss gap $L(f_{D+\Delta}) - L(f_{D^*})$ does not scale as $\Omega(\epsilon^2\|x\|^2)$ for small $\epsilon$, or if the projected-gradient iterates with $m = \Omega(n^6/(\sigma^2 k^5))$ fail to drive the column-wise error toward zero, then Theorem 2.6 or Theorem 3.4 would be contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the accuracy-interpretability trade-off in concept-based explainable AI can be eliminated by refining concept embeddings inside a constraint set rather than by adding black-box modules. The paper first establishes a negative result (Theorem 2.6): for column-orthogonal ground-truth concepts $D^*$ and $k$-sparse inputs, any deviation $\epsilon$ in the query-feature matrix forces a loss gap of order $\Omega(\epsilon^2\|x\|_2^2)$. It then proves the positive result (Theorem 3.4): with $m = \Omega(n^6/(\sigma^2 k^5))$ samples, a full-rank column-orthogonal $D^*$, and initial error at most $\gamma/(8\sqrt{k}\Gamma)$, the CCR update (gradient step plus projection onto $\|\Delta D\|_{1,2} \le \rho$) satisfies $\|D(t+1)-D^*\|_{1,2} \le \tau \|D(t)-D^*\|_{1,2}$ and $L_m(\tilde{f}_{D(t)}) \le \frac{k\sum_h \|x_h\|_2^2}{m}\|D(t)-D^*\|_{1,2}^2$, hence zero loss and progressively better interpretability. In the deployed image-classification version, the squared-loss surrogate is replaced by hard-thresholded CLIP inner products and a linear classifier trained with cross-entropy, and the same constrained update beats the two strong baselines on CIFAR-10/100, ImageNet, and Places365.

Load-bearing premise

The convergence proof assumes the surrogate estimator (a top-$k$ linear combination with the true coefficients $\beta_i = \langle d_i^*, x\rangle$) faithfully represents the actual IP-OMP predictor, so if the surrogate stops tracking the real estimator when $D$ moves away from $D^*$, the zero-loss guarantee does not apply to the deployed cross-entropy version of the algorithm.

Editorial extensions

If this is right

Even small errors in CLIP or human-supplied concept embeddings measurably hurt prediction, so refining embeddings is a necessary step for interpretable-by-design models, not an optional extra.
With enough samples, CCR reaches zero training loss while the concept matrix converges linearly to the ground-truth concepts, giving provable accuracy and provable interpretability at the same time.
The radius $\rho$ is a tunable dial between interpretability and accuracy: $\rho = 0$ reproduces IP-OMP, while a modest $\rho$ (0.1 in the experiments) improves accuracy across CIFAR-10/100, ImageNet, and Places365.
CCR reduces the training cost of interpretable image classification on ImageNet from tens of hours to about two hours, roughly a tenfold speedup over CLIP-IP-OMP and lf-CBM at similar or better accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the surrogate-to-real-estimator fidelity is the fragile link, the most direct testable extension is to replace the top-$k$ surrogate with task-driven or unrolled dictionary learning (alternatives the paper cites) and check whether the convergence still holds; the paper does not establish this.
The full-rank assumption suggests CCR's theoretical guarantees operate in the complete-dictionary regime; on highly overcomplete concept sets the column-wise contraction may be partial, so performance gains would likely be smaller than on benchmarks where concepts nearly span the feature space.
The empirical version swaps squared loss for cross-entropy, so the theorem does not literally cover the deployed model; if the linear-convergence behavior persists there, it would indicate a broader alignment mechanism than the proof captures.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a real contribution. CCR—optimizing concept embeddings under a per-column l2 radius—is a natural idea, and as far as I can tell it is new relative to residual modules and dual embeddings. The paper also ships the first convergence guarantee for task-driven dictionary updates in a constrained-interpretability setting, and the lower bound (Theorem 2.6) is a clean worst-case construction. The empirical story is refreshingly concrete: on CIFAR, ImageNet, and Places365, CCR beats CLIP-IP-OMP and lf-CBM at comparable sparsity, often with a 10x runtime cut, and code is linked. That alone makes it worth a careful read.

The weaknesses are real but not fatal, and I want to be precise about them. First, the stress-test note is correct: Theorem 3.4's proof of Lemma C.6 relies on d = n ('we recall that d = n in the full-rank and orthogonal setting'), but the theorem and Assumption 3.2 never state that. If n < d, the d-th eigenvalue of (1/m)∑ x_h x_hᵀ is zero and the claimed contraction cannot hold. The paper's own synthetic experiments actually confirm the boundary: with n < d the dictionary error plateaus. So the theorem as written is false in general; it needs an explicit d = n or a surrogate-gradient argument. Second, even with d = n, the proof analyzes the surrogate f̃ with squared loss and oracle responses r_i, while Algorithm 2 uses cross-entropy, hard-thresholding, dispersion, and normalization. The authors acknowledge this in Section 3 but do not bridge the gap. So the headline guarantee is narrower than the abstract suggests. That said, the paper is honest about some of this—it explicitly notes that a non-full-rank D* leaves orthogonal components uncorrected. I do not read this as intentional overreach; I read it as an idealized theory with a scoped application.

Minor: the empirical comparison gives five-run min-max shading for CCR but only point estimates for baselines, so the margin is not fully quantified. Also, the interpretability claim rests on the heuristic that staying within a small l2 radius preserves human meaning. That is a reasonable prior, not a proof.

Bottom line: the method is new, the theory is a genuine first step even if narrower than claimed, and the empirical gains are substantial. It deserves a serious referee who will push for the d = n statement and a spelled-out transfer argument. I would accept it for review and recommend the authors fix the theorem statement and add a remark on the surrogate gap.

Referee Report

3 major / 4 minor

Summary. This paper studies the effect of deviations between observed concept embeddings D and ground-truth latent query features D* in interpretable-by-design models based on Information Pursuit / OMP. It proves a worst-case lower bound (Theorem 2.6) showing that small column-wise perturbations of D can cause a non-vanishing squared-loss gap, and proposes Constrained Concept Refinement (CCR): constrained projected gradient descent on a differentiable surrogate f̃ of the IP-OMP estimator. Under a generative model with column-orthogonal D* and sparse x=D*β, Theorems 3.3 and 3.4 claim linear convergence of the surrogate loss and, with multiple samples, contraction of D(t) toward D*. The paper then instantiates CCR as Algorithm 2 for image classification with CLIP embeddings, hard thresholding, a learned linear layer and cross-entropy loss, reporting accuracy improvements over CLIP-IP-OMP and lf-CBM on CIFAR-10/100, ImageNet and Places365 with roughly an order of magnitude lower runtime. A repository link and ablations are provided.

Significance. If the theoretical guarantees held for the deployed method, this would be a valuable contribution: Theorem 2.6 gives a clean, self-contained lower-bound construction, and Theorem 3.4 is proved in detail with explicit sample complexity and contraction rate. The empirical evaluation is extensive, the runtime comparison is concrete, and the release of code and ablations of the dispersion/projection steps strengthens reproducibility. However, the gap between the analyzed surrogate (squared population loss with oracle access to β_i or r_i) and the deployed Algorithm 2 (cross-entropy on labels, hard thresholding, learned linear layer), together with the missing d=n condition in Theorem 3.4, means the current manuscript does not substantiate the abstract's claim that 'our algorithm' provably achieves zero loss in the setting where it is evaluated.

major comments (3)

[Theorem 3.4 and Lemma C.6 (Appendix C.4.7).] Theorem 3.4 is stated for a 'fixed full-rank column-orthogonal D*' with no assumption d=n, but the proof of Lemma C.6 relies on d=n ('we recall that d = n in the full-rank and orthogonal setting'). When d>n, D* is full column rank but not square, every x_h=D* β_h lies in the n-dimensional column space of D*, and the d×d matrix (1/m)∑_{h∈Q_i} x_h x_h^T has rank at most n; its d-th smallest eigenvalue is zero, so the lower bound in Lemma C.6 cannot hold. Components of d_i−d*_i orthogonal to range(D*) are annihilated by the gradient term x_h x_h^T(d_i−d*_i), so the claimed contraction ∥D(t+1)−D*∥_{1,2}≤τ∥D(t)−D*∥_{1,2} is impossible in general. This is not merely technical: the empirical deployment uses CLIP embeddings of dimension much larger than the number of concepts, and the paper's own Appendix D.1 treats n<d as the 'rank-deficient' setting. The theorem statement and Assumption 3.2 must either require d=n explicitly or provide a different argument for the non-square case.
[Section 3 (Eqs. (4)-(6)) and Algorithm 2.] The convergence guarantees are proved for the differentiable surrogate f̃ of Eq. (4) using squared population loss whose gradient in Eq. (5) requires the true coefficients β_i=⟨d*_i,x⟩ (or exact query responses r_i), whereas the deployed Algorithm 2 replaces the sparse code and prediction with hard-thresholded CLIP scores s_i=HT_λ(D^T x_i), trains a linear layer L with cross-entropy on labels, and never observes r_i or β_i. Section 3 itself notes that cross-entropy is 'a more common choice' but provides no argument that any of Theorems 3.3-3.4 transfers to this surrogate. Consequently the abstract's claim that 'our algorithm' is rigorously proved to achieve zero loss is accurate only for Algorithm 1 in a generative setting with an oracle, not for the algorithm whose empirical performance is reported. Either the analysis must be extended to the deployed loss and thresholding operation, or the claims must be explicitly scoped to the theoretical surrogate.
[Section 3.1 and Appendix D.1.] The interpretability guarantee in Theorem 3.4 is recovery of D* in the square orthonormal case; the paper's practical interpretability argument rests on the heuristic assumption that a small ℓ2 radius around CLIP embeddings preserves human-meaningful concepts. No experiment measures human agreement with the learned concepts, and the ACED values in Table 1 only measure distance from initialization, not semantic validity. If the authors want to claim that CCR 'preserves interpretability' in the empirical setting, a human-study or concept-alignment metric is needed; otherwise interpretability should be described as inherited from CLIP-IP-OMP's framework.

minor comments (4)

[Throughout the manuscript.] There are several typographical errors, including 'expainable' in the Introduction, 'suppose that that' in Theorem 3.3, and 'rank defficient' in Figure 4; these should be corrected.
[Appendix C.4.7, Lemma C.6.] The concentration displays in Eqs. (24)-(25) mix σ and σ² and are hard to verify; please restate the normalized bound from Theorem C.7 before multiplying by |Q_i|, so that the reader can check the final sample-complexity condition.
[Appendix D.2, Algorithm 3.] The dispersion factor r is not reported or discussed in the experiments; since Figure 6 shows that removing dispersion affects sparsity, please specify the value of r used for each benchmark.
[References.] The reference for Vaswani is incomplete; please provide the full author list and publication venue.

Circularity Check

0 steps flagged · score 1.0 of 10

No load-bearing circularity: the theoretical claims are self-contained proofs under an explicit generative model; only minor, non-load-bearing self-citations appear.

full rationale

The derivation chain is not circular. The main guarantees (Theorems 2.6, 3.3, 3.4) are proved from the stated generative model (Assumptions 2.3 and 3.2) using explicit lemmas (C.1–C.8), with no fitted parameter reused as a prediction and no external benchmark value imported into the proof. The surrogate f̃ in Eq. (4) is not fitted: it is the algebraic form of the IP-OMP estimator under column-orthogonality (Lemmas 3.1, C.1, C.2), and the proof demonstrates convergence of gradient descent on that surrogate. Self-citations to Liang et al. (2022, 2024) are motivational or stylistic ('proof strategy similar to Lemma 3.1 in (Liang et al., 2022)'), and the relevant proof is reproduced in full, so they are not load-bearing. Flagged limitations, weighed here as non-circularity: the paper explicitly states that if D* is not full-rank, deviations orthogonal to its column space 'cannot be eliminated by Equation (5)' (Section 3), and the proof of Lemma C.6 invokes an unstated squareness condition ('we recall that d = n in the full-rank and orthogonal setting'), which is a correctness and assumption gap for d > n, not a circular reduction. Similarly, the paper acknowledges that Algorithm 2's cross-entropy, hard-thresholding, and normalization are modifications not covered by Theorem 3.4; this is a transfer gap, not circularity. Empirical comparisons use external baselines CLIP-IP-OMP and lf-CBM. Overall, the central derivation does not reduce to its inputs.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The theory rests on the generative assumptions of Chattopadhyay et al. plus column-orthogonality and full-rank conditions; the deployed Algorithm 2 is a heuristic adaptation whose signal differs from the theoretical loss. No new physical entities are introduced. Hyperparameters ρ, λ, η, and the dispersion factor are hand-chosen and not all reported.

free parameters (5)

ρ (correction radius) = 0.1 fixed for all image experiments
Hand-chosen; controls the interpretability-accuracy trade-off; ablation shows performance saturates beyond 0.1.
λ (hard-threshold parameter) = not reported per dataset
Controls sparsity and explanation length; tuned in Appendix D.4 but final values are not given.
ηD, ηL (learning rates) = not reported
Step sizes for D and the linear layer in Algorithm 2; essential for convergence but unreported.
dispersion factor r = not reported
Scales concept angles in Algorithm 3; changes D(0) and therefore the whole refinement trajectory.
ρ (initial error bound in theory) = ≤ γ/(8√kΓ)
Both an assumption and the projection radius; central to support-recovery Lemma C.4 and convergence.

assumptions (7)

domain assumption Assumption 2.3: y=⟨x,z⟩ and q_i=⟨v_i,z⟩ with z~N(0,I_d).
Defines the theoretical testbed; real image classification does not follow this linear generative model.
domain assumption Assumption 3.2: x=D*β with column-orthogonal D*, support drawn uniformly, β_i zero-mean with γ≤|β_i|≤Γ.
Sparse-coding generative model needed for exact support recovery and linear convergence; not satisfied in the experiments.
domain assumption D* is full-rank (d=n) in Theorem 3.4.
Without full rank, deviations in the nullspace of D* cannot be corrected; the paper itself notes convergence fails for rank-deficient D*.
ad hoc to paper Surrogate estimator f̃ (Eq. 4) behaves like the actual IP-OMP estimator f_D.
All convergence theorems are for f̃; no bound relates their losses when D ≠ D*.
ad hoc to paper The update in Eq. (5) can access the true population loss or the coefficients β_i=⟨d*_i,x⟩.
The gradient formula used in Theorem 3.3 contains β_i; Algorithm 2 uses cross-entropy labels instead. This is an oracle assumption that is not stated.
domain assumption CLIP embeddings (after dispersion) retain human-meaningful concept semantics within radius ρ.
Underpins the interpretability-preservation claim; no human study verifies it.
standard math Standard concentration inequalities (Vershynin Thm C.7, Chernoff Thm C.8).
Used in Lemma C.6 to control covariance estimation and support sizes; accepted background.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing Performance of Explainable AI Models with Constrained Concept Refinement." pith.science (2026). https://pith.science/paper/H3M6B5WE

@misc{pith2026250206775,
  author       = {Pith},
  title        = {Pith review of: Enhancing Performance of Explainable AI Models with Constrained Concept Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3M6B5WE}},
  note         = {Machine review of arXiv:2502.06775}
}

read the original abstract

The trade-off between accuracy and interpretability has long been a challenge in machine learning (ML). This tension is particularly significant for emerging interpretable-by-design methods, which aim to redesign ML algorithms for trustworthy interpretability but often sacrifice accuracy in the process. In this paper, we address this gap by investigating the impact of deviations in concept representations-an essential component of interpretable models-on prediction performance and propose a novel framework to mitigate these effects. The framework builds on the principle of optimizing concept embeddings under constraints that preserve interpretability. Using a generative model as a test-bed, we rigorously prove that our algorithm achieves zero loss while progressively enhancing the interpretability of the resulting model. Additionally, we evaluate the practical performance of our proposed framework in generating explainable predictions for image classification tasks across various benchmarks. Compared to existing explainable methods, our approach not only improves prediction accuracy while preserving model interpretability across various large-scale benchmarks but also achieves this with significantly lower computational cost.

Figures

Figures reproduced from arXiv: 2502.06775 by the authors.

**Figure 1.** The red arrows represent the backpropagation training process for classic explainable AI models. This paper extends the training process to refine concept embeddings with constraints on their deviation from initial embeddings, represented by green arrows and box. low accuracy (Zarlenga et al., 2022; Chowdhury et al., 2024). For embeddings generated with pretrained models, their encoders are reported to face issues … view at source ↗

**Figure 2.** Prediction accuracy of CCR and its baseline across iterations, with the final test accuracy of CLIP-IP-OMP and lf-CBM indicated for reference. For CCR and its baseline, we run each experiment for five times and present the average test accuracy at each time step. The shaded area is bounded by the maximum and minimum accuracy obtained over five runs. the concept refinement step, in the context of explainable image c… view at source ↗

**Figure 3.** The first example illustrates a simple case where CCR successfully learns the correct concepts. The second example represents a misleading case, where the image contains concepts like “a flag” that, while relevant and visually apparent, could potentially mislead classification. However, CCR effectively extracts both useful and misleading information, assigning appropriate weights to ensure the correct prediction. 4… view at source ↗

Figures from the paper (25 more)

**Figure 4.** Figure 4: Results on synthetic dataset. D.2. More Details on Algorithm 2 Here we provide a detailed description for the concept dispersion step and the embedding normalization and projection step in Algorithm 2. In Algorithm 3, we present the pseudo-code for concept dispersion. …

**Figure 5.** Figure 5: We calculate the correlation matrix (D⊤D) for dictionaries before and after Algorithm 3, and present them in the format of heatmaps. As can be seen, the proposed dispersion process effectively reduces the correlation between concept embeddings generated by CLIP. 30 [P…

**Figure 6.** Figure 6: Ablation study for Algorithm 3 and Algorithm 4. D.3. Ablation Study In this section, we present empirical findings from our ablation study. As a preliminary note, we emphasize that the comparison between CCR and its baseline shown in [PITH_FULL_IMAGE:figures/full_fig_…

**Figure 7.** Figure 7: Average sparsity and test accuracy for varying thresholds λ. in the left column of [PITH_FULL_IMAGE:figures/full_fig_p032_7.png]

**Figure 8.** Figure 8: Average sparsity and test accuracy for varying radius bounds ρ. • Reliance on the richness of the concept set. The effectiveness of the algorithm is inherently influenced by the quality of the concept set, a phenomenon that aligns with the fundamental motivation behind…

**Figure 9.** Figure 9: CIFAR-10 (a) [PITH_FULL_IMAGE:figures/full_fig_p034_9.png]

**Figure 10.** Figure 10: CIFAR-10 (b) [PITH_FULL_IMAGE:figures/full_fig_p034_10.png]

**Figure 11.** Figure 11: CIFAR-10 (c) 34 [PITH_FULL_IMAGE:figures/full_fig_p034_11.png]

**Figure 12.** Figure 12: CIFAR-100 (a) [PITH_FULL_IMAGE:figures/full_fig_p035_12.png]

**Figure 13.** Figure 13: CIFAR-100 (b) [PITH_FULL_IMAGE:figures/full_fig_p035_13.png]

**Figure 14.** Figure 14: CIFAR-100 (c) 35 [PITH_FULL_IMAGE:figures/full_fig_p035_14.png]

**Figure 15.** Figure 15: CUB-200 (a) [PITH_FULL_IMAGE:figures/full_fig_p036_15.png]

**Figure 16.** Figure 16: CUB-200 (b) [PITH_FULL_IMAGE:figures/full_fig_p036_16.png]

**Figure 17.** Figure 17: CUB-200 (c) 36 [PITH_FULL_IMAGE:figures/full_fig_p036_17.png]

**Figure 18.** Figure 18: ImageNet (a) [PITH_FULL_IMAGE:figures/full_fig_p037_18.png]

**Figure 19.** Figure 19: ImageNet (b) [PITH_FULL_IMAGE:figures/full_fig_p037_19.png]

**Figure 20.** Figure 20: ImageNet (c) 37 [PITH_FULL_IMAGE:figures/full_fig_p037_20.png]

**Figure 21.** Figure 21: Places365 (a) [PITH_FULL_IMAGE:figures/full_fig_p038_21.png]

**Figure 22.** Figure 22: Places365 (b) [PITH_FULL_IMAGE:figures/full_fig_p038_22.png]

**Figure 23.** Figure 23: Places365 (c) 38 [PITH_FULL_IMAGE:figures/full_fig_p038_23.png]

**Figure 24.** Figure 24: Baseline comparison to the top figure in [PITH_FULL_IMAGE:figures/full_fig_p039_24.png]

**Figure 25.** Figure 25: Baseline comparison to the bottom figure in [PITH_FULL_IMAGE:figures/full_fig_p039_25.png]

**Figure 26.** Figure 26: Baseline comparison to [PITH_FULL_IMAGE:figures/full_fig_p039_26.png]

**Figure 27.** Figure 27: Baseline comparison to [PITH_FULL_IMAGE:figures/full_fig_p040_27.png]

**Figure 28.** Figure 28: Baseline comparison to [PITH_FULL_IMAGE:figures/full_fig_p040_28.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 43 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Sanity checks for saliency maps

Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018

2018
[3]

Learning sparsely used overcomplete dictionaries via alternating minimization

Agarwal, A., Anandkumar, A., Jain, P., and Netrapalli, P. Learning sparsely used overcomplete dictionaries via alternating minimization. SIAM Journal on Optimization, 26 0 (4): 0 2775--2799, 2016

work page 2016
[4]

K-svd: An algorithm for designing overcomplete dictionaries for sparse representation

Aharon, M., Elad, M., and Bruckstein, A. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing, 54 0 (11): 0 4311--4322, 2006

work page 2006
[5]

W., Anderson, T

Anderson, T. W., Anderson, T. W., Anderson, T. W., and Anderson, T. W. An introduction to multivariate statistical analysis, volume 2. Wiley New York, 1958

work page 1958
[6]

Simple, efficient, and neural algorithms for sparse coding

Arora, S., Ge, R., Ma, T., and Moitra, A. Simple, efficient, and neural algorithms for sparse coding. In Conference on learning theory, pp.\ 113--149. PMLR, 2015

work page 2015
[7]

How to explain individual classification decisions

Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., and M \"u ller, K.-R. How to explain individual classification decisions. The Journal of Machine Learning Research, 11: 0 1803--1831, 2010

work page 2010
[8]

o m, M., L \

Barrag \'a n-Montero, A., Javaid, U., Vald \'e s, G., Nguyen, D., Desbordes, P., Macq, B., Willems, S., Vandewinckele, L., Holmstr \"o m, M., L \"o fman, F., et al. Artificial intelligence and machine learning for medical imaging: A technology review. Physica Medica, 83: 0 242--256, 2021

work page 2021

Show all 70 references

[9]

What, indeed, is an achievable provable guarantee for learning-enabled safety-critical systems

Bensalem, S., Cheng, C.-H., Huang, W., Huang, X., Wu, C., and Zhao, X. What, indeed, is an achievable provable guarantee for learning-enabled safety-critical systems. In International Conference on Bridging the Gap between AI and Reality, pp.\ 55--76. Springer, 2023

2023
[10]

Language models are few-shot learners

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33: 0 1877--1901, 2020

1901
[11]

Semantic bottleneck for computer vision tasks

Bucher, M., Herbin, S., and Jurie, F. Semantic bottleneck for computer vision tasks. In Computer Vision--ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2--6, 2018, Revised Selected Papers, Part II 14, pp.\ 695--712. Springer, 2019

2018
[12]

D., Vidal, R., and Geman, D

Chattopadhyay, A., Slocum, S., Haeffele, B. D., Vidal, R., and Geman, D. Interpretable by design: Learning predictors by composing interpretable queries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 0 (6): 0 7430--7443, 2022

2022
[13]

Chattopadhyay, A., Chan, K. H. R., Haeffele, B. D., Geman, D., and Vidal, R. Variational information pursuit for interpretable predictions. arXiv preprint arXiv:2302.02876, 2023

2023 arXiv
[14]

Information maximization perspective of orthogonal matching pursuit with applications to explainable ai

Chattopadhyay, A., Pilgrim, R., and Vidal, R. Information maximization perspective of orthogonal matching pursuit with applications to explainable ai. Advances in Neural Information Processing Systems, 36, 2024

2024
[15]

F., Phan, V

Chowdhury, T. F., Phan, V. M. H., Liao, K., To, M.-S., Xie, Y., van den Hengel, A., Verjans, J. W., and Liao, Z. Adacbm: An adaptive concept bottleneck model for explainable and accurate diagnosis. In International Conference on Medical Image Computing and Computer-Assisted In...

2024
[16]

R., Oksuz, I., Puyol-Ant \'o n, E., Ruijsink, B., King, A

Clough, J. R., Oksuz, I., Puyol-Ant \'o n, E., Ruijsink, B., King, A. P., and Schnabel, J. A. Global and local interpretability for cardiac mri classification. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp.\ 656--664. Springer, 2019

2019
[17]

Imagenet: A large-scale hierarchical image database

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

2009
[18]

Dutta, S., Wei, H., van der Laan, L., and Alaa, A. M. Estimating uncertainty in multimodal foundation models using public internet data. arXiv preprint arXiv:2310.09926, 2023

2023 arXiv
[19]

Rethinking domain adaptation and generalization in the era of clip

Feng, R., Yu, T., Jin, X., Yu, X., Xiao, L., and Chen, Z. Rethinking domain adaptation and generalization in the era of clip. In 2024 IEEE International Conference on Image Processing (ICIP), pp.\ 2585--2591. IEEE, 2024

2024
[20]

Can we constrain concept bottleneck models to learn semantically meaningful input features? arXiv preprint arXiv:2402.00912, 2024

Furby, J., Cunnington, D., Braines, D., and Preece, A. Can we constrain concept bottleneck models to learn semantically meaningful input features? arXiv preprint arXiv:2402.00912, 2024

2024 arXiv
[21]

and Jedynak, B

Geman, D. and Jedynak, B. An active testing model for tracking roads in satellite images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18 0 (1): 0 1--14, 1996

1996
[22]

Interpretation of neural networks is fragile

Ghorbani, A., Abid, A., and Zou, J. Interpretation of neural networks is fragile. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp.\ 3681--3688, 2019

2019
[23]

W., Gast, J., Ruiz, I

Gondal, M. W., Gast, J., Ruiz, I. A., Droste, R., Macri, T., Kumar, S., and Staudigl, L. Domain aligned clip for few-shot classification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp.\ 5721--5730, 2024

2024
[24]

Regression concept vectors for bidirectional explanations in histopathology

Graziani, M., Andrearczyk, V., and M \"u ller, H. Regression concept vectors for bidirectional explanations in histopathology. In Understanding and Interpreting Machine Learning in Medical Image Computing Applications: First International Workshops, MLCN 2018, DLF 2018, and iM...

2018
[25]

Identifying implicit social biases in vision-language models

Hamidieh, K., Zhang, H., Gerych, W., Hartvigsen, T., and Ghassemi, M. Identifying implicit social biases in vision-language models. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 7, pp.\ 547--561, 2024

2024
[26]

The elements of statistical learning: data mining, inference, and prediction, 2009

Hastie, T. The elements of statistical learning: data mining, inference, and prediction, 2009

2009
[27]

Probabilistic concept bottleneck models

Kim, E., Jung, D., Park, S., Kim, S., and Yoon, S. Probabilistic concept bottleneck models. arXiv preprint arXiv:2306.01574, 2023

2023 arXiv
[28]

u tt, K. T., D \

Kindermans, P.-J., Hooker, S., Adebayo, J., Alber, M., Sch \"u tt, K. T., D \"a hne, S., Erhan, D., and Kim, B. The (un) reliability of saliency methods. Explainable AI: Interpreting, explaining and visualizing deep learning, pp.\ 267--280, 2019

2019
[29]

W., Nguyen, T., Tang, Y

Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. Concept bottleneck models. In International conference on machine learning, pp.\ 5338--5348. PMLR, 2020

2020
[30]

A., Levie, R., Bruna, J., and Kutyniok, G

Kolek, S., Nguyen, D. A., Levie, R., Bruna, J., and Kutyniok, G. A rate-distortion framework for explaining black-box model decisions. In International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers, pp.\ 91--115. Springer, 2020

2020
[31]

Learning multiple layers of features from tiny images

Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009

2009
[32]

H., Nickisch, H., and Harmeling, S

Lampert, C. H., Nickisch, H., and Harmeling, S. Learning to detect unseen object classes by between-class attribute transfer. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 951--958. IEEE, 2009

2009
[33]

Deep learning

LeCun, Y., Bengio, Y., and Hinton, G. Deep learning. nature, 521 0 (7553): 0 436--444, 2015

2015
[34]

Liang, G., Zhang, G., Fattahi, S., and Zhang, R. Y. Simple alternating minimization provably solves complete dictionary learning. arXiv preprint arXiv:2210.12816, 2022

2022 arXiv
[35]

Personalized dictionary learning for heterogeneous datasets

Liang, G., Shi, N., Al Kontar, R., and Fattahi, S. Personalized dictionary learning for heterogeneous datasets. Advances in Neural Information Processing Systems, 36, 2024

2024
[36]

Task-driven dictionary learning

Mairal, J., Bach, F., and Ponce, J. Task-driven dictionary learning. IEEE transactions on pattern analysis and machine intelligence, 34 0 (4): 0 791--804, 2011

2011
[37]

Understanding approximate and unrolled dictionary learning for pattern recovery

Mal \'e zieux, B., Moreau, T., and Kowalski, M. Understanding approximate and unrolled dictionary learning for pattern recovery. arXiv preprint arXiv:2106.06338, 2021

2021 arXiv
[38]

Interpretability is in the mind of the beholder: A causal framework for human-interpretable representation learning

Marconato, E., Passerini, A., and Teso, S. Interpretability is in the mind of the beholder: A causal framework for human-interpretable representation learning. Entropy, 25 0 (12): 0 1574, 2023

2023
[39]

Do concept bottleneck models learn as intended? arXiv preprint arXiv:2105.04289, 2021

Margeloiu, A., Ashman, M., Bhatt, U., Chen, Y., Jamnik, M., and Weller, A. Do concept bottleneck models learn as intended? arXiv preprint arXiv:2105.04289, 2021

2021 arXiv
[40]

and Raghavan, P

Motwani, R. and Raghavan, P. Randomized algorithms. ACM Computing Surveys (CSUR), 28 0 (1): 0 33--37, 1996

1996
[41]

M., and Weng, T.-W

Oikarinen, T., Das, S., Nguyen, L. M., and Weng, T.-W. Label-free concept bottleneck models. arXiv preprint arXiv:2304.06129, 2023

2023 arXiv
[42]

Olshausen, B. A. and Field, D. J. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research, 37 0 (23): 0 3311--3325, 1997

1997
[43]

C., Rezaiifar, R., and Krishnaprasad, P

Pati, Y. C., Rezaiifar, R., and Krishnaprasad, P. S. Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Proceedings of 27th Asilomar conference on signals, systems and computers, pp.\ 40--44. IEEE, 1993

1993
[44]

Quinlan, J. R. Induction of decision trees. Machine learning, 1: 0 81--106, 1986

1986
[45]

W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.\ 8748--8763. PMLR, 2021

2021
[46]

Unveiling glitches: A deep dive into image encoding bugs within clip

Ranjan, A., Wen, D., and Bhat, K. Unveiling glitches: A deep dive into image encoding bugs within clip. arXiv preprint arXiv:2407.00592, 2024

2024 arXiv
[47]

Analysis of fast structured dictionary learning

Ravishankar, S., Ma, A., and Needell, D. Analysis of fast structured dictionary learning. Information and Inference: A Journal of the IMA, 9 0 (4): 0 785--811, 2020

2020
[48]

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1 0 (5): 0 206--215, 2019

2019
[49]

and Nakamura, K

Sawada, Y. and Nakamura, K. Concept bottleneck model with additional unsupervised concepts. IEEE Access, 10: 0 41758--41765, 2022

2022
[50]

R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D

Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp.\ 618--626, 2017

2017
[51]

Investigating the limitation of clip models: The worst-performing categories

Shao, J.-J., Shi, J.-X., Yang, X.-W., Guo, L.-Z., and Li, Y.-F. Investigating the limitation of clip models: The worst-performing categories. arXiv preprint arXiv:2310.03324, 2023

2023 arXiv
[52]

Learning important features through propagating activation differences

Shrikumar, A., Greenside, P., and Kundaje, A. Learning important features through propagating activation differences. In International conference on machine learning, pp.\ 3145--3153. PMlR, 2017

2017
[53]

Deep inside convolutional networks: Visualising image classification models and saliency maps

Simonyan, K., Vedaldi, A., and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013

2013 arXiv
[54]

Fooling lime and shap: Adversarial attacks on post hoc explanation methods

Slack, D., Hilgard, S., Jia, E., Singh, S., and Lakkaraju, H. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp.\ 180--186, 2020

2020
[55]

Smoothgrad: removing noise by adding noise

Smilkov, D., Thorat, N., Kim, B., Vi \'e gas, F., and Wattenberg, M. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825, 2017

2017 arXiv
[56]

Fooling network interpretation in image classification

Subramanya, A., Pillai, V., and Pirsiavash, H. Fooling network interpretation in image classification. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 2020--2029, 2019

2020
[57]

Complete dictionary recovery over the sphere i: Overview and the geometric picture

Sun, J., Qu, Q., and Wright, J. Complete dictionary recovery over the sphere i: Overview and the geometric picture. IEEE Transactions on Information Theory, 63 0 (2): 0 853--884, 2016

2016
[58]

and Ba, D

Tolooshams, B. and Ba, D. Stable and interpretable unrolled dictionary learning. arXiv preprint arXiv:2106.00058, 2021

2021 arXiv
[59]

Tropp, J. A. Greed is good: Algorithmic results for sparse approximation. IEEE Transactions on Information theory, 50 0 (10): 0 2231--2242, 2004

2004
[60]

Attention is all you need

Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017

2017
[61]

High-dimensional probability: An introduction with applications in data science, volume 47

Vershynin, R. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018

2018
[62]

The caltech-ucsd birds-200-2011 dataset

Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd birds-200-2011 dataset. 2011

2011
[63]

and Kang, G

Wang, Y. and Kang, G. Attention head purification: A new perspective to harness clip for domain generalization. arXiv preprint arXiv:2412.07226, 2024

2024 arXiv
[64]

Watson, D. S. Conceptual challenges for interpretable machine learning. Synthese, 200 0 (2): 0 65, 2022

2022
[65]

Neural-symbolic vqa: Disentangling reasoning from vision and language understanding

Yi, K., Wu, J., Gan, C., Torralba, A., Kohli, P., and Tenenbaum, J. Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. Advances in neural information processing systems, 31, 2018

2018
[66]

Post-hoc concept bottleneck models

Yuksekgonul, M., Wang, M., and Zou, J. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205.15480, 2022

2022 arXiv
[67]

E., Barbiero, P., Ciravegna, G., Marra, G., Giannini, F., Diligenti, M., Precioso, F., Melacci, S., Weller, A., Lio, P., et al

Zarlenga, M. E., Barbiero, P., Ciravegna, G., Marra, G., Giannini, F., Diligenti, M., Precioso, F., Melacci, S., Weller, A., Lio, P., et al. Concept embedding models. In NeurIPS 2022-36th Conference on Neural Information Processing Systems, 2022

2022
[68]

Zeiler, M. D. and Fergus, R. Visualizing and understanding convolutional networks. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pp.\ 818--833. Springer, 2014

2014
[69]

Places: A 10 million image database for scene recognition

Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40 0 (6): 0 1452--1464, 2017

2017
[70]

Interpretable basis decomposition for visual explanation

Zhou, B., Sun, Y., Bau, D., and Torralba, A. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), pp.\ 119--134, 2018

2018

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Sanity checks for saliency maps

Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018

2018

[3] [3]

Learning sparsely used overcomplete dictionaries via alternating minimization

Agarwal, A., Anandkumar, A., Jain, P., and Netrapalli, P. Learning sparsely used overcomplete dictionaries via alternating minimization. SIAM Journal on Optimization, 26 0 (4): 0 2775--2799, 2016

work page 2016

[4] [4]

K-svd: An algorithm for designing overcomplete dictionaries for sparse representation

Aharon, M., Elad, M., and Bruckstein, A. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing, 54 0 (11): 0 4311--4322, 2006

work page 2006

[5] [5]

W., Anderson, T

Anderson, T. W., Anderson, T. W., Anderson, T. W., and Anderson, T. W. An introduction to multivariate statistical analysis, volume 2. Wiley New York, 1958

work page 1958

[6] [6]

Simple, efficient, and neural algorithms for sparse coding

Arora, S., Ge, R., Ma, T., and Moitra, A. Simple, efficient, and neural algorithms for sparse coding. In Conference on learning theory, pp.\ 113--149. PMLR, 2015

work page 2015

[7] [7]

How to explain individual classification decisions

Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., and M \"u ller, K.-R. How to explain individual classification decisions. The Journal of Machine Learning Research, 11: 0 1803--1831, 2010

work page 2010

[8] [8]

o m, M., L \

Barrag \'a n-Montero, A., Javaid, U., Vald \'e s, G., Nguyen, D., Desbordes, P., Macq, B., Willems, S., Vandewinckele, L., Holmstr \"o m, M., L \"o fman, F., et al. Artificial intelligence and machine learning for medical imaging: A technology review. Physica Medica, 83: 0 242--256, 2021

work page 2021