Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

DCBM: Data-Efficient Visual Concept Bottleneck Models

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that a concept bottleneck model can be built from image regions proposed by segmentation or detection models, needing only 50 images per class for concept generation, and that this visual concept bank is accurate enough to…

desk verdict Useful data-efficient CBM with thorough ablations; interpretability claim needs grounding before it fully lands. read the letter →

arxiv 2412.11576 v3 pith:6QVWZ2YB submitted 2024-12-16 cs.CV

Katharina Prasse , Patrick Knab , Sascha Marton , Christian Bartelt , Margret Keuper This is my paper · ORCID
classification cs.CV
keywords conceptbottleneckmodelsvisualconceptssegmentationfoundationdetectiondata-efficientgenerationfine-grainedclassificationout-of-distributiongeneralizationlocalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Concept Bottleneck Models usually get their concepts from large language model text descriptions or large image corpora, which makes them hard to use when data is scarce. The paper claims that a CBM can instead define concepts as image regions cut out by off-the-shelf segmentation or detection models, cluster those regions in CLIP embedding space, and train a sparse linear layer over the cluster centroids. With only 50 images per class, this produces a concept bank that yields competitive accuracy, including the best CBM accuracy in its comparison on the fine-grained CUB dataset (82.4% with the ViT-L/14 backbone) and smaller accuracy losses on ImageNet-R than a task-agnostic text-named CBM. The significance is that interpretability no longer depends on a predefined vocabulary: the concepts are dataset-specific, can be localized in test images, can be named after the fact, and can be removed by a text prompt if they are spurious.

What carries the argument

The central object is a concept bank $C=\{c_j\}_{j=1}^{k}$ of $k$ cluster centroids in the CLIP embedding space, each centroid being the median embedding of a cluster of image-region crops proposed by a segmentation or detection foundation model. Concept activations are computed as the normalized projection $a(x_i)_j = \langle f(x_i), c_j\rangle / \|c_j\|_2^2$, and a sparse linear layer $t(x_i)=\omega^{\top}a(x_i)$ with $\ell^1$ regularization turns these activations into class predictions. The machinery is: foundation-model proposals give multi-granularity visual concepts; k-means clustering collapses redundant crops into a few thousand centroids; and the frozen CLIP encoder plus linear layer keeps the whole pipeline inspectable at every step.

What would settle it

On the CUB bird dataset used in the paper, run DCBM twice, once with part-sensitive segment proposals and once with only whole-image crops as proposals, keeping cluster counts, centroids, and training identical; if the whole-image variant matches the segment variant in accuracy, the claim that image-region concepts are doing the work would fail. A complementary check is to compute the Grid Pointing Game score after ablating all concepts whose Grad-CAM peak in test images falls outside their own proposal bounding box; if the localization score stays near the 0.25 random baseline, the localization claim fails.

Watch

Extended reading notes

Core claim

The paper's discovery is that a concept bottleneck model does not need text descriptions, large concept corpora, or task-agnostic pretraining to be accurate and interpretable. If the image is first decomposed by a segmentation or detection foundation model into region proposals, and those proposals are embedded, clustered, and reduced to a few thousand centroids, then a single sparse linear layer over the centroid activations can classify competitively. The paper reports that this holds across several generic segmenters and a promptable detector, with 2048 clusters as the default, and that the approach is strongest where predefined text concepts are weakest: fine-grained classes such as CUB (82.4% with ViT-L/14, the best CBM in its comparison) and out-of-distribution renditions, where every DCBM variant keeps a smaller IID-to-OOD error gap than the task-agnostic DN-CBM. The visual concepts are also localizable, as measured by Grad-CAM and the Grid Pointing Game, and they can be named post hoc by matching cluster centroids to text in CLIP space.

Load-bearing premise

The load-bearing premise is that the image regions cut out by a pretrained segmenter or detector from as few as 50 images per class, after clustering, capture the visual distinctions the target classes differ on; if the proposals miss the discriminative parts, the linear classifier has no way to recover the lost accuracy.

Editorial extensions

If this is right

  • DCBM achieves 82.4% top-1 accuracy on CUB with the ViT-L/14 backbone, the best among the CBMs compared, indicating that visual part-level concepts fit fine-grained classification.
  • On ImageNet-R, DCBM variants show IID-to-OOD error gaps of 22.4–27.4 percentage points, versus 38.8 for DN-CBM, indicating that visual dataset-specific concepts transfer to out-of-distribution renditions better than task-agnostic text-named concepts.
  • Building the concept bank from 50 images per class cuts concept-generation data by about 96% on ImageNet compared with the task-agnostic baseline and needs no extra image-caption corpus or pretraining.
  • On MiT-States and ClimateTV, DCBM outperforms the linear probe (e.g., 42.8–43.3 vs. 37.3 on MiT-States and 85.6–87.9 vs. 84.5 on ClimateTV), suggesting the pipeline adapts to novel domains quickly.
  • Because concepts are visual and clustered, unwanted concepts can be removed after clustering via a text prompt, giving users a way to suppress spurious correlations (e.g., removing stone concepts from a gull classifier leaves accuracy unchanged or slightly better).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the frozen concept bank doubles as a fixed visual vocabulary for post-hoc explanation: project any black-box model's embeddings onto the same centroids to obtain concept attributions without retraining the pipeline.
  • A stress test beyond the paper's ablations would vary segment size thresholds far beyond the tested range; if accuracy stays flat across very different proposal distributions, the data-efficiency claim is robust, and if it changes sharply, the choice of segmentation model is the real load-bearing component.
  • The discussion sketches interpretable regression through concept activations as spatial extent; this is testable, for instance by predicting lesion severity from how strongly a single damage concept activates across an image.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes Data-efficient Concept Bottleneck Models (DCBMs), which build a concept bottleneck from image regions proposed by segmentation or detection foundation models (SAM, SAM2, Mask R-CNN, DETR, GroundingDINO). The proposals are embedded with a frozen CLIP image encoder, clustered with k-means, and the cluster centroids serve as concepts for a sparse linear classifier. Concepts are optionally named via CLIP text alignment. The authors claim that DCBMs need only 50 images per class for concept generation, yield dataset-specific multi-granularity concepts, perform well on fine-grained tasks and under domain shift, and preserve interpretability by allowing visual concepts to be localized in test images. The evaluation covers ImageNet, Places365, CUB, CIFAR-10/100, ImageNet-R, MiT-States, ClimateTV, AwA2, and CelebA, with extensive ablations and a public code release.

Significance. If the claims were fully established, DCBM would be a practically valuable recipe: foundation-model segmentation or detection plus k-means in CLIP space and a sparse linear layer yields an interpretable CBM from very few images, with released code and unusually thorough ablations. The paper is also honest about limitations, explicitly acknowledging spurious correlations and the possibility of decisions based on concepts not present in the image. However, the central interpretability claim is not yet supported: concept activations in Eq. (2) are never validated as concept-presence indicators, and the only quantitative localization evidence, the Grid Pointing Game, reports internally inconsistent numbers. The contribution is therefore promising but currently overclaimed relative to the evidence.

major comments (4)
  1. [Section 3.3, Eq. (2); Sections 4.3, 4.4.2, and 5] The central claim that DCBMs 'preserve interpretability' is not established because concept activations in Eq. (2) are projections of the whole-image CLIP embedding onto cluster centroids, and the paper never shows that high activation implies the concept's visual presence in the image. Figure 4 itself shows an ambulance correctly classified with top concepts 'med', 'police', 'kit', and 'injection' that are not visible in the input, and Section 5 concedes that decisions can be based on 'semantically close concepts that are not present in the test image.' The Grid Pointing Game in Section 4.4.2 validates Grad-CAM localization of the linear layer's chosen concepts, not the correspondence between Eq. (2) activations and concept presence. I request a direct validation of Eq. (2) as a presence indicator, for example through concept-presence annotations or intervention experiments showing that activation is high only when the concept is actually present, or a revised claim that restricts interpretability to a linear probe with a named visual dictionary.
  2. [Section 4.4.2 and Table 4] The quantitative localization evidence is internally inconsistent and therefore not usable as support. The text reports 'Abs: 0.7028 vs. 0.4988' while Table 4 lists Abs values of 0.6551 and 0.3445. The text also lists three metrics (Gini Index, relative percentage, maximum score), whereas Table 4 has columns labeled Gini, Percentage, and Abs, with no definition of the difference between 'Percentage' and 'Abs' or of 'maximum score.' No error bars or confidence intervals are reported. These numbers should be reconciled, the metrics precisely defined, and the evaluation rerun or reported with variance.
  3. [Section 4.2.1 and Table 3] The claim that 'On both datasets, DCBMs outperform the linear probes' is contradicted by Table 3: on ClimateTV, DCBM-GDINO achieves 81.8% versus the linear probe's 84.5%. Similarly, the 'within 6%' statement in Section 4.2.1 conflicts with the 'within 5%' statement in Section 5, given that the ImageNet gap in Table 1 is exactly 6.0%. These contradictions should be corrected, and the performance comparisons should be accompanied by error bars or significance statements, since many differences in Table 1 and Table 3 are smaller than typical run-to-run variation.
  4. [Related Work and Table 1] The paper motivates itself against parallel visual-concept CBMs (Kowal et al., 2024; Sun et al., 2024; Zhu et al., 2024) and cites them in Section 2, but Table 1 includes only text-based CBMs and DN-CBM. Without a quantitative comparison to at least one of these closest visual-concept methods, the claim of 'extensively evaluate' and the positioning against parallel visual-concept CBMs are incomplete. Please add such comparisons where feasible, or explicitly state and justify why they are not comparable.
minor comments (7)
  1. [Section 4.2.1 vs. Section 5] Please reconcile the 'within 6%' and 'within 5%' statements; Table 1 shows an ImageNet gap of 6.0%, so the Discussion's 'within 5%' is inaccurate.
  2. [Section 4.4.2] The sentence 'Percentage and absolute scores are lower bounded at 0.25' is unclear because the text earlier defines only Gini, relative percentage, and maximum score; please define every metric precisely.
  3. [Figure 2 caption] The caption reads 'effectively (Step 3)' without a verb; please rephrase to complete the sentence.
  4. [Table 13 in Appendix D.2] The value '82.' for DCBM-MASK-RCNN with CLIP ViT-L/14 appears to be missing a digit; please correct it.
  5. [Appendix D, first paragraph] The appendix states that 'Each DCBM was trained with 1024 clusters,' whereas the main text uses k = 2048; please clarify that the ablations use a different cluster count and explain any implications.
  6. [Appendix F.2] The text 'A W A2' appears to be a typo for 'AwA2'.
  7. [Section 5] The phrase 'as in(Wang et al., 2023)' is missing a space before the citation.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: DCBM is an empirical pipeline with unsupervised concept extraction; its interpretability limitations are evidentiary, not circular.

full rationale

This paper is an empirical method paper, not a derivation chain. DCBM builds concepts by (1) running segmentation/detection foundation models on a subset of training images, (2) clustering the CLIP embeddings of the resulting crops, and (3) training a sparse linear layer on the projections of whole-image embeddings onto those cluster centroids (Eqs. 1-2). The accuracy results in Tables 1-3 are standard held-out evaluations: concept banks are constructed from training images and the linear layer is fit to training labels, then evaluated on test splits. No reported 'prediction' reduces by construction to a fitted constant. The concept naming step (Section 3.4) is explicitly post-hoc and the paper cautions that 'the image should always be attributed more weight than the text, as the image-text alignment may be imperfect,' so the nearest-text assignment is not dressed up as an independent prediction. Self-citations (ClimateTV from Prasse et al. 2023; Knab et al. 2025 for a LIME variant) are evaluation resources or background, not load-bearing evidence for the central claim; the main benchmarks are external and the code is released. No uniqueness theorem or ansatz is imported from the authors' prior work. Two evidentiary weaknesses are flagged but are not circularity: (i) Eq. (2) activations are global projections, and the paper's own Figure 4 and Discussion concede that top concepts such as 'med', 'police' and 'injection' can be absent from the image, so the localization/interpretability claim is weaker than the abstract suggests; (ii) the Grid Pointing Game numbers are internally inconsistent (Table 4 reports Abs 0.6551/0.3445 while Section 4.4.2 states Abs 0.7028/0.4988). These are correctness concerns, not reductions of the method to its inputs.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method does not introduce new physical or theoretical entities. It relies on existing foundation models and CLIP embeddings. The free parameters are mostly standard hyperparameters chosen by validation or hand, and the axioms are standard assumptions in the CBM and interpretability literature. The main dependency is that the quality of the visual concepts is inherited entirely from the segmentation/detection model and the CLIP embedding space.

free parameters (6)
  • n (images per class for concept generation) = 50
    Chosen as a balance between efficiency and accuracy; ablations show 25 and 50 are similar. Central to the data-efficiency claim.
  • k (number of clusters) = 2048
    Ablated over 128 to 4096; chosen based on validation and observed tendency toward larger k.
  • lambda (sparsity regularization) = 1e-4
    Found optimal in hyperparameter search on CUB, ImageNette, and ImageWoof.
  • learning rate = 1e-4
    Found optimal in hyperparameter search.
  • segmentation and detection hyperparameters = SAM/SAM2: points_per_side=64, pred_iou_thresh=0.88, stability_score_thresh=0.95, box_nms_thresh=0.5…
    Hand-set to retrieve sub-part concepts; not systematically ablated in the main text.
  • GroundingDINO prompts = CUB parts, AWA attributes, Part-ImageNet labels, Pascal parts
    External part and attribute label sets used to steer detection; influence concept proposals.
assumptions (5)
  • domain assumption CLIP embedding space captures semantic similarity between image regions and enables text-based naming
    Invoked in Section 3.2 for embedding proposals and Section 3.4 for naming; if CLIP is poorly aligned for a domain, concept quality and naming suffer.
  • domain assumption Segmentation and detection foundation models produce meaningful and diverse concept proposals
    Section 3.1 builds the entire concept bank from these proposals; if they miss discriminative parts, the CBM cannot recover.
  • domain assumption K-means clustering of segment embeddings preserves concept identity and linear separability
    Section 3.2 uses cluster centroids directly as concepts; clustering errors remove granularity needed for fine-grained classification.
  • domain assumption A linear combination of concept activations is sufficient for classification
    Standard CBM assumption from Koh et al., 2020; used in Section 3.3. If concepts are not linearly separable, accuracy drops.
  • domain assumption Grad-CAM on the vision encoder localizes concept activations in the input image
    Section 4.4.2 uses the Grid Pointing Game and assumes Grad-CAM maps concept importance to spatial regions; this is not proven for the CLIP backbone.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DCBM: Data-Efficient Visual Concept Bottleneck Models." pith.science (2026). https://pith.science/paper/6QVWZ2YB

@misc{pith2026241211576,
  author       = {Pith},
  title        = {Pith review of: DCBM: Data-Efficient Visual Concept Bottleneck Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6QVWZ2YB}},
  note         = {Machine review of arXiv:2412.11576}
}
read the original abstract

Concept Bottleneck Models (CBMs) enhance the interpretability of neural networks by basing predictions on human-understandable concepts. However, current CBMs typically rely on concept sets extracted from large language models or extensive image corpora, limiting their effectiveness in data-sparse scenarios. We propose Data-efficient CBMs (DCBMs), which reduce the need for large sample sizes during concept generation while preserving interpretability. DCBMs define concepts as image regions detected by segmentation or detection foundation models, allowing each image to generate multiple concepts across different granularities. This removes reliance on textual descriptions and large-scale pre-training, making DCBMs applicable for fine-grained classification and out-of-distribution tasks. Attribution analysis using Grad-CAM demonstrates that DCBMs deliver visual concepts that can be localized in test images. By leveraging dataset-specific concepts instead of predefined ones, DCBMs enhance adaptability to new domains.

Figures

Figures reproduced from arXiv: 2412.11576 by the authors.

Figure 1
Figure 1. DCBMs extract image regions as concepts. Using vision foundation models, we use crop image regions as concepts for CBM training. Based on few concept samples (50 imgs / class), DCBMs offer interpretability even for fine-grained classification. cepts (Koh et al., 2020), recent advances in vision-language models have enabled text-aligned CBMs, which leverage textual descriptions for concept selection and interpretabil… view at source ↗
Figure 2
Figure 2. DCBM framework. The DCBM framework generates concept proposals through foundation models (Step 1). These proposals are then clustered, each represented by its centroid (Step 2). Finally, the unique concepts are utilized to train a sparse CBM, effectively (Step 3). We leverage the image-text alignment to map the visual concept to the corresponding textual concept (Step 4). We can remove undesired concepts after Step … view at source ↗
Figure 4
Figure 4. illustrates two examples of challenges arising when using DCBM. The first image is correctly classified using concepts not contained in the image, whereas the second one is incorrectly classified even though the important concepts are contained in the image. For instance, in the case of the ambulance image from ImageNet, concepts such as med, police, kit, or injection are not explicitly visible in the input image. T… view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Clustered proposals as concepts. We show four concept clusters for ImageNet, Places365, and CUB, illustrating that clustering concept proposal embeddings produce semantically coherent clusters across datasets. The center image (orange border) represents the final conce…
Figure 6
Figure 6. Figure 6: Grid pointing game. GPG applied to an ImageNet-R instance using a visual concept from ImageNet to demonstrate the effectiveness of our approach in ood (SAM2 and ResNet-50). test image in a 2×2 grid with three randomly chosen images from the same dataset (see [PITH_FUL…
Figure 7
Figure 7. Figure 7: Backbone NMI scores for the CUB dataset. NMI scores for three CLIP model combinations: B16, L14 (blue), B16, RN50 (orange), and L14, RN50 (green), showing clustering performance across different backbones across cluster sizes (128, 256, 512, 1024, 2048). Similarly, [P…
Figure 8
Figure 8. Figure 8: PCA NMI scores for the CUB dataset. NMI scores illustrate clustering consistency with and without PCA preprocessing (100 components) for each CLIP backbone across varying cluster sizes (128, 256, 512, 1024, 2048). 19 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: DCBM ablations. Our technique DCBM was applied to ImageWoof and ImageNette (SAM2 and ViT L/14). G.1. DCBM concept comparison GDINO Mask-RCNN SAM2 Faucet Toilet Floors Closet Towel Shower Ankle Condosaver Tiles Faucets Shower Jacuzzi Sink Baths Plaster Input x Top 5 Con…
Figure 10
Figure 10. Figure 10: Diverse concept proposals. Concept sets differ in terms of granularity depending on the segmentation or detection method employed, shown on shower (Places365). The choice of the concept proposal model determines the retrieved concept set. The generic SAM2 model create…
Figure 11
Figure 11. Figure 11: Concept intervention in CUB. The first instance shows the top five concepts of a gull without concept intervention. The second one shows the same instance, but with a trained DCBM with the removal of ’stone’ concepts. The third one is a DCBM with the removal of the co…
Figure 12
Figure 12. Figure 12: Comparison of concept explanations across CBMs. This figure shows concept-based explanations for two images from the Places365 dataset, as generated by four different CBMs: CDM (Panousis et al., 2023), LF-CBM (Oikarinen et al., 2023), DNCBM (Rao et al., 2024), and our…
Figure 13
Figure 13. Figure 13: Concept explanations from DCBM and DNCBM. This figure presents a comparison of concept explanations for two images from the Places365 dataset, illustrating outputs from DNCBM (Rao et al., 2024) and our DCBM. For each DCBM example, we show the concept attribution acros…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Are Synthetic Corruptions A Reliable Proxy For Real-World Corruptions?

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Synthetic corruptions are a strong proxy for real-world corruptions when ranking semantic segmentation models on average, but individual corruption types like fog and night show weak correlation.

Reference graph

Works this paper leans on

72 extracted references · 61 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    and Jaakkola, T

    Alvarez Melis, D. and Jaakkola, T. Towards robust interpretability with self-explaining neural networks. Advances in neural information processing systems, 31, 2018

    work page 2018

  3. [3]

    Convolutional dynamic alignment networks for interpretable classifications

    Bohle, M., Fritz, M., and Schiele, B. Convolutional dynamic alignment networks for interpretable classifications. In Conference on Computer Vision and Pattern Recognition, 2021

    work page 2021

  4. [4]

    B-cos networks: Alignment is all we need for interpretability

    B \"o hle, M., Fritz, M., and Schiele, B. B-cos networks: Alignment is all we need for interpretability. In Conference on Computer Vision and Pattern Recognition, 2022

    work page 2022

  5. [5]

    End-to-end object detection with transformers

    Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers. In European Conference on Computer Vision, 2020

    work page 2020

  6. [6]

    Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers

    Chefer, H., Gur, S., and Wolf, L. Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers. In International Conference on Computer Vision, October 2021

    work page 2021

  7. [7]

    Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., and Su, J. K. This looks like that: Deep learning for interpretable image recognition. In Advances in Neural Information Processing Systems, 2019

    work page 2019

  8. [8]

    Detect what you can: Detecting and representing objects using holistic models and body parts

    Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., and Yuille, A. Detect what you can: Detecting and representing objects using holistic models and body parts. In Conference on Computer Vision and Pattern Recognition, 2014

    work page 2014

Show all 72 references
  1. [9]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Conference on Computer Vision and Pattern Recognition. Ieee, 2009

    2009

  2. [10]

    J., and Chen, C

    Donnelly, J., Barnett, A. J., and Chen, C. Deformable protopnet: An interpretable image classifier using deformable prototypes. In Conference on Computer Vision and Pattern Recognition, 2022

    2022

  3. [11]

    Craft: Concept recursive activation factorization for explainability

    Fel, T., Picard, A., Bethune, L., Boissin, T., Vigouroux, D., Colin, J., Cad \`e ne, R., and Serre, T. Craft: Concept recursive activation factorization for explainability. In Conference on Computer Vision and Pattern Recognition, 2023

    2023

  4. [12]

    Y., and Kim, B

    Ghorbani, A., Wexler, J., Zou, J. Y., and Kim, B. Towards automatic concept-based explanations. Advances in Neural Information Processing Systems, 32, 2019

    2019

  5. [13]

    and contributors

    Gildenblat, J. and contributors. Pytorch library for cam methods. https://github.com/jacobgil/pytorch-grad-cam, 2021

    2021

  6. [14]

    Partimagenet: A large, high-quality dataset of parts

    He, J., Yang, S., Yang, S., Kortylewski, A., Yuan, X., Chen, J.-N., Liu, S., Yang, C., Yu, Q., and Yuille, A. Partimagenet: A large, high-quality dataset of parts. In European Conference on Computer Vision, 2022

    2022

  7. [15]

    Mask r-cnn

    He, K., Gkioxari, G., Doll \'a r, P., and Girshick, R. Mask r-cnn. In International Conference on Computer Vision, 2017

    2017

  8. [16]

    Concept correlation and its effects on concept-based models

    Heidemann, L., Monnet, M., and Roscher, K. Concept correlation and its effects on concept-based models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp.\ 4780--4788, 2023

    2023

  9. [17]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In International Conference on Computer Vision, 2021

    2021

  10. [18]

    Imagenette: A smaller subset of 10 easily classified classes from imagenet

    Howard, J. Imagenette: A smaller subset of 10 easily classified classes from imagenet. https://github.com/fastai/imagenette, March 2019 a

    2019

  11. [19]

    Imagewoof: a subset of 10 classes from imagenet that aren't so easy to classify

    Howard, J. Imagewoof: a subset of 10 classes from imagenet that aren't so easy to classify. https://github.com/fastai/imagenette\#imagewoof, March 2019 b

    2019

  12. [20]

    R., Ewart, A., and Sharkey, L

    Huben, R., Cunningham, H., Smith, L. R., Ewart, A., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=F76bwRSLeK

    2024

  13. [21]

    J., and Adelson, E

    Isola, P., Lim, J. J., and Adelson, E. H. Discovering states and transformations in image collections. In Conference on Computer Vision and Pattern Recognition, 2015

    2015

  14. [22]

    google-10000-english, 2012

    Kaufman, J. google-10000-english, 2012. URL https://github.com/first20hours/google-10000-english. Accessed: 2024-11-12

    2012

  15. [23]

    C., Lo, W.-Y., et al

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. Segment anything. In International Conference on Computer Vision, 2023

    2023

  16. [24]

    Beyond pixels: Enhancing LIME with hierarchical features and segmentation foundation models

    Knab, P., Marton, S., and Bartelt, C. Beyond pixels: Enhancing LIME with hierarchical features and segmentation foundation models. In ICLR 2025 Workshop on Foundation Models in the Wild, 2025. URL https://openreview.net/forum?id=JHs5p6nPbG

    2025

  17. [25]

    W., Nguyen, T., Tang, Y

    Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. Concept bottleneck models. In International Conference on Machine Learning, 2020

    2020

  18. [26]

    P., and Derpanis, K

    Kowal, M., Wildes, R. P., and Derpanis, K. G. Visual concept connectome (vcc): Open world concept discovery and their interlayer connections in deep models. In Conference on Computer Vision and Pattern Recognition, 2024

    2024

  19. [27]

    Learning multiple layers of features from tiny images

    Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009

    2009

  20. [28]

    H., Nickisch, H., and Harmeling, S

    Lampert, C. H., Nickisch, H., and Harmeling, S. Learning to detect unseen object classes by between-class attribute transfer. In Conference on Computer Vision and Pattern Recognition, 2009

    2009

  21. [29]

    Segment and recognize anything at any granularity

    Li, F., Zhang, H., Sun, P., Zou, X., Liu, S., Li, C., Yang, J., Zhang, L., and Gao, J. Segment and recognize anything at any granularity. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XLVIII, pp.\ 467–48...

    2024 doi

  22. [30]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Li, F., Zhang, R., Zhang, H., Zhang, Y., Li, B., Li, W., Ma, Z., and Li, C. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024 b

    2024 arXiv

  23. [31]

    Deal: Disentangle and localize concept-level explanations for vlms

    Li, T., Ma, M., and Peng, X. Deal: Disentangle and localize concept-level explanations for vlms. In European Conference on Computer Vision, 2025

    2025

  24. [32]

    W., Zhang, Y., Kwon, Y., Yeung, S., and Zou, J

    Liang, V. W., Zhang, Y., Kwon, Y., Yeung, S., and Zou, J. Y. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Advances in Neural Information Processing Systems, 35, 2022

    2022

  25. [33]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, 2024

    2024

  26. [34]

    Deep learning face attributes in the wild

    Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015

    2015

  27. [35]

    Explainable artificial intelligence (xai) to enhance trust management in intrusion detection systems using decision tree model

    Mahbooba, B., Timilsina, M., Sahal, R., and Serrano, M. Explainable artificial intelligence (xai) to enhance trust management in intrusion detection systems using decision tree model. Complexity, 2021 0 (1): 0 6634811, 2021

    2021

  28. [36]

    and Vondrick, C

    Menon, S. and Vondrick, C. Visual classification via description from large language models. In International Conference on Learning Representations, 2023

    2023

  29. [37]

    Neural prototype trees for interpretable fine-grained image recognition

    Nauta, M., Van Bree, R., and Seifert, C. Neural prototype trees for interpretable fine-grained image recognition. In Conference on Computer Vision and Pattern Recognition, 2021

    2021

  30. [38]

    Label-free concept bottleneck models

    Oikarinen, T., Das, S., Nguyen, L., and Weng, L. Label-free concept bottleneck models. In International Conference on Learning Representations, 2023

    2023

  31. [39]

    P., Ienco, D., and Marcos, D

    Panousis, K. P., Ienco, D., and Marcos, D. Sparse linear concept discovery models. In International Conference on Computer Vision, 2023

    2023

  32. [40]

    P., Ienco, D., and Marcos, D

    Panousis, K. P., Ienco, D., and Marcos, D. Hierarchical concept discovery models: A concept pyramid scheme, 2024. URL https://openreview.net/forum?id=gM8X6RbXkV

    2024

  33. [41]

    The sun attribute database: Beyond categories for deeper scene understanding

    Patterson, G., Xu, C., Su, H., and Hays, J. The sun attribute database: Beyond categories for deeper scene understanding. International Journal of Computer Vision, 108 0 (1-2), 2014

    2014

  34. [42]

    PEEB : Part-based image classifiers with an explainable and editable language bottleneck

    Pham, T., Chen, P., Nguyen, T., Yoon, S., Bui, T., and Nguyen, A. PEEB : Part-based image classifiers with an explainable and editable language bottleneck. In Duh, K., Gomez, H., and Bethard, S. (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp....

    2024 doi

  35. [43]

    B., Walter, S., and Keuper, M

    Prasse, K., Jung, S., Bravo, I. B., Walter, S., and Keuper, M. Towards understanding climate change perceptions: A social media dataset. In NeurIPS Workshop on Tackling Climate Change with Machine Learning. climatechange.ai, 2023

    2023

  36. [44]

    Task-driven modular networks for zero-shot compositional learning

    Purushwalkam, S., Nickel, M., Gupta, A., and Ranzato, M. Task-driven modular networks for zero-shot compositional learning. In International Conference on Computer Vision, 2019

    2019

  37. [45]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.\ 8748--8763. PmLR, 2021

    2021

  38. [46]

    E., Heo, J., and Jamnik, M

    Raman, N., Zarlenga, M. E., Heo, J., and Jamnik, M. Do concept bottleneck models obey locality? In NeurIPS Workshop: XAI in Action, 2023

    2023

  39. [47]

    Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery

    Rao, S., Mahajan, S., B \"o hle, M., and Schiele, B. Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision, 2024. First 2 authors contribute equally

    2024

  40. [48]

    V., Carion, N., Wu, C.-Y., Girshick, R., Dollar, P., and Feichtenhofer, C

    Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., R \"a dle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K. V., Carion, N., Wu, C.-Y., Girshick, R., Dollar, P., and Feichtenhofer, C. SAM 2: Segment anything in images and videos. In The Thi...

    2025

  41. [49]

    why should i trust you?

    Ribeiro, M. T., Singh, S., and Guestrin, C. “why should i trust you?” explaining the predictions of any classifier. In International Conference on Knowledge Discovery and Data Mining, 2016

    2016

  42. [50]

    M., Koepke, A., Vinyals, O., Schmid, C., and Akata, Z

    Roth, K., Kim, J. M., Koepke, A., Vinyals, O., Schmid, C., and Akata, Z. Waffling around for performance: Visual classification with random words and broad concepts. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 15746--15757, 2023

    2023

  43. [51]

    Concept bottleneck models without predefined concepts

    Schrodi, S., Schur, J., Argus, M., and Brox, T. Concept bottleneck models without predefined concepts. CoRR, 2024

    2024

  44. [52]

    R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D

    Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, 128 0 (2), October 2019

    2019

  45. [53]

    Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning

    Sharma, P., Ding, N., Goodman, S., and Soricut, R. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 2556--...

    2018

  46. [54]

    Explain any concept: Segment anything meets concept-based explanation

    Sun, A., Ma, P., Yuan, Y., and Wang, S. Explain any concept: Segment anything meets concept-based explanation. Advances in Neural Information Processing Systems, 36, 2024

    2024

  47. [55]

    Caltech-ucsd birds 200

    Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. Caltech-ucsd birds 200. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011

    2011

  48. [56]

    Learning bottleneck concepts in image classification

    Wang, B., Li, L., Nakashima, Y., and Nagahara, H. Learning bottleneck concepts in image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 10962--10971, June 2023

    2023

  49. [57]

    Interpretable image recognition by constructing transparent embedding space

    Wang, J., Liu, H., Wang, X., and Jing, L. Interpretable image recognition by constructing transparent embedding space. In International Conference on Computer Vision, 2021

    2021

  50. [58]

    H., Schiele, B., and Akata, Z

    Xian, Y., Lampert, C. H., Schiele, B., and Akata, Z. Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly. IEEE transactions on pattern analysis and machine intelligence, 41 0 (9): 0 2251--2265, 2018

    2018

  51. [59]

    Energy-based concept bottleneck models: Unifying prediction, concept intervention, and probabilistic interpretations

    Xu, X., Qin, Y., Mi, L., Wang, H., and Li, X. Energy-based concept bottleneck models: Unifying prediction, concept intervention, and probabilistic interpretations. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=I1quoTXZzc

    2024

  52. [60]

    Protopformer: Concentrating on prototypical parts in vision transformers for interpretable image recognition

    Xue, M., Huang, Q., Zhang, H., Hu, J., Song, J., Song, M., and Jin, C. Protopformer: Concentrating on prototypical parts in vision transformers for interpretable image recognition. In Larson, K. (ed.), Proceedings of the Thirty-Third International Joint Conference on Artificia...

    2024 doi

  53. [61]

    Language in a bottle: Language model guided concept bottlenecks for interpretable image classification

    Yang, Y., Panagopoulou, A., Zhou, S., Jin, D., Callison-Burch, C., and Yatskar, M. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Conference on Computer Vision and Pattern Recognition, 2023

    2023

  54. [62]

    Post-hoc concept bottleneck models

    Yuksekgonul, M., Wang, M., and Zou, J. Post-hoc concept bottleneck models. In International Conference on Learning Representations, 2023

    2023

  55. [63]

    Do vision-language pretrained models learn composable primitive concepts? Transactions on Machine Learning Research, 2023

    Yun, T., Bhalla, U., Pavlick, E., and Sun, C. Do vision-language pretrained models learn composable primitive concepts? Transactions on Machine Learning Research, 2023

    2023

  56. [64]

    Benchmarking and enhancing disentanglement in concept-residual models

    Zabounidis, R., Oguntola, I., Zhao, K., Campbell, J., Stepputtis, S., and Sycara, K. Benchmarking and enhancing disentanglement in concept-residual models. arXiv preprint arXiv:2312.00192, 2023

    2023 arXiv

  57. [65]

    A Playground for CLIP-like Models , 7 2021

    Zakka, K. A Playground for CLIP-like Models , 7 2021. URL https://github.com/kevinzakka/clip_playground

    2021

  58. [66]

    A., Lin, Z., Brandt, J., Shen, X., and Sclaroff, S

    Zhang, J., Bargal, S. A., Lin, Z., Brandt, J., Shen, X., and Sclaroff, S. Top-down neural attention by excitation backprop. International Journal of Computer Vision, 126 0 (10): 0 1084--1102, 2018

    2018

  59. [67]

    A., and Rubinstein, B

    Zhang, R., Madumal, P., Miller, T., Ehinger, K. A., and Rubinstein, B. I. Invertible concept-based explanations for cnn models with non-negative concept activation vectors. In Conference on Artificial Intelligence, volume 35,13, 2021

    2021

  60. [68]

    The decoupling concept bottleneck model

    Zhang, R., Du, X., Yan, J., and Zhang, S. The decoupling concept bottleneck model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 a

    2024

  61. [69]

    Recognize anything: A strong image tagging model

    Zhang, Y., Huang, X., Ma, J., Li, Z., Luo, Z., Xie, Y., Qin, Y., Luo, T., Li, Y., Liu, S., et al. Recognize anything: A strong image tagging model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 1724--1732, 2024 b

    2024

  62. [70]

    Places: A 10 million image database for scene recognition

    Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. Places: A 10 million image database for scene recognition. Transactions on Pattern Analysis and Machine Intelligence, 40 0 (6), 2017

    2017

  63. [71]

    Where you see is what you know: A visual-semantic conceptual explainer

    Zhu, L., Kong, X., Li, R., and Guo, G. Where you see is what you know: A visual-semantic conceptual explainer. In International Conference on Multimedia in Asia, 2024

    2024

  64. [72]

    J., Wang, Z., Mallen, A., Basart, S., Koyejo, S., Song, D., Fredrikson, M., Kolter, J

    Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., Goel, S., Li, N., Byun, M. J., Wang, Z., Mallen, A., Basart, S., Koyejo, S., Song, D., Fredrikson, M., Kolter, J. Z., and Hendrycks, D. Representation engineering: A ...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.