Pith. sign in

REVIEW 3 major objections 6 minor 44 references

Interpretable Failure Detection with Human-Level Concepts

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Asking about visual attributes—not just class names—reveals overconfident mistakes, cutting false-positive failure-detection rates by 3.7% on ImageNet and 9% on EuroSAT.

desk verdict The ordinal-ranking idea is genuinely new, but the headline FPR claim is not established because concept sets are selected on evaluation data and ORCA changes the prediction rule. read the letter →

arxiv 2502.05275 v2 pith:CSHNPZRF submitted 2025-02-07 cs.CV

Kien X. Nguyen , Tang Li , Xi Peng This is my paper · ORCID
classification cs.CV
keywords failuredetectionmisclassificationvision-languagemodelsCLIPconceptbottleneckinterpretabilityconfidencescoringordinalranking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Failure detectors usually judge confidence from a model's category-level outputs, the class logits, which stay overconfident on misclassified images. This paper argues that a finer-grained signal, the activation of human-readable visual concepts, separates correct from incorrect predictions more reliably. Its ORCA method sorts the CLIP similarity scores of per-category attributes, takes the top K, and treats the mix of categories among those top activations as the confidence signal. On ImageNet the approach lowers the false positive rate at 95% true-positive detection by 3.7%, and on EuroSAT by about 9%, with no training. The same concept activations provide an explanation of what visual cues led the model astray.

What carries the argument

ORCA (Ordinal Ranking of Concept Activation) is the central mechanism: a training-free confidence function that replaces category-level logits with the sorted list of concept-level CLIP similarity scores. Each category is described by K human-level concepts (e.g., 'humps on back' for camel), the image's similarity to every concept is computed by CLIP, and the top-K concepts among all C×K scores form an ordered set. The confidence score is then the fraction (ORCA-B) or the logarithmically weighted sum (ORCA-R) of the predicted category's concepts within that top-K set. The ordering is what makes the method work: a concentrated top-K is a sign of confident recognition, while a scattered top-K signals confusion between categories.

What would settle it

Use only the training split of ImageNet or EuroSAT to pick the per-category concepts (highest average similarity to training images), then measure ORCA's FPR@95TPR on the validation split; if the margin over maximum softmax probability collapses or reverses, the reported 3.7% and 9% improvements are largely artifacts of concept-selection leakage.

Watch

Extended reading notes

Core claim

The paper's central claim is that the ordinal ranking of concept activations is a better confidence score for zero-shot vision-language classifiers than the maximum softmax of class logits. ORCA scores an image against K text concepts per category via CLIP, sorts all C×K similarity scores, and examines the top K. If the predicted category's concepts occupy most of the top K, the prediction is likely correct; if concepts from many categories are intermixed, it is likely wrong. ORCA-B counts the predicted category's concepts in the top K, while ORCA-R weights each rank logarithmically before summing. This ranking-based score reduces FPR@95TPR across CIFAR, ImageNet, EuroSAT, and RESISC45 in zero-shot settings, and the top-K concepts themselves reveal spurious correlations and cross-category resemblances behind failures.

Load-bearing premise

The reported FPR reductions depend on the concept set being selected from the same benchmark images whose failures are then measured; if that selection leaks label information, the gains could shrink once concepts are chosen from a separate training split.

Editorial extensions

If this is right

  • Zero-shot vision-language classifiers can get a reliable failure detector for free, with no fine-tuning or access to the model's internal logits.
  • The confidence score is inherently interpretable: the concepts that intrude into the top-K say which visual features are pulling the model toward a wrong category.
  • The method transfers to domains outside CLIP's training distribution, such as satellite imagery, where logit-based baselines degrade sharply.
  • Adding more concepts per category monotonically improves AUROC and FPR@95TPR on CIFAR-100, so richer attribute vocabularies strengthen the confidence signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because concept selection on large datasets uses the same images that later measure performance, part of the reported gain may come from label leakage; a held-out concept-selection protocol would show how large that part is.
  • ORCA's ranking signal could be combined with classical uncertainty estimates (entropy, logit margin) to build a two-factor confidence score, an extension the paper does not test.
  • The approach should transfer to other vision-language alignments beyond CLIP, provided the text encoder can be prompted with attribute phrases; the rank distribution's shape may differ, so its two parameters (concept set and K) would need recalibration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ORCA, a training-free failure-detection method for vision-language classifiers. The idea is to represent each class by a set of human-level concepts, compute concept-activation similarities via CLIP, and derive both the prediction and the confidence from the ordinal ranking of the top-K concept activations. Two variants are introduced: ORCA-B (counting) and ORCA-R (rank-weighted). The authors report reduced FPR@95TPR on ImageNet, CIFAR-100, EuroSAT, and RESISC45 relative to MSP, ODIN, and DOCTOR baselines, and illustrate failure interpretation with concept-level evidence.

Significance. If validated under a sound evaluation protocol, the concept-ordinal-ranking idea is a useful and interpretable addition to failure detection for VLMs. The method is simple, the code is promised, and the interpretability examples are compelling. However, the current experimental protocol for large datasets selects concept sets using the same images on which the metrics are computed, so the headline ImageNet/CIFAR-100 improvements are not yet attributable to the ordinal-ranking mechanism. The EuroSAT result, based on manually selected concepts, is less affected by this issue and gives the idea more credibility, but the central claim requires a corrected evaluation before it can be accepted.

major comments (3)
  1. [Experiment, Implementation Details] For datasets with a larger number of categories (CIFAR-100, ImageNet, and likely RESISC45), the concept set A is selected by choosing concepts with the highest average similarity to images within each category, using the same benchmark images on which AUROC and FPR@95TPR are later computed. Since the confidence score in Eqs. (7)-(9) is computed from the membership of the predicted class's concepts in A_top-K, this selection injects label information from the evaluation set into the confidence score. The reported ImageNet reduction of 3.72% in FPR95 (Table 1, ViT-B/32, ORCA-B vs. Zero-shot+MSP) may therefore be an artifact of transduction rather than a property of ordinal ranking. The paper must either select concepts on a held-out split, use random-concept controls, or perform cross-validation to establish the claim.
  2. [Experiment, Ablation Studies] The claim that ORCA is 'training-free' (Baselines section) is misleading for the large-dataset protocol, because the concept-selection step is a data-dependent optimization over the evaluation set. The ablation on the number of concepts (Fig. 3) is performed on CIFAR-100 using this same selection procedure; the observed monotonic improvement with K could reflect increasing overfitting to the evaluation set rather than a genuine property of the ranking method. A control with a fixed concept set or a proper train/validation split is needed to interpret this trend.
  3. [Abstract and Table 1/2] The abstract claims 'our method significantly reduce the false positive rate ... specifically by 3.7% on ImageNet and 9% on EuroSAT.' These numbers correspond to different variants (ORCA-B on ImageNet, ORCA-R on EuroSAT) and the method does not consistently reduce FPR across all settings: on CIFAR-10 with ResNet-101, ORCA-B increases FPR95 from 62.98 to 66.09 relative to Zero-shot+MSP (Table 1). The claim of consistency should be qualified by variant, backbone, and dataset, and the abstract should identify which variant produces each headline number.
minor comments (6)
  1. [Abstract] The phrase 'our method significantly reduce' should be 'our method significantly reduces'.
  2. [Background] In the failure-detection definition, the symbol 'Y' appears to be a typo; it should likely be 'y' or 'c' for the label variable.
  3. [Table 2 caption] The caption spells 'RESICS45'; the correct spelling is 'RESISC45'.
  4. [Figure 3 caption] The caption reads 'we can an increase'; it should be 'we can see an increase'.
  5. [References] The reference to 'Menon and V ondrick' contains a stray space; this appears in multiple places.
  6. [Methods, Eq. (9)] The relationship between the prediction f(x) from Eq. (8) and the confidence g(f,x) from Eq. (9) should be stated explicitly: because g takes the maximum over classes, it always equals the score of the predicted class, but this is not immediately obvious and should be spelled out.

Circularity Check

1 steps flagged · score 4.0 of 10

Data-dependent concept selection on evaluation images, not ordinal ranking alone, may drive reported ImageNet/CIFAR-100 FPR gains; EuroSAT/RESISC45 results remain independent.

  1. fitted input called prediction [Experiments > Implementation Details, with Eqs. (6)-(9)]
    "For datasets with a larger number of categories, we use the concept collection provided by Yang et al. (Yang et al. 2022). This collection contains up to 500 concept candidates per category; we then select the top concepts that yield the highest average similarity score with the images within each category to form A. We include the number of concepts used for each dataset in Table 1 and 2."

    The paper reports FPR@95TPR and AUROC on the same benchmark images (e.g., ImageNet, CIFAR-100) and defines ORCA's confidence g(f,x) in Eqs. (7)/(9) from the top-K concept set Atop-K, which is derived from the concept collection A. The quoted protocol selects A by ranking concept candidates according to average similarity with 'the images within each category'—that is, using label-dependent statistics from the evaluation images themselves. No train/validation split for this selection is specified. Consequently, the reported FPR reduction (e.g., 3.7% on ImageNet) is not a clean out-of-sample prediction; it is a transductive fit in which the evaluation labels influence the very concept set used to compute the confidence score.

full rationale

The core ORCA formulation—sorting concept activation scores and deriving confidence from the rank-weighted membership of the predicted class's concepts in the top-K set—is self-contained given a fixed concept collection; it is not equivalent to its inputs by construction. Nor is there load-bearing self-citation: the cited works by the same authors (Li, Ma, and Peng 2024a,b) are contextual references for concept-based explanations, not uniqueness theorems or ansatz justifications. The central concern is empirical leakage: for large-category datasets, concept set A is selected using the same benchmark images on which failure-detection metrics are computed, so the headline ImageNet/CIFAR-100 improvements may reflect transduction rather than the ordinal-ranking mechanism. However, the EuroSAT/RESISC45 results use manually selected concepts and are unaffected by this particular selection leakage, showing that the method has independent content. The appropriate score is therefore moderate: partial circularity in the large-dataset evaluation protocol, not a derivation that reduces to its inputs by definition.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method relies on data-dependent concept selection and hand-chosen hyperparameters. The only invented entities are the concept sets, which are curated vocabularies rather than theoretical postulates, so no independent physical or mathematical entity is introduced.

free parameters (3)
  • Number of concepts K per dataset = K=10 for CIFAR-10, EuroSAT, RESISC45; K=20 for CIFAR-100; K=25 for ImageNet
    Chosen by hand for each dataset and used to define the top-K ranking and the confidence score. The ablation on CIFAR-100 shows performance changes with K.
  • Concept sets A = GPT-3.5-generated and manually filtered for small datasets; top concepts by average similarity with per-category…
    The exact prompt, manual filtering, and data-dependent selection affect all concept scores; the paper does not use a single fixed public concept bank.
  • Logarithmic weighting function = wi = log(1+ri)/sum_j log(1+rj)
    The choice of logarithmic versus linear or exponential weights is determined by an ablation on EuroSAT, not by a theoretical derivation.
assumptions (3)
  • domain assumption A correct prediction implies the image's top-K concept activations are dominated by concepts of the predicted category; mixing across categories signals likely failure.
    Core intuition in the Methods section under Ordinal Ranking; if false, ORCA's confidence score does not track correctness.
  • domain assumption CLIP similarity scores are comparable across arbitrary concept descriptions, so sorting them yields a meaningful ordinal ranking.
    ORCA sorts concept logits from Eq. 3 across heterogeneous text prompts; if scales differ by phrasing, the ranking is noise.
  • domain assumption Human-level attributes provide sufficient signal to discriminate categories in CLIP's embedding space.
    The method relies on concept sets from GPT-3.5 or Yang et al.; if the concepts are not informative for CLIP, the top-K ranking degenerates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpretable Failure Detection with Human-Level Concepts." pith.science (2026). https://pith.science/paper/CSHNPZRF

@misc{pith2026250205275,
  author       = {Pith},
  title        = {Pith review of: Interpretable Failure Detection with Human-Level Concepts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CSHNPZRF}},
  note         = {Machine review of arXiv:2502.05275}
}
read the original abstract

Reliable failure detection holds paramount importance in safety-critical applications. Yet, neural networks are known to produce overconfident predictions for misclassified samples. As a result, it remains a problematic matter as existing confidence score functions rely on category-level signals, the logits, to detect failures. This research introduces an innovative strategy, leveraging human-level concepts for a dual purpose: to reliably detect when a model fails and to transparently interpret why. By integrating a nuanced array of signals for each category, our method enables a finer-grained assessment of the model's confidence. We present a simple yet highly effective approach based on the ordinal ranking of concept activation to the input image. Without bells and whistles, our method significantly reduce the false positive rate across diverse real-world image classification benchmarks, specifically by 3.7% on ImageNet and 9% on EuroSAT.

Figures

Figures reproduced from arXiv: 2502.05275 by the authors.

Figure 1
Figure 1. Comparison between standard (MSP) and our approaches. MSP relies solely on class logits to predict failures, which [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the ORCA framework. We first prompt GPT-3.5 to construct the concept collection [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Failure detection accuracy (AUROC) and false [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Failure detection capabilities of each weighting [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Failure interpretation with human-level concepts. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 25 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Arjovsky, M.; Bottou, L.; Gulrajani, I.; and Lopez-Paz, D. 2019. Invariant Risk Minimization. ArXiv, abs/1907.02893

    arXiv 2019

  4. [4]

    Bernhardt, M.; Ribeiro, F. D. S.; and Glocker, B. 2022. Failure Detection in Medical Image Classification: A Reality Check and Benchmarking Testbed. TMLR

    work page 2022

  5. [5]

    Blundell, C.; Cornebise, J.; Kavukcuoglu, K.; and Wierstra, D. 2015. Weight uncertainty in neural networks. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML'15, 1613–1622. JMLR.org

    work page 2015

  6. [6]

    B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T

    Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T. J.; Child, R.; Ramesh, A.; Ziegler, D. M.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.; Chess, B.; Clark, J.; Berner, C.; McCandlish, S.; Radfor...

    arXiv 2020

  7. [7]

    Cheng, G.; Han, J.; and Lu, X. 2017. Remote Sensing Image Scene Classification: Benchmark and State of the Art. Proceedings of the IEEE, 105(10): 1865–1883

    work page 2017

  8. [8]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. ImageNet: A large-scale hierarchical image database. 248--255

    work page 2009

Show all 44 references
  1. [9]

    Dinari, O.; and Freifeld, O. 2022. Variational- and metric-based deep latent space for out-of-distribution detection. In Cussens, J.; and Zhang, K., eds., Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, volume 180 of Proceedings of Machin...

    2022

  2. [10]

    Fisch, A.; Jaakkola, T.; and Barzilay, R. 2022. Calibrated Selective Classification. arXiv:2208.12084

    2022 arXiv

  3. [11]

    Gal, Y.; and Ghahramani, Z. 2016. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Balcan, M. F.; and Weinberger, K. Q., eds., Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Lear...

    2016

  4. [12]

    Geifman, Y.; and El-Yaniv, R. 2017. Selective Classification for Deep Neural Networks. arXiv:1705.08500

    2017 arXiv

  5. [13]

    Geng, C.; Huang, S.-J.; and Chen, S. 2021. Recent Advances in Open Set Recognition: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10): 3614–3631

    2021

  6. [14]

    Granese, F.; Romanelli, M.; Gorla, D.; Palamidessi, C.; and Piantanida, P. 2021. DOCTOR: A Simple Method for Detecting Misclassification Errors. In Neural Information Processing Systems

    2021

  7. [15]

    R.; and Borth, D

    Helber, P.; Bischke, B.; Dengel, A. R.; and Borth, D. 2017. EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12: 2217--2226

    2017

  8. [16]

    Hendrycks, D.; and Gimpel, K. 2016. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. ArXiv, abs/1610.02136

    2016 arXiv

  9. [17]

    F.; L \"u th, C

    Jaeger, P. F.; L \"u th, C. T.; Klein, L.; and Bungert, T. J. 2023. A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification. In The Eleventh International Conference on Learning Representations

    2023

  10. [18]

    V.; Sung, Y.-H.; Li, Z.; and Duerig, T

    Jia, C.; Yang, Y.; Xia, Y.; Chen, Y.-T.; Parekh, Z.; Pham, H.; Le, Q. V.; Sung, Y.-H.; Li, Z.; and Duerig, T. 2021. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision. In International Conference on Machine Learning

    2021

  11. [19]

    W.; Nguyen, T.; Tang, Y

    Koh, P. W.; Nguyen, T.; Tang, Y. S.; Mussmann, S.; Pierson, E.; Kim, B.; and Liang, P. 2020. Concept Bottleneck Models. ArXiv, abs/2007.04612

    2020 arXiv

  12. [20]

    Krizhevsky, A. 2009. Learning Multiple Layers of Features from Tiny Images

    2009

  13. [21]

    Lakshminarayanan, B.; Pritzel, A.; and Blundell, C. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, 6405–6416. Red Hook, NY, USA: Curran Ass...

    2017

  14. [22]

    Lee, K.; Lee, K.; Lee, H.; and Shin, J. 2018. A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. arXiv:1807.03888

    2018 arXiv

  15. [23]

    LeVine, W.; Pikus, B.; Raja, P.; and Gil, F. A. 2023. Enabling Calibration In The Zero-Shot Inference of Large Vision-Language Models. arXiv:2303.12748

    2023 arXiv

  16. [24]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. C. H. 2023. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. ArXiv, abs/2301.12597

    2023 arXiv

  17. [25]

    Li, J.; Li, D.; Xiong, C.; and Hoi, S. C. H. 2022. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. In International Conference on Machine Learning

    2022

  18. [26]

    Li, T.; Ma, M.; and Peng, X. 2024 a . Beyond Accuracy: Ensuring Correct Predictions With Correct Rationales. arXiv preprint arXiv:2411.00132

    2024 arXiv

  19. [27]

    Li, T.; Ma, M.; and Peng, X. 2024 b . Deal: Disentangle and localize concept-level explanations for vlms. In European Conference on Computer Vision, 383--401. Springer

    2024

  20. [28]

    Liang, S.; Li, Y.; and Srikant, R. 2018. Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. ICLR

    2018

  21. [29]

    Menon, S.; and Vondrick, C. 2023. Visual Classification via Description from Large Language Models. ICLR

    2023

  22. [30]

    A.; Zhai, X.; Houlsby, N.; Tran, D.; and Lucic, M

    Minderer, M.; Djolonga, J.; Romijnders, R.; Hubis, F. A.; Zhai, X.; Houlsby, N.; Tran, D.; and Lucic, M. 2021. Revisiting the Calibration of Modern Neural Networks. In Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems

    2021

  23. [31]

    Moon, J.; Kim, J.; Shin, Y.; and Hwang, S. 2020. Confidence-Aware Learning for Deep Neural Networks. ArXiv, abs/2007.01458

    2020 arXiv

  24. [32]

    H.; and Gal, Y

    Mukhoti, J.; Kirsch, A.; van Amersfoort, J.; Torr, P. H.; and Gal, Y. 2023. Deep Deterministic Uncertainty: A New Simple Baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 24384--24394

    2023

  25. [33]

    Mukhoti, J.; Kulharia, V.; Sanyal, A.; Golodetz, S.; Torr, P. H. S.; and Dokania, P. K. 2020. Calibrating deep neural networks using focal loss. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20. Red Hook, NY, USA: Curran Ass...

    2020

  26. [34]

    P.; Das, S.; Nguyen, L

    Oikarinen, T. P.; Das, S.; Nguyen, L. M.; and Weng, T.-W. 2023. Label-Free Concept Bottleneck Models. ArXiv, abs/2304.06129

    2023 arXiv

  27. [35]

    Peng, A.; Wu, M.; Allard, J.; Kilpatrick, L.; and Heidel, S. 2023. GPT-3.5 Turbo Fine-Tuning and API Updates. https://openai.com/blog/gpt-3-5-turbo/

    2023

  28. [36]

    Pereyra, G.; Tucker, G.; Chorowski, J.; Łukasz Kaiser; and Hinton, G. 2017. Regularizing Neural Networks by Penalizing Confident Output Distributions. arXiv:1701.06548

    2017 arXiv

  29. [37]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I. 2021. Learning Transferable Visual Models From Natural Language Supervision. In International Conference on Machine Learning

    2021

  30. [38]

    W.; Hashimoto, T

    Sagawa, S.; Koh, P. W.; Hashimoto, T. B.; and Liang, P. 2019. Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization. ArXiv, abs/1911.08731

    2019 arXiv

  31. [39]

    Singh, A.; Hu, R.; Goswami, V.; Couairon, G.; Galuba, W.; Rohrbach, M.; and Kiela, D. 2021. FLAVA: A Foundational Language And Vision Alignment Model. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15617--15629

    2021

  32. [40]

    Vaze, S.; Han, K.; Vedaldi, A.; and Zisserman, A. 2022. Open-Set Recognition: A Good Closed-Set Classifier is All You Need. In International Conference on Learning Representations

    2022

  33. [41]

    Yang, Y.; Panagopoulou, A.; Zhou, S.; Jin, D.; Callison-Burch, C.; and Yatskar, M. 2022. Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification. CVPR

    2022

  34. [42]

    Yuksekgonul, M.; Wang, M.; and Zou, J. Y. 2022. Post-hoc Concept Bottleneck Models. ArXiv, abs/2205.15480

    2022 arXiv

  35. [43]

    Zhu, F.; Cheng, Z.; Zhang, X.-Y.; and Liu, C.-L. 2023 a . OpenMix: Exploring Outlier Samples for Misclassification Detection. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12074--12083

    2023

  36. [44]

    Zhu, F.; Cheng, Z.; Zhang, X.-Y.; and Liu, C.-L. 2023 b . Rethinking Confidence Calibration for Failure Prediction. ArXiv, abs/2303.02970

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.