Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations

Florian Buettner; Thomas Decker; Volker Tresp

REVIEW 4 major objections 6 minor 57 references

Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Miscalibration under perturbations directly degrades perturbation-based explanations, and recalibrating there improves them.

desk verdict The paper makes a real and clearly-stated point—calibration under the explanation's own perturbations matters—but the empirical support needs a stated train/eval split, a proof for Theorem 3.4, and a baseline before the causal claim is solid. read the letter →

arxiv 2506.19630 v1 pith:QLECBHEI submitted 2025-06-24 cs.LG cs.AI

Thomas Decker , Volker Tresp , Florian Buettner This is my paper · ORCID

classification cs.LGcs.AI

keywords uncertaintycalibrationperturbation-basedexplanationsShapleyvaluesLIMEtemperaturescalingerrormutualinformationexplainableAI

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that uncertainty calibration is a component of explanation quality for perturbation-based methods such as Shapley values and LIME, not a separate nicety. It proves that for a classifier trained with cross-entropy, the predictive power of a feature subset under perturbation decomposes as baseline bias plus mutual information between the perturbed predictions and the label minus a calibration-error term. Because the calibration error is subtracted, miscalibrated probabilities under the exact perturbations used by an explanation directly lower the explanatory signal the attributions can carry. The paper then introduces ReCalX, a perturbation-level-dependent temperature scaling that recalibrates the model on those perturbations, and shows on ImageNet models that it cuts calibration error across perturbation strengths and makes Shapley and LIME attributions align better with human attention and object locations. If the claim is right, recalibrating under the explanation's own perturbations is a concrete way to make model explanations more trustworthy.

What carries the argument

The load-bearing identity is the decomposition of Theorem 3.2, which makes calibration error a term inside the explanation objective rather than an external quality score. The method that carries the empirical part is ReCalX, an extension of temperature scaling: it splits the perturbation level $\lambda(S) = (d-|S|)/d$ into ten bins, learns one temperature per bin by minimizing cross-entropy on randomly perturbed validation samples, and applies the bin-specific temperature during explanation generation. This preserves each prediction's class ranking while recalibrating the confidence under exactly the perturbation intensities the explanation uses.

What would settle it

On a held-out set, compute the three terms of Theorem 3.2 before and after ReCalX under the same perturbations: predictive power $v_f^\pi(S)$ with cross-entropy, the mutual-information term $I(f_S^\pi(X), Y)$, and the calibration error $\mathrm{CE}_{\mathrm{KL}}(f_S^\pi)$. If reducing the calibration error does not increase predictive power by roughly the size of the reduction, or if human-alignment and localization gains appear while mutual information is unchanged, then the decomposition is not the mechanism driving the observed improvement.

Watch

Extended reading notes

Core claim

The central discovery is Theorem 3.2: for cross-entropy loss, the global predictive power of a feature subset $S$ under perturbation $\pi$ decomposes as $$v_f^\pi(S) = D_{\mathrm{KL}}(P_Y \| f_\emptyset^\pi(X)) + I(f_S^\pi(X), Y) - \mathrm{CE}_{\mathrm{KL}}(f_S^\pi).$$ The first term is the bias introduced by the perturbation baseline, the second is the mutual information between the restricted model's output and the label, and the third is the calibration error of the restricted model. Since the calibration error enters with a minus sign, a model that gives unreliable probabilities on perturbed inputs has less predictive power available to its explanations, and explanations built from those probabilities inherit the loss. A corollary states that a model perfectly calibrated under every subset perturbation has predictive power exactly equal to the mutual-information term $I(f_S^\pi(X), Y)$, which the paper offers as the idealized target for perturbation-based explanations. A second theorem bounds the mean-squared difference between a local explanation and the one a perfectly calibrated model would give by twice the maximum per-subset calibration error plus a sampling term, motivating calibration at every perturbation level rather than only on unperturbed images.

Load-bearing premise

The empirical claims rest on treating human attention maps and object bounding boxes as measures of explanation quality; if those proxies do not track the mutual-information objective in Theorem 3.2, the experiments show that recalibration changes attributions in ways humans prefer rather than improving explanation fidelity in the theorem's sense.

Editorial extensions

If this is right

Perfect calibration under all subset perturbations makes global predictive power equal to $I(f_S^\pi(X), Y)$, giving perturbation-based explanations an explicit information-theoretic target.
Calibrating on unperturbed data alone is insufficient; reliable explanations require the model to be calibrated under the perturbation distribution that the explanation method uses, which is exactly what ReCalX's bin-wise temperatures address.
Because temperature scaling preserves the argmax class, ReCalX can change explanation values without changing the classifications being explained, separating explanation reliability from decision behavior.
In the reported experiments, ReCalX reduces the maximum calibration error under perturbations by roughly 85--96 percent across DenseNet, ViT, and SigLip, and consistently improves Shapley-value and LIME alignment with human attention and object locations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A diagnostic follows from the same identity: explanation tools could report the calibration error under their own perturbation distribution alongside each attribution, giving users a direct signal for when attributions are degraded.
The same perturbation-level recalibration idea could be tested with other perturbation families, such as Gaussian blur, inpainting, or counterfactual edits, where the shape of miscalibration may differ from zero-pixel replacement.
Because the decomposition is tied to cross-entropy as a proper scoring rule, analogous identities should hold for other proper losses, meaning the choice of training loss determines which miscalibration term explanations pay for.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the core observation is real and useful. If your classifier is miscalibrated under the perturbations a removal-based explanation uses, that miscalibration directly subtracts from the predictive-power definition behind those explanations. Theorem 3.2 is a clean restatement of the proper-scoring-rule decomposition from Bröcker, Kull & Flach, and Gruber & Buettner, and the paper says so; the genuinely new piece is pointing out that the calibration term should be evaluated under the explanation's perturbation distribution. ReCalX, bin-wise temperature scaling by perturbation strength, is a simple and sensible lightweight fix.

What the paper does well: it states the decomposition clearly, draws the corollary that a model perfectly calibrated under all perturbations has predictive power equal to mutual information, and the ReCalX algorithm is cheap, preserves rankings, and is easy to implement. The empirical direction is plausible, and the results on human alignment and localization are consistent across three architectures and two explanation methods. The citation pattern looks honest; the authors attribute the decomposition and engage with the recent proper-score literature.

Weak spots, in order of severity. Theorem 3.4 is asserted without proof, and the statement mixes a population expectation with a high-probability term while the left-hand side is deterministic given the sampled subsets; the (1−δ) claim needs a probability space and sampling assumptions that are not supplied. The empirical section never states that the 200 fitting images in Section 5.1 and the 200 evaluation images in Section 5.2 are disjoint. If they overlap, the headline calibration improvements are partly in-sample: ReCalX minimizes cross-entropy on those images' labels, so the CE reduction is an optimized quantity. The paper says the CE curves are estimated on 5000 fresh instances, which helps, but the explanation-quality tables report no error bars and no held-out split. There is also no baseline comparing ReCalX against plain temperature scaling under the same perturbations, so the added value of binning is not isolated. Human attention maps and bounding boxes are reasonable proxies, but they are proxies; the paper should be clearer that they test semantic alignment, not the mutual information objective directly.

My take: the load-bearing idea is sound, Theorem 3.2 is correct and properly attributed, and ReCalX is worth testing further. But as written, the empirical support does not yet establish the causal chain from recalibration to better explanations. At minimum, the split must be stated, Theorem 3.4 must be proved or removed, and a temperature-scaling baseline must be added. This is solid workshop material and deserves a serious referee, but it needs revision before I would rely on the empirical claims.

Recommendation: send it to peer review with major-revision expectations, and treat it as a useful pointer when discussing calibration in XAI.

Referee Report

4 major / 6 minor

Summary. The paper investigates the relationship between uncertainty calibration and perturbation-based explanations. It presents a theoretical decomposition (Theorem 3.2) showing that predictive power under perturbation decomposes into baseline bias, mutual information, and calibration error; introduces a local explanation error bound (Theorem 3.4); and proposes ReCalX, a bin-wise temperature scaling method that recalibrates a model under the specific perturbation levels used in explanation. Experiments on ImageNet with DenseNet, ViT, and SigLip show that ReCalX reduces calibration error under perturbations and improves human-alignment and localization scores for Shapley values and LIME.

Significance. The theoretical link between calibration and explanation fidelity is a useful and timely contribution. The decomposition in Theorem 3.2 correctly builds on established proper-score decompositions and gives a clear mechanistic story: miscalibration under perturbation directly subtracts from explanation-relevant predictive power. ReCalX is a simple, post-hoc, prediction-preserving method that is easy to adopt. The experimental section evaluates diverse architectures and two explanation methods, which is commendable. However, the empirical validation is not yet at the level needed to support the causal claim: the split between calibration-fitting and evaluation data is not stated, no alternative calibration baseline is tested, and no uncertainty estimates are provided. These gaps prevent the reader from attributing the observed explanation improvements to perturbation-specific recalibration as opposed to in-sample optimization, generic temperature scaling, or sampling noise.

major comments (4)

[Section 5.1 and 5.2] The paper never states whether the 200 images used to fit ReCalXTS in Section 5.1 are disjoint from the 200 images used for explanation evaluation in Section 5.2. If the sets overlap, the bin temperatures are optimized on the evaluation images' labels, so the calibration-error reduction in Figure 1 and the explanation improvements in Tables 1 and 2 partly reflect in-sample optimization rather than a transferable recalibration effect. This is load-bearing because Theorem 3.2 is invoked as the mechanism explaining the improvement. Please state the split explicitly and, if the sets overlap, re-run the explanation-quality evaluation on a held-out disjoint set.
[Section 5.2] Tables 1 and 2 compare only the uncalibrated model ('Initial') and ReCalXTS. No temperature-scaling baseline trained on unperturbed validation data is included, nor any other post-hoc calibration method. Without such a baseline, the results do not establish that the perturbation-specific binning in ReCalX is the operative ingredient; a single global temperature may produce similar or even larger improvements. Please add a standard temperature scaling baseline (and ideally at least one other calibrator) evaluated under the same perturbation protocol.
[Section 3.2, Theorem 3.4] Theorem 3.4 is stated without a proof and without specifying the probability space over which the (1−δ) claim holds. The left-hand side, (1/d)||ϕ(x)−ϕ*(x)||², is a deterministic quantity for a given x and the sampled subsets S, whereas CE_max^KL is a population expectation over the data distribution; mixing a deterministic left-hand side with a probabilistic right-hand side requires explicit conditions on the randomness of S (e.g., how many subsets are sampled, whether they are independent, and how δ relates to those samples). Since Theorem 3.4 is the stated motivation for ReCalX, please provide a complete statement with the probabilistic model and a proof.
[Section 5.2, Tables 1-2] The explanation-quality results are reported as point estimates from 200 images with no error bars, confidence intervals, or significance tests. For example, the Shapley human-alignment improvement for DenseNet is 0.164 to 0.240, but without an estimate of variability it is unclear whether this difference is reliable. Please report bootstrap confidence intervals or standard errors across multiple random subsets, and test the significance of the calibration-error reductions and explanation-metric improvements.

minor comments (6)

[Section 3.1, Theorem 3.2] The notation D_KL(P_Y ∥ f_∅^π(X)) is not well-defined because f_∅^π(X) is a random variable; it should be E_X[D_KL(P_Y ∥ f_∅^π(X))] or an equivalent explicit form.
[General] The proof of Theorem 3.2 is deferred to an appendix that is not present in the arXiv version; consider including a short derivation in the main text or, at minimum, a precise pointer to the exact theorem in the cited decomposition literature (e.g., Gruber & Buettner, 2022).
[Algorithm 1] The sampling of subsets S with λ(S) ∈ B_b is not specified; state the distribution (e.g., uniform over subsets with that perturbation level) and the resulting number of samples per bin, as this affects the temperature estimates.
[Figure 1] The text notes that calibration error is estimated using 5000 fresh instances, but the caption does not mention this; add the sample size and the exact estimator and binning parameters of Popordanoska et al. (2024) for reproducibility.
[Tables 1 and 2] It would be useful to state whether the 200 evaluation images are the same across models and explanation methods or re-sampled independently, and to report the standard deviations of the metrics.
[Section 4] The phrase 'preserving their original predictions' should be clarified: temperature scaling preserves the argmax class, but it changes the probability vector, so 'predictions' should be understood as class labels only.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 3.2 is a standard external proper-scoring decomposition, and ReCalX's main calibration results are evaluated on held-out fresh instances.

full rationale

The load-bearing derivation, Theorem 3.2, is not defined in terms of its conclusion. Predictive power v_f^pi(S) is defined via expected cross-entropy loss in Definition 3.1, and CEKL is the independently defined KL-based calibration error from Popordanoska et al.; the theorem is the standard proper-scoring-rule decomposition previously established by Brocker, Kull and Flach, and Gruber and Buettner, so it is a genuine mathematical identity rather than a restatement of an input. The ReCalX calibration improvements in Figure 1 are explicitly estimated on 5000 fresh instances, not on the 200 instances used to fit the temperatures, so the main reported effect is a genuine out-of-sample result rather than a fitted input called a prediction. The explanation-quality results in Tables 1 and 2 use external human-attention and bounding-box proxies, and ReCalX optimizes only bin-wise temperatures by cross-entropy; the improvements are therefore not forced by construction. Two support concerns are noted but are not circularity: the paper does not state that the 200 images used for explanation evaluation in Section 5.2 are disjoint from the 200 images used to fit ReCalX in Section 5.1, and Theorem 3.4 is asserted without a proof in the main text. Both are evaluation or completeness issues, not reductions of the claimed result to its own inputs.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central derivation relies on standard proper-scoring-rule math and on the common practice of perturbation with a fixed baseline. The only fitted quantities are the ten bin temperatures; the bin count and sample count are chosen by hand. No new entities are introduced. The main unstated assumptions are the reliability of the KL calibration error estimator and the use of human attention and bounding boxes as ground truth for explanation quality.

free parameters (3)

Bin-wise temperatures T_b (b=1..10) = not reported
Ten temperature parameters are fitted by minimizing cross-entropy on perturbed validation samples, one per perturbation-level bin. The fitted values are not listed in the text.
Number of bins B=10 = 10
Chosen by hand; affects the granularity of perturbation-specific calibration and is not derived from theory.
Samples per bin M=10 = 10
Chosen by hand; controls how many random subsets are sampled per validation sample for temperature fitting.

assumptions (6)

standard math Proper-score decomposition of cross-entropy into mutual information and KL calibration error
Used to prove Theorem 3.2; known from Brocker (2009), Kull & Flach (2015), Gruber & Buettner (2022).
domain assumption Complete perturbation with a fixed baseline makes f_emptyset a constant model, so the baseline term equals D_KL(P_Y || f_emptyset)
Needed for the first term of Theorem 3.2; holds for zero-value replacement used in experiments but not for all perturbation strategies.
domain assumption The KL calibration error estimator of Popordanoska et al. (2024) is consistent and asymptotically unbiased
Used in Section 5.1 to measure calibration error before and after ReCalX.
ad hoc to paper A concentration bound on local explanation error in terms of the global maximum calibration error (Theorem 3.4)
The theorem is stated in Section 3.2 with proof deferred to the appendix; the probability space and concentration inequality are not specified in the main text.
domain assumption Human attention maps and bounding boxes are valid proxies for explanation quality
Section 5.2 uses these metrics as evidence that ReCalX improves explanations.
domain assumption The validation images used to fit ReCalX are disjoint from those used for evaluation
The paper does not state the split; without it, evaluation may be optimistically biased.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations." pith.science (2026). https://pith.science/paper/QLECBHEI

@misc{pith2026250619630,
  author       = {Pith},
  title        = {Pith review of: Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QLECBHEI}},
  note         = {Machine review of arXiv:2506.19630}
}

read the original abstract

Perturbation-based explanations are widely utilized to enhance the transparency of modern machine-learning models. However, their reliability is often compromised by the unknown model behavior under the specific perturbations used. This paper investigates the relationship between uncertainty calibration - the alignment of model confidence with actual accuracy - and perturbation-based explanations. We show that models frequently produce unreliable probability estimates when subjected to explainability-specific perturbations and theoretically prove that this directly undermines explanation quality. To address this, we introduce ReCalX, a novel approach to recalibrate models for improved perturbation-based explanations while preserving their original predictions. Experiments on popular computer vision models demonstrate that our calibration strategy produces explanations that are more aligned with human perception and actual object locations.

Figures

Figures reproduced from arXiv: 2506.19630 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 38 canonical work pages

[1]

Explainable machine learning in deployment

Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, Jos \'e MF Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pp.\ 648--657, 2020

work page 2020
[2]

Reliability, sufficiency, and the decomposition of proper scores

Jochen Br \"o cker. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 135 0 (643): 0 1512--1519, 2009

work page 2009
[3]

Overinterpretation reveals image classification model pathologies

Brandon Carter, Siddhartha Jain, Jonas W Mueller, and David Gifford. Overinterpretation reveals image classification model pathologies. Advances in Neural Information Processing Systems, 34: 0 15395--15407, 2021

work page 2021
[4]

Learning to explain: An information-theoretic perspective on model interpretation

Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: An information-theoretic perspective on model interpretation. In International conference on machine learning, pp.\ 883--892. PMLR, 2018

work page 2018
[5]

Understanding global feature contributions with additive importance measures

Ian Covert, Scott M Lundberg, and Su-In Lee. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems, 33: 0 17212--17223, 2020

work page 2020
[6]

Explaining by removing: A unified framework for model explanation

Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22 0 (209): 0 1--90, 2021

work page 2021
[7]

Does your model think like an engineer? explainable ai for bearing fault detection with deep learning

Thomas Decker, Michael Lebacher, and Volker Tresp. Does your model think like an engineer? explainable ai for bearing fault detection with deep learning. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2023

work page 2023
[8]

Explanatory model monitoring to understand the effects of feature shifts on performance

Thomas Decker, Alexander Koebler, Michael Lebacher, Ingo Thon, Volker Tresp, and Florian Buettner. Explanatory model monitoring to understand the effects of feature shifts on performance. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 550--561, 2024

work page 2024

Show all 57 references

[9]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Con...

2020
[10]

Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis

Thomas Fel, R \'e mi Cad \`e ne, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, and Thomas Serre. Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis. Advances in neural information processing systems, 34: 0 26005--26014, 2021

2021
[11]

Harmonizing the object recognition strategies of deep neural networks with humans

Thomas Fel, Ivan F Rodriguez Rodriguez, Drew Linsley, and Thomas Serre. Harmonizing the object recognition strategies of deep neural networks with humans. Advances in neural information processing systems, 35: 0 9432--9446, 2022

2022
[12]

Pathologies of neural models make interpretations difficult

Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber. Pathologies of neural models make interpretations difficult. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp.\ 3719--3728, 2018

2018
[13]

Shapley explainability on the data manifold

Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley explainability on the data manifold. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=OPyWRrcjVQw

2021
[14]

Strictly proper scoring rules, prediction, and estimation

Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102 0 (477): 0 359--378, 2007

2007
[15]

Better uncertainty calibration via proper scores for classification and beyond

Sebastian Gruber and Florian Buettner. Better uncertainty calibration via proper scores for classification and beyond. Advances in Neural Information Processing Systems, 35: 0 8618--8632, 2022

2022
[16]

On calibration of modern neural networks

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp.\ 1321--1330. PMLR, 2017

2017
[17]

The out-of-distribution problem in explainability and search methods for feature importance explanations

Peter Hase, Harry Xie, and Mohit Bansal. The out-of-distribution problem in explainability and search methods for feature importance explanations. Advances in neural information processing systems, 34: 0 3650--3666, 2021

2021
[18]

o m, Leander Weber, Daniel Krakowczyk, Dilyara Bareeva, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, and Marina M-C H \

Anna Hedstr \"o m, Leander Weber, Daniel Krakowczyk, Dilyara Bareeva, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, and Marina M-C H \"o hne. Quantus: An explainable ai toolkit for responsible evaluation of neural network explanations and beyond. Journal of Machine Lear...

2023
[19]

Densely connected convolutional networks

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 4700--4708, 2017

2017
[20]

Openclip, July 2021

Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. URL https://doi.org/10.5281/zenodo.5143773

2021 doi
[21]

Perturbation-based methods for explaining deep neural networks: A survey

Maksims Ivanovs, Roberts Kadikis, and Kaspars Ozols. Perturbation-based methods for explaining deep neural networks: A survey. Pattern Recognition Letters, 150: 0 228--234, 2021

2021
[22]

Missingness bias in model debugging

Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, and Aleksander Madry. Missingness bias in model debugging. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=Te5ytkqsnl

2022
[23]

Trustworthy artificial intelligence: a review

Davinder Kaur, Suleyman Uslu, Kaley J Rittichier, and Arjan Durresi. Trustworthy artificial intelligence: a review. ACM computing surveys (CSUR), 55 0 (2): 0 1--38, 2022

2022
[24]

Captum: A unified and generic model interpretability library for pytorch

Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, et al. Captum: A unified and generic model interpretability library for pytorch. arXiv preprint arXiv:2009.07896, 2020

2009 arXiv
[25]

Novel decompositions of proper scoring rules for classification: Score adjustment as precursor to calibration

Meelis Kull and Peter Flach. Novel decompositions of proper scoring rules for classification: Score adjustment as precursor to calibration. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Pro...

2015
[26]

On the robustness of removal-based feature attributions

Chris Lin, Ian Covert, and Su-In Lee. On the robustness of removal-based feature attributions. Advances in Neural Information Processing Systems, 36, 2024

2024
[27]

Learning what and where to attend

Drew Linsley, Dan Shiebler, Sven Eberhardt, and Thomas Serre. Learning what and where to attend. International Conference on Learning Representations (ICLR), 2019

2019
[28]

On the limited memory bfgs method for large scale optimization

Dong C Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45 0 (1): 0 503--528, 1989

1989
[29]

o fstr \

Helena L \"o fstr \"o m, Tuwe L \"o fstr \"o m, Ulf Johansson, and Cecilia S \"o nstr \"o d. Calibrated explanations: With uncertainty information and counterfactuals. Expert Systems with Applications, 246: 0 123154, 2024

2024
[30]

A unified approach to interpreting model predictions

Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 2017

2017
[31]

From local explanations to global understanding with explainable ai for trees

Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable ai for trees. Nature machine intelligence, 2 0 (1): 0 56--67, 2020

2020
[32]

Local calibration: metrics and recalibration

Rachel Luo, Aadyot Bhatnagar, Yu Bai, Shengjia Zhao, Huan Wang, Caiming Xiong, Silvio Savarese, Stefano Ermon, Edward Schmerling, and Marco Pavone. Local calibration: metrics and recalibration. In Uncertainty in Artificial Intelligence, pp.\ 1286--1295. PMLR, 2022

2022
[33]

Revisiting the calibration of modern neural networks

Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems, 34: 0 15682--15694, 2021

2021
[34]

Obtaining well calibrated probabilities using bayesian binning

Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015

2015
[35]

Predicting good probabilities with supervised learning

Alexandru Niculescu-Mizil and Rich Caruana. Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning, pp.\ 625--632, 2005

2005
[36]

Making sense of dependence: Efficient black-box explanations using dependence measure

Paul Novello, Thomas Fel, and David Vigouroux. Making sense of dependence: Efficient black-box explanations using dependence measure. Advances in Neural Information Processing Systems, 35: 0 4344--4357, 2022

2022
[37]

Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processi...

2019
[38]

Rise: Randomized input sampling for explanation of black-box models

V Petsiuk. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421, 2018

2018 arXiv
[39]

Consistent and asymptotically unbiased estimation of proper calibration errors

Teodora Popordanoska, Sebastian Gregor Gruber, Aleksei Tiulpin, Florian Buettner, and Matthew B Blaschko. Consistent and asymptotically unbiased estimation of proper calibration errors. In International Conference on Artificial Intelligence and Statistics, pp.\ 3466--3474. PMLR, 2024

2024
[40]

why should i trust you?

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp.\ 1135--1144, 2016

2016
[41]

Perturbation-based explanations of prediction models

Marko Robnik- S ikonja and Marko Bohanec. Perturbation-based explanations of prediction models. Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent, pp.\ 159--175, 2018

2018
[42]

Calibrate to interpret

Gregory Scafarto, Nicolas Posocco, and Antoine Bonnefoy. Calibrate to interpret. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.\ 340--355. Springer, 2022

2022
[43]

Restricting the flow: Information bottlenecks for attribution

Karl Schulz, Leon Sixt, Federico Tombari, and Tim Landgraf. Restricting the flow: Information bottlenecks for attribution. arXiv preprint arXiv:2001.00396, 2020

2001 arXiv
[44]

Classifier calibration: a survey on how to assess and improve predicted class probabilities

Telmo Silva Filho, Hao Song, Miquel Perello-Nieto, Raul Santos-Rodriguez, Meelis Kull, and Peter Flach. Classifier calibration: a survey on how to assess and improve predicted class probabilities. Machine Learning, 112 0 (9): 0 3211--3260, 2023

2023
[45]

Visualizing the impact of feature attribution baselines

Pascal Sturmfels, Scott Lundberg, and Su-In Lee. Visualizing the impact of feature attribution baselines. Distill, 5 0 (1): 0 e22, 2020

2020
[46]

The many shapley values for model explanation

Mukund Sundararajan and Amir Najmi. The many shapley values for model explanation. In International conference on machine learning, pp.\ 9269--9278. PMLR, 2020

2020
[47]

Post-hoc uncertainty calibration for domain drift scenarios

Christian Tomani, Sebastian Gruber, Muhammed Ebrar Erdem, Daniel Cremers, and Florian Buettner. Post-hoc uncertainty calibration for domain drift scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 10124--10132, 2021

2021
[48]

Calibration in deep learning: A survey of the state-of-the-art

Cheng Wang. Calibration in deep learning: A survey of the state-of-the-art. arXiv preprint arXiv:2308.01222, 2023

2023 arXiv
[49]

Pytorch image models

Ross Wightman. Pytorch image models. https://github.com/rwightman/pytorch-image-models, 2019

2019
[50]

Robust calibration with multi-domain temperature scaling

Yaodong Yu, Stephen Bates, Yi Ma, and Michael Jordan. Robust calibration with multi-domain temperature scaling. Advances in Neural Information Processing Systems, 35: 0 27510--27523, 2022

2022
[51]

Visualizing and understanding convolutional networks

Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pp.\ 818--833. Springer, 2014

2014
[52]

Sigmoid loss for language image pre-training

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 11975--11986, 2023

2023
[53]

Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning

Jize Zhang, Bhavya Kailkhura, and T Yong-Jin Han. Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning. In International conference on machine learning, pp.\ 11117--11128. PMLR, 2020

2020
[54]

Individual calibration with randomized forecasting

Shengjia Zhao, Tengyu Ma, and Stefano Ermon. Individual calibration with randomized forecasting. In International Conference on Machine Learning, pp.\ 11387--11397. PMLR, 2020

2020
[55]

@esa (Ref

\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
[56]

\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
[57]

P;!־ ֪PTRޖP ՖQJ TIK ĒD Id_52d uz<s<ys B

@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Explainable machine learning in deployment

Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, Jos \'e MF Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pp.\ 648--657, 2020

work page 2020

[2] [2]

Reliability, sufficiency, and the decomposition of proper scores

Jochen Br \"o cker. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 135 0 (643): 0 1512--1519, 2009

work page 2009

[3] [3]

Overinterpretation reveals image classification model pathologies

Brandon Carter, Siddhartha Jain, Jonas W Mueller, and David Gifford. Overinterpretation reveals image classification model pathologies. Advances in Neural Information Processing Systems, 34: 0 15395--15407, 2021

work page 2021

[4] [4]

Learning to explain: An information-theoretic perspective on model interpretation

Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: An information-theoretic perspective on model interpretation. In International conference on machine learning, pp.\ 883--892. PMLR, 2018

work page 2018

[5] [5]

Understanding global feature contributions with additive importance measures

Ian Covert, Scott M Lundberg, and Su-In Lee. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems, 33: 0 17212--17223, 2020

work page 2020

[6] [6]

Explaining by removing: A unified framework for model explanation

Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22 0 (209): 0 1--90, 2021

work page 2021

[7] [7]

Does your model think like an engineer? explainable ai for bearing fault detection with deep learning

Thomas Decker, Michael Lebacher, and Volker Tresp. Does your model think like an engineer? explainable ai for bearing fault detection with deep learning. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2023

work page 2023

[8] [8]

Explanatory model monitoring to understand the effects of feature shifts on performance

Thomas Decker, Alexander Koebler, Michael Lebacher, Ingo Thon, Volker Tresp, and Florian Buettner. Explanatory model monitoring to understand the effects of feature shifts on performance. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 550--561, 2024

work page 2024