REVIEW 3 major objections 2 minor 1 cited by
Informative Post-Hoc Explanations Only Exist for Simple Functions
T0 review · 3 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Post-hoc explanations are uninformative for complex functions
desk verdict A useful theoretical frame for when local explanations can't be informative, but the definition of 'informative' is what a referee should put under a microscope. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The framework defines informativeness through complexity reduction: an explanation of a decision function on an input is informative if, among all functions consistent with the explanation, the space of plausible decision functions has strictly lower learning-theoretic complexity (for example, lower VC dimension) than before the explanation was given. This definition turns the question of whether an explanation 'says something' into a precise statement about the size of a hypothesis class, and the impossibility results are proven by showing that the explained functions remain flexible enough to keep the complexity unchanged.
What would settle it
Find a concrete pair of differentiable functions that agree with a given gradient explanation at every queried point yet differ on some other input, and show that no such pair exists—that is, show that the gradient explanation forces all consistent differentiable functions to agree everywhere, which would contradict the claimed non-informativeness with respect to the class of differentiable functions.
Extended reading notes
Core claim
The central claim is a rigorous mathematical rejection of the idea that it should be possible to explain any model. Concretely, the paper proves that gradient and counterfactual explanations carry no information when the space of plausible decision functions is the set of differentiable functions, and SHAP and anchor explanations carry no information when the space is the set of decision trees. An explanation is called informative if it serves to reduce the complexity of the space of plausible decision functions; the impossibility results show that these algorithms fail to do so for the natural function classes they are meant to explain. The paper then derives conditions under which the same
Load-bearing premise
The entire argument depends on defining 'informative' as 'reducing the complexity of the space of plausible decision functions'; if that is not what people need from an explanation in practice, the impossibility results do not directly apply to their use case.
Editorial extensions
If this is right
- Local post-hoc explanations should not be treated as reliable windows into a complex model's reasoning, because for common function classes they do not reduce the set of functions that could have produced the data.
- Explanation algorithms can be made informative only by restricting the hypothesis class or by adding enough assumptions about the model, and those assumptions may be stronger than practitioners typically realize.
- Auditing and regulatory uses of explanations need to specify a plausible function class explicitly; without such a class, an explanation cannot be said to convey information.
- Gradient-based and counterfactual methods are informative for simple function classes, so their use may still be justified when the model is known to be simple, but not for arbitrary deep networks.
- The results provide a formal guide for designing explanation methods: to be informative, a method must be paired with a hypothesis class it actually narrows, not just with a local feature attribution.
Reading between the lines
- If this definition of informativeness is accepted, then a practical test for any explanation method is whether it changes the set of models a regulator would need to check; methods that only shuffle weights without shrinking that set are informationally vacuous, even if they look plausible.
- The impossibility results likely extend to other explanation families, such as saliency maps or integrated gradients, because they all take the same form: they constrain the local behavior of a function without constraining its global behavior enough to reduce class complexity.
- One testable extension would be to measure, for a given real dataset and model family, how much the VC dimension or Rademacher complexity actually drops after constraining by a specific explanation; the paper's theorems predict drops close to zero for rich function classes.
- The framework could be read as shifting the burden of proof: a practitioner who claims an explanation conveys insight must first name the hypothesis class and then show the explanation narrows it, rather than relying on the intuitive appeal of feature weights.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a learning-theory-based notion of informativeness for local post-hoc explanations: an explanation is informative if it reduces the complexity of the space of plausible decision functions. Using this definition, the authors claim impossibility results for popular algorithms on complex function classes, namely that gradient and counterfactual explanations are non-informative with respect to differentiable functions and that SHAP and anchor explanations are non-informative with respect to decision trees. They also state that conditions can be derived under which such algorithms become informative, and discuss practical implications for auditing and regulation. The provided manuscript is abstract-only, so no formal definitions, theorem statements, or proofs are available for inspection.
Significance. If the central theorem is rigorously proven, the paper would make a valuable theoretical contribution by showing a precise sense in which post-hoc explainability is impossible for complex model classes. The abstract names concrete explanation algorithms and function classes, which gives the claim falsifiable specificity. The strength of the contribution, however, rests entirely on the choice of the informativeness operationalization and on whether that operationalization captures what practitioners need from explanations. The paper's practical conclusions are only as strong as its defense of that definition.
major comments (3)
- [Abstract, definition of informativeness] The central definition—'an explanation is informative if it serves to reduce the complexity of the space of plausible decision functions'—is stipulated rather than derived. The impossibility results are relative to this definition. For infinite function classes, any finite local datum (gradient, counterfactual, SHAP value) leaves uncountably many globally consistent functions, so complexity reduction may fail almost by construction. The paper needs to justify why global complexity reduction is the right or necessary notion of practical informativeness, or to explicitly restrict its conclusions to that definition. Without such a justification, the abstract's leap to 'strong implications for auditing, regulation, and high-risk applications' is not supported.
- [Abstract, central theorem statement] The abstract asserts the main result without stating its formal assumptions. No explicit definitions are given for 'complexity,' 'plausible function space,' or the formal input of each explanation algorithm. The claims about 'differentiable functions' and 'decision trees' need precise statements: which norm/topology on differentiable functions, which tree class (depth, leaf count, splitting thresholds), and what complexity measure (VC dimension, Rademacher complexity, description length, etc.) is used. These choices determine whether the theorem is nontrivial. The omitted derivations are load-bearing for the central claim, and the current abstract-only text does not allow verification.
- [Abstract, practical conclusions] The paper argues that non-informativeness under its complexity-reduction definition has direct consequences for practical explainability. This inference is not automatic. Practitioners may care about local predictive accuracy, counterfactual truthfulness, monotonicity verification, or detection of spurious correlations—none of which require reducing global complexity of the entire function space. The abstract provides no argument that complexity reduction is necessary for these tasks. A competing operationalization could plausibly find gradient or counterfactual explanations informative in a practical sense. The paper should either weaken the practical claims or supply a separate argument connecting the mathematical definition to application requirements.
minor comments (2)
- [Abstract, notation and terminology] The abstract uses 'SHAP and anchor explanations' without specifying the exact algorithms or providing references; adding citations would help situate the claims. Also, the hyphenation of 'non-informative' and 'post-hoc' should be made consistent.
- [General] Because this review is based solely on the abstract, the manuscript should make the formal definition of informativeness and the main theorem statement visible early, ideally in the introduction, so that readers can assess the scope of the claims without reading the full proof.
Circularity Check
Central non-informativeness results are entailed by the paper's stipulated definition of 'informative' as complexity reduction.
-
self definitional
[Abstract, definition of 'informative' and non-informativeness examples]
"We call an explanation informative if it serves to reduce the complexity of the space of plausible decision functions. ... gradient explanations and counterfactual explanations are non-informative with respect to the space of differentiable functions, and SHAP and anchor explanations are not informative with respect to the space of decision trees."
The paper defines 'informative' as reducing the complexity of the space of plausible decision functions. Thus 'non-informative' is definitionally equivalent to 'not reducing that complexity'. The claimed non-informativeness of finite local explanations (gradient, counterfactual, SHAP, anchor) for complex function classes is exactly the statement that a finite local constraint does not shrink an infinite-dimensional hypothesis class. This conclusion is an immediate unfolding of the stipulated definition plus the infinite cardinality of the classes, not an independent empirical or mathematical discovery. The 'rigorous mathematical rejection' of universal explainability therefore rests on the definition being accepted as the right notion of informativeness; the abstract does not argue for tha
full rationale
The score is 6 because the central negative claims of the abstract reduce by construction to the paper's explicit definition of 'informative' as complexity reduction. For an infinite-dimensional function class, a finite-dimensional local explanation cannot change the global complexity of the plausible-function space, so non-informativeness follows immediately from the definition plus this elementary observation. The paper is transparent about the definition ('We call...'), and the framework may still be internally valid; it also derives conditions under which explanations become informative, so the paper is not entirely vacuous. However, the strongest headline claim—that popular explanation algorithms are not informative for complex functions, a 'rigorous mathematical rejection' of explain-any-model—is a definitional artifact: the result holds only under the stipulated complexity-reduction criterion, and the abstract does not independently justify that this criterion captures practical informativeness for auditing or regulation. No self-citation or citation-based circularity is evident from the abstract; the concern is purely the self-definitional loading of the core concept. Full text is unavailable, so the assessment is based on the abstract.
Assumptions & free parameters
assumptions (2)
- domain assumption Informativeness of an explanation is defined as reduction in complexity of the space of plausible decision functions.
- domain assumption The function classes considered (e.g., differentiable functions, decision trees) are the relevant 'complex model' spaces.
Cite this review
Pith. "Pith review of Informative Post-Hoc Explanations Only Exist for Simple Functions." pith.science (2026). https://pith.science/paper/4M6EEAN7
@misc{pith2026250811441,
author = {Pith},
title = {Pith review of: Informative Post-Hoc Explanations Only Exist for Simple Functions},
year = {2026},
howpublished = {\url{https://pith.science/paper/4M6EEAN7}},
note = {Machine review of arXiv:2508.11441}
}
read the original abstract
Many researchers have suggested that local post-hoc explanation algorithms can be used to gain insights into the behavior of complex machine learning models. However, theoretical guarantees about such algorithms only exist for simple decision functions, and it is unclear whether and under which assumptions similar results might exist for complex models. In this paper, we introduce a general, learning-theory-based framework for what it means for an explanation to provide information about a decision function. We call an explanation informative if it serves to reduce the complexity of the space of plausible decision functions. With this approach, we show that many popular explanation algorithms are not informative when applied to complex decision functions, providing a rigorous mathematical rejection of the idea that it should be possible to explain any model. We then derive conditions under which different explanation algorithms become informative. These are often stronger than what one might expect. For example, gradient explanations and counterfactual explanations are non-informative with respect to the space of differentiable functions, and SHAP and anchor explanations are not informative with respect to the space of decision trees. Based on these results, we discuss how explanation algorithms can be modified to become informative. While the proposed analysis of explanation algorithms is mathematical, we argue that it holds strong implications for the practical applicability of these algorithms, particularly for auditing, regulation, and high-risk applications of AI.
Forward citations
Cited by 1 Pith paper
-
Tractable Shapley Values and Interactions via Tensor Networks
TN-SHAP extracts exact Shapley values and k-way interactions of a multilinear tensor-network surrogate from O(n) probe evaluations, replacing O(2^n) coalition enumeration.
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.