pith. sign in

arxiv: 2606.12646 · v1 · pith:G7EOAE6Unew · submitted 2026-06-10 · 📊 stat.ML · cs.IT· cs.LG· math.IT

Epistemic Uncertainty Is Not the Reducible Kind

Robin Young This is my paper

Pith reviewed 2026-06-27 07:48 UTC · model grok-4.3

classification 📊 stat.ML cs.ITcs.LGmath.IT
keywords epistemic uncertaintymutual informationreducible uncertaintyaleatoric uncertaintyensemble disagreementpredictive uncertaintyacquisition function
0
0 comments X

The pith

The mutual-information measure of epistemic uncertainty contradicts the definition that more data removes it.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proves that the standard definition of epistemic uncertainty as the reducible part of predictive uncertainty and the standard mutual-information measure of it are extensionally inconsistent. An explicit construction is given in which the measure assigns every bit of uncertainty to the epistemic class, yet no amount of additional training data reduces that uncertainty. Reducibility turns out to be a property of the pair consisting of the uncertainty and the class of data acquisition, which splits epistemic uncertainty into a sample-reducible part and a mechanism-reducible part. An exact identity for the value of an observation shows that in-distribution data never reduces mechanism-irreducible uncertainty and generically increases it. Ensemble disagreement, the quantity actually used in practice, tracks the training procedure rather than the epistemic term and collapses to zero under consistent training.

Core claim

We prove the definition and the measure are extensionally inconsistent. On an explicit construction, the measure assigns all uncertainty to the epistemic class, yet no quantity of training data reduces it. Reducibility is instead a property of the pair (uncertainty, acquisition class), and the dichotomy resolves into three parts: aleatoric, sample-reducible epistemic, and mechanism-reducible epistemic uncertainty. An exact identity for the value of an observation shows that in-distribution data never reduces mechanism-irreducible uncertainty and generically increases it. Ensemble disagreement, the deployed epistemic estimate, tracks the training procedure rather than the epistemic term.

What carries the argument

The explicit construction in which the mutual-information measure labels all uncertainty epistemic while it remains irreducible by any amount of additional training data.

If this is right

  • Reducibility depends on the pair of uncertainty and the class of data acquisition rather than on uncertainty alone.
  • In-distribution data never reduces mechanism-irreducible uncertainty and generically increases it.
  • Ensemble disagreement collapses to zero beneath a positive truth under consistent training and equals hyperparameter-scaled initialization noise under interpolation.
  • The three-way split replaces the two-way aleatoric-epistemic taxonomy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Practical uncertainty estimates in deployed models may be driven more by training choices than by any intrinsic epistemic quantity.
  • Acquisition functions based on mutual information may systematically over- or under-estimate reducible uncertainty depending on the data source.
  • Distinguishing sample-reducible from mechanism-reducible uncertainty could change how active-learning loops are designed.

Load-bearing premise

There exists an explicit construction where the mutual-information measure assigns all uncertainty to the epistemic class yet no quantity of training data reduces it.

What would settle it

A calculation or experiment on the explicit construction showing that additional training data does reduce the uncertainty that the mutual-information measure labels epistemic.

Figures

Figures reproduced from arXiv: 2606.12646 by Robin Young.

Figure 2
Figure 2. Figure 2: The statistic on the positive control is identi [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 1
Figure 1. Figure 1: The epistemic term strictly increases with in-distribution sample size under exact unidentifiability [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: EU(n) profiles for the two controls at x⋆ = 8. The unidentifiable amplitude prior is flat (strictly increasing below numerical visibility) and the test fails to reject in 10/10 runs. The identifiable output-scale prior decays and the test rejects in 10/10 runs at the proven Hoeffding threshold [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ensemble predictive variance at an off￾support query equals α 2∥PN x⋆∥ 2 (solid) and crosses the true epistemic variance (dashed) only where the initialization scale happens to equal the prior scale. Inflation. In the overparametrized linear regime (d = 60, n = 20), gradient-descent ensembles of 256 mem￾bers match the closed form α 2∥PN x⋆∥ 2 across two and a half decades of initialization scale with media… view at source ↗
read the original abstract

The standard taxonomy of predictive uncertainty defines epistemic uncertainty as the part removable by collecting more data, while the standard measure identifies it with a mutual-information term. We prove the definition and the measure are extensionally inconsistent. On an explicit construction, the measure assigns all uncertainty to the epistemic class, yet no quantity of training data reduces it. Reducibility is instead a property of the pair (uncertainty, acquisition class), and the dichotomy resolves into three parts: aleatoric, sample-reducible epistemic, and mechanism-reducible epistemic uncertainty. An exact identity for the value of an observation shows that in-distribution data never reduces mechanism-irreducible uncertainty and generically increases it. Ensemble disagreement, the deployed epistemic estimate, tracks the training procedure rather than the epistemic term. It collapses to zero beneath a positive truth under consistent training, and equals hyperparameter-scaled initialization noise under interpolation. A finite-sample falsification test and seed-swept experiments confirm the theory.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims that the standard definition of epistemic uncertainty (as the component of predictive uncertainty reducible by additional data) is extensionally inconsistent with the standard mutual-information (MI) measure of epistemic uncertainty. It demonstrates this via an explicit construction in which the MI measure assigns all uncertainty to the epistemic class, yet the uncertainty remains irreducible by any amount of training data. Reducibility is reframed as a property of the pair (uncertainty, acquisition class), yielding a three-way taxonomy: aleatoric, sample-reducible epistemic, and mechanism-reducible epistemic uncertainty. The paper derives an exact identity for the value of an observation (showing in-distribution data never reduces mechanism-irreducible uncertainty and generically increases it), argues that ensemble disagreement tracks the training procedure rather than the epistemic term (collapsing to zero under consistent training and equaling hyperparameter-scaled initialization noise under interpolation), and supports the claims with a finite-sample falsification test and seed-swept experiments.

Significance. If the explicit construction and derivations hold without circularity, the result would be significant for uncertainty quantification in machine learning. It challenges a foundational assumption underlying much work on epistemic uncertainty estimation, active learning, and Bayesian neural networks by showing that the MI-based measure does not align with the reducibility definition. The three-way taxonomy and observation-value identity could reshape how acquisition functions and uncertainty decompositions are designed. The finite-sample falsification test is a positive feature, offering a concrete, testable prediction rather than purely theoretical claims.

major comments (2)
  1. [Abstract] Abstract (paragraph beginning 'On an explicit construction'): The central inconsistency claim rests on an explicit construction in which the MI measure labels all uncertainty epistemic while remaining irreducible by additional data. The manuscript must provide the full mathematical specification of this construction (model class, data-generating process, exact MI computation, and proof that no finite or infinite training set reduces the assigned epistemic component) in the main text; without it the extensional inconsistency cannot be verified and the subsequent taxonomy and identity derivations lack a load-bearing foundation.
  2. [Derivation of observation-value identity] Section deriving the exact identity for the value of an observation: The identity is presented as following from the construction, yet the reader's circularity concern is live: if the identity reduces to quantities already fixed by the choice of mutual-information functional, it is definitional rather than an independent result. The derivation must be expanded to show the steps that establish independence from the MI definition itself.
minor comments (1)
  1. [Abstract] The abstract states that 'ensemble disagreement ... equals hyperparameter-scaled initialization noise under interpolation,' but the precise scaling relation and the interpolation regime (e.g., overparameterized regime, specific loss) should be stated explicitly with the relevant equation or proposition number.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thorough review and insightful comments. We address each major comment point by point below, providing clarifications and committing to revisions where necessary to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract (paragraph beginning 'On an explicit construction'): The central inconsistency claim rests on an explicit construction in which the MI measure labels all uncertainty epistemic while remaining irreducible by additional data. The manuscript must provide the full mathematical specification of this construction (model class, data-generating process, exact MI computation, and proof that no finite or infinite training set reduces the assigned epistemic component) in the main text; without it the extensional inconsistency cannot be verified and the subsequent taxonomy and identity derivations lack a load-bearing foundation.

    Authors: We agree that the explicit construction is central and must be fully specified in the main text for verifiability. The construction is presented in Section 3, with the model class being a deterministic neural network with parameters drawn from a prior that induces mechanism uncertainty, the data-generating process involving a fixed but unknown mechanism, the MI computed exactly as the difference between predictive entropy and expected conditional entropy, and the proof showing that the epistemic term remains constant under any additional data because the mechanism is irreducible. To fully address this, we will expand the main text to include all mathematical details currently possibly in supplementary material, ensuring the construction is self-contained. revision: yes

  2. Referee: [Derivation of observation-value identity] Section deriving the exact identity for the value of an observation: The identity is presented as following from the construction, yet the reader's circularity concern is live: if the identity reduces to quantities already fixed by the choice of mutual-information functional, it is definitional rather than an independent result. The derivation must be expanded to show the steps that establish independence from the MI definition itself.

    Authors: The concern about circularity is valid to address explicitly. The observation-value identity is derived by first defining the three-way split (aleatoric, sample-reducible epistemic, mechanism-reducible epistemic) based on reducibility properties, then showing how the value of an observation (change in uncertainty) interacts with each component. While MI is used to measure the epistemic part, the identity relies on the acquisition class (in-distribution vs out) and the mechanism distinction, which are not part of the standard MI definition. We will revise the section to include a step-by-step breakdown that isolates the new contributions from the MI functional, demonstrating independence. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper advances a proof of extensional inconsistency between the standard reducibility definition of epistemic uncertainty and the mutual-information measure, via an explicit construction that assigns all uncertainty to the epistemic class while showing it remains irreducible. The subsequent three-way split (aleatoric, sample-reducible epistemic, mechanism-reducible epistemic) and the observation-value identity are derived directly from that construction. No load-bearing step reduces by definition to its inputs, no fitted parameters are relabeled as predictions, and no self-citation chain or imported uniqueness theorem is invoked. The finite-sample falsification test and seed-swept experiments function as independent checks rather than tautological confirmations. The derivation is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The argument rests on standard information-theoretic definitions and one explicit counter-example construction; no free parameters or new physical entities are introduced in the abstract.

axioms (2)
  • domain assumption Mutual information is the standard quantitative measure of epistemic uncertainty
    Invoked when the paper equates the measure with the epistemic term
  • standard math Standard axioms of probability and conditional entropy
    Required for any mutual-information identity
invented entities (1)
  • mechanism-reducible epistemic uncertainty no independent evidence
    purpose: Distinguishes reducibility by model mechanism change from reducibility by additional samples
    Introduced to resolve the observed inconsistency between definition and measure

pith-pipeline@v0.9.1-grok · 5690 in / 1447 out tokens · 23929 ms · 2026-06-27T07:48:33.914321+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 9 canonical work pages · 2 internal anchors

  1. [1]

    Proceedings of the 34th International Conference on Neural Information Processing Systems , articleno =

    Amini, Alexander and Schwarting, Wilko and Soleimany, Ava and Rus, Daniela , title =. Proceedings of the 34th International Conference on Neural Information Processing Systems , articleno =. 2020 , isbn =

    2020

  2. [2]

    Uncertainty in

    Baan, Joris and Daheim, Nico and Ilia, Evgenia and Ulmer, Dennis and Li, Haau-Sing and Fernández, Raquel and Plank, Barbara and Sennrich, Rico and Zerva, Chrysoula and Aziz, Wilker , month = jul, year =. Uncertainty in. doi:10.48550/arXiv.2307.15703 , urldate =

    work page doi:10.48550/arxiv.2307.15703

  3. [3]

    Advances in Neural Information Processing Systems , editor=

    Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation , author=. Advances in Neural Information Processing Systems , editor=. 2022 , url=

    2022

  4. [4]

    Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , pages =

    Prediction-Oriented Bayesian Active Learning , author =. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , pages =. 2023 , editor =

    2023

  5. [5]

    Decomposition of Uncertainty in

    Depeweg, Stefan and Hernandez-Lobato, Jose-Miguel and Doshi-Velez, Finale and Udluft, Steffen , booktitle =. Decomposition of Uncertainty in. 2018 , editor =

    2018

  6. [6]

    Aleatoric or epistemic? Does it matter? Structural Safety 2009;31:105–12

    Aleatory or epistemic?. Structural Safety , author =. 2009 , pages =. doi:10.1016/j.strusafe.2008.06.020 , language =

    work page doi:10.1016/j.strusafe.2008.06.020 2009

  7. [7]

    Garg, S., Jung, C., Reingold, O., and Roth, A

    Asymptotic calibration , volume =. Biometrika , author =. 1998 , pages =. doi:10.1093/biomet/85.2.379 , language =

    work page doi:10.1093/biomet/85.2.379 1998

  8. [8]

    Yarin Gal and Riashat Islam and Zoubin Ghahramani , booktitle =. Deep. 2017 , editor =

    2017

  9. [9]

    Sources of

    Gruber, Cornelia and Schenk, Patrick Oliver and Schierholz, Malte and Kreuter, Frauke and Kauermann, Göran , month = jan, year =. Sources of. doi:10.48550/arXiv.2305.16703 , urldate =

    work page doi:10.48550/arxiv.2305.16703

  10. [10]

    On out-of-distribution detection with

    D'Angelo, Francesco and Henning, Christian , month = feb, year =. On out-of-distribution detection with. doi:10.48550/arXiv.2110.06020 , urldate =

    work page doi:10.48550/arxiv.2110.06020

  11. [11]

    Bayesian Active Learning for Classification and Preference Learning

    Houlsby, Neil and Huszár, Ferenc and Ghahramani, Zoubin and Lengyel, Máté , month = dec, year =. Bayesian. doi:10.48550/arXiv.1112.5745 , urldate =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1112.5745

  12. [12]

    Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods , journal =

    H. Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods , journal =. 2021 , doi =

    2021

  13. [13]

    Proceedings of the 31st International Conference on Neural Information Processing Systems , url =

    Kendall, Alex and Gal, Yarin , title =. Proceedings of the 31st International Conference on Neural Information Processing Systems , url =. 2017 , isbn =

    2017

  14. [14]

    Proceedings of the 33rd International Conference on Neural Information Processing Systems , articleno =

    Kirsch, Andreas and Amersfoort, Joost van and Gal, Yarin , title =. Proceedings of the 33rd International Conference on Neural Information Processing Systems , articleno =. 2019 , publisher =

    2019

  15. [15]

    Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages =

    Malinin, Andrey and Gales, Mark , title =. Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages =. 2018 , url =

    2018

  16. [16]

    2023 , isbn =

    Meinert, Nis and Gawlikowski, Jakob and Lavin, Alexander , title =. 2023 , isbn =. doi:10.1609/aaai.v37i8.26096 , booktitle =

    work page doi:10.1609/aaai.v37i8.26096 2023

  17. [17]

    ICLR Blogposts 2025 , year =

    Kirchhof, Michael and Kasneci, Gjergji and Kasneci, Enkelejda , title =. ICLR Blogposts 2025 , year =

    2025

  18. [18]

    Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages =

    Osband, Ian and Aslanides, John and Cassirer, Albin , title =. Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages =. 2018 , url =

    2018

  19. [19]

    Rügamer, David , month = may, year =. On the. doi:10.48550/arXiv.2605.25234 , urldate =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.25234

  20. [20]

    Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence (UAI 2018) , pages =

    Smith, Lewis and Gal, Yarin , title =. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence (UAI 2018) , pages =. 2018 , url =

    2018

  21. [21]

    Monthly Notices of the Royal Astronomical Society , volume =

    Walmsley, Mike and Smith, Lewis and Lintott, Chris and Gal, Yarin and Bamford, Steven and Dickinson, Hugh and Fortson, Lucy and Kruk, Sandor and Masters, Karen and Scarlata, Claudia and Simmons, Brooke and Smethurst, Rebecca and Wright, Darryl , title =. Monthly Notices of the Royal Astronomical Society , volume =. 2020 , month = jan, doi =

    2020

  22. [22]

    The Fourteenth International Conference on Learning Representations , year=

    Query-Level Uncertainty in Large Language Models , author=. The Fourteenth International Conference on Learning Representations , year=

  23. [23]

    Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence , pages =

    Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? , author =. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence , pages =. 2023 , editor =

    2023

  24. [24]

    IEEE Transactions on Information Theory , author =

    Minimum. IEEE Transactions on Information Theory , author =. 2022 , pages =. doi:10.1109/TIT.2022.3176056 , number =

    work page doi:10.1109/tit.2022.3176056 2022