{"id":"3c296711-d9e4-440f-8f47-7d958c533738","arxiv_id":"2602.04596","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A supervised predictive CLT with a volatility-based variance estimator separates aleatoric from epistemic uncertainty in Bayes-filtered transformers like TabPFN.","lead":"This paper proves a predictive central limit theorem for supervised learning under quasi-martingale conditions and uses it to separate TabPFN's predictive uncertainty into aleatoric and epistemic parts. The method is fast on tabular data and gives credible bands whose coverage is near nominal in many, but not all, test settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TabPFN's own Appendix B diagnostics violate Theorem 4.3's sufficient condition (7), so the theoretical guarantee does not formally cover the paper's central application; empirical coverage is also below nominal in several settings.","rationale":"The reader's weakest assumption and my own most load-bearing concern coincide: the application of Theorem 4.3 to TabPFN depends on quasi-martingale conditions that the paper's own diagnostics suggest are violated. Appendix B.5 is unusually honest in reporting fitted exponents β≈0.73–0.87, which are inconsistent with the β>1.5 needed for the √n-weighted condition (7). Since Theorem 4.3 is the only theoretical route from the volatility of TabPFN's predictive updates to a Gaussian posterior for the limiting predictive, a genuine violation would leave the uncertainty decomposition formally unsupported for the central application. The authors' caveat that (7) is sufficient and not necessary is legitimate, which is why I do not escalate the verdict to REJECT; the empirical coverage results provide partial alternative support. However, those coverage results are themselves mixed—simultaneous bands for Categorical and Poisson DGPs remain well below nominal at n=1000. A high-powered re-run of the diagnostic is the most direct way to settle whether the condition failure is real or a numerical artifact. If it is real, the paper should be read as a heuristic but well-tested uncertainty-decomposition method for TabPFN plus a theorem whose conditions are not verified on the flagship example; if the diagnostic reverses at larger scale, the theoretical gap narrows. This matches the reader's CONDITIONAL verdict, so no change is needed.","tokens_in":32865,"tokens_out":14774,"duration_ms":163285,"concrete_test":"Independently re-run the Appendix B.5 diagnostic at production settings (n_estimators=64, R≥500 rollouts, N_end=5000 or more, double precision), and compute both the deep-tail power-law exponent β and the partial sums T(n)=Σ_{m≤n} √m E|b_m| over the extended range. If the 95% confidence interval for β remains strictly below 1.5 and T(n) continues to grow, condition (7) is genuinely violated for TabPFN, confirming that the theorem does not justify the application. If instead the deep-tail β crosses 1.5 or T(n) stabilizes, the current diagnostic is a numerical artifact and this particular concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Theorem 4.3's predictive CLT applies to TabPFN, making V_n/n a valid posterior variance for the limiting predictive \\widetilde{F} and yielding credible bands with near-nominal coverage. This requires condition (7): Σ √n E|E[Δ_n | Z_{1:n-1}]| < ∞, plus the uniform jump bound (ii) and convergence of R_n. Appendix B.5 is the only direct evidence about these conditions for TabPFN. It fits E|b_n| ≈ C n^{-β} with β̂ = 0.87 (95% CI [0.77,0.98]) for n_estimators=8 and β̂ = 0.73 ([0.67,0.80]) for n_estimators=16. Under a clean power law, the unweighted quasi-martingale condition (5) needs β > 1, and the √n-weighted condition (7) needs β > 1.5; the fitted exponents are below even the unweighted threshold. The paper's partial-sum plots (Figures 8–9) do not flatten over the probed range, and the authors state they “do not see behaviour consistent with β > 1.5.” Thus the theorem's sufficient conditions are not established for TabPFN—the Gaussian form of the posterior and the interpretation of V_n/n as epistemic variance have no formal justification for the target model. The authors correctly note that (7) is sufficient, not necessary, but this shifts the burden onto empirical coverage. That evidence is incomplete: Table 1 reports simultaneous-band coverage of 0.53–0.87 for the Categorical DGP and 0.80 for Poisson with U_n at n=1000, which is not “near-nominal” 0.95. The central claim therefore rests on a sufficiency gap plus partial, somewhat mixed empirical validation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a Bayesian predictive inference (BPI) framework for uncertainty decomposition in TabPFN, a supervised Bayes-filtered transformer. The main theoretical contribution is a predictive central limit theorem (Theorem 4.3) for supervised settings under quasi-martingale conditions. The theorem states that, conditionally on an observed context, the limiting predictive distribution is asymptotically Gaussian with covariance determined by the volatility of predictive updates; the normalized update covariance V_n/n is then interpreted as epistemic uncertainty. The authors use this to build pointwise and simultaneous credible bands for TabPFN's limiting predictive probabilities/CDFs, and propose an entropy-based aleatoric/epistemic decomposition. The method is validated on seven synthetic DGPs and illustrated on two real datasets.","tokens_in":33192,"tokens_out":10859,"duration_ms":119700,"significance":"If the theoretical bridge to TabPFN were secure, this would be a genuinely useful contribution: it is one of the first attempts to give a principled, black-box epistemic/aleatoric decomposition for a foundation model that never explicitly represents a posterior. The paper is honest about the gap between its sufficient conditions and practical verification, and the appendix proofs are detailed. It also ships code and includes an external coverage benchmark. However, the central application-level claim is not formally supported: the paper's own diagnostics in Appendix B indicate that TabPFN violates the CLT's sufficient condition (7), and several simultaneous-band coverage rates in Table 1 are far from nominal. The theoretical CLT is valuable in its own right, but the advertised TabPFN uncertainty decomposition currently rests on an empirical heuristic rather than on Theorem 4.3.","major_comments":[{"comment":"Theorem 4.3 requires condition (7): Σ √n E|E[Δ_n|Z_{1:n-1}]| < ∞. Under a power-law model E|b_n| ≈ C n^{-β}, this requires β > 1.5. Appendix B.5 reports β̂ ≈ 0.87 (95% CI [0.77,0.98]) for n_estimators=8 and β̂ ≈ 0.73 ([0.67,0.80]) for n_estimators=16. The authors explicitly state they do not see behaviour consistent with β > 1.5. The unweighted quasi-martingale condition (5) would require β > 1, which is also not supported by the point estimates. Thus the Gaussian posterior approximation and the interpretation of V_n/n as epistemic variance are not formally justified for TabPFN. The paper's caveat that (7) is sufficient, not necessary, is correct but shifts the burden onto empirical validation. That validation is partial: coverage of f_0 is a frequentist property and can hold even when V_n/n is not a correct posterior variance. Please either develop a theorem with weaker, verifiable cond","section":"§4.3 and Appendix B.5"},{"comment":"The abstract and Section 3.2 claim that the credible bands achieve near-nominal frequentist coverage as context length grows. Table 1 does not support this uniformly. For the Categorical DGP, simultaneous V_n bands have coverage 0.53 at n=200, 0.53 at n=500, and 0.87 at n=1000; simultaneous U_n bands at n=1000 are 0.80. With R=100 replicates, the Monte Carlo standard error of a simultaneous coverage rate is about 0.05, so 0.87 and 0.80 are substantially below the nominal 0.95. The paper acknowledges deviations but the abstract's wording is too strong. Moreover, coverage of the true f_0 does not directly validate the epistemic variance: conservative intervals can overcover even if V_n/n overstates or misstates the posterior variance. A more direct check of the epistemic variance (e.g., comparing V_n/n to a known posterior variance in a simple Bayesian model) would be needed to support the","section":"§6, Table 1"},{"comment":"The entropy-based decomposition is presented as a contribution, but the moment-matched Beta/Dirichlet approximation is not derived from the predictive CLT and has no stated error bound. In the multiclass extension, the Dirichlet parameters are set by matching only the sum of variances across classes (Eq. 16), not the individual class variances, so the estimated aleatoric entropy may be inaccurate when class variances differ. Since this decomposition is one of the three advertised contributions, it needs either a theoretical justification (e.g., an error bound or a proof that the approximation is asymptotically equivalent to the CLT-based delta method) or at least a numerical sanity check against a known posterior.","section":"§7 and Appendix C"}],"minor_comments":[{"comment":"Condition (iii) assumes V(x,t) is positive definite. For atomic events, the limiting variance can be zero or singular; positive semidefinite would be the more natural assumption, with the Gaussian statement interpreted in the possibly degenerate sense.","section":"§4.2, Theorem 4.3"},{"comment":"In the proof of Theorem 4.3, the step from R_n(x,t) to n Σ_{k≥n}(M_k−M_{k−1})^2 silently drops the k=n term. This is harmless only if nΔ_nΔ_n^T → 0, which follows from convergence of R_n; please make this explicit.","section":"§H.3"},{"comment":"The row-permutation procedure is described briefly. Since V_n depends on the trajectory, the choice of random permutation is an additional stochastic ingredient. For i.i.d. data this is justified, but a formal statement about the permutation's effect on the CLT conditions would be helpful, especially for non-exchangeable real-data settings.","section":"§3.2"},{"comment":"The column headers are misaligned: 'Vn Point.', 'Un Point.', 'Vn Simul.', 'Un Simul.' should be associated with their Rate/Width columns. Also, 'n estimators' and 'n_estimators' are used inconsistently across the paper.","section":"§6, Table 1"},{"comment":"The diagnostics are run with n_estimators=8 and 16, while the rest of the paper uses n_estimators=64. The relation between ensemble size and the fitted exponents β̂ should be discussed; the observed decrease from 0.87 to 0.73 as the ensemble size increases is itself informative about the stability of the diagnostic.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid theoretical core (the predictive CLT and its proofs) but the authors' own diagnostics undermine the formal applicability to TabPFN, which is the paper's central selling point. I would ask the editor to insist on either closing that gap or substantially reframing the paper so that the TabPFN decomposition is presented as an empirical method with theoretical motivation, not as a theorem-backed guarantee. The coverage claims should also be recalibrated to match Table 1. With those changes, the manuscript could be a valuable contribution; in its current form, the load-bearing claim is not supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about this paper. First, the theoretical contribution is real: a predictive CLT for supervised settings under quasi-martingale conditions, extending BPI beyond the martingale or unsupervised cases. The proof adapts Berti et al. (2011) carefully, and the variance estimator V_n from predictive updates is simple and computationally cheap. That is worth having.\n\nSecond, the TabPFN application is shakier than the headline suggests. The paper's own Appendix B diagnostics estimate the drift decay exponent β at 0.73–0.87, while the CLT's sufficient condition (7) needs β > 1.5. The authors openly say they do not see behavior consistent with β > 1.5. So the theorem does not formally justify the Gaussian posterior for V_n/n in the target model. They are honest that (7) is sufficient, not necessary, but that shifts the burden onto empirical coverage, and Table 1 shows simultaneous bands at 0.53–0.87 for the Categorical DGP even at n=1000. Calling that 'near-nominal' is a stretch.\n\nWhat the paper does well: the exposition is clear, the proofs in Appendix H are detailed, and the authors flag their own limitations instead of burying them. The entropy decomposition via a moment-matched Beta/Dirichlet is sensible and avoids the boundary issues of a naive delta method. The code is available, and the frequentist coverage check is an external benchmark, which is more than many papers in this area do.\n\nThe soft spots are real but not fatal. The qualitative result that epistemic uncertainty shrinks with context length is partly by construction—updates decay, so V_n/n decays—but that does not make the decomposition useless. The diagnostics use a discrete covariate in a controlled setting, so they may not reflect real TabPFN use, but they are the only direct evidence about the conditions. The pointwise coverage is mostly good at n=1000, and the method is fast and practical.\n\nWho is this for? Practitioners who want a cheap epistemic/aleatoric split for TabPFN, and BPI theorists interested in quasi-martingale CLTs. It deserves a serious referee: the theory is likely sound, and the application, while not formally justified, is honest and potentially useful. A referee should focus on whether the empirical coverage is enough to carry the sufficiency gap, and the authors should soften the 'near-nominal' claim in the abstract. I would engage with it and cite the CLT.","headline":"A genuinely new supervised predictive CLT under quasi-martingale conditions, honestly applied to TabPFN, but the theorem's sufficient conditions are not met by the target model and the coverage evidence is mixed.","tokens_in":33779,"tokens_out":2423,"would_cite":true,"duration_ms":25249,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves a predictive central limit theorem under quasi-martingale conditions and uses it to decompose TabPFN's total predictive uncertainty into aleatoric and epistemic components with credible bands that meet frequentist coverage.","keywords":["uncertainty decomposition","Bayesian predictive inference","predictive central limit theorem","quasi-martingale","TabPFN","in-context learning","aleatoric uncertainty","epistemic uncertainty"],"falsifier":"Under the paper's discrete-covariate logistic DGP, run TabPFN rollouts with larger ensembles (n_estimators≥64) and fit the drift exponent β for E|b_n|; if β is consistently below 1.5 (or below 1), the quasi-martingale condition underlying Theorem 4.3 fails for TabPFN, and the decomposition's validity reduces to the empirical coverage alone.","tokens_in":32638,"feed_emoji":"📊","tokens_out":7933,"duration_ms":75344,"temperature":0.7,"pith_summary":"TabPFN, a transformer meta-learned on Bayesian-filtered data, returns total predictive uncertainty in one forward pass but never exposes a posterior, so the usual Bayesian route to separating aleatoric from epistemic uncertainty is blocked. The authors prove a predictive central limit theorem for supervised settings under quasi-martingale conditions—weaker than the martingale assumption—showing that the posterior of the limiting predictive distribution given the observed context is asymptotically Gaussian. The Gaussian variance, estimated from the volatility of in-context prediction updates, serves as a measure of epistemic uncertainty. Applied to TabPFN, this yields credible bands with near-nominal frequentist coverage and an entropy-based decomposition that behaves sensibly: epistemic uncertainty shrinks with context and peaks in sparsely observed regions, while aleatoric uncertainty dominates near decision boundaries.","feed_headline":"New theorem splits TabPFN's uncertainty into aleatoric and epistemic","feed_subtitle":"In-context update volatility becomes a variance estimate whose credible bands hit near-nominal coverage.","key_machinery":"The central object is the quasi-martingale predictive rule: a sequence of one-step-ahead predictive distributions P_k(x,A) whose expected absolute drift is summable. The load-bearing identity is the predictive CLT (Theorem 4.3), which makes the posterior of the limiting predictive distribution asymptotically Gaussian; the workhorse estimator is V_n = (1/n)Σ_{k=1}^n k² Δ_k Δ_kᵀ, the inflated outer product of prediction updates along the context. It requires only n forward passes and no simulation of future covariates. A moment-matched Beta (or Dirichlet) approximation converts the Gaussian law into an expected-entropy estimate of aleatoric uncertainty.","core_discovery":"Theorem 4.3 is the central claim: under the quasi-martingale condition (5) and the stronger √n-weighted variant (7), the posterior law of the limiting predictive distribution F̃(x,t) given Z_{1:n} is asymptotically N_m(0, V(x,t)). The covariance is consistently estimated by V_n = (1/n)Σ k² Δ_k Δ_kᵀ, computed from the one-step predictive updates along the observed context. Since TabPFN is, in an idealized limit, a Bayesian in-context learner, this variance serves as epistemic uncertainty. The authors report near-nominal frequentist coverage across seven DGPs and an entropy-based decomposition (via a moment-matched Beta/Dirichlet approximation) that places aleatoric uncertainty near class boun","pith_inferences":["The size of V_n/n could serve as an in-context diagnostic of how far a given dataset is from the 'idealized' Bayesian limit: large persistent epistemic variance flags contexts where the predictive rule is still fluctuating and where extra data would shift predictions.","Because V_n depends on row order while TabPFN's prediction itself is permutation-invariant, the estimator captures a trajectory-dependent quantity; comparing V_n across random permutations could yield a permutation-based sensitivity measure for epistemic uncertainty.","If the quasi-martingale conditions fail for TabPFN (as the paper's power-law diagnostics hint), a natural next step is a CLT with a slower-than-√n rate; such a result would also sharpen the two estimators and explain the small-n coverage deficits visible in the Dependent and Categorical DGPs.","The credible bands target the limiting predictive distribution F̃ rather than the true conditional law; the near-nominal coverage suggests the two coincide in the tested regimes, but applications to misspecified or non-stationary data would be the natural stress test."],"forward_implications":["The predictive CLT extends Bayesian predictive inference to supervised settings under quasi-martingale conditions, covering predictive rules that are neither exchangeable nor exact martingales.","For TabPFN, epistemic uncertainty bands are computable in-context—n forward passes on expanding prefixes—avoiding the need for a generative model of covariates or ensembles of rollouts.","The asymptotic credible intervals and sup-t bands achieve near-nominal frequentist coverage at moderate context lengths across linear, dependent, Poisson, probit, and categorical DGPs.","The entropy-based decomposition is numerically stable near p=0 or 1 and generalizes to multiclass problems via a Dirichlet moment match.","Because the CLT only requires a compact response space and a fixed event/covariate pair, the framework applies beyond TabPFN to other supervised black-box predictive rules."],"fun_headline_variants":["Theorem turns TabPFN's in-context updates into epistemic uncertainty","TabPFN's uncertainty decomposition gets a Bayesian CLT","Predictive CLT gives TabPFN a principled aleatoric/epistemic split","Near-nominal credible bands from TabPFN's update volatility","How TabPFN separates known from unknown uncertainty"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The theorem's formal validity rests on the √n-weighted quasi-martingale condition (7), and the paper's own diagnostics for TabPFN fit drift exponents of about 0.73–0.87, below the 1.5 threshold the condition requires—so if those estimates are accurate, the CLT is not formally guaranteed for the exact model.","fun_headline_variants_meta":{"raw":{"variants":["Theorem turns TabPFN's in-context updates into epistemic uncertainty","TabPFN's uncertainty decomposition gets a Bayesian CLT","Predictive CLT gives TabPFN a principled aleatoric/epistemic split","Near-nominal credible bands from TabPFN's update volatility","How TabPFN separates known from unknown uncertainty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000813,"raw_usage":{"total_tokens":3401,"prompt_tokens":742,"completion_tokens":2659,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":2570}},"tokens_in":486,"tokens_out":2659,"duration_ms":16870,"temperature":1.0,"reasoning_tokens":2570,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T04:31:16.000085+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Under the paper's discrete-covariate logistic DGP, run TabPFN rollouts with larger ensembles (n_estimators≥64) and fit the drift exponent β for E|b_n|; if β is consistently below 1.5 (or below 1), the quasi-martingale condition underlying Theorem 4.3 fails for TabPFN, and the decomposition's validity reduces to the empirical coverage alone.","supporting_citations":[],"review_version":1}