Marginal likelihoods for finite-support Huber contamination

Jaehoan Kim

arxiv: 2605.26723 · v1 · pith:6YNBX7OVnew · submitted 2026-05-26 · 📊 stat.ME · math.ST· stat.CO· stat.TH

Marginal likelihoods for finite-support Huber contamination

Jaehoan Kim This is my paper

Pith reviewed 2026-06-29 15:59 UTC · model grok-4.3

classification 📊 stat.ME math.STstat.COstat.TH

keywords Huber contaminationmarginal likelihoodDirichlet priorBeta priordynamic programmingrobust estimationfinite support

0 comments

The pith

Dirichlet and Beta priors yield an exact marginal likelihood for the structural parameter under finite-support Huber contamination after analytic integration of the nuisances.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that when the sample space is finite and known, the contaminating distribution under Huber contamination reduces to a probability vector on the atoms with atomwise domination constraints. A Dirichlet prior on that vector paired with a Beta prior on the contamination proportion permits exact analytic integration of both nuisance quantities. The resulting marginal likelihood for the structural parameter is a finite weighted sum over the possible allocations of the observed counts to the structural versus contaminating parts. For fixed support size this sum and its gradient can be obtained by dynamic programming at quadratic cost in the number of observations, which supports gradient-based posterior sampling.

Core claim

For Huber contamination on a known finite sample space, the unrestricted contaminating law is a probability vector on the support atoms, and domination over all measurable subsets reduces to atomwise inequalities. Placing a Dirichlet prior on this probability vector and a Beta prior on the contamination proportion gives an exact marginal likelihood for the structural parameter after analytic integration of both nuisance quantities. The likelihood is a finite weighted sum over allocations of the observed counts between the structural and contaminating components. For fixed support size, this sum and its score can be evaluated by a dynamic program with quadratic cost in the sample size, enabli

What carries the argument

The exact marginal likelihood expressed as a finite weighted sum over allocations of observed counts between structural and contaminating components, obtained after analytic integration via Dirichlet and Beta priors.

If this is right

The marginal likelihood and its score admit quadratic-time evaluation by dynamic programming when support size is fixed.
Gradient-based posterior sampling for the structural parameter becomes feasible without numerical integration over the nuisances.
The construction applies precisely when the sample space is known and finite, reducing measurable-set domination to atomwise inequalities.
The marginal is obtained by exact analytic integration rather than approximation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same conjugate structure might be exploitable in other discrete robust models where the contamination law lives on a finite known support.
The quadratic dynamic program could be adapted to related allocation problems in categorical data analysis beyond contamination.
Practitioners working with categorical observations could obtain exact robust Bayesian posteriors without resorting to MCMC over the full contamination parameters.

Load-bearing premise

The sample space must be known and finite so the contaminating law reduces to a probability vector on fixed atoms.

What would settle it

For a small finite support and small sample, numerically integrate the joint prior over the contamination proportion and vector to obtain the marginal likelihood and compare the value to the closed-form weighted sum; any systematic discrepancy would falsify the exact-integration claim.

Figures

Figures reproduced from arXiv: 2605.26723 by Jaehoan Kim.

**Figure 1.** Figure 1: Contaminated binomial example. Left: observed count histogram with fitted structural probability masses. Right: [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

**Figure 2.** Figure 2: Sensitivity of bias, 95% credible interval length and coverage to the prior parameter [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗

**Figure 3.** Figure 3: Trace plots of the Metropolis-adjusted Langevin sampler for different values of [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

read the original abstract

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper gives an exact marginal likelihood for the structural parameter under finite known support Huber contamination by integrating Dirichlet and Beta priors, then evaluates the sum with a DP.

read the letter

The main takeaway is that for Huber contamination on a known finite support, analytic integration of a Dirichlet prior on the contaminating probabilities and a Beta prior on the contamination fraction produces an exact marginal likelihood as a finite sum over count allocations between clean and contaminated components. A dynamic program then computes that sum and its score in quadratic time in the sample size.

What stands out is the reduction to a tractable exact expression. The binomial expansion of the mixture terms followed by the multivariate beta integrals yields the weighted sum, and the DP processes atoms one by one while tracking only the running contamination count. That matches the stress-test description and turns an otherwise exponential sum into something usable for gradient-based sampling.

The approach is narrow by design. Everything rests on the support being known and finite so that domination reduces to atomwise inequalities; outside that setting the same steps do not apply. The abstract supplies no simulation results or comparisons against approximate robust methods, so the practical payoff over existing techniques remains unshown. Soundness looks internally consistent from the given derivation outline, but the full recurrence and numerical stability would need checking in the manuscript.

This is for methodologists who need exact marginal likelihoods in discrete robust problems rather than a broad audience. A reader already working on conjugate Bayesian models for contaminated multinomials could extract the DP and try it on their data. It is worth sending to peer review because the central construction is self-contained and the computational claim is specific enough to be verified or refuted by referees.

Referee Report

1 major / 2 minor

Summary. The manuscript claims that for Huber contamination on a known finite sample space, placing a Dirichlet prior on the contaminating probability vector and a Beta prior on the contamination proportion yields an exact marginal likelihood for the structural parameter after analytic integration of the nuisances. The resulting likelihood is a finite weighted sum over allocations of observed counts to structural versus contaminating components, which for fixed support size K can be evaluated (along with its score) by a dynamic program in O(K n^2) time.

Significance. If the analytic integration and dynamic program hold, the work supplies an exact, non-approximate marginal likelihood for Bayesian inference under finite-support Huber contamination. The explicit reduction to a computable sum via Dirichlet integrals and binomial expansions, together with the DP that tracks only the running contamination count, is a concrete strength that enables gradient-based posterior sampling without MCMC over the nuisance parameters.

major comments (1)

[dynamic program description] The section describing the dynamic program states that atoms are processed sequentially while tracking only the running total contamination count and yields O(K n^2) time, but does not supply the explicit recurrence relation, base cases, or inductive argument establishing that the DP correctly computes the finite sum over allocation vectors; this recurrence is load-bearing for the central claim of exact, efficient marginal-likelihood evaluation.

minor comments (2)

[Abstract] The abstract and introduction could state more explicitly that the known finite support is an essential modeling assumption that reduces subset domination to atomwise inequalities.
[main derivation] Notation for the allocation vectors (one integer per atom) and the resulting multivariate-beta ratios after integration could be introduced with a small illustrative example for K=2 to improve readability.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their positive evaluation of the significance of the work and for the constructive comment on the dynamic program. We address the major comment below and will revise the manuscript to incorporate the requested details.

read point-by-point responses

Referee: The section describing the dynamic program states that atoms are processed sequentially while tracking only the running total contamination count and yields O(K n^2) time, but does not supply the explicit recurrence relation, base cases, or inductive argument establishing that the DP correctly computes the finite sum over allocation vectors; this recurrence is load-bearing for the central claim of exact, efficient marginal-likelihood evaluation.

Authors: We agree that the current description of the dynamic program is high-level and lacks the explicit recurrence, base cases, and inductive argument needed to rigorously establish correctness. This is a valid observation that affects the clarity of the central computational claim. In the revised manuscript we will add a new subsection that defines the state f(k, m) as the partial weighted sum after processing the first k atoms with exactly m observations allocated to contamination, supplies the base cases f(0, 0) = 1 and f(0, m) = 0 for m > 0, gives the recurrence that branches on whether the count for atom k+1 is assigned to the structural or contaminating component (using the appropriate Beta-Dirichlet integral factors), and includes a short inductive argument on k showing that f(K, ·) recovers the desired finite sum over allocation vectors. The revised text will also confirm the O(K n^2) complexity under the standard implementation that iterates over possible m values. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained analytic integration

full rationale

The paper's central derivation places standard Dirichlet and Beta priors on the nuisance contamination vector and proportion, then analytically integrates them out via the Dirichlet integral and binomial expansions to obtain an explicit finite sum over allocation vectors. This sum is independent of any fitted parameters or self-cited results; the subsequent dynamic program is presented purely as an efficient evaluation method (O(K n^2)) rather than as part of the likelihood definition itself. No self-definitional steps, fitted-input predictions, load-bearing self-citations, uniqueness theorems, or smuggled ansatzes appear in the provided derivation chain. The finite-support reduction to atomwise inequalities follows directly from the problem statement without external author-specific theorems.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on standard properties of Dirichlet and Beta distributions for analytic integration and on the reduction of setwise domination to atomwise inequalities when the support is finite; no free parameters or invented entities are introduced in the abstract.

axioms (2)

standard math Dirichlet and Beta priors admit closed-form marginalization when integrated against multinomial likelihoods on finite support.
Invoked to obtain the exact marginal likelihood after integrating the contaminating probability vector and contamination proportion.
domain assumption For a known finite sample space, domination of the contaminating law over all measurable subsets reduces to atomwise inequalities.
Stated in the abstract as the key simplification that allows the contaminating law to be represented as a probability vector on the atoms.

pith-pipeline@v0.9.1-grok · 5619 in / 1470 out tokens · 29821 ms · 2026-06-29T15:59:50.130280+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages

[1]

Florian Br¨ uhlmann, Serge Petralito, Lena F

doi: 10.1111/rssb.12158. Florian Br¨ uhlmann, Serge Petralito, Lena F. Aeschbach, and Klaus Opwis. The quality of data collected online: An investigation of careless responding in a crowdsourced sample.Methods in Psychology, 2:100022,

work page doi:10.1111/rssb.12158
[2]

Mengjie Chen, Chao Gao, and Zhao Ren

doi: 10.1016/j.metip.2020.100022. Mengjie Chen, Chao Gao, and Zhao Ren. A general decision theory for Huber’s𝜖-contamination model.Electronic Journal of Statistics, 10(2):3752–3774,

work page doi:10.1016/j.metip.2020.100022 2020
[3]

Mengjie Chen, Chao Gao, and Zhao Ren

doi: 10.1214/16-EJS1217. Mengjie Chen, Chao Gao, and Zhao Ren. Robust covariance and scatter matrix estimation under Huber’s contamination model.The Annals of Statistics, 46(5):1932–1960,

work page doi:10.1214/16-ejs1217 1932
[4]

doi: 10.1214/aoms/1177703732. Peter J. Huber and Elvezio M. Ronchetti.Robust Statistics. John Wiley & Sons, Hoboken, NJ, 2 edition,

work page doi:10.1214/aoms/1177703732
[5]

Jeffrey W

doi: 10.1002/9780470434697. Jeffrey W. Miller and David B. Dunson. Robust bayesian inference via coarsening.Journal of the American Statistical Association, 114(527):1113–1125,

work page doi:10.1002/9780470434697
[6]

doi: 10.1080/01621459.2018. 1469995. Andreas F. Ruckstuhl and Alan H. Welsh. Robust fitting of the binomial model.The Annals of Statistics, 29(4):1117–1136,

work page doi:10.1080/01621459.2018 2018
[7]

Matthew S

doi: 10.1214/aos/1013699996. Matthew S. Shotwell and Elizabeth H. Slate. Bayesian outlier detection with Dirichlet process mixtures.Bayesian Analysis, 6(4):665–690,

work page doi:10.1214/aos/1013699996
[8]

Douglas G

doi: 10.1214/11-BA625. Douglas G. Simpson, Raymond J. Carroll, and David Ruppert. M-estimation for discrete data: asymptotic distribution theory and implications.The Annals of Statistics, 15(2):657–669,

work page doi:10.1214/11-ba625
[9]

URL https: //doi.org/10.1214/aos/1176350263

doi: 10.1214/aos/1176350367. Isabella Verdinelli and Larry Wasserman. Bayesian analysis of outlier problems using the Gibbs sampler.Statistics and Computing, 1(2):105–117,

work page doi:10.1214/aos/1176350367
[10]

doi: 10.1007/BF01889985. M. K. Ward and Adam W. Meade. Dealing with careless responding in survey data: Prevention, identification, and recommended best practices.Annual Review of Psychology, 74:577–596,

work page doi:10.1007/bf01889985
[11]

doi: 10.1146/annurev-psych-040422-045007. 16

work page doi:10.1146/annurev-psych-040422-045007

[1] [1]

Florian Br¨ uhlmann, Serge Petralito, Lena F

doi: 10.1111/rssb.12158. Florian Br¨ uhlmann, Serge Petralito, Lena F. Aeschbach, and Klaus Opwis. The quality of data collected online: An investigation of careless responding in a crowdsourced sample.Methods in Psychology, 2:100022,

work page doi:10.1111/rssb.12158

[2] [2]

Mengjie Chen, Chao Gao, and Zhao Ren

doi: 10.1016/j.metip.2020.100022. Mengjie Chen, Chao Gao, and Zhao Ren. A general decision theory for Huber’s𝜖-contamination model.Electronic Journal of Statistics, 10(2):3752–3774,

work page doi:10.1016/j.metip.2020.100022 2020

[3] [3]

Mengjie Chen, Chao Gao, and Zhao Ren

doi: 10.1214/16-EJS1217. Mengjie Chen, Chao Gao, and Zhao Ren. Robust covariance and scatter matrix estimation under Huber’s contamination model.The Annals of Statistics, 46(5):1932–1960,

work page doi:10.1214/16-ejs1217 1932

[4] [4]

doi: 10.1214/aoms/1177703732. Peter J. Huber and Elvezio M. Ronchetti.Robust Statistics. John Wiley & Sons, Hoboken, NJ, 2 edition,

work page doi:10.1214/aoms/1177703732

[5] [5]

Jeffrey W

doi: 10.1002/9780470434697. Jeffrey W. Miller and David B. Dunson. Robust bayesian inference via coarsening.Journal of the American Statistical Association, 114(527):1113–1125,

work page doi:10.1002/9780470434697

[6] [6]

doi: 10.1080/01621459.2018. 1469995. Andreas F. Ruckstuhl and Alan H. Welsh. Robust fitting of the binomial model.The Annals of Statistics, 29(4):1117–1136,

work page doi:10.1080/01621459.2018 2018

[7] [7]

Matthew S

doi: 10.1214/aos/1013699996. Matthew S. Shotwell and Elizabeth H. Slate. Bayesian outlier detection with Dirichlet process mixtures.Bayesian Analysis, 6(4):665–690,

work page doi:10.1214/aos/1013699996

[8] [8]

Douglas G

doi: 10.1214/11-BA625. Douglas G. Simpson, Raymond J. Carroll, and David Ruppert. M-estimation for discrete data: asymptotic distribution theory and implications.The Annals of Statistics, 15(2):657–669,

work page doi:10.1214/11-ba625

[9] [9]

URL https: //doi.org/10.1214/aos/1176350263

doi: 10.1214/aos/1176350367. Isabella Verdinelli and Larry Wasserman. Bayesian analysis of outlier problems using the Gibbs sampler.Statistics and Computing, 1(2):105–117,

work page doi:10.1214/aos/1176350367

[10] [10]

doi: 10.1007/BF01889985. M. K. Ward and Adam W. Meade. Dealing with careless responding in survey data: Prevention, identification, and recommended best practices.Annual Review of Psychology, 74:577–596,

work page doi:10.1007/bf01889985

[11] [11]

doi: 10.1146/annurev-psych-040422-045007. 16

work page doi:10.1146/annurev-psych-040422-045007