Pith. sign in

REVIEW 2 major objections 1 minor 1 cited by

Defining a Minimum Resolution for Unbinned Analyses

T0 review · 2 major / 1 minor · reviewed 2026-06-30 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The Minimum Resolution Likelihood method defines a Fiducial Signal Region that converts systematic effects from machine learning background estimation into statistical uncertainties.

desk verdict MRL defines a fiducial region to convert ML background systematics into statistical errors, shown unbiased in toy models and one di-Higgs case but without general proof. read the letter →

arxiv 2606.28498 v1 pith:OZT5X4SZ submitted 2026-06-26 hep-ph hep-ex

Manuel Szewc This is my paper · ORCID
classification hep-phhep-ex
keywords MinimumResolutionLikelihoodFiducialSignalRegionmachinelearningbackgroundestimationstrengthunbinnedanalysessystematicuncertaintiesdi-Higgssearches
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Collider analyses use machine learning models to estimate background likelihoods, but these can introduce hard-to-quantify systematic biases when estimating signal parameters. The paper presents the Minimum Resolution Likelihood method to define a Fiducial Signal Region that turns those systematic effects into statistical uncertainties. Demonstrations in toy examples and a realistic di-Higgs search with the HI-SIGMA technique show that signal strength estimates end up unbiased or consistent with zero. A reader would care because the approach addresses a practical obstacle to using advanced machine learning in unbinned collider data analysis without leaving hidden biases.

What carries the argument

The Minimum Resolution Likelihood (MRL) method, which defines a Fiducial Signal Region to convert ML-induced systematic effects from background estimation into statistical uncertainties.

What would settle it

A simulation or data analysis after MRL application in which the extracted signal strength exhibits a bias exceeding the size of the converted statistical uncertainties.

Watch

Extended reading notes

Core claim

We present the Minimum Resolution Likelihood (MRL) method, which defines a Fiducial Signal Region that effectively turns the systematic effects into statistical uncertainties. We show with examples that the resulting signal strength estimation is either unbiased or consistent with zero. We consider both toy examples and a realistic application based on the HI-SIGMA technique applied to di-Higgs searches.

Load-bearing premise

Defining a Fiducial Signal Region via the Minimum Resolution Likelihood method can systematically convert all relevant systematic effects arising from ML background estimation into statistical uncertainties without introducing new unaccounted biases or selection effects.

Editorial extensions

If this is right

  • Signal strength estimation remains unbiased when the MRL-defined region is applied.
  • When no signal is present, the estimated strength is consistent with zero.
  • Systematic effects from machine learning background models become statistical uncertainties.
  • The method works for both simplified toy cases and full di-Higgs searches using HI-SIGMA.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The approach could extend to other machine-learning-driven searches in high-energy physics beyond the di-Higgs channel.
  • It might reduce reliance on separate systematic uncertainty modeling in future unbinned analyses.
  • Further validation in additional final states would test whether the conversion from systematics to statistics holds more broadly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces the Minimum Resolution Likelihood (MRL) method, which defines a Fiducial Signal Region to convert systematic effects arising from machine-learning estimates of background likelihoods into statistical uncertainties in unbinned collider analyses. The central claim is that the resulting signal-strength estimation is either unbiased or consistent with zero, supported by toy-model examples and one realistic application to di-Higgs searches via the HI-SIGMA technique.

Significance. If the MRL construction can be shown to avoid new selection biases in general, the approach would offer a practical way to mitigate hard-to-quantify ML systematics in unbinned fits, strengthening the reliability of signal-parameter extraction. The provided examples supply initial evidence, but the absence of a general derivation or exhaustive validation against correlated error modes reduces the assessed impact.

major comments (2)
  1. [Abstract] Abstract: the assertion that signal-strength estimation is unbiased or consistent with zero is presented without derivation, validation details, or quantitative checks, leaving open whether the toy models and single application actually support the claim against ML systematics.
  2. [Toy examples and HI-SIGMA application] Toy examples and HI-SIGMA application: the demonstrations are confined to specific cases; no exhaustive scan over ML error modes (e.g., shape distortions correlated with signal-like kinematics) is reported, so the assumption that the fiducial-region definition introduces no new selection biases remains untested and load-bearing for the unbiased claim.
minor comments (1)
  1. The term 'Fiducial Signal Region' is introduced as a new construct; a brief comparison to existing fiducial definitions in the literature would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive review and for identifying areas where the presentation of our results can be strengthened. We respond to each major comment below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the assertion that signal-strength estimation is unbiased or consistent with zero is presented without derivation, validation details, or quantitative checks, leaving open whether the toy models and single application actually support the claim against ML systematics.

    Authors: The abstract summarizes the empirical findings reported in Sections 3 and 4. While no general analytic derivation is provided, the toy-model studies and HI-SIGMA application contain explicit quantitative checks (bias measurements, pull distributions, and coverage tests) demonstrating that the extracted signal strength is unbiased or consistent with zero under the tested ML-induced systematics. We will revise the abstract to state explicitly that the property is shown empirically in the presented examples and to reference the relevant validation sections. revision: yes

  2. Referee: [Toy examples and HI-SIGMA application] Toy examples and HI-SIGMA application: the demonstrations are confined to specific cases; no exhaustive scan over ML error modes (e.g., shape distortions correlated with signal-like kinematics) is reported, so the assumption that the fiducial-region definition introduces no new selection biases remains untested and load-bearing for the unbiased claim.

    Authors: We agree that the validation is limited to the specific toy configurations and the single realistic di-Higgs application described. An exhaustive scan over all conceivable ML error modes, including arbitrary correlated shape distortions, lies outside the scope of this work. The MRL construction is intended to convert ML systematics into statistical uncertainties by construction of the fiducial region; the examples illustrate this mechanism but do not constitute a proof against every possible bias. We will add a dedicated limitations paragraph discussing the representativeness of the tested cases and the possibility of residual selection effects, while retaining the claim that no new biases were observed in the reported studies. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MRL method and unbiasedness shown via examples, not by construction

full rationale

The paper introduces the Minimum Resolution Likelihood (MRL) method as a definition for a fiducial signal region that converts ML background systematics into statistical uncertainties. It then validates the resulting signal strength estimation (unbiased or consistent with zero) through explicit toy examples and one HI-SIGMA di-Higgs application. No load-bearing step reduces by the paper's own equations to a fitted input renamed as prediction, a self-referential definition, or a self-citation chain; the central claim rests on empirical demonstration rather than algebraic identity or imported uniqueness theorems. This is the most common honest outcome for a methods paper whose result is externally falsifiable via the provided examples.

Assumptions & free parameters 0 free parameters · 0 assumptions · 1 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities beyond the named method itself can be identified.

invented entities (1)
  • Fiducial Signal Region
    purpose: Region in which ML systematic effects are redefined as statistical uncertainties
    Central construct of the MRL method introduced to achieve the claimed bias reduction

how reviews work

0 comments
Cite this review

Pith. "Pith review of Defining a Minimum Resolution for Unbinned Analyses." pith.science (2026). https://pith.science/paper/OZT5X4SZ

@misc{pith2026260628498,
  author       = {Pith},
  title        = {Pith review of: Defining a Minimum Resolution for Unbinned Analyses},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZT5X4SZ}},
  note         = {Machine review of arXiv:2606.28498}
}
read the original abstract

Collider analyses combine rigorous statistical techniques with state-of-the-art Machine Learning models. However, when the latter are used directly to estimate the likelihood function of the background, hard to quantify systematic effects may bias the estimation of the relevant signal parameters. To address this problem, we present the Minimum Resolution Likelihood (MRL) method, which defines a Fiducial Signal Region that effectively turns the systematic effects into statistical uncertainties. We show with examples that the resulting signal strength estimation is either unbiased or consistent with zero. We consider both toy examples and a realistic application based on the HI-SIGMA technique applied to di-Higgs searches.

Figures

Figures reproduced from arXiv: 2606.28498 by the authors.

Figure 1
Figure 1. The three fits done on a single pseudo-experiment for the 1D Gaussian [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Lmin determination obtained by comparing a Calibration Region and a set of toys for the 1D Gaussian example. Left: The probability distribution of the model likelihood under the background model and the true background. Right: The fraction difference distribution as a function of the critical likelihood Lc obtained from comparing the toy and Calibration datasets. We observe how Lmin captures the crossing between bac… view at source ↗
Figure 3
Figure 3. 100 pseudo-experiment runs for different signal injection for the 1D [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The three fits done on a single pseudo-experiment for the 2D example [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Lmin determination obtained by comparing a Calibration Region and a set of toys for the 2D Gaussian example. Left: The probability distribution of the model likelihood under the background model and the true background. Right: The fraction difference distribution as a …
Figure 6
Figure 6. Figure 6: 100 pseudo-experiment runs for different signal injection for the 2D [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: A single run of HI-SIGMA+MRL , where p(⃗x|m ∈ SB) has not been masked. The figures show, in clockwise order: the mγγ distribution for the Signal Region and when applying the Fiducial Signal Region cuts, the signal strength test statistic landscape using four diffferent…
Figure 8
Figure 8. Figure 8: A single run of HI-SIGMA+MRL , where p(⃗x|m ∈ SB) has been masked. The figures show, in clockwise order: the mγγ distribution for the Signal Region and when applying the Fiducial Signal Region cuts, the signal strength test statis￾tic landscape using four diffferent st…
Figure 9
Figure 9. Figure 9: 5 pseudo-experiment runs for the di-Higgs dataset, with sideband mask [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Well-Tempered Likelihood: Honest Confidence Intervals for Misspecified Models

    stat.ME 2026-07 conditional novelty 6.0 of 10

    Dividing the likelihood-ratio statistic by a goodness-of-fit statistic stops confidence intervals from collapsing under model misspecification, making them self-limit at a floor set by the model's inadequacy.

Reference graph

Works this paper leans on

39 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    Behnke, K

    O. Behnke, K. Kröninger, T. Schörner-Sadenius and G. Schott, eds.,Data analysis in high energy physics: A practical guide to statistical methods, Wiley-VCH, Weinheim, Germany, ISBN 978-3-527-41058-3, 978-3-527-65344-7, 978-3-527-65343-0 (2013)

    2013

  2. [2]

    M. Aaboudet al.,Search for pair production of Higgs bosons in theb¯bb¯bfinal state using proton-proton collisions at√s= 13TeV with the ATLAS detector, JHEP01, 030 (2019), doi:10.1007/JHEP01(2019)030,1804.06174

    work page Pith review arXiv doi:10.1007/jhep01(2019)030 2019

  3. [3]

    Search for ZZ and ZH production in the b bb b final state using proton-proton collisions at √s=13 TeV

    A. Hayrapetyanet al.,Search for ZZ and ZH production in the b¯bb¯b final state us- ing proton-proton collisions at √s= 13TeV, Eur. Phys. J. C84(7), 712 (2024), doi:10.1140/epjc/s10052-024-13021-z,2403.20241. [4]Improved results on Higgs boson pair production in the 4b final state(2025). 20 SciPost Physics Submission

    work page doi:10.1140/epjc/s10052-024-13021-z 2024

  4. [4]

    Hallin, J

    A. Hallin, J. Isaacson, G. Kasieczka, C. Krause, B. Nachman, T. Quadfasel, M. Schlaf- fer, D. Shih and M. Sommerhalder,Classifying anomalies through outer density es- timation, Phys. Rev. D106(5), 055006 (2022), doi:10.1103/PhysRevD.106.055006, 2109.00546

    work page doi:10.1103/physrevd.106.055006 2022

  5. [5]

    J. A. Raine, S. Klein, D. Sengupta and T. Golling,CURTAINs for your Sliding Win- dow: Constructing Unobserved Regions by Transforming Adjacent Intervals, Front.Big Data6, 899345 (2022), doi:10.3389/fdata.2023.899345,2203.09470

    work page doi:10.3389/fdata.2023.899345 2022

  6. [6]

    Flow-enhanced transportation for anomaly detection

    T. Golling, S. Klein, R. Mastandrea and B. Nachman,Flow-enhanced transportation for anomaly detection, Phys. Rev. D107(9), 096025 (2023), doi:10.1103/PhysRevD.107.096025,2212.11285

    work page doi:10.1103/physrevd.107.096025 2023

  7. [7]

    Resonant anomaly detection without background sculpting

    A. Hallin, G. Kasieczka, T. Quadfasel, D. Shih and M. Sommerhalder,Resonant anomaly detection without background sculpting, Phys.Rev.D107(11), 114012(2023), doi:10.1103/PhysRevD.107.114012,2210.14924

    work page doi:10.1103/physrevd.107.114012 2023

  8. [8]

    Sengupta, S

    D. Sengupta, S. Klein, J. A. Raine and T. Golling,CURTAINs flows for flows: Constructing unobserved regions with maximum likelihood estimation, SciPost Phys. 17(2), 046 (2024), doi:10.21468/SciPostPhys.17.2.046,2305.04646

    work page doi:10.21468/scipostphys.17.2.046 2024

Show all 39 references
  1. [9]

    R. Das, G. Kasieczka and D. Shih,Residual ANODE(2023),2312.11629. [11]Recommendations for the Modeling of Smooth Backgrounds(2020)

    2023

  2. [10]

    P. D. Dauncey, M. Kenzie, N. Wardle and G. J. Davies,Handling uncertainties in background shapes: the discrete profiling method, JINST10(04), P04015 (2015), doi:10.1088/1748-0221/10/04/P04015,1408.6865

    2015 doi arXiv

  3. [11]

    Haußmann, R

    M. Haußmann, R. Winterhalder and M. Ubiali,Uncertainty in Physics and AI: Tax- onomy, Quantification, and Validation(2026),2605.10378

    2026 arXiv

  4. [12]

    Brehmer, K

    J. Brehmer, K. Cranmer, G. Louppe and J. Pavez,Constraining Effective Field Theories with Machine Learning, Phys. Rev. Lett.121(11), 111801 (2018), doi:10.1103/PhysRevLett.121.111801,1805.00013

    2018 doi arXiv

  5. [13]

    Brehmer, K

    J. Brehmer, K. Cranmer, G. Louppe and J. Pavez,A Guide to Constraining Ef- fective Field Theories with Machine Learning, Phys. Rev. D98(5), 052004 (2018), doi:10.1103/PhysRevD.98.052004,1805.00020

    2018 doi arXiv

  6. [14]

    Cranmer, J

    K. Cranmer, J. Brehmer and G. Louppe,The frontier of simulation-based inference, Proceedings of the National Academy of Sciences117(48), 30055 (2020), doi:10.1073/pnas.1912789117,https://www.pnas.org/doi/pdf/10.1073/ pnas.1912789117

    2020 doi

  7. [15]

    171–176 (2020)

    A.Ghosh,Measuring quantum interference in the off-shell Higgs to four leptons process with Machine Learning, pp. 171–176 (2020)

    2020

  8. [16]

    Gomez Ambrosio, J

    R. Gomez Ambrosio, J. ter Hoeve, M. Madigan, J. Rojo and V. Sanz,Unbinned multivariate observables for global SMEFT analyses from machine learning, JHEP 03, 033 (2023), doi:10.1007/JHEP03(2023)033,2211.02058

    2023 doi

  9. [17]

    Bahl and S

    H. Bahl and S. Brass,ConstrainingCP-violation in the Higgs-top-quark interaction using machine-learning-based inference, JHEP03, 017 (2022), doi:10.1007/JHEP03(2022)017,2110.10177. 21 SciPost Physics Submission

    2022 doi

  10. [18]

    Barrué, P

    R. Barrué, P. Conde-Muíño, V. Dao and R. Santos,Simulation-based inference in the search for CP violation in leptonic WH production, JHEP04, 014 (2024), doi:10.1007/JHEP04(2024)014,2308.02882

    2024 doi

  11. [19]

    Schöfbeck,Refinable modeling for unbinned SMEFT analyses, Mach

    R. Schöfbeck,Refinable modeling for unbinned SMEFT analyses, Mach. Learn. Sci. Tech.6(1), 015007 (2025), doi:10.1088/2632-2153/ad9fd1,2406.19076

    2025 doi

  12. [20]

    S. Chai, J. Gu and L. Li,From optimal observables to machine learning: an effective- field-theory analysis of e+e−→W +W− at future lepton colliders, JHEP05, 292(2024), doi:10.1007/JHEP05(2024)292,2401.02474

    2024 doi

  13. [21]

    Mastandrea, B

    R. Mastandrea, B. Nachman and T. Plehn,Constraining the Higgs potential with neu- ral simulation-based inference for di-Higgs production, Phys. Rev. D110(5), 056004 (2024), doi:10.1103/PhysRevD.110.056004,2405.15847

    2024 doi

  14. [22]

    Benato, C

    L. Benato, C. Giordano, C. Krause, A. Li, R. Schöfbeck, D. Schwarz, M. Shooshtari and D. Wang,Unbinned inclusive cross-section measurements with machine-learned systematic uncertainties(2025),2505.05544

    2025

  15. [23]

    Diefenbacher, S

    S. Diefenbacher, S. Palacios Schweitzer and G. Kasieczka,Generative Models and Statistical Validation(2026),2605.30453

    2026 arXiv

  16. [24]

    Amram and M

    O. Amram and M. Szewc,Data-driven high-dimensional statistical inference with generative models, JHEP11, 129 (2025), doi:10.1007/JHEP11(2025)129,2506.06438

    2025 doi

  17. [25]

    Cranmer,Practical Statistics for the LHC, In2011 European School of High-Energy Physics, pp

    K. Cranmer,Practical Statistics for the LHC, In2011 European School of High-Energy Physics, pp. 267–308, doi:10.5170/CERN-2014-003.267 (2014),1503.07622

    2014 doi arXiv

  18. [26]

    Buchner,Nested sampling methods, Statistics Surveys17(none) (2023), doi:10.1214/23-ss144

    J. Buchner,Nested sampling methods, Statistics Surveys17(none) (2023), doi:10.1214/23-ss144

    2023 doi

  19. [27]

    A. L. Read,Linear interpolation of histograms, Nucl. Instrum. Meth. A425, 357 (1999), doi:10.1016/S0168-9002(98)01347-3

    1999 doi

  20. [28]

    Cranmer, G

    K. Cranmer, G. Lewis, L. Moneta, A. Shibata and W. Verkerke,HistFactory: A tool for creating statistical models for use with RooFit and RooStats(2012)

    2012

  21. [29]

    Dembinski and P

    H. Dembinski and P. O. et al.,scikit-hep/iminuit(2020), doi:10.5281/zenodo.3949207

    2020 doi

  22. [30]

    Dawsonet al.,Report of the Topical Group on Higgs Physics for Snowmass 2021: The Case for Precision Higgs Physics, InSnowmass 2021(2022),2209.07510

    S. Dawsonet al.,Report of the Topical Group on Higgs Physics for Snowmass 2021: The Case for Precision Higgs Physics, InSnowmass 2021(2022),2209.07510

    2021

  23. [31]

    Alwall, R

    J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer, H. S. Shao, T. Stelzer, P. Torrielli and M. Zaro,The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations, JHEP07, 0...

    2014 doi arXiv

  24. [32]

    Sjöstrand, S

    T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Rasmussen and P. Z. Skands,An introduction to PYTHIA 8.2, Comput. Phys. Commun.191, 159 (2015), doi:10.1016/j.cpc.2015.01.024,1410.3012

    2015 doi arXiv

  25. [33]

    Bierlichet al.,A comprehensive guide to the physics and usage of PYTHIA 8.3, SciPostPhys.Codeb.2022, 8(2022), doi:10.21468/SciPostPhysCodeb.8,2203.11601

    C. Bierlichet al.,A comprehensive guide to the physics and usage of PYTHIA 8.3, SciPostPhys.Codeb.2022, 8(2022), doi:10.21468/SciPostPhysCodeb.8,2203.11601

    2022 doi arXiv

  26. [34]

    de Favereau, C

    J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens and M. Selvaggi,DELPHES 3, A modular framework for fast simulation of a generic collider experiment, JHEP02, 057(2014), doi:10.1007/JHEP02(2014)057,1307.6346. 22 SciPost Physics Submission

    2014 doi arXiv

  27. [35]

    M. J. Oreglia,A study of the reactionsψ′ →γγψ, Ph.D. thesis, Stanford University, SLAC-R-236 (1980)

    1980

  28. [36]

    J. E. Gaiser,Charmonium Spectroscopy From Radiative Decays of theJ/ψandψ ′, Ph.D. thesis, Stanford University, SLAC-R-255 (1982)

    1982

  29. [37]

    Aadet al.,An implementation of neural simulation-based inference for parameter estimation in ATLAS(2024),2412.01600

    G. Aadet al.,An implementation of neural simulation-based inference for parameter estimation in ATLAS(2024),2412.01600

    2024

  30. [38]

    G. Aadet al.,Measurement of off-shell Higgs boson production in theH ∗ → ZZ→4ℓdecay channel using a neural simulation-based inference technique in 13TeV pp collisions with the ATLAS detector, Rept. Prog. Phys.88(5), 057803 (2025), doi:10.1088/1361-6633/adcd9a,2412.01548

    2025 doi

  31. [39]

    Amram, D

    O. Amram, D. A. Faroughy, T. Gerdes, A. Hallin, G. Kasieczka, M. Krämer, H. Reyes- Gonzalez and D. Shih,Neural Scaling Laws for Jet Generation(2026),2605.28940. 23

    2026 arXiv

Pith tools

Reviewed June 30, 2026 · model on record in the stance chip above.