Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation

T0 review · 2 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that the estimator with the variance-minimising additive control variate (β★-IPS) asymptotically dominates self-normalised IPS (SNIPS) in mean squared error, and that the same holds at every position for ranking evaluation

desk verdict The paper's main theorem is true but classical; the proof has a repairable hole in the tail bound, and the ranking claims overreach. read the letter →

arxiv 2602.14914 v3 pith:HVZQWB6D submitted 2026-02-16 cs.LG cs.IR

Olivier Jeunen , Shashank Gupta This is my paper · ORCID
classification cs.LGcs.IR
keywords off-policyevaluationself-normalisedimportancesamplingcontrolvariatesvariancereductionrankingpoliciesmeansquarederrorweightsbaselinecorrections
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Off-policy evaluation estimates how a new policy would perform using logged data, without running online tests. SNIPS is a standard variance-reduction trick that divides by the average importance weight, trading bias for lower variance. The paper proves that an additive baseline correction—subtracting an optimally chosen constant from each reward—is asymptotically at least as good as SNIPS, and strictly better unless the optimal baseline happens to equal the true policy value. By deriving the exact variance gap, it shows SNIPS is asymptotically equivalent to using the generally sub-optimal baseline equal to the policy value. This gives practitioners a theoretical reason to prefer baseline corrections over self-normalisation, including for ranked outputs.

What carries the argument

The key mechanism is the exact decomposition SNIPS = β-IPS(V) + R_n, meaning self-normalisation is asymptotically nothing but an additive-baseline estimator whose baseline is the true policy value. The variance-gap identity Δ = (V(π)σ²_w − σ_{w,wr})² / (nσ²_w) ≥ 0 quantifies the loss from this sub-optimal baseline, and the O(n⁻²) bound on R_n, obtained via Hoeffding and Rosenthal inequalities under bounded weights and rewards, ensures the gap is the leading term in the MSE comparison.

What would settle it

Simulate off-policy evaluation with heavy-tailed importance weights (e.g., log-normal) and compare finite-sample MSE of β★-IPS and SNIPS at large n; if SNIPS ever has strictly lower MSE, the bounded-weight remainder bound in the proof is the point of failure. Alternatively, compute the exact asymptotic variance gap from the data and check whether empirical MSE differences converge to it at the claimed rate.

Watch

Extended reading notes

Core claim

The central claim is Theorem 3.1: for i.i.d. logged data with bounded rewards and bounded importance weights, MSE(β★-IPS) ≤ MSE(SNIPS) + O(n⁻²), and the inequality is strict for all sufficiently large n whenever β★ ≠ V(π) and Var(w) > 0. The proof constructs the exact identity SNIPS = β-IPS(V) + R_n, where β-IPS(V) is the additive-baseline estimator with baseline fixed to the true policy value V(π), and shows that the remainder R_n has mean-squared size O(n⁻²). Proposition 3.2 then gives the exact asymptotic variance gap: Δ = (V(π)σ²_w − σ_{w,wr})² / (nσ²_w) ≥ 0, so SNIPS's variance equals that of the additive estimator with baseline V(π), which is generally not the variance-minimising β★ =

Load-bearing premise

The theorem requires bounded importance weights (0 ≤ w ≤ W < ∞) and bounded rewards to control the O(n⁻²) remainder; if weights are heavy-tailed, the declared asymptotic dominance is not proven and may fail.

Editorial extensions

If this is right

  • Practitioners can replace SNIPS with β★-IPS and provably reduce asymptotic MSE whenever the optimal baseline differs from the policy value.
  • For ranking evaluation under the Item-Position Model, per-position additive baselines dominate the self-normalised SNIPM estimator at every rank.
  • Cross-fitting the empirically estimated β★ restores exact finite-sample unbiasedness while retaining the variance advantage.
  • Because the remainder decays as O(n⁻²), the variance gap dominates except in very small samples, so gains appear precisely in the policy-mismatch regimes where variance reduction matters most.
  • The exact gap formula gives a diagnostic: estimating β★ and V(π) from data reveals how much variance SNIPS is leaving on the table.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to check whether the dominance survives when importance weights are heavy-tailed; the O(n⁻²) remainder bound relies on bounded weights, and simulation could reveal whether a modified theorem holds in the unbounded case.
  • The same decomposition might apply to other ratio estimators beyond SNIPS, such as weighted importance sampling with different normalisers, suggesting additive control variates are a general recipe for variance reduction.
  • The paper leaves cross-position dependencies in rankings open; a globally optimal vector of baselines accounting for these dependencies could dominate the per-position approximation β★⊥⊥ even further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies off-policy evaluation (OPE) and compares two variance-reduction strategies: self-normalised importance sampling (SNIPS) and additive baseline corrections (β-IPS). The main theorem (Theorem 3.1) claims that the estimator using the population-optimal additive baseline, β*-IPS, asymptotically dominates SNIPS in mean squared error: MSE(β*-IPS) ≤ MSE(SNIPS) + O(n^{-2}), with strict dominance for sufficiently large n when β* ≠ V(π) and Var(w) > 0. The proof decomposes SNIPS as β-IPS(V(π)) plus a remainder, bounds the remainder, and then compares the optimal baseline with V(π). Proposition 3.2 gives an exact asymptotic variance gap between the two estimators. Section 3.3 extends the argument per-position to ranking under the Item-Position Model, defining β⊥⊥-IPM and claiming position-wise dominance over SNIPM. Section 3.4 discusses the practical use of an empirically estimated baseline and its finite-sample bias.

Significance. If correct, the result provides a clean theoretical justification for preferring additive control variates over self-normalisation in OPE, filling a gap left by prior empirical work (Gupta et al., RecSys 2024). The paper's main strengths are the exact decomposition in Eq. (7), the closed-form variance gap in Eq. (13), and the extension to ranking via a per-position argument. The dominance inequality is a genuine in-paper derivation rather than a restatement of known results. The result is also falsifiable: the variance gap has an explicit algebraic form and the strict-dominance condition is testable from data. The main caveats are that the proof of the O(n^{-2}) remainder is incomplete as written and the practical plug-in claim is not formally established.

major comments (2)
  1. [§3.1, Step 2 (Eqs. (8)–(11))] The proof of the O(n^{-2}) remainder is incomplete. The event E^c = {W̄ < 1/2} is dismissed via Hoeffding's inequality, but the text only says 'W̄ = 0 occurs with exponentially vanishing probability'. The dangerous event for the factor 1/W̄ in R_n = L_n(1-W̄)/W̄ is {0 < W̄ < 1/2}, where the factor is large and is not controlled by the fourth-moment bound on L_n. Thus E[R_n^2 1_{E^c}] is not bounded as written. This is load-bearing: the central inequality MSE(β*-IPS) ≤ MSE(SNIPS) + O(n^{-2}) and the strict-dominance conclusion both depend on this remainder being O(n^{-2}). The gap is repairable under the stated assumptions: on {W̄>0}, |X̄/W̄| ≤ max|r_i| ≤ R, hence |R_n| ≤ (R+|V|)(1+W), so E[R_n^2 1_{E^c}] ≤ C^2 P(E^c) = O(exp(-n/(2W^2))). The manuscript should state this argument or an equivalent; it currently addresses the wrong event. The cross-term analysis in Step 3 also uses 'R_n ≈ L
  2. [§3.4 (plug-in β̂*)] The practical claim that the empirically estimated β̂*-IPS 'maintains the same asymptotic MSE convergence rate as SNIPS whilst benefiting from the superior variance constant' is asserted without proof. Theorem 3.1 concerns the population-optimal β*, not the data-dependent β̂*. The extra term introduced by plug-in estimation is (β̂*-β*)(1-W̄), which is O_p(n^{-1}) if β̂* is n^{1/2}-consistent, so the asymptotic variance constant is indeed unchanged; but the mean-squared-error claim, including the O(n^{-1}) bias, should be stated as a lemma or given a rigorous derivation. Since the paper's practical recommendation is to replace SNIPS with β̂*-IPS, this step should not be left as an informal note.
minor comments (5)
  1. [§3.1, Assumptions and Theorem 3.1] The optimal baseline β* = σ_{w,wr}/σ_w^2 is undefined when Var(w)=0. The theorem and Proposition 3.2 should explicitly assume Var(w_i) > 0, or treat the degenerate case separately.
  2. [§3.1, Eq. (10)] The one-sided Hoeffding bound is fine, but the sentence 'W̄ = 0 occurs with exponentially vanishing probability' is misleading because P(E^c) is not the same as P(W̄=0), and the latter is not the event that needs controlling. Rephrase once the E^c bound is fixed.
  3. [§3.3, Eq. (17)] The notation for the per-position optimal baseline is inconsistent: the theorem and text use β*_{⊥⊥,j}, while Eq. (17) writes β_{⊥⊥,j} without the star. Please align the notation.
  4. [§3.4] The reference to cross-fitting [2] is appropriate but the 'negligible reduction in effective sample size' claim is informal; a precise statement of the variance cost would be useful.
  5. [General] The abstract and introduction state the result as a 'definitive answer' and 'theoretically justify shifting' from SNIPS. Given that Theorem 3.1 is asymptotic and relies on bounded importance weights and the oracle β*, the wording could be softened to reflect the scope of the assumptions.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the SNIPS-vs-β*-IPS comparison is derived in-paper from first-principles algebra; remaining self-citations are contextual and not load-bearing.

full rationale

The central claim does not reduce to its inputs. Eq. (7)-(9) derive an exact algebraic identity SNIPS = β-IPS(V)+R_n, and the proof attempts to bound the remainder under stated boundedness assumptions rather than assuming the comparison. Proposition 3.2 independently computes Var(β-IPS), derives β*=σ_{w,wr}/σ²_w as the minimizer of that quadratic, and obtains the variance gap Δ=(V(π)σ²_w−σ_{w,wr})²/(nσ²_w) directly from those expressions; no term in the derivation is fitted to the SNIPS MSE or imported as the conclusion. Theorem 3.3 applies Theorem 3.1 position-wise, not via [6]. The cited [6] provides the β-IPS estimator and the β* formula, but the paper re-derives the variance expression and does not lean on [6] for the dominance. §3.4 openly acknowledges the plug-in bias of empirical β̂*; this limits the practical extrapolation but is not a circular step. Per the reviewing rule, I flag the proof gap at Theorem 3.1 Step 2: the text dismisses only P(W̄=0) and does not bound E[R_n² 1_{0<W̄<1/2}], so E[R_n²]=O(n^{-2}) is not established as written; this is a repairable correctness gap, not circularity, and does not change the score. Minor self-citations ([6], [14]) in the motivation and practical takeaway are not load-bearing for the theorems.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper's claims rest on standard OPE regularity conditions: i.i.d. logging data with full support (E[w]=1), bounded rewards and bounded importance weights (needed for the concentration/moment bounds in Theorem 3.1), the Delta method for SNIPS's asymptotic variance, and the Item-Position Model for the ranking extension. The practical recommendation additionally depends on estimating β* from data, whose bias the paper acknowledges (§3.4). No ad-hoc constants are fitted; β* is the variance-minimizing baseline derived in-paper. No new entities are postulated.

free parameters (2)
  • β* (optimal additive baseline) = σ_{w,wr}/σ²_w (population); β̂* estimated from logged data in practice
    The entire recommendation hinges on estimating this variance-minimizing baseline from the same logged data; §3.4 concedes plug-in introduces O(n^{−1}) bias, and cross-fitting is offered as a fix. The theoretical dominance is proven only for the population value.
  • β*⊥⊥,j (per-position baselines, ranking) = Cov(w_j, w_j r_j)/Var(w_j) per position (Eq. 17)
    Per-position optimal baselines for the IPM extension; chosen independently (off-diagonal dependencies ignored), which the paper acknowledges is not globally optimal.
assumptions (5)
  • domain assumption Data i.i.d. from logging policy π0 with E[w_i]=1 (target policy covered by logging support)
    Assumptions (1)–(3) of Theorem 3.1; fails when target policy has mass outside logging support; standard support condition in OPE.
  • domain assumption Bounded rewards |r_i| ≤ R and bounded importance weights 0 ≤ w_i ≤ W
    Theorem 3.1 Assumptions (1)–(2); Step 2 needs these for Hoeffding/Rosenthal bounds and the exponential failure probability; heavy-tailed weights (near-zero propensities) invalidate the O(n^{−2}) argument as written.
  • domain assumption Item-Position Model: reward at position j of a ranked list is independent of other items
    Theorem 3.3 / Eq. (15) inherit the IPM from London et al. [20]; without it the position-marginal decomposition in Eq. (15) does not define the target value.
  • standard math Delta-method asymptotic variance of ratio estimator X̄/W̄ equals Var(X − V W)/n
    Proposition 3.2 cites Owen [21, Eq. 9.8]; this is the bridge making SNIPS equivalent to β-IPS(β=V) at first order.
  • standard math Hoeffding and Rosenthal inequalities for bounded zero-mean sums
    Theorem 3.1 Step 2 moment bounds, cited to Boucheron–Lugosi–Massart [1].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation." pith.science (2026). https://pith.science/paper/HVZQWB6D

@misc{pith2026260214914,
  author       = {Pith},
  title        = {Pith review of: Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HVZQWB6D}},
  note         = {Machine review of arXiv:2602.14914}
}
abstract

Off-policy evaluation (OPE) is essential for assessing ranking and recommendation systems without costly online interventions. Self-Normalised Inverse Propensity Scoring (SNIPS) is a standard tool for variance reduction in OPE, leveraging a multiplicative control variate. Recent advances in off-policy learning suggest that additive control variates (baseline corrections) may offer superior performance, yet theoretical guarantees for evaluation are lacking. This paper provides a definitive answer: we prove that $\beta^\star$-IPS, an estimator with an optimal additive baseline, asymptotically dominates SNIPS in Mean Squared Error. By analytically decomposing the variance gap, we show that SNIPS is asymptotically equivalent to using a specific -- but generally sub-optimal -- additive baseline. Our results theoretically justify shifting from self-normalisation to optimal baseline corrections for both ranking and recommendation.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Accelerating A/B-Tests with Counterfactual Estimation: Reducing Variance through Policy Overlap

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Reframing A/B assignment as a mixture policy and applying Δ-off-policy estimators yields an unbiased ATE estimator with variance provably no larger than difference-in-means whenever the tested policies overlap.

Reference graph

Works this paper leans on

31 extracted references · cited by 1 Pith paper

  1. [1]

    2013.Concentration Inequalities: A Nonasymptotic Theory of Independence

    Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. 2013.Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press. Additive Control Variates Dominate Self-Normalisation SIGIR ’26, July 20–24, 2026, Melbourne, VIC, Australia doi:10.1093/acprof:oso/9780199535255.001.0001

    arXiv 2013

  2. [2]

    Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. 2018. Dou- ble/debiased machine learning for treatment and structural parameters. The Econometrics Journal21, 1 (01 2018), C1–C68. doi:10.1111/ectj.12097 arXiv:https://academic.oup.com/ectj/article-pdf/21/1/C1/27684918/ectj00c1.pdf

    doi:10.1111/ectj.12097 2018

  3. [3]

    Alexandre Gilotte, Clément Calauzènes, Thomas Nedelec, Alexandre Abraham, and Simon Dollé. 2018. Offline A/B Testing for Recommender Systems. InProc. of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM ’18). ACM, 198–206. https://doi.org/10.1145/3159652.3159687

    arXiv 2018

  4. [4]

    Shashank Gupta. 2025. Safe, Efficient, and Robust Reinforcement Learning for Ranking and Diffusion Models.arXiv preprint arXiv:2510.15429(2025)

    arXiv 2025

  5. [5]

    Shashank Gupta, Philipp Hager, Jin Huang, Ali Vardasbi, and Harrie Oosterhuis

  6. [7]

    Shashank Gupta, Harrie Oosterhuis, and Maarten de Rijke. 2023. Safe deployment for counterfactual learning to rank with exposure-based risk minimization. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 249–258

    2023

  7. [8]

    an essay on the logical foundations of survey sampling, part one

    Jaroslav Hájek. 1971. Comment on “an essay on the logical foundations of survey sampling, part one”.The foundations of survey sampling236 (1971)

    1971

  8. [9]

    Daniel G Horvitz and Donovan J Thompson. 1952. A generalization of sampling without replacement from a finite universe.Journal of the American statistical Association47, 260 (1952), 663–685

    1952

Show all 31 references
  1. [10]

    2021.Offline Approaches to Recommendation with Online Success

    Olivier Jeunen. 2021.Offline Approaches to Recommendation with Online Success. Ph. D. Dissertation. University of Antwerp

    2021

  2. [11]

    Olivier Jeunen, Thorsten Joachims, Harrie Oosterhuis, Yuta Saito, and Flavian Vasile. 2022. CONSEQUENCES — Causality, Counterfactuals and Sequential Decision-Making for Recommender Systems. InProc. of the 16th ACM Confer- ence on Recommender Systems (RecSys ’22). ACM, 654–657....

    2022 doi

  3. [13]

    Olivier Jeunen, Ivan Potapov, and Aleksei Ustimenko. 2024. On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recom- mendation. InProc. of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’24). ACM, 1222–1233. doi:...

    2024

  4. [14]

    Olivier Jeunen and Aleksei Ustimenko. 2024. Δ-OPE: Off-Policy Estimation with Pairs of Policies. InProc. of the 18th ACM Conference on Recommender Systems (RecSys ’24). ACM, 878–883. doi:10.1145/3640457.3688162

    2024

  5. [15]

    Thorsten Joachims, Ben London, Yi Su, Adith Swaminathan, and Lequn Wang

  6. [16]

    Thorsten Joachims, Adith Swaminathan, and Maarten de Rijke. 2018. Deep Learning with Logged Bandit Feedback. InInternational Conference on Learning Representations. https://openreview.net/forum?id=SJaP_-xAb

    2018

  7. [17]

    2020.Trustworthy online controlled experi- ments: A practical guide to A/B testing

    Ron Kohavi, Diane Tang, and Ya Xu. 2020.Trustworthy online controlled experi- ments: A practical guide to A/B testing. Cambridge University Press

    2020

  8. [18]

    Augustine Kong. 1992. A note on importance sampling using standardized weights.University of Chicago, Dept. of Statistics, Tech. Rep348 (1992)

    1992

  9. [19]

    Muthukrishnan, Vishwa Vinay, and Zheng Wen

    Shuai Li, Yasin Abbasi-Yadkori, Branislav Kveton, S. Muthukrishnan, Vishwa Vinay, and Zheng Wen. 2018. Offline Evaluation of Ranking Policies with Click Models. InProc. of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’18). ACM, 1685–16...

    2018 doi

  10. [20]

    Ben London, Alexander Buchholz, Giuseppe Di Benedetto, Jan Malte Lichtenberg, Yannik Stein, and Thorsten Joachims. 2023. Self-Normalized Off-Policy Estimators for Ranking. InCONSEQUENCES Workshop at ACM RecSys ’23 (CONSEQUENCES ’23)

    2023

  11. [21]

    Art B. Owen. 2013.Monte Carlo theory, methods and examples

    2013

  12. [22]

    Hitesh Sagtani, Madan Gopal Jhawar, Rishabh Mehrotra, and Olivier Jeunen

  13. [23]

    Yuta Saito and Thorsten Joachims. 2021. Counterfactual Learning and Evaluation for Recommender Systems: Foundations, Implementations, and Recent Advances. InProc. of the 15th ACM Conference on Recommender Systems (RecSys ’21). ACM, 828–830. doi:10.1145/3460231.3473320

    2021

  14. [24]

    Yuta Saito, Takuma Udagawa, Haruka Kiyohara, Kazuki Mogi, Yusuke Narita, and Kei Tateno. 2021. Evaluating the Robustness of Off-Policy Evaluation. In Proc. of the 15th ACM Conference on Recommender Systems (RecSys ’21). ACM, 114–123. doi:10.1145/3460231.3474245

    2021

  15. [25]

    Ad-load Balancing via Off-policy Learning in a Content Marketplace. In Proc. of the 17th ACM International Conference on Web Search and Data Mining (WSDM ’24). ACM, 586–595. doi:10.1145/3616855.3635846

  16. [26]

    Adith Swaminathan, Akshay Krishnamurthy, Alekh Agarwal, Miro Dudik, John Langford, Damien Jose, and Imed Zitouni. 2017. Off-policy evaluation for slate recommendation. InAdvances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Ferg...

    2017

  17. [27]

    Bram van den Akker, Olivier Jeunen, Ying Li, Ben London, Zahra Nazari, and Devesh Parekh. 2024. Practical Bandits: An Industry Perspective. InProc. of the 17th ACM International Conference on Web Search and Data Mining (WSDM ’24). ACM, 1132–1135. doi:10.1145/3616855.3636449

    2024

  18. [28]

    Adith Swaminathan and Thorsten Joachims. 2015. The Self-Normalized Estimator for Counterfactual Learning. InAdvances in Neural Information Processing Systems, Vol. 28. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/ 2015/file/39027dfad5138c9ca0c474d71...

    2015

  19. [29]

    Nikos Vlassis, Ashok Chandrashekar, Fernando Amat, and Nathan Kallus

  20. [31]

    Flavian Vasile, David Rohde, Olivier Jeunen, and Amine Benhalloum. 2020. A Gentle Introduction to Recommendation as Counterfactual Policy Learning. In Proc. of the 28th ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’20). ACM, 392–393. doi:10.1145/334063...

    2020

  21. [33]

    InAdvances in Neu- ral Information Processing Systems, M

    Control Variates for Slate Off-Policy Evaluation. InAdvances in Neu- ral Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 3667–3679. https://proceedings.neurips.cc/paper_files/p...

    2021

  22. [2021]

    2021), 19–30

    Recommendations as Treatments.AI Magazine42, 3 (Nov. 2021), 19–30. doi:10.1609/aimag.v42i3.18141

    2021 doi

  23. [2024]

    InProceedings of the 17th ACM International Conference on Web Search and Data Mining

    Unbiased Learning to Rank: On Recent Advances and Practical Applications. InProceedings of the 17th ACM International Conference on Web Search and Data Mining. 1118–1121

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.