Pith. sign in

REVIEW 2 major objections 2 minor 43 references

Ranking Treatment Saturations under Clustered Network Interference

T0 review · 2 major / 2 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An empirical success ranking rule bounds maximum regret for choosing treatment saturations using only one combinatorial summary of cluster network dependency.

desk verdict The paper gives a clean decision-theoretic ranking rule for saturation levels under clustered interference, with non-asymptotic regret bounds that reduce to one combinatorial network summary, but the isolation step in the proof is the part that needs direct verification. read the letter →

arxiv 2606.18590 v1 pith:ZWW2TGWQ submitted 2026-06-17 econ.EM

Seungjin Han , Julius Owusu , Youngki Shin This is my paper · ORCID
classification econ.EM
keywords treatmentsaturationrankingclusterednetworkinterferenceempiricalsuccessruleregretboundstwo-stagerandomizeddesignstatisticaldecisiontheoryasymptoticoptimality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper develops a way to rank a finite set of treatment saturation levels when units interfere through networks inside clusters. It proposes an empirical success ranking rule that picks the higher-welfare saturation for each pair by comparing estimates from a two-stage randomized saturation experiment. The central technical result is a set of non-asymptotic upper bounds on the rule's maximum regret; these bounds depend on the interference structure only through a single combinatorial summary of the within-cluster dependency graph. The same bounds are used to identify a quasi-optimal allocation of first-stage saturation levels and to prove that the rule is asymptotically optimal among all threshold ranking rules when judged by worst-case regret. The approach therefore lets researchers rank saturations without needing the complete network adjacency matrix.

What carries the argument

The empirical success (ES) ranking rule that pairwise compares estimated welfares, together with the single combinatorial summary of within-cluster dependency structure that governs the derived regret bounds.

What would settle it

Empirical observation that two distinct within-cluster networks sharing the identical combinatorial summary produce materially different realized regrets for the ES rule on the same data-generating process.

Watch

Extended reading notes

Core claim

We propose an empirical success (ES) ranking rule that, for each pair of saturations, selects the saturation level with the higher estimated welfare using data from a two-stage randomized saturation design. We adopt the statistical decision theory framework with additively separable regret loss to assess the performance of the ES ranking rule. We derive non-asymptotic upper bounds on the maximum regret of the ES ranking rule that depend on the within-cluster network only through a single combinatorial summary of its dependency structure. We exploit these bounds to characterize a quasi-optimal first-stage saturation distribution within the two-stage randomized saturation design. We further sh

Load-bearing premise

The within-cluster network dependency structure can be summarized by a single combinatorial measure that suffices for the regret bounds.

Editorial extensions

If this is right

  • The regret bounds directly characterize a quasi-optimal distribution of first-stage saturation levels in the two-stage design.
  • The ES ranking rule is asymptotically optimal among threshold ranking rules with respect to an upper bound on worst-case regret.
  • Performance is evaluated inside the statistical decision theory framework using additively separable regret loss.
  • Ranking decisions remain feasible when only the single combinatorial summary is known rather than the full network.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the combinatorial summary can itself be estimated from pilot data, the bounds could support sequential redesign of the first-stage allocation.
  • The single-summary reduction may extend to other partial-observation interference models provided an analogous combinatorial quantity exists.
  • Threshold rules could be ranked against one another more generally by comparing the tightness of their respective regret upper bounds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes an empirical success (ES) ranking rule to select among a finite set of treatment saturations for populations subject to clustered network interference. Using data from a two-stage randomized saturation design and an additively separable regret loss, it derives non-asymptotic upper bounds on the maximum regret of the ES rule that depend on the within-cluster network solely through a single combinatorial summary of its dependency structure. These bounds are then used to characterize a quasi-optimal first-stage saturation distribution, and the ES rule is shown to be asymptotically optimal among threshold ranking rules in the sense of minimizing an upper bound on worst-case regret.

Significance. If the non-asymptotic bounds indeed isolate network dependence to a single combinatorial summary, the result would strengthen statistical decision theory approaches to interference by delivering finite-sample guarantees that do not require full knowledge of the adjacency matrix. The explicit use of a two-stage design and the focus on ranking rather than estimation are constructive contributions; the asymptotic optimality result among threshold rules provides a clear benchmark.

major comments (2)
  1. [Section deriving the non-asymptotic regret bounds] The central claim that the non-asymptotic upper bounds on maximum regret depend on the within-cluster network only through a single combinatorial summary (abstract) is load-bearing for the entire contribution. The derivation must explicitly isolate this summary and show that all other features of the adjacency matrix can be majorized or bounded away without inflating the finite-sample regret expression under the two-stage saturation design and additively separable loss; if the argument instead absorbs extra graph structure into universal constants or if the summary fails to dominate relevant interference terms for some admissible graphs, the reduction does not hold.
  2. [Section on first-stage design optimization] The characterization of the quasi-optimal first-stage saturation distribution (abstract) relies on the regret bounds; any looseness in the combinatorial summary would propagate directly into the recommended first-stage allocation and undermine the claim that the design is quasi-optimal.
minor comments (2)
  1. Notation for the combinatorial summary of the dependency structure should be introduced with an explicit definition and an example computation on a small graph to clarify what information is retained versus discarded.
  2. The abstract states that the ES rule is asymptotically optimal 'in the sense of minimizing an upper bound on the worst-case regret'; the precise sense in which this upper bound is minimized (e.g., rate, constant, or both) should be stated clearly in the introduction.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive feedback on our manuscript. Below we respond point by point to the major comments.

read point-by-point responses

  1. Referee: [Section deriving the non-asymptotic regret bounds] The central claim that the non-asymptotic upper bounds on maximum regret depend on the within-cluster network only through a single combinatorial summary (abstract) is load-bearing for the entire contribution. The derivation must explicitly isolate this summary and show that all other features of the adjacency matrix can be majorized or bounded away without inflating the finite-sample regret expression under the two-stage saturation design and additively separable loss; if the argument instead absorbs extra graph structure into universal constants or if the summary fails to dominate relevant interference terms for some admissible graphs, the reduction does not hold.

    Authors: We thank the referee for underscoring the centrality of this isolation. In the derivation (Section 3), the two-stage design and additively separable regret loss allow us to write the cluster-level regret as a sum of terms whose dependence on the adjacency matrix is controlled solely by the size of the largest interference neighborhood within each cluster; this quantity serves as the combinatorial summary. All other adjacency features are majorized by this summary because the separability of the loss and the randomization in the design bound any additional interference paths by the worst-case neighborhood size. No extra graph structure is absorbed into universal constants, and the bound holds uniformly over admissible graphs by construction of the summary. To address the referee's concern about explicitness, we will insert a dedicated remark immediately after the main bound statement that walks through this majorization step. revision: partial

  2. Referee: [Section on first-stage design optimization] The characterization of the quasi-optimal first-stage saturation distribution (abstract) relies on the regret bounds; any looseness in the combinatorial summary would propagate directly into the recommended first-stage allocation and undermine the claim that the design is quasi-optimal.

    Authors: We agree that the first-stage optimization in Section 4 is derived directly from the regret bounds of Section 3. Because those bounds isolate network dependence through the single combinatorial summary (as detailed in the response to the preceding comment), the resulting characterization of the quasi-optimal saturation distribution inherits the same isolation property and does not introduce additional looseness. We will add an explicit cross-reference in the revised Section 4 linking the optimization back to the isolation argument. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: theoretical regret bounds derived independently from external decision-theoretic framework.

full rationale

The paper derives non-asymptotic upper bounds on maximum regret of the ES ranking rule under additively separable regret loss and a two-stage randomized saturation design. The claimed reduction of network dependence to a single combinatorial summary is presented as the output of that derivation rather than an input assumption or fitted quantity. No equations or steps are shown to reduce by construction to self-citations, parameter fits, or definitional equivalences; the central result is self-contained against the external statistical decision theory framework with no load-bearing self-citation chains or ansatz smuggling.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper relies on standard decision theory and randomized design assumptions; no free parameters or invented entities are introduced in the abstract.

assumptions (2)
  • domain assumption Additively separable regret loss framework from statistical decision theory
    Invoked to assess ES ranking rule performance (abstract).
  • domain assumption Two-stage randomized saturation design is feasible and identifies relevant welfare quantities
    Used to generate data for the ranking rule (abstract).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ranking Treatment Saturations under Clustered Network Interference." pith.science (2026). https://pith.science/paper/ZWW2TGWQ

@misc{pith2026260618590,
  author       = {Pith},
  title        = {Pith review of: Ranking Treatment Saturations under Clustered Network Interference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZWW2TGWQ}},
  note         = {Machine review of arXiv:2606.18590}
}
read the original abstract

In this paper, we study how to rank a finite set of treatment saturations for a target population with clustered network interference. We propose an empirical success (ES) ranking rule that, for each pair of saturations, selects the saturation level with the higher estimated welfare using data from a two-stage randomized saturation design. We adopt the statistical decision theory framework with additively separable regret loss to assess the performance of the ES ranking rule. We derive non-asymptotic upper bounds on the maximum regret of the ES ranking rule that depend on the within-cluster network only through a single combinatorial summary of its dependency structure. We exploit these bounds to characterize a quasi-optimal first-stage saturation distribution within the two-stage randomized saturation design. We further show that the ES ranking rule is asymptotically optimal among threshold ranking rules in the sense of minimizing an upper bound on the worst-case regret.

Figures

Figures reproduced from arXiv: 2606.18590 by the authors.

Figure 1
Figure 1. Closed-form risk bounds vs. Monte Carlo risk [PITH_FULL_IMAGE:figures/full_fig_p033_1.png] view at source ↗
Figure 2
Figure 2. Manski bound validity across η1 with σϵ = 0.3. MC risk (black), Manski bound (orange), and the paper’s complete-graph bound (purple). K = 4, C = 12, η2 + η3 = 0.05. 0.00 0.05 0.10 0.15 0.20 0.25 η1 = 0.5 cluster size ni maximum regret 20 50 100 200 MC risk Manski bound Paper bound 0.0 0.1 0.2 0.3 0.4 0.5 0.6 η1 = 0.8 cluster size ni maximum regret 20 50 100 200 MC risk Manski bound Paper bound 0.0 0.2 0.4 0.6 0.8 1.… view at source ↗
Figure 3
Figure 3. Risk curves by allocation design (C = 60). The balanced design (black solid) achieves the lowest peak risk. K = 4, ni = 20, η1 = 0.3, σϵ = 0.2. 0.1 0.2 0.3 0.4 0.5 0.000 0.005 0.010 0.015 0.020 0.025 signal strength η2 + η3 MC maximum regret Balanced (1/4 each) Extreme tilt (.50/.20/.20/.10) Random Dirichlet(1,1,1,1) 7 Conclusion We develop a decision-theoretic framework for ranking treatment saturations under clus￾… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 1 canonical work pages

  1. [1]

    Kitagawa, and A

    Andrews, I., T. Kitagawa, and A. McCloskey (2024). Inference on winners. Quarterly Journal of Economics\/ 139\/ (1), 305--358

    2024

  2. [2]

    Athey, S. and S. Wager (2021). Policy learning with observational data. Econometrica\/ 89\/ (1), 133--161

    2021

  3. [3]

    Baird, S., J. A. Bohren, C. McIntosh, and B. \"O zler (2018). Optimal design of experiments in the presence of interference. Review of Economics and Statistics\/ 100\/ (5), 844--860

    2018

  4. [4]

    Banerjee, A., A. G. Chandrasekhar, E. Duflo, and M. O. Jackson (2013). The diffusion of microfinance. Science\/ 341\/ (6144), 1236498

    2013

  5. [5]

    Mogstad, J

    Bazylik, S., M. Mogstad, J. P. Romano, A. M. Shaikh, and D. Wilhelm (2025). Finite- and large-sample inference for ranks using multinomial data with an application to ranking political parties. Journal of Econometrics\/ 250 , 106010

    2025

  6. [6]

    Djebbari, and B

    Bramoull \'e , Y., H. Djebbari, and B. Fortin (2009). Identification of peer effects through social networks. Journal of Econometrics\/ 150\/ (1), 41--55

    2009

  7. [7]

    Chen, H. and P. Guggenberger (2025). A note on minimax regret rules with multiple treatments in finite samples. Econometric Theory\/ , 1--27

    2025

  8. [8]

    Cohen, A. and H. B. Sackrowitz (2005). Decision theory results for one-sided multiple comparison procedures. The Annals of Statistics\/ 33\/ (1), 126--144

    2005

Show all 43 references
  1. [9]

    Duflo, M

    Cr \'e pon, B., E. Duflo, M. Gurgand, R. Rathelot, and P. Zamora (2013). Do labor market policies have displacement effects? E vidence from a clustered randomized experiment. The Quarterly Journal of Economics\/ 128\/ (2), 531--580

    2013

  2. [10]

    Deaton, A. and N. Cartwright (2018). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine\/ 210 , 2--21

    2018

  3. [11]

    Duflo, E. (2004). Scaling up and evaluation. In Annual World Bank Conference on Development Economics , pp.\ 341--369

    2004

  4. [12]

    Haushofer, E

    Egger, D., J. Haushofer, E. Miguel, P. Niehaus, and M. Walker (2022). General equilibrium effects of cash transfers: experimental evidence from K enya. Econometrica\/ 90\/ (6), 2603--2643

    2022

  5. [13]

    Feller, W. (1991). An introduction to probability theory and its applications, Volume 2 , Volume 2. John Wiley & Sons

    1991

  6. [14]

    Godsil, C. and G. F. Royle (2001). Algebraic Graph Theory , Volume 207 of Graduate Texts in Mathematics . New York: Springer

    2001

  7. [15]

    Gu, J. and R. Koenker (2023). Invidious comparisons: Ranking and selection as compound decisions. Econometrica\/ 91\/ (1), 1--41

    2023

  8. [16]

    Hirano, K. and J. R. Porter (2009). Asymptotics for statistical treatment rules. Econometrica\/ 77\/ (5), 1683--1701

    2009

  9. [17]

    Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American statistical association\/ 58\/ (301), 13--30

    1963

  10. [18]

    Hudgens, M. G. and M. E. Halloran (2008). Toward causal inference with interference. Journal of the American Statistical Association\/ 103\/ (482), 832--842

    2008

  11. [19]

    Janson, S. (2004). Large deviations for sums of partly dependent random variables. Random Structures & Algorithms\/ 24\/ (3), 234--248

    2004

  12. [20]

    Kitagawa, T. and A. Tetenov (2018). Who should be treated? empirical welfare maximization methods for treatment choice. Econometrica\/ 86\/ (2), 591--616

    2018

  13. [21]

    Lehmann, E. L. (1957). A theory of some multiple decision problems, i. The Annals of Mathematical Statistics\/ 28\/ (1), 1--25

    1957

  14. [22]

    Liu, L. and M. G. Hudgens (2014). Large sample randomization inference of causal effects in the presence of interference. Journal of the American Statistical Association\/ 109\/ (505), 288--301

    2014

  15. [23]

    Manski, C. F. (1993). Identification of endogenous social effects: The reflection problem. The Review of Economic Studies\/ 60\/ (3), 531--542

    1993

  16. [24]

    Manski, C. F. (2004). Statistical treatment rules for heterogeneous populations. Econometrica\/ 72\/ (4), 1221--1246

    2004

  17. [25]

    Manski, C. F. and A. Tetenov (2016). Sufficient trial size to inform clinical practice. Proceedings of the National Academy of Sciences\/ 113\/ (38), 10518--10523

    2016

  18. [26]

    Manski, C. F. and A. Tetenov (2019). Trial size for near-optimal choice between surveillance and aggressive treatment: Reconsidering MSLT-II . The American Statistician\/ 73\/ (sup1), 305--311

    2019

  19. [27]

    Masten, M. A. (2023). Minimax-regret treatment rules with many treatments. The Japanese Economic Review\/ 74\/ (4), 501--537

    2023

  20. [28]

    Mbakop, E. and M. Tabord-Meehan (2021). Model selection for treatment choice: Penalized welfare maximization. Econometrica\/ 89\/ (2), 825--848

    2021

  21. [29]

    Miguel, E. and M. Kremer (2004). Worms: identifying impacts on education and health in the presence of treatment externalities. Econometrica\/ 72\/ (1), 159--217

    2004

  22. [30]

    Mogstad, M., J. P. Romano, A. M. Shaikh, and D. Wilhelm (2024). Inference for ranks with applications to mobility across neighbourhoods and academic achievement across countries. Review of Economic Studies\/ 91\/ (1), 476--518

    2024

  23. [31]

    Neyman, J. (1923). Sur les applications de la th \'e orie des probabilit \'e s aux experiences agricoles: Essai des principes. Roczniki Nauk Rolniczych\/ 10 , 1--51

    1923

  24. [32]

    Park, C., G. Chen, M. Yu, and H. Kang (2024). Minimum resource threshold policy under partial interference. Journal of the American Statistical Association\/ 119\/ (548), 2881--2894

    2024

  25. [33]

    Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology\/ 66\/ (5), 688--701

    1974

  26. [34]

    Scheinerman, E. R. and D. H. Ullman (2011). Fractional graph theory: a rational approach to the theory of graphs . Courier Corporation

    2011

  27. [35]

    Stoye, J. (2009). Minimax regret treatment choice with finite samples. Journal of Econometrics\/ 151\/ (1), 70--81

    2009

  28. [36]

    Tchetgen, E. J. T. and T. J. VanderWeele (2012). On causal inference in the presence of interference. Statistical methods in medical research\/ 21\/ (1), 55--75

    2012

  29. [37]

    Tetenov, A. (2012). Statistical treatment choice based on asymmetric minimax regret criteria. Journal of Econometrics\/ 166\/ (1), 157--165

    2012

  30. [38]

    Van der Vaart, A. (1991). An asymptotic representation theorem. International Statistical Review\/ 59\/ (1), 97--121

    1991

  31. [39]

    Van der Vaart, A. W. (1998). Asymptotic Statistics . Number 3 in Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press

    1998

  32. [40]

    Viviano, D. (2022). Policy design in experiments with unknown interference. Technical report, working paper

    2022

  33. [41]

    Viviano, D. (2025). Policy targeting under network interference. Review of Economic Studies\/ 92\/ (2), 1257--1292

    2025

  34. [42]

    Wald, A. (1950). Statistical Decision Functions . New York: Wiley

    1950

  35. [43]

    Zhang, Y. and K. Imai (2023). Individualized policy evaluation and learning under clustered network interference. arXiv preprint arXiv:2311.02467\/

    2023

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.