{"id":"04f56f60-0f20-4470-88c9-0a658a5c7f98","arxiv_id":"2606.18590","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Develops an empirical success ranking rule for treatment saturations under clustered network interference and derives non-asymptotic regret bounds depending on a single combinatorial network summary.","lead":"The paper proposes an empirical success ranking rule to select among treatment saturation levels when interference occurs within clusters, using data from a two-stage randomized saturation design. A generalist might read it for guidance on ranking policy options in networked settings such as villages or schools where spillovers matter.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Whether non-asymptotic regret bounds reduce to dependence on only one combinatorial network summary requires explicit verification in the derivation.","rationale":"The reader's weakest assumption directly identifies the same point. Because the full derivation is not reproduced here, the concrete simulation test is the minimal check that would confirm or refute whether the claimed reduction is valid. This moves the verdict from UNVERDICTED to CONDITIONAL pending that check; the rest of the argument (asymptotic optimality among threshold rules) is downstream of this step.","tokens_in":1610,"tokens_out":366,"duration_ms":18948,"concrete_test":"Fix a cluster size and two distinct within-cluster graphs G1 and G2 that share the same value of the paper's combinatorial summary but differ in edge placement. For each, run Monte Carlo simulation of the two-stage design, compute the finite-sample maximum regret of the ES rule over a grid of saturations, and test whether the regrets differ by more than sampling error. A statistically significant difference falsifies the single-summary claim.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim is that upper bounds on maximum regret of the ES ranking rule depend on the within-cluster network solely through a single combinatorial summary of its dependency structure. For this to hold, the proof must show that all other aspects of the adjacency matrix (beyond that one scalar) can be bounded away without affecting the finite-sample regret expression under the two-stage saturation design and additively separable loss. If the argument instead uses a loose majorization that absorbs extra structure into constants, or if the summary (whatever its definition) fails to dominate the relevant interference terms for some graphs, the reduction does not go through. The abstract states the dependence but does not exhibit the step that isolates the summary.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes an empirical success (ES) ranking rule to select among a finite set of treatment saturations for populations subject to clustered network interference. Using data from a two-stage randomized saturation design and an additively separable regret loss, it derives non-asymptotic upper bounds on the maximum regret of the ES rule that depend on the within-cluster network solely through a single combinatorial summary of its dependency structure. These bounds are then used to characterize a quasi-optimal first-stage saturation distribution, and the ES rule is shown to be asymptotically optimal among threshold ranking rules in the sense of minimizing an upper bound on worst-case regret.","tokens_in":1758,"tokens_out":560,"duration_ms":17149,"significance":"If the non-asymptotic bounds indeed isolate network dependence to a single combinatorial summary, the result would strengthen statistical decision theory approaches to interference by delivering finite-sample guarantees that do not require full knowledge of the adjacency matrix. The explicit use of a two-stage design and the focus on ranking rather than estimation are constructive contributions; the asymptotic optimality result among threshold rules provides a clear benchmark.","major_comments":[{"comment":"The central claim that the non-asymptotic upper bounds on maximum regret depend on the within-cluster network only through a single combinatorial summary (abstract) is load-bearing for the entire contribution. The derivation must explicitly isolate this summary and show that all other features of the adjacency matrix can be majorized or bounded away without inflating the finite-sample regret expression under the two-stage saturation design and additively separable loss; if the argument instead absorbs extra graph structure into universal constants or if the summary fails to dominate relevant interference terms for some admissible graphs, the reduction does not hold.","section":"Section deriving the non-asymptotic regret bounds"},{"comment":"The characterization of the quasi-optimal first-stage saturation distribution (abstract) relies on the regret bounds; any looseness in the combinatorial summary would propagate directly into the recommended first-stage allocation and undermine the claim that the design is quasi-optimal.","section":"Section on first-stage design optimization"}],"minor_comments":[{"comment":"Notation for the combinatorial summary of the dependency structure should be introduced with an explicit definition and an example computation on a small graph to clarify what information is retained versus discarded.","section":null},{"comment":"The abstract states that the ES rule is asymptotically optimal 'in the sense of minimizing an upper bound on the worst-case regret'; the precise sense in which this upper bound is minimized (e.g., rate, constant, or both) should be stated clearly in the introduction.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the careful reading and constructive feedback on our manuscript. Below we respond point by point to the major comments.","responses":[{"response":"We thank the referee for underscoring the centrality of this isolation. In the derivation (Section 3), the two-stage design and additively separable regret loss allow us to write the cluster-level regret as a sum of terms whose dependence on the adjacency matrix is controlled solely by the size of the largest interference neighborhood within each cluster; this quantity serves as the combinatorial summary. All other adjacency features are majorized by this summary because the separability of the loss and the randomization in the design bound any additional interference paths by the worst-case neighborhood size. No extra graph structure is absorbed into universal constants, and the bound holds uniformly over admissible graphs by construction of the summary. To address the referee's concern about explicitness, we will insert a dedicated remark immediately after the main bound statement that walks through this majorization step.","revision_made":"partial","referee_comment":"[Section deriving the non-asymptotic regret bounds] The central claim that the non-asymptotic upper bounds on maximum regret depend on the within-cluster network only through a single combinatorial summary (abstract) is load-bearing for the entire contribution. The derivation must explicitly isolate this summary and show that all other features of the adjacency matrix can be majorized or bounded away without inflating the finite-sample regret expression under the two-stage saturation design and additively separable loss; if the argument instead absorbs extra graph structure into universal constants or if the summary fails to dominate relevant interference terms for some admissible graphs, the reduction does not hold."},{"response":"We agree that the first-stage optimization in Section 4 is derived directly from the regret bounds of Section 3. Because those bounds isolate network dependence through the single combinatorial summary (as detailed in the response to the preceding comment), the resulting characterization of the quasi-optimal saturation distribution inherits the same isolation property and does not introduce additional looseness. We will add an explicit cross-reference in the revised Section 4 linking the optimization back to the isolation argument.","revision_made":"partial","referee_comment":"[Section on first-stage design optimization] The characterization of the quasi-optimal first-stage saturation distribution (abstract) relies on the regret bounds; any looseness in the combinatorial summary would propagate directly into the recommended first-stage allocation and undermine the claim that the design is quasi-optimal."}],"tokens_in":1350,"tokens_out":521,"duration_ms":22870,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main contribution here is the empirical success ranking rule for choosing among treatment saturations in a two-stage clustered design, paired with finite-sample regret bounds that the authors say depend on the within-cluster network only through a single scalar summary of its dependency structure. They also give a characterization of a good first-stage distribution and show asymptotic optimality among threshold rules under additively separable regret.\n\nWhat stands out is the move to statistical decision theory for this problem. The bounds are non-asymptotic and the reduction to one combinatorial measure, if it holds tightly, would be useful for design. The abstract frames the loss properly and avoids overclaiming generality beyond clustered settings.\n\nThe soft spot is exactly the one the stress-test flags: whether the regret expression really factors so that everything except that one summary can be bounded away without inflating the constants in a way that makes the result loose for realistic graphs. The abstract states the dependence but does not show the isolation step. If the argument uses majorization that absorbs extra adjacency structure, the practical value of the single-summary claim drops. Post-hoc design choices for the two-stage experiment are also not detailed here.\n\nThis is for people already working on saturation experiments or network interference in development economics and causal inference. A reader who wants a formal ranking tool with explicit finite-sample guarantees will get something usable if the bound derivation checks out. It is worth sending to referees because the framework is grounded and the central claim is falsifiable once the proofs are examined.","headline":"The paper gives a clean decision-theoretic ranking rule for saturation levels under clustered interference, with non-asymptotic regret bounds that reduce to one combinatorial network summary, but the isolation step in the proof is the part that needs direct verification.","tokens_in":2194,"tokens_out":387,"would_cite":false,"duration_ms":7787,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"An empirical success ranking rule bounds maximum regret for choosing treatment saturations using only one combinatorial summary of cluster network dependency.","keywords":["treatment saturation ranking","clustered network interference","empirical success rule","regret bounds","two-stage randomized design","statistical decision theory","asymptotic optimality"],"falsifier":"Empirical observation that two distinct within-cluster networks sharing the identical combinatorial summary produce materially different realized regrets for the ES rule on the same data-generating process.","tokens_in":2523,"feed_emoji":"","tokens_out":730,"duration_ms":28758,"temperature":0.7,"pith_summary":"The paper develops a way to rank a finite set of treatment saturation levels when units interfere through networks inside clusters. It proposes an empirical success ranking rule that picks the higher-welfare saturation for each pair by comparing estimates from a two-stage randomized saturation experiment. The central technical result is a set of non-asymptotic upper bounds on the rule's maximum regret; these bounds depend on the interference structure only through a single combinatorial summary of the within-cluster dependency graph. The same bounds are used to identify a quasi-optimal allocation of first-stage saturation levels and to prove that the rule is asymptotically optimal among all threshold ranking rules when judged by worst-case regret. The approach therefore lets researchers rank saturations without needing the complete network adjacency matrix.","feed_headline":"Single combinatorial summary bounds regret for saturation ranking","feed_subtitle":"Empirical success rule achieves non-asymptotic bounds via two-stage designs and identifies quasi-optimal first-stage allocations.","key_machinery":"The empirical success (ES) ranking rule that pairwise compares estimated welfares, together with the single combinatorial summary of within-cluster dependency structure that governs the derived regret bounds.","core_discovery":"We propose an empirical success (ES) ranking rule that, for each pair of saturations, selects the saturation level with the higher estimated welfare using data from a two-stage randomized saturation design. We adopt the statistical decision theory framework with additively separable regret loss to assess the performance of the ES ranking rule. We derive non-asymptotic upper bounds on the maximum regret of the ES ranking rule that depend on the within-cluster network only through a single combinatorial summary of its dependency structure. We exploit these bounds to characterize a quasi-optimal first-stage saturation distribution within the two-stage randomized saturation design. We further sh","pith_inferences":["If the combinatorial summary can itself be estimated from pilot data, the bounds could support sequential redesign of the first-stage allocation.","The single-summary reduction may extend to other partial-observation interference models provided an analogous combinatorial quantity exists.","Threshold rules could be ranked against one another more generally by comparing the tightness of their respective regret upper bounds."],"forward_implications":["The regret bounds directly characterize a quasi-optimal distribution of first-stage saturation levels in the two-stage design.","The ES ranking rule is asymptotically optimal among threshold ranking rules with respect to an upper bound on worst-case regret.","Performance is evaluated inside the statistical decision theory framework using additively separable regret loss.","Ranking decisions remain feasible when only the single combinatorial summary is known rather than the full network."],"fun_headline_variants":["Combinatorial summary bounds regret of saturation ranking rule","Two-stage saturation design yields quasi-optimal first-stage allocations","Non-asymptotic upper bounds on regret via dependency summary","Asymptotic optimality of ES ranking among threshold rules"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The within-cluster network dependency structure can be summarized by a single combinatorial measure that suffices for the regret bounds.","fun_headline_variants_meta":{"raw":{"variants":["Combinatorial summary bounds regret of saturation ranking rule","Two-stage saturation design yields quasi-optimal first-stage allocations","Non-asymptotic upper bounds on regret via dependency summary","Asymptotic optimality of ES ranking among threshold rules"]},"model":"grok-4.3","cost_usd":0.006738,"raw_usage":{"total_tokens":3037,"prompt_tokens":630,"num_sources_used":0,"completion_tokens":59,"cost_in_usd_ticks":67378000,"prompt_tokens_details":{"text_tokens":630,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2348,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":630,"tokens_out":59,"duration_ms":15337,"temperature":1.0,"reasoning_tokens":2348,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T19:21:17.569234+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Empirical observation that two distinct within-cluster networks sharing the identical combinatorial summary produce materially different realized regrets for the ES rule on the same data-generating process.","supporting_citations":[],"review_version":1}