{"id":"27fae8b6-77b2-4dad-9aa8-5eee6c95acd5","arxiv_id":"2606.08923","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Develops scalable clustering methods for network-aware A/B testing in two-sided markets that cut spillover while boosting sample size and power, plus a theoretical bias correction.","lead":"This paper introduces EgoCluster V3 and MultiEgoCluster, iterative clustering algorithms for designing experiments in networked two-sided marketplaces to reduce spillover effects from treatment interference. The methods aim to improve statistical power and include a graph-based bias correction for average treatment effect estimation, with reported production use at LinkedIn.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Whether iterative clustering on the ego graph can isolate spillover without substantial coverage loss or new bias depends on unstated graph properties.","rationale":"The reader’s weakest_assumption directly identifies the same structural precondition on the graph that must hold for both the algorithmic gains and the bias correction to be valid. Because the full text was not available to the reader, the current UNVERDICTED status already reflects this uncertainty; the concrete test above would resolve it without changing the verdict category.","tokens_in":1750,"tokens_out":352,"duration_ms":11730,"concrete_test":"Re-run the EgoCluster V3 and MultiEgoCluster procedures on the paper’s own production graph (or a faithful subsample) while recording the exact spillover metric, number of qualifying clusters, and effective sample size before and after each iteration; if any of the three headline percentages deviates by more than 20% from the reported figures, the isolation-without-loss assumption fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline empirical claims (3× spillover reduction, doubled power, +38% sample size) and the bias-correction derivation both presuppose that the underlying bipartite graph admits an iterative clustering procedure whose output clusters are sufficiently isolated yet numerous. If the graph is dense, has heavy-tailed degrees, or contains many cross-side bridges, the procedure either retains high residual spillover or discards too many nodes, violating the “preserving node coverage” part of the claim. The theoretical correction is stated to be graph-based, so the same structural assumption is required for it to remove the remaining bias rather than introduce a new one. No independent verification of these graph-dependent conditions is supplied beyond the LinkedIn deployment.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript claims that EgoCluster V3, an iterative clustering algorithm on ego graphs, reduces spillover by 3x relative to prior versions while preserving node coverage and doubling test power in two-sided marketplace experiments. MultiEgoCluster extends this via a two-stage multi-ego grouping procedure, yielding an additional ~56% spillover reduction and ~38% sample-size increase. Both algorithms are deployed in production at LinkedIn. The paper further derives a graph-structure-based theoretical bias correction for ATE estimation to address residual interference and proposes a method to generalize results to the broader population.","tokens_in":1892,"tokens_out":507,"duration_ms":18314,"significance":"If the empirical performance claims and the bias-correction derivation hold under the stated graph assumptions, the work would be significant for causal inference in networked two-sided markets: it directly tackles SUTVA violations via scalable clustering that trades off spillover against power, and the production deployment supplies real-world evidence of utility. The attempt to supply a graph-based correction is a positive step toward generalizability. The manuscript receives credit for the LinkedIn deployment demonstrating practical impact.","major_comments":[{"comment":"Abstract and bias-correction section: the central claim that a 'theoretical bias correction method for ATE estimation based on graph structure' enables generalization to the general population is load-bearing, yet the manuscript supplies no equations, derivation steps, or proof sketch showing whether the correction is independent of quantities fitted from the same experimental data or reduces to a data-dependent adjustment.","section":"Abstract / bias-correction section"},{"comment":"EgoCluster V3 and MultiEgoCluster sections: the headline claims (3× spillover reduction, doubled power, +38% sample size, preservation of node coverage) presuppose that the underlying bipartite graph admits iterative clustering whose output clusters remain sufficiently isolated yet numerous; no analysis of graph properties (degree distribution, density, cross-side bridges) or sensitivity checks is provided to establish that these conditions hold beyond the specific LinkedIn deployment.","section":"EgoCluster V3 / MultiEgoCluster sections"}],"minor_comments":[{"comment":"Abstract: numerical claims (3×, ~56%, ~38%, doubled power) are stated without reference to the specific baseline methods, tables, or figures that support them.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We are grateful for the referee's positive evaluation of the significance and practical impact of our work on network-aware experiment design in two-sided marketplaces. We provide point-by-point responses to the major comments below.","responses":[{"response":"We agree that the bias-correction section would benefit from greater mathematical detail. The manuscript derives the correction from the observed graph structure to address residual interference after clustering, but we will expand it in revision to include the explicit equations, derivation steps, and clarification that the adjustment depends only on graph properties rather than quantities estimated from the experimental outcomes.","revision_made":"yes","referee_comment":"[Abstract / bias-correction section] Abstract and bias-correction section: the central claim that a 'theoretical bias correction method for ATE estimation based on graph structure' enables generalization to the general population is load-bearing, yet the manuscript supplies no equations, derivation steps, or proof sketch showing whether the correction is independent of quantities fitted from the same experimental data or reduces to a data-dependent adjustment."},{"response":"The reported performance metrics are empirical results obtained on LinkedIn's production bipartite graph, where the algorithms were deployed at scale. The clustering procedures are constructed around ego-graph isolation properties that are characteristic of two-sided marketplace networks. We will add a discussion of the relevant graph properties (degree distribution, density, and cross-side connectivity) observed in the LinkedIn data and the conditions under which the iterative procedure preserves isolation and coverage.","revision_made":"partial","referee_comment":"[EgoCluster V3 / MultiEgoCluster sections] EgoCluster V3 and MultiEgoCluster sections: the headline claims (3× spillover reduction, doubled power, +38% sample size, preservation of node coverage) presuppose that the underlying bipartite graph admits iterative clustering whose output clusters remain sufficiently isolated yet numerous; no analysis of graph properties (degree distribution, density, cross-side bridges) or sensitivity checks is provided to establish that these conditions hold beyond the specific LinkedIn deployment."}],"tokens_in":1450,"tokens_out":403,"duration_ms":26760,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that the authors built EgoCluster V3 as an iterative clustering procedure on ego graphs that they report reduces spillover by 3x over prior versions, keeps node coverage, and doubles test power. MultiEgoCluster adds a two-stage step that first bundles highly connected egos, then applies the iteration, for another 56% spillover drop and 38% sample size gain. They also derive a bias correction for ATE that uses the graph structure to adjust for residual interference, and both methods are live in LinkedIn production for two-sided marketplace tests.\n\nWhat works is the direct attack on the coverage-versus-isolation trade-off that cluster randomization usually faces in bipartite settings like job seekers and posters. The iterative approach and the multi-ego pre-grouping are specific extensions that target the practical constraint of keeping enough units while limiting cross-side spillovers. The production deployment supplies some external grounding that the methods scale and deliver usable experiments.\n\nThe soft spots sit in the missing technical layer. The abstract states the performance numbers and the existence of a derived correction but shows no equations, no pseudocode for the clustering, no simulation or sensitivity results, and no breakdown of how the bias term is computed or whether it stays independent of the experimental outcomes. The stress-test point about graph properties lands: if the underlying network is dense or has many bridges, the procedure could either leave more spillover than claimed or drop too many clusters, and the correction would then rest on the same untested structural assumptions. The LinkedIn case is useful but does not substitute for checks on other graphs or explicit error bounds.\n\nThis is for applied causal-inference people working on platform experiments with network interference. A reader running marketplace tests could pull the algorithmic ideas and try them directly; a theorist would want the derivations filled in first.\n\nIt deserves a serious referee because the problem is real, the production evidence is concrete, and the extensions are specific enough to review. I would send it to peer review and ask for the clustering details, the exact form of the bias correction, and some robustness checks on graph variation.","headline":"EgoCluster V3 and MultiEgoCluster give concrete iterative clustering steps that cut spillover while holding coverage and power, plus a graph-based bias fix, already running at LinkedIn, but the abstract leaves the derivations and checks thin.","tokens_in":2347,"tokens_out":516,"would_cite":false,"duration_ms":19715,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Iterative ego clustering cuts spillover threefold in two-sided marketplace experiments while doubling test power.","keywords":["causal inference","experiment design","two-sided marketplaces","spillover effects","network interference","clustering algorithms","average treatment effect","A/B testing"],"falsifier":"A controlled deployment or simulation in which direct measurement of cross-side interactions shows no reduction in spillover after applying EgoCluster V3 or MultiEgoCluster, or in which the bias-corrected ATE differs from the known true effect.","tokens_in":2646,"feed_emoji":"📊","tokens_out":686,"duration_ms":17388,"temperature":0.7,"pith_summary":"The paper addresses the problem that treatment applied to one side of a marketplace spills over through interactions to the untreated side, violating SUTVA and biasing causal estimates. Standard cluster randomization reduces spillover only by shrinking the number of usable clusters and thereby losing power. The authors introduce EgoCluster V3, which iteratively refines clusters around ego nodes to achieve three times lower spillover without loss of node coverage and with twice the statistical power. They then present MultiEgoCluster, a two-stage extension that first forms multi-ego groups and yields an extra 56 percent spillover cut plus 38 percent larger sample size. A graph-structure correction for remaining bias in average treatment effect estimation is also derived so results can be generalized beyond the experiment sample.","feed_headline":"Clustering cuts spillover 3x in marketplace experiments","feed_subtitle":"Iterative ego grouping preserves coverage and doubles power while a graph correction removes residual bias from treatment-effect estimates.","key_machinery":"EgoCluster V3, an iterative clustering procedure that repeatedly refines ego-centered clusters to isolate cross-side interactions.","core_discovery":"EgoCluster V3 is an iterative clustering algorithm that reduces spillover by a factor of three relative to earlier versions while preserving node coverage and doubling test power; MultiEgoCluster extends this via a two-stage multi-ego grouping procedure to obtain a further 56 percent spillover reduction and 38 percent sample-size increase; a theoretical bias-correction formula based on the observed graph structure then removes residual interference bias from the ATE estimator.","pith_inferences":["The same iterative clustering logic could be tested on time-varying graphs where edges appear and disappear during the experiment window.","If the bias correction proves accurate, the method may reduce the minimum cluster size required in other interference settings such as social or recommendation networks.","The two-stage multi-ego grouping could be combined with stratification on node attributes to further improve balance across treatment arms."],"forward_implications":["Marketplace experiments can maintain higher statistical power at any given level of allowable interference.","One-sided treatments can be tested with reduced contamination of the opposite side.","The bias-correction step allows extrapolation of results from the clustered sample to the full population.","Production systems can run more frequent, smaller, or more precise tests without violating interference assumptions."],"fun_headline_variants":["EgoCluster V3 reduces spillover 3x doubles test power","MultiEgoCluster cuts spillover 56% raises sample size 38%","Iterative ego clustering mitigates interference in markets","Graph bias correction removes residual ATE interference","Network-aware design boosts power in two-sided experiments"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The underlying network must permit iterative ego clustering to isolate spillover without losing too many qualifying clusters or introducing new selection biases, and the graph-based correction must fully capture any remaining interference.","fun_headline_variants_meta":{"raw":{"variants":["EgoCluster V3 reduces spillover 3x doubles test power","MultiEgoCluster cuts spillover 56% raises sample size 38%","Iterative ego clustering mitigates interference in markets","Graph bias correction removes residual ATE interference","Network-aware design boosts power in two-sided experiments"]},"model":"grok-4.3","cost_usd":0.00375,"raw_usage":{"total_tokens":1950,"prompt_tokens":685,"num_sources_used":0,"completion_tokens":77,"cost_in_usd_ticks":37499500,"prompt_tokens_details":{"text_tokens":685,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1188,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":685,"tokens_out":77,"duration_ms":10356,"temperature":1.0,"reasoning_tokens":1188,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T14:55:35.791667+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A controlled deployment or simulation in which direct measurement of cross-side interactions shows no reduction in spillover after applying EgoCluster V3 or MultiEgoCluster, or in which the bias-corrected ATE differs from the known true effect.","supporting_citations":[],"review_version":1}