REVIEW 2 major objections 4 minor 1 cited by
Causal inference with dyadic data in randomized experiments
T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Pairwise outcomes, weighted by their randomization probabilities, give unbiased estimates of the global average treatment effect even when unit-level estimators fail.
desk verdict Solid new framework for dyadic outcomes under unit-level randomization, but the complete-randomization CLT proof has a gap that needs fixing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the dyadic interference assumption $Y_{ij}(\mathbf{z})=Y_{ij}(z_i,z_j)$, which converts network interference into a pair-local structure while still allowing arbitrary interference at the unit level. Estimation rests on dyad inclusion probabilities $p_{ij}=\Pr(Z_i=Z_j=1)$ and $q_{ij}=\Pr(Z_i=Z_j=0)$, which are known from the randomization design, making the Horvitz-Thompson estimator a difference of two inverse-probability-weighted dyad sums. The rates are governed by the counterfactual network degree summaries $d_1(z)$, $d_2(z)$, and $d_\infty(z)$—average, mean-squared, and maximum degree of the network that would form if all units received $z$—because dyads sharing a unit are dependent. The central limit theorem is proved with Stein's method on dependency neighbourhoods for Bernoulli randomization and transferred to complete randomization by Hajek coupling; the conservative variance estimator $\widehat{V}(\Delta)$ uses Young's inequality to dominate cross-world product terms, selecting $\Delta$ at least as large as the maximum counterfactual degree.
What would settle it
Take a small population, say $n=3$, choose bounded potential outcomes $Y_{ij}(z_i,z_j)$ satisfying Assumption 1, and enumerate all $2^3=8$ treatment assignments under Bernoulli randomization with $p_i=0.3$. Theorem 2 requires the exact expectation of $\widehat{\tau}_{\mathrm{HT}}$ over these assignments to equal $\tau=n^{-1}\sum_i\{U_i(1)-U_i(0)\}$; if the enumeration shows any nonzero discrepancy, the unbiasedness claim is false. To test the robustness boundary, repeat the enumeration with $Y_{ij}$ shifted by $\zeta Z_k$ for a third unit $k$, and compare the estimator's expectation with $\tau$: the gap quantifies the bias that higher-order interference would introduce.
Extended reading notes
Core claim
The central claim, stated on the paper's own terms, is that under Assumption 1, $Y_{ij}(\mathbf{z})=Y_{ij}(z_i,z_j)$ for $i\neq j$, the global average treatment effect $\tau=n^{-1}\sum_i\{U_i(1)-U_i(0)\}$ is estimable from dyadic outcomes. The estimator $\widehat{\tau}_{\mathrm{HT}}=n^{-1}\sum_i\sum_{j\in D_i}(Z_{ij}/p_{ij}-\bar{Z}_{ij}/q_{ij})Y_{ij}$ contrasts treated-treated dyads with control-control dyads and is unbiased under Bernoulli and complete randomization, with convergence rate $r_n=\max_{z\in\{0,1\}}\{P_z^{-1}d_1(z)^{1/2}, P_z^{-1/2}d_2(z)^{1/2}\}n^{-1/2}$; the Hajek version has the same rate. A central limit theorem holds under a maximum-degree condition, and the paper constructs asymptotically conservative variance estimators using a sensitivity parameter $\Delta$. The same framework shows that unit-level estimators $\tilde{\tau}(Y)=n^{-1}\sum_i(Z_i/a_i-(1-Z_i)/b_i)Y_i$ are biased and inconsistent in general, becoming unbiased only in the special case $a_i=b_i=p_i=1/2$.
Load-bearing premise
The load-bearing premise is that each pairwise outcome is affected only by the treatments of the two people in the pair; if influences from friends-of-friends or wider parts of the network reach that outcome, the proposed estimators are generally biased, and the paper leaves formal sensitivity bounds to future work.
Editorial extensions
If this is right
- Unit-level Horvitz-Thompson estimators are generally biased and inconsistent for the global average treatment effect under dyadic interference, while the dyadic estimators are unbiased and consistent in the same settings.
- Under Bernoulli and complete randomization, both estimators converge at rate $r_n$; the supplement shows cluster randomization attains the same rate when inter-cluster leakage is negligible.
- The convergence rate worsens as the network becomes denser: when the mean squared degree $d_2(z)$ grows with $n$, consistency slows, and a star-like degree distribution can destroy consistency entirely.
- With $\Delta\ge\max_z d_\infty(z)$, the Wald confidence intervals based on $\widehat{V}(\Delta)$ are asymptotically conservative, so they protect nominal coverage even when the unadjusted variance estimator undercovers.
- In the WeChat Channels experiment, dyad-based estimates of relative treatment effects (1.012%, 0.624%, 1.124%) exceed unit-level estimates (0.076%, -0.120%, 0.285%), changing the statistical conclusion for two of the three metrics when the unadjusted variance estimator is used.
Reading between the lines
- Because the dyadic estimators tolerate arbitrary known assignment probabilities, dyad-level data weaken the experimental-design pressure toward balanced 50/50 splits; this is directly relevant when treatment is costly and skewed allocation is standard.
- The same inverse-probability weighting logic should carry over to observational studies with estimated assignment probabilities—the paper lists this as future work—and would give dyadic analogues of propensity-score weighting.
- A formal sensitivity analysis for violations of dyadic interference, which the paper only sketches, could be built by bounding the higher-order spillover coefficient $\zeta$ and propagating the resulting worst-case bias through the same Young's-inequality machinery used for the variance estimator.
- Before applying the method, checking the observed degree distribution is wise: large hubs and near-star networks signal both inflated variance and potential failure of the central limit theorem, and the paper's Assumptions 4 and 5 make this diagnostic transparent.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a design-based causal inference framework for randomized experiments with dyadic (pairwise) outcomes. Under a dyadic-interference assumption (Assumption 1), it defines the global average treatment effect, constructs Horvitz-Thompson and Hajek estimators based on dyadic outcomes, and derives convergence rates under Bernoulli, complete, and cluster randomization. It further develops conservative variance estimators and Wald-type confidence intervals, and reports simulation studies and a WeChat field experiment. The main theorems are stated for both Bernoulli and complete randomization, with complete-randomization results proved via Hajek coupling in the supplementary material.
Significance. If the results hold, the paper is a useful contribution to causal inference under network interference: it provides a clean formulation of dyadic interference, shows the bias of naive unit-level estimators, gives explicit convergence rates in terms of network degree statistics, and provides a conservative variance estimator with a transparent tuning parameter. The proofs in the supplement are detailed and the main consistency and unbiasedness results are directly tied to the stated assumptions. The empirical application and simulations are appropriate for illustrating the method. The main weakness is a specific gap in the proof of the complete-randomization central limit theorem (Theorem 4), which the stress-test concern identifies correctly; this does not undermine the Bernoulli-randomization results or the consistency claims, but it does affect a load-bearing theoretical claim.
major comments (2)
- [Supplementary B.4, Theorem 4] The Hajek-coupling step for complete randomization is not established. The proof asserts that combining (S.14) and (S.16) gives var(hat_tau_HT - tilde_tau_HT)/var(hat_tau_HT) = o(1), but (S.16) only shows the numerator is O(max_z d1(z)^2 n^{-1}) (up to lower-order terms). Under Assumption 4, the denominator only satisfies var(hat_tau_HT) >> max_z d_infty(z)^2 n^{-4/3}. For a regular graph with degree d = n^alpha and 0 < alpha < 1/3, one has d1 = d_infty = d, the natural variance scale is d^2/n, Assumption 4 holds because d n^{-2/3} / (d n^{-1/2}) = n^{-1/6} = o(1), and the coupling bound is also of order d^2/n, so the displayed inequalities do not force the required o(1) ratio. The complete-randomization half of Theorem 4 therefore rests on an unproved assertion. The Bernoulli-randomization CLT and the consistency results appear valid; the gap is specific to this coupling argument.
- [Section 4, Theorem 6] The statement of Theorem 6 includes Assumption 5, but the proof of asymptotic conservativeness requires the additional condition Delta = o(max_z n^{1/2} d2(z)^{1/2}) in addition to Delta >= max_z d_infty(z). These two conditions can be incompatible: in a star network, max_z d_infty(z) is of order n while max_z n^{1/2} d2(z)^{1/2} is also of order n, so no Delta can satisfy both strict inequalities, and the theorem as stated does not cover such cases. The paper should either state the additional growth condition explicitly in the theorem or clarify the scope of the guarantee.
minor comments (4)
- [Section 1] The text contains a typographical error: 'SUTV A' should read 'SUTVA'.
- [Section 7] The section heading 'Dicussion' should be 'Discussion'.
- [Supplementary B.2] In the complete-randomization variance calculation, the denominator in one covariance expression is written as p_ij p_jk; this appears to be a typo for p_ij p_kl. The surrounding argument is otherwise clear.
- [Section 2, equation (2)] The equality tau = n^{-1} sum_i {U_i(1) - U_i(0)} = n^{-1} sum_i {D_i(1) - D_i(0)} is used without proof. It is true by exchanging the order of summation, but a one-line justification would help readers because the neighbor sets are treatment-dependent.
Circularity Check
No circularity: the dyadic Horvitz-Thompson and Hajek estimators are derived directly from the potential-outcome model and known randomization weights, with no fitted parameters or load-bearing self-citations.
full rationale
The paper's derivation chain is self-contained and does not reduce any prediction to its inputs by construction. The estimators tau_hat_HT and tau_hat_HA in equations (4) and (5) are explicit linear functionals of observed dyadic outcomes with known design-based inclusion probabilities; unbiasedness follows from E[Z_ij/p_ij] = 1 and E[bar-Z_ij/q_ij] = 1 under Assumptions 1 and 2. The convergence rates in Theorems 2 and 3 are obtained by bounding the variance of these fixed-weight estimators in terms of degree statistics d_1(z) and d_2(z); no parameter is fitted to data and no estimated quantity is later relabeled as a prediction. The variance estimator bV(Delta) in equation (8) uses a user-specified sensitivity parameter Delta, and Theorem 6 proves conservatism under the explicit condition Delta >= max_z d_infty(z); the suggested choices for Delta (a known graph maximum degree or a plug-in estimate) are practical calibrations, not fitted predictions that the theorem then redisovers. Assumption 4 is a high-level variance-growth condition used to obtain a central limit theorem, not an assumption that already contains the conclusion. The paper also does not rely on load-bearing self-citations: references such as Ross (2011) and Hajek (1960) are standard external mathematical tools. The only substantive weakness suggested by a critical reading, a possible gap in the complete-randomization Hajek-coupling proof in Supplementary B.4 around equation (S.16), is a mathematical correctness concern rather than a circular dependency: the disputed step concerns whether the displayed bounds imply the required o(1) variance ratio, not whether an input was defined in terms of an output. Accordingly, no circularity is found.
Assumptions & free parameters
free parameters (1)
- Delta (tuning parameter in variance estimator bV(Delta)) =
user-specified, e.g., maximum observed degree
assumptions (5)
- domain assumption Assumption 1 (Dyadic interference): Y_ij(z) = Y_ij(z_i, z_j) for i != j, and Y_ii(z) = 0.
- domain assumption Assumption 2 (Positivity): P1 = min_i p_i > 0 and P0 = min_i q_i > 0 for all n.
- domain assumption Assumption 3 (Bounded potential outcomes): max_{i,j,z} |Y_ij(z)| <= K for some constant K.
- domain assumption Assumption 4 (Degree condition for CLT): max_{z in {0,1}} d_infinity(z) n^{-2/3} var(tau_hat_HT)^{-1/2} = o(1).
- domain assumption Assumption 5 (Degree condition for variance estimator): max_{z in {0,1}} {d_infinity(z) d_2(z)^{-1/2}} n^{-1/2} = o(1).
Cite this review
Pith. "Pith review of Causal inference with dyadic data in randomized experiments." pith.science (2026). https://pith.science/paper/LKJQNPY7
@misc{pith2026250520780,
author = {Pith},
title = {Pith review of: Causal inference with dyadic data in randomized experiments},
year = {2026},
howpublished = {\url{https://pith.science/paper/LKJQNPY7}},
note = {Machine review of arXiv:2505.20780}
}
read the original abstract
Estimating treatment effects in networked settings is a central challenge in online controlled experiments, particularly on social media platforms. We investigate a scenario where the unit-level outcome of interest comprises a series of dyadic outcomes that record pairwise interactions between units, spanning from point-to-point messaging at the microscale to bilateral trade flows at the macroscale. Because the response is defined at the dyadic level, the treatment assigned to one unit can affect the outcomes of all dyads that involve it, inducing a form of network interference. We propose a design-based causal inference framework for randomized experiments with dyadic outcomes. Within this framework, we propose estimators of the global average treatment effect under Bernoulli, complete, and cluster randomization, derive the convergence rates, and establish a central limit theorem for Bernoulli randomization. We further construct a class of variance estimators that are asymptotically conservative under transparent degree conditions. Numerical studies show that the proposed estimators can reduce bias and mean squared error relative to estimators based on unit-level outcomes in a range of finite-sample settings. We illustrate the methods using two large-scale experiments on WeChat, evaluating the impact of a recommendation algorithm and a calling feature.
Figures
Forward citations
Cited by 1 Pith paper
-
Causal Estimation of Share-Induced Engagement with Flywheel Effects
A flow-balance identity yields a closed-form geometric-amplification estimator of the global treatment effect of sharing features under network flywheel interference, with consistency under homogeneity and valid A/A i...
Reference graph
Works this paper leans on
-
[1]
For the upstream aggregated potential outcomeU i(z), it can be rewritten as Ui(z) = X j̸=i Yji(zj, zi) =U i(0) +α izi + X j̸=i (ηjizj +λ jizizj). The expectation of the estimator ˜τ(U) in (3) is E{˜τ(U)}= 1 n nX i=1 E ZiUi ai − (1−Z i)Ui bi = 1 n nX i=1 E Zi ai − 1−Z i bi Ui(Z) = 1 n nX i=1 E Zi ai − 1−Z i bi Ui(0) +α iZi + X j̸=i (ηjiZj +λ jiZji) = 1 n n...
-
[2]
Similarly, for the downstream aggregated potential outcomeD i(z), we have Di(z) = X j̸=i Yij(zi, zj) =D i(0) +β izi + X j̸=i (γijzj +λ ijzizj), and the expectation of ˜τ(D) is E{˜τ(D)}= 1 n nX i=1 (ai +b i)pi aibi − 1 bi Di(0) + pi ai βi + X j̸=i (ai +b i)pij aibi − pj bi γij + X j̸=i pij ai λij . We also match each coefficient from the above expectation ...
-
[3]
For the unit-level aggregated potential outcomesY i(z), Yi(z) =Y i(0) + (αi +β i)zi + X j̸=i (ηjizj +λ jizizj +γ ijzj +λ ijzizj). Then the expectation of ˜τ(Y) is E{˜τ(Y)}=E{˜τ(D) + ˜τ(U)}= 1 n nX i=1 (ai +b i)pi aibi − 1 bi Yi(0) + pi ai (αi +β i) + X j̸=i (ai +b i)pij aibi − pj bi (γij +η ji) + X j̸=i pij ai (λij +λ ji) . The following equations hold du...
work page 2021
-
[4]
The variance decomposition terms in (S.3) can be categorised by subscript overlap patterns: (A)
(Bernoulli randomization) We letζ i =q i/pi denote the odds of treated probability of unitiandζ ij = (1−p ij)/pij denote the odds of treated probability that unit (i, j) are both treated. The variance decomposition terms in (S.3) can be categorised by subscript overlap patterns: (A). Both subscripts overlap: (a). Ifi=kandj=l, we have cov ZijYij pij , ZklY...
-
[5]
Next, we consider each covariance term in (S.3) under complete randomization
(Complete randomization) It is direct to show that the estimator ˆτHT is unbiased. Next, we consider each covariance term in (S.3) under complete randomization. Akin to the proof of Bernoulli randomization, there are three categories in terms of subscript overlap: (A). The subscripts are identical: (a). Ifi=kandj=l, we have cov ZijYij pij , ZklYkl pkl = v...
work page 2011
-
[6]
(Bernoulli randomization) If the treatment probabilitiesP 1, P0 are bounded, then the upper bound in Lemma S.2 can be further reduced into d(ˆτHT, Z)2 ≲var(ˆτHT)−2n−3 max z∈{0,1} {d1(z)d∞(z)3}max 1,var(ˆτHT)−1n−1 max z∈{0,1} {d1(z)d∞(z)} . Sinced 1(z)≤d ∞(z), and var(ˆτHT)−1/2n−2/3 maxz∈{0,1} d∞(z) =o(1), the Wasserstein dis- tance d(ˆτHT, Z)≲var(ˆτHT)−3/...
-
[7]
Given some fixedn1, we letT= (T 1, T2,
(Complete randomization) For complete randomized treatment vectorZ, we first intro- duce the H´ ajek coupling technique between Bernoulli sampling and simple random sampling 46 (H´ ajek, 1960). Given some fixedn1, we letT= (T 1, T2, . . . , Tn)∈ {0,1}n be the indicator vector for a Bernoulli random sample with inclusion probabilityp i =n −1n1 for all unit...
work page 1960
-
[8]
If none of the subscripts is identical, i.e.,{i, j} ∩ {k, l}=∅, then the covariance is cov ZijS2 ij 2p2 ij , ZklS2 kl 2p2 kl = 0, sinceZ ij andZ kl are independent random variables under Bernoulli randomization
Show all 16 references
-
[9]
Combining (S.18) and (S.19), we have var 1 n nX i=1 X j̸=i ZijS2 ij 2p2 ij ! ≤ 1 n2 nX i=1 |Ni(1)|2 + 1 n2 nX i=1 X j∈Ni(1) |Nj(1)| C5K 4 =O d2(1)n−1
Otherwise, if at least one of the subscripts in{i, j}and{k, l}overlaps, for instance, (i=k, j̸=l), it can be upper bounded by cov ZijS2 ij 2p2 ij , ZklS2 kl 2p2 kl = Sij(1,1) 2Skl(1,1) 2 cov Zij 2p2 ij , Zkl 2p2 kl ≤C 5K 4, (S.19) whereC 5 is a constant. Combining (S.18) and (...
-
[10]
The covariance is cov ZijkSijSik pijpik , ZlmnSlmSln plmpln = 0, 50 sinceZ ijk andZ lmn are independent random variables under Bernoulli randomization
If none of the subscripts is identical, i.e.,{i, j, k} ∩ {l, m, n}=∅. The covariance is cov ZijkSijSik pijpik , ZlmnSlmSln plmpln = 0, 50 sinceZ ijk andZ lmn are independent random variables under Bernoulli randomization
-
[11]
For (S.20), we have var 1 n nX i=1 X j̸=i X k̸=i k̸=j ZijkSijSik pijpik ≤ 1 n2 nX i=1 X j∈Ni(1) X k∈Ni(1)\{j} C6K 4d2 ∞(1) =O d2(1)d2 ∞(1)n−1
Otherwise, if at least one of the subscripts overlaps, i.e.,{i, j, k} ∩ {l, m, n} ̸=∅, then it can be upper bounded by cov ZijkSijSik pijpik , ZlmnSlmSln plmpln ≤16K 4 cov Zijk pijpik , Zlmn plmpln ≤C 6K 4, whereC 6 is another constant. For (S.20), we have var 1 n nX i=1 X j̸=...
-
[12]
There are PL u=1 ICu (1) 2 choices of dyads (i, j) and (k, l) in this case
When units (i, j, k, l) are from the same clusterCu, we have cov ZijYij pij , ZklYkl pkl ≤ (1−p u) pu ≤P −1 1 . There are PL u=1 ICu (1) 2 choices of dyads (i, j) and (k, l) in this case
-
[13]
There are PL u=1 ICu(1)∂Cu(1) choices of dyads (i, j) and (k, l) in this case
When three of the subscripts (i, j, k, l) are from the same clusterCu while the remaining one is from the another clusterC v, we have cov ZijYij pij , ZklYkl pkl ≤ (1−p u) p2 u ≤P −2 1 . There are PL u=1 ICu(1)∂Cu(1) choices of dyads (i, j) and (k, l) in this case
-
[14]
(i, k) are from the same cluster while and (j, l) are from another cluster: cov ZijYij pij , ZklYkl pkl ≤ (1−p upv) pupv ≤P −2 1
When two of the subscripts are from the same clusterC u while the other two are from another clusterC v: (a). (i, k) are from the same cluster while and (j, l) are from another cluster: cov ZijYij pij , ZklYkl pkl ≤ (1−p upv) pupv ≤P −2 1 . 53 (b). (i, j) are from the same clu...
-
[15]
Case 3.(a) and 4 together have less than PL u=1 ∂Cu(1)2 choices of dyads (i, j) and (k, l) in this case
When two of the subscripts are from the same clusterC u while the other two are from the different clustersC v andC w, e.g., (i, k) are from the same cluster while and (j, l) are from different clusters: cov ZijYij pij , ZklYkl pkl ≤ (1−p u) pu ≤P −1 1 . Case 3.(a) and 4 toget...
-
[16]
When four subscripts are from the four different clusters: cov ZijYij pij , ZklYkl pkl = 0. Combining the above five cases, the variance of ˆµ1 under cluster randomization can be upper bounded by var(ˆµ1)≲n −2 LX u=1 ICu(1) 2 +∂ Cu(1)2 P −1 1 + LX u=1 ICu(1)∂Cu(1) +∂ Cu(1)2 P ...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.