Lost in Aggregation: The Causal Interpretation of the IV Estimand

Danielle Tsao; Emilija Perkovi\'c; Frederick Eberhardt; Krikamol Muandet

Instrumental-variable estimates of a treatment that is an aggregate of components with heterogeneous effects generally cannot be interpreted as the causal effect of the treatment; only proportional aggregation or instrument-tuned interventi

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 06:19 UTC pith:BC5CUFXA

load-bearing objection A clean, honest formalization of when IV estimands match aggregate causal effects, with the central conditional claim holding and limitations explicitly scoped.

arxiv 2601.12120 v2 pith:BC5CUFXA submitted 2026-01-17 stat.ME stat.ML

Lost in Aggregation: The Causal Interpretation of the IV Estimand

Danielle Tsao , Krikamol Muandet , Frederick Eberhardt , Emilija Perkovi\'c This is my paper

classification stat.ME stat.ML MSC 62D2062P20

keywords instrumental variablesaggregate treatmentcausal effect ambiguityexclusion restrictionstructural causal modelstwo-stage least squarestreatment variation invarianceACID

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that when a treatment like education, GDP, or caloric intake is a sum of finer-grained components with different causal effects, the standard IV estimand is not, in general, the causal effect of the aggregate. The authors formalize the aggregate causal effect through a distribution over how the intervention is realized at the component level (the ACID) and prove that the IV estimand matches this effect only if either the components have proportional effects on the outcome or the intervention distribution is deliberately tuned to the instrument's component-wise effects. Outside these two scenarios, the gap between the IV estimate and the aggregate causal effect can be arbitrarily large. Because the aggregate setting is distributionally equivalent to a violation of the exclusion restriction in linear Gaussian models, standard diagnostics cannot detect the problem. The paper concludes that IV analyses with aggregate treatments require explicit, defensible justification of one of these two conditions before estimates can guide policy.

Core claim

The paper formalizes a linear structural causal model in which the treatment A is a weighted sum of unobserved components A_j, each with its own effect β_j on Y and its own response δ_j to the instrument I. The standard IV estimand is β_IV = (Σ_j β_j δ_j) / (Σ_j α_j δ_j). Defining the aggregate causal effect ACE(A,Y) as a unit intervention effect under a 'valid ACID', the paper shows ACE = Σ_j β_j d_j for Gaussian ACIDs, and that β_IV equals ACE only if the components satisfy proportional aggregation (β_j/α_j constant for all j) or the ACID's slopes d_j are aligned with the instrument effects δ_j (an 'instrument-tuned intervention'). Otherwise the difference between β_IV and ACE is unbounded

What carries the argument

The aggregate-constrained component intervention distribution (ACID) is the distribution over the unobserved components given an intervention on the aggregate. A 'valid' ACID must be surgical (each component set independently of its causes), respect the aggregation rule, and yield value-independent effects. The ACID carries the argument because the aggregate causal effect is defined as an expectation under it, and matching the IV estimand to that effect reduces to a constraint on the ACID's slope parameters d_j.

Load-bearing premise

The aggregate causal effect is defined only under a 'valid ACID' whose surgicality requires that an intervention on the aggregate acts as a clean, independent intervention on each component, severing their links to the instrument and confounder; if real policy interventions on the aggregate do not have this surgical character, the aggregate causal effect is not well-defined and the paper's contrast does not apply.

What would settle it

Generate data from a linear Gaussian SCM with two components, α1=α2=1, β1=1, β2=2, δ1=δ2=1, and a confounder U affecting both components. Define the aggregate causal effect using the natural Gaussian ACID that sets components to their observational conditional distribution given A=a. The population IV estimand is (1+2)/(1+1)=1.5; the ACE computed from the conditional mean of components given A is (β1 cov(A1,A)+β2 cov(A2,A))/var(A), which depends on the confounder and instrument strengths and is generally not 1.5. A reader can pick parameter values and compute both quantities to see the mismatc

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

In applied IV studies with aggregate treatments—education, GDP, caloric intake, BMI—the reported coefficient cannot be read as the causal effect of the treatment unless proportional aggregation is assumed or an instrument-tuned ACID is justified.
Under proportional aggregation, the IV estimand equals the aggregate effect regardless of how the intervention is instantiated; this is the only ACID-independent scenario.
When aggregation is not proportional, the linear Gaussian aggregate setting is distributionally equivalent to a violation of the exclusion restriction, so standard tests of instrument validity cannot detect the problem.
The Sargan test can offer evidence for proportional aggregation only when instruments are strong and the type-I error is set high; a non-rejection does not imply proportional aggregation.
Non-linear models do not rescue the interpretation: they lose value independence and make the aggregate causal effect still more sensitive to the ACID.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

An editorial extension: the failure described is an identifiability failure at the population level, not a finite-sample artifact, so collecting more data or adding stronger instruments will not close the gap.
A practical diagnostic follows: if component-level data are available, one can directly estimate β_j and δ_j to check whether β_j/α_j are equal; if they are not, the IV estimate should be reported together with an explicit ACID assumption.
The equivalence to an exclusion violation suggests that sensitivity analyses for direct instrument effects could double as aggregation-robustness checks, provided the plausible magnitude of component heterogeneity can be bounded.
For Mendelian randomization, where instruments are genetic variants and treatments like BMI are aggregates, the paper implies that gene-exposure effects on components must be homogeneous or the ACID must be variant-specific—an unlikely condition across multiple genetic instruments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A clean, honest formalization of when IV estimands match aggregate causal effects, with the central conditional claim holding and limitations explicitly scoped.

read the letter

Colleague,

This paper deserves a serious referee. The core contribution is the explicit formalization of the aggregate-constrained component intervention distribution (ACID) and the derivation of two precise conditions under which the standard linear IV estimand equals the aggregate causal effect: proportional aggregation and instrument-tuned interventions. The negative result—that outside these conditions the difference between the IV estimand and the aggregate causal effect is unbounded—is what applied people will remember. It is correct, and it is genuinely new in this form. The existing literature on treatment variation invariance and aggregation bias does not make the IV identification conditions explicit, and Proposition 1 cleanly shows that aggregation is observationally equivalent to a violation of the exclusion restriction. That is a useful bridge to a familiar diagnostic.

The math in Sections 2–4 is straightforward linear algebra and checks out. The covariance matching in Appendix D is careful. The paper is also honest about what it does not do: the definition of the aggregate causal effect depends on a researcher-chosen ACID, and the surgicality assumption is strong. Both limitations are acknowledged in Section 3.3, and they delimit the scope rather than undermine the conditional claim. The simulations are illustrative rather than load-bearing; the lack of error bars is a minor presentation issue, not a substantive flaw. I would not hold that against the paper.

The Sargan test simulation is a nice practical addition. It confirms that with weak instruments, passing Sargan gives almost no support for proportional aggregation—useful caution for empirical work.

Who should read this: applied IV users in economics and epidemiology, and methodologists working on treatment effect heterogeneity and multiple versions of treatment. The writing is clear enough that an applied reader can follow the main argument without grinding through every derivation.

My recommendation: send it to peer review. The theory is sound, the claims are scoped, and the practical implications are important. Minor revision would be appropriate—mostly to make the simulation presentation a bit more standard—but the core should be accepted.

In short: a solid paper, not a flashy one. Worth engaging with.

Referee Report

0 major / 5 minor

Summary. The paper studies instrumental variable (IV) estimation when the treatment A is an aggregate of k unobserved components A_1,...,A_k with possibly heterogeneous effects on the outcome. In the linear SCM of Section 2, the standard IV estimand is derived as β_IV = (Σ_j β_j δ_j)/(Σ_j α_j δ_j). The paper introduces the Aggregate-Constrained Component Intervention Distribution (ACID) to define the aggregate causal effect (ACE) as the effect of a unit intervention on A instantiated through the components, requiring surgicality, the aggregation restriction, and value independence. Under a Gaussian ACID the ACE is Σ_j β_j d_j with Σ_j α_j d_j = 1. The central result is that β_IV equals ACE only under two special conditions: proportional aggregation (β_j/α_j constant) or an 'instrument-tuned' ACID in which the component slopes d_j align with the normalized instrument effects δ_j. Otherwise, the difference can be arbitrarily large. Section 5 shows that, in the linear Gaussian case, the aggregate setting is observationally equivalent to a violation of the exclusion restriction, and demonstrates that the Sargan test cannot generally certify proportional aggregation. The paper concludes that IV estimates of aggregate treatments require strong, usually unjustified, assumptions.

Significance. If accepted, this paper makes a substantive contribution to the causal-inference literature on treatments with multiple versions and aggregation. It provides a clean formalization of the aggregate causal effect, a precise algebraic characterization of when IV identifies that effect, and a striking equivalence result connecting aggregation to exclusion-restriction violation in the Gaussian case. The derivation of β_IV and ACE is transparent and reproducible from the manuscript; the identification condition in Eq. (19) is a genuine constraint rather than a tautology. The Sargan-test analysis is a useful practical warning that passing an overidentification test does not establish proportional aggregation. The main message—that IV analyses with aggregate treatments need either proportional aggregation or an explicit, defensible ACID—is likely to influence applied practice. The principal limitation, acknowledged by the authors, is that the ACE is defined only under a valid ACID, and the surgicality requirement may not match real interventions on aggregates; however, this caveat is stated explicitly and does not undermine the conditional theoretical claim.

minor comments (5)

[§4.3, Figure 2] The simulation results appear to be based on a single dataset per configuration. Reporting the number of replications and adding confidence intervals or standard-error bands would make the convergence claims visually and statistically supported. The same applies to Figure 4, where 100 datasets are used but no uncertainty measures are shown.
[§5, first paragraph] The phrase 'the difference ... is unbounded' could be misread as a fixed-model property. What is established is that, across the class of models satisfying the aggregate setting but not the two sufficient conditions, the difference can be made arbitrarily large. Rewording to 'can be made arbitrarily large' or adding a formal statement would prevent overstatement.
[§4.2, Eq. (20)] The instrument-tuned solution d_j = δ_j/(Σ α_j δ_j) is one particular solution to Eq. (19), but the text says 'the most straightforward to see.' It may be worth noting explicitly that other solutions exist and are equally contrived, so that the argument does not depend on this specific choice.
[§5.1, Figure 4] The recommendation to use a large type-I error (0.5) for the Sargan test as a diagnostic tool is based on simulation evidence without formal justification. Adding a sentence that this is a heuristic diagnostic, not a significance test with calibrated error control, would improve the presentation.
[Throughout] There are a few typographical and formatting issues: 'highights' in the introduction, 'Hernan' without the accent in the reference list is inconsistent with standard spelling, and some equations have slightly awkward inline notation. These do not affect the technical content.

Circularity Check

0 steps flagged

No significant circularity: the IV-vs-ACE mismatch is a derived conditional characterization, not a fitted prediction or self-citation chain.

full rationale

The derivation chain is self-contained. Section 2 defines the aggregate SCM (1) and computes the IV estimand β_IV = (Σ β_j δ_j)/(Σ α_j δ_j) in Eq. (3) by direct covariance algebra. Section 3 defines the target aggregate causal effect via an explicit ACID, giving ACE = Σ β_j d_j under the Gaussian ACID in Eq. (15). Section 4 then solves the equality β_IV = ACE as Eq. (19); proportional aggregation and instrument-tuned interventions (Eq. (20)) are consequences of that equality, not assumptions relabeled as conclusions. The paper explicitly calls the instrument-tuned ACID 'highly contrived,' so it is not presenting the match as an empirical prediction. The unbounded-difference claim in Section 5 follows from the freedom in d_j subject only to Σ α_j d_j = 1, which is a mathematical fact about the defined target, not a fitted input. The simulations in Sections 4.3 and 5.1 are illustrative diagnostics, not load-bearing for the theorems. The only self-citation (Beckers et al. 2020, involving an author) appears in related-work discussion and is not used to justify any assumption or result. Acknowledged limitations—Section 3.3's discussion of non-surgical interventions and Section 6's discussion of nonlinearity—are scope conditions on the definition of ACE, not circular moves; the paper's central claim is explicitly conditional on a valid ACID. No step reduces to its own input.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical or empirical entity. The ACID is a formal distributional construct chosen by the researcher to represent an intervention policy; it is a modeling assumption rather than a postulated object with falsifiable consequences. Therefore the graviton problem does not apply.

free parameters (1)

d_j (ACID component slopes) = d_j = δ_j / Σ_j α_j δ_j for instrument-tuned interventions (eq. 20); otherwise unspecified in the general ACID
The Gaussian ACID in Section 3.2 parameterizes the intervention on components by c_j + a d_j. The aggregate causal effect is Σ β_j d_j. For the IV estimand to equal the ACE, d_j must satisfy the linear constraint (19); the simplest solution (20) ties d_j to the instrument effects δ_j. These slopes are chosen by the modeller, not estimated from data, and the paper's negative result largely restates

axioms (4)

domain assumption Linear structural causal model with additive independent errors (SCM (1), eqs. 1a-1d)
The entire analysis is conducted in a linear IV setting; the paper states this is the regime underlying much applied practice.
domain assumption The aggregation rule A = Σ α_j A_j is definitional and cannot be broken by intervention (eq. 1c)
The paper distinguishes constitutive aggregation from causal relations; if the aggregation rule could change under intervention, the target quantity would differ.
ad hoc to paper A valid ACID must satisfy surgicality, aggregation restriction, and value independence (Section 3.1)
These three desiderata are introduced by the authors to make the aggregate causal effect well-defined. They are strong assumptions and are not derived from or tested against any external benchmark.
domain assumption For Proposition 1, errors are mutually independent standard Gaussians (Section 5, Proposition 1)
The covariance-matching proof in Appendix D relies on Gaussianity; non-Gaussian distributions are not covered by the equivalence result.

pith-pipeline@v1.3.0-alltime-deepseek · 20348 in / 12308 out tokens · 123052 ms · 2026-08-04T06:19:01.656924+00:00 · methodology

0 comments

read the original abstract

Instrumental variable estimation has emerged as a standard approach to mitigating confounding bias in the social sciences and epidemiology, where conducting randomized experiments can be too costly or infeasible. However, justifying the validity of the instrument is frequently challenging. We highlight a problem generally neglected in arguments for instrumental variable validity: the presence of an "aggregate treatment variable", where the treatment (e.g., education, GDP, caloric intake) is composed of finer-grained, unobserved components that each may have a different effect on the outcome. While the aggregation problem itself is general, our focus is on instrumental variable estimation in a linear setting, the regime underlying much of applied IV practice. We show that the causal effect of an aggregate treatment is generally ambiguous, as it depends on how an intervention on the aggregate is instantiated at the component level. We formalize this relation using the aggregate-constrained component intervention distribution (ACID). We then identify two key conditions under which standard instrumental variable estimators identify the aggregate effect. The contrived nature of these conditions implies major limitations on the interpretation of instrumental variable estimates based on aggregate treatments and highlights the need for a broader justificatory base for the exclusion restriction in such settings.

Figures

Figures reproduced from arXiv: 2601.12120 by Danielle Tsao, Emilija Perkovi\'c, Frederick Eberhardt, Krikamol Muandet.

**Figure 2.** Figure 2: A: IV estimates for different samples sizes [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

**Figure 3.** Figure 3: A violation of the exclusion restriction that is equivalent to the aggregate setting [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

**Figure 4.** Figure 4: Empirical power of the Sargan test for two strong instruments (red), one strong and [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗

**Figure 5.** Figure 5: Distribution of F-statistics for pairs of instruments of different strengths. The typical cut-off of 11 is marked in yellow where applicable. E.1 Calculations for [PITH_FULL_IMAGE:figures/full_fig_p030_5.png] view at source ↗

**Figure 6.** Figure 6: Aggregate setting with aggregate outcome [PITH_FULL_IMAGE:figures/full_fig_p031_6.png] view at source ↗

**Figure 7.** Figure 7: Aggregate setting with aggregate instrument [PITH_FULL_IMAGE:figures/full_fig_p032_7.png] view at source ↗

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Estimating Causal Effects from Data Generated by Stochastic Algorithms
stat.ME 2026-07 accept novelty 8.0

Logging the features and relative probability of one unexposed item alongside the exposed item identifies causal effects of content features from stochastic algorithms even with unobserved confounders.

Reference graph

Works this paper leans on

3 extracted references · cited by 1 Pith paper

[1]

Acemoglu, D., Johnson, S., Robinson, J. A. and Yared, P. (2008) Income and democracy. American Economic Review,98, 808–842. 20 Angrist, J. D., Imbens, G. W. and Rubin, D. B. (1996) Identification of causal effects using instrumental variables.Journal of the American Statistical Association,91, 444–455. Angrist, J. D. and Krueger, A. B. (1991) Does compuls...

arXiv 2008
[1217]

reduced stage

Zhu, Y., Gultchin, L., Gretton, A., Kusner, M. J. and Silva, R. (2022) Causal inference with treatment measurement error: A nonparametric instrumental variable approach. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, 2414–2424. A Full IV Calculations In classic (non-aggregate) 2SLS estimation,Ais first regressed onIand the lin...

2022
[1223]

(2019) Two-stage least squares as minimum distance.The Econometrics Journal,22, 1–9

Windmeijer, F. (2019) Two-stage least squares as minimum distance.The Econometrics Journal,22, 1–9. Ye, T., Ertefaie, A., Flory, J., Hennessy, S. and Small, D. S. (2023) Instrumented difference- in-differences.Biometrics,79, 569–581. Zhu, Y., Budhathoki, K., K¨ ubler, J. M. and Janzing, D. (2024) Meaningful causal aggregation and paradoxical confounding. ...

2019

[1] [1]

Acemoglu, D., Johnson, S., Robinson, J. A. and Yared, P. (2008) Income and democracy. American Economic Review,98, 808–842. 20 Angrist, J. D., Imbens, G. W. and Rubin, D. B. (1996) Identification of causal effects using instrumental variables.Journal of the American Statistical Association,91, 444–455. Angrist, J. D. and Krueger, A. B. (1991) Does compuls...

arXiv 2008

[2] [1217]

reduced stage

Zhu, Y., Gultchin, L., Gretton, A., Kusner, M. J. and Silva, R. (2022) Causal inference with treatment measurement error: A nonparametric instrumental variable approach. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, 2414–2424. A Full IV Calculations In classic (non-aggregate) 2SLS estimation,Ais first regressed onIand the lin...

2022

[3] [1223]

(2019) Two-stage least squares as minimum distance.The Econometrics Journal,22, 1–9

Windmeijer, F. (2019) Two-stage least squares as minimum distance.The Econometrics Journal,22, 1–9. Ye, T., Ertefaie, A., Flory, J., Hennessy, S. and Small, D. S. (2023) Instrumented difference- in-differences.Biometrics,79, 569–581. Zhu, Y., Budhathoki, K., K¨ ubler, J. M. and Janzing, D. (2024) Meaningful causal aggregation and paradoxical confounding. ...

2019