Pith. sign in

REVIEW 3 major objections 4 minor 21 references

Positive-Unlabeled Learning for Control Group Construction in Observational Causal Inference

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Positive-unlabeled learning can identify control units from an unlabeled pool using only treated units, and the recovered controls yield average treatment effect estimates that closely track the true effect—if the PU learner is allowed to…

desk verdict The paper asks a good question, but its headline evidence is built on a setup that cannot expose its own failure mode, and the selection-on-outcome issue is real and unresolved. read the letter →

arxiv 2507.14528 v1 pith:XRTVNXWV submitted 2025-07-19 cs.LG

Ilias Tsoumas , Dimitrios Bormpoudakis , Vasileios Sitokonstantinou , Athanasios Askitopoulos , Andreas Kalogeras , Charalampos Kontoes , Ioannis Athanasiadis This is my paper · ORCID
classification cs.LG MSC 62D2062H30
keywords positive-unlabeledlearningcontrolgroupconstructionaveragetreatmenteffectobservationalcausalinferencepropensityscoregraphdigitalagricultureearthobservation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes using positive-unlabeled (PU) learning to build a control group when no confirmed control units exist, so that an average treatment effect can still be estimated from treated units plus an unlabeled pool. The central claim is that a two-step PU procedure—SPY followed by iterative SVM—recovers control units with high precision, and that training the PU learner on an expanded feature set that includes the outcome and other post-treatment variables works far better than using only the back-door adjustment set. On simulated linear data the method yields an ATE of 3.021 against a true value of 3.000, and on nonlinear data 9.719 against 9.525; on real agricultural data it reproduces the expected positive effect of optimal sowing and of digestate fertilization. If this holds, it enables quasi-experiments in earth, environmental, and agricultural sciences wherever treatment has been recorded but controls have not.

What carries the argument

The machinery is a two-stage PU pipeline. In stage one, the SPY method selects some real treated units as 'spies', hides them in the unlabeled set, trains a Naive Bayes classifier treating unlabeled units as controls, and labels as reliable controls any unit whose control posterior is lower than the lowest spy posterior. In stage two, iterative SVM (iSVM) retrains an SVM on treated plus the reliable controls, and repeatedly absorbs unlabeled units classified as controls until convergence. The feature set X is the key design choice: it is a superset of the adjustment set Z and may contain 'bad controls' and the outcome, which the paper argues yields much better separation of true controls from treated units.

What would settle it

In the fertilization dataset, SPY+iSVM trained on X flags 8 of 11 hidden treated units as controls, a treated leakage of 0.727; if removing those leaked units from the selected 'reliable controls' changes the reported ATE of 1.750 (linear regression, p=0.049) to non-significance, the claim that PU controls yield valid estimates in this real case fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that a control group can be constructed from unlabeled data using only the treated/positive units, by treating control identification as a PU learning task and deliberately allowing the PU learner to use a feature set X that goes beyond the back-door adjustment set Z—including mediators, colliders, and the outcome Y itself. The paper shows that SPY+iSVM trained on X recovers control groups with near-zero contamination in the two simulations, and that ATE estimates computed from these recovered controls closely track the estimates obtained from true controls. The authors frame this as a preparatory step that is decoupled from effect estimation, so any causal estimator and any adjustment set can be used downstream.

Load-bearing premise

The load-bearing premise is that the PU learner can use the outcome variable—and other post-treatment covariates—to identify controls without thereby biasing the later effect estimate; if picking controls on the outcome secretly selects units with low potential outcomes, the near-true ATEs are not credible.

Editorial extensions

If this is right

  • In settings with no labeled controls, PU learning can supply a control group from unlabeled units, making ATE estimation possible where it was previously infeasible.
  • Using the full informative feature set X, rather than only the adjustment set Z, is what makes control recovery accurate; restricting the PU learner to Z leads to severe underestimation and loss of statistical significance.
  • The approach is estimator-agnostic: after controls are recovered, any propensity-score-based or outcome-based causal estimator can be applied.
  • In agricultural applications, the method recovers expected positive effects of sowing-date recommendations and digestate fertilization, pointing toward quasi-experiments on earth-observation data.
  • The reported ATEs are local, not population-level: propensity trimming confines the estimand to a low- to moderate-propensity subpopulation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the best results come from training the PU learner on the outcome Y itself, an open risk is that 'reliable controls' are simply units with low outcomes; a natural test is whether a placebo treatment with no true effect yields a spuriously non-zero ATE under this pipeline.
  • The same PU scores could be used as continuous weights for the control group instead of a hard label threshold, potentially avoiding some of the trimming-induced selection bias the paper acknowledges.
  • The method is likely most trustworthy when the unlabeled pool genuinely contains many controls, as in the simulations; when the pool is dominated by treated units—as in the fertilization run with 8 of 11 hidden treated units leaked into the selected controls—the ATE estimates should be treated with caution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a two-step framework in which positive-unlabeled (PU) learning is used to construct a control group from unlabeled units when no confirmed controls are available, after which a standard ATE estimator is applied. The method combines the SPY heuristic with iterative SVM (iSVM), and the authors evaluate it on two synthetic datasets (linear and nonlinear DGPs with a known causal graph) and two real agricultural datasets (sowing decision and digestate fertilization). The headline results are that, with an expanded feature set X that includes the outcome variable Y, SPY+iSVM yields ATE estimates close to the true effects in the simulations (3.021 vs. 3.000; 9.719 vs. 9.525), while real-data results are less stable and are qualified by the authors as uncertain due to treated leakage.

Significance. The conceptual contribution is useful and practically motivated: decoupling control-group construction from effect estimation would allow any causal estimator to be used when confirmed controls are absent, and the paper tests the idea on real agricultural data with standard estimators. The simulation DGP in Appendix B is transparently specified, and the use of four different ATE estimators is a strength. However, the current evidence for the central claim is confined to a regime where PU selection is nearly perfect; the paper does not address the selection-on-outcome bias that can arise when Y is included in the PU feature set and selection is imperfect. The oracle benchmark in the linear simulation also fails to recover the true effect, which weakens the calibration of the comparison. These issues are central rather than cosmetic, so the manuscript needs substantive revision before the claim can be accepted.

major comments (3)
  1. [§3.2, Tables 1–2] The central claim that PU-selected controls yield ATE estimates close to the truth is supported only in the two simulation rows where SPY+iSVM is trained on the expanded feature set X that includes the outcome Y (Table 3 lists Yield and ndvi_trapezoidal_area in X). In those rows the selection is near-perfect (Table 1: control recall 1.000/0.992, control precision 1.000, treated leakage 0), so the selected set is essentially the full control population and any dependence of selection on Y cannot manifest as selection bias. In the real-data rows with imperfect selection (sowing and fertilization: recall 0.835/0.878, leakage 0.333/0.727), the ATE estimates in Table 2 are unstable. Formally, since S=1 (selected as a reliable control) is a function of X and X contains Y, E[Y|T=0,S=1,Z] need not equal E[Y|T=0,Z]; the two-step de-linking described in Section 3.1 does not repair this because the bias is in the selected sample, not in the estimator. The paper needs either a formal condition under which selection on Y is harmless, or a simulation that exercises imperfect selection with Y in X.
  2. [§4 and Table 2] The comparison of trimmed PU-based estimates to the global true effect is not apples-to-apples. Section 4 states that asymmetric trimming to the propensity-score range [0.1, 0.6] is applied in the sowing and fertilization datasets, and it acknowledges that this changes the estimand to a local ATE; the text also indicates that all PU rows in Table 2 are reported after trimming, while the 'real controls' row is labeled 'no trim'. For the simulations, the target quantity is the global ATE (3.000 and 9.525), yet the trimmed estimates may correspond to a different subpopulation. A local ATE can differ from the global ATE even in the absence of selection bias, so the reported closeness of 3.021 and 9.719 to the true values is not by itself evidence of unbiasedness. The authors should either report the estimand corresponding to each trimmed sample or avoid using the global true effect as the benchmark in the trimmed rows.
  3. [Table 2, Linear row] The oracle benchmark itself does not recover the true effect in the linear simulation: with real controls and no trimming, linear regression gives 3.664 (CI 3.447–3.895), IPW 3.848, matching 3.927, and T-learner 4.029, all excluding the true value 3.000. If the adjustment set Z satisfies the back-door criterion and the DGP in Appendix B is linear, this should not happen; either the adjustment set is misspecified, the estimators are applied incorrectly, or the reported 'true effect' does not correspond to the estimated sample. This undermines the calibration of the whole simulation comparison, because closeness of the PU-based estimates to 3.000 or 9.525 may then reflect offsetting biases rather than unbiased control construction. The authors should diagnose the linear real-controls discrepancy before claiming that the PU estimates approximate the true ATE.
minor comments (4)
  1. [§3.1] The sentence 'we train the propensity score estimator only using this subset of X, the adjustment set Z' is ambiguous; it should state explicitly that the propensity score model uses only Z, not all of X, and that Z is a subset of X.
  2. [§4] The claim that 'the full 2-step PU learning method outperforms the use of SPY alone in most cases' is not supported by Table 1 in every row: for example, in the Non-linear dataset SPY with X has higher recall than SPY+iSVM with X (0.996 vs. 0.992) and in the Sowing dataset SPY with X has lower treated leakage than SPY+iSVM with X (0.267 vs. 0.333). Please quantify 'most cases' by specifying the metrics and datasets.
  3. [Table 2] P-values reported as 0.000 should be formatted as <0.001, and the header contains the typo 'Ferilization' instead of 'Fertilization'.
  4. [Abstract] The phrase 'using only the available treated (positive) units' is misleading; the method also uses a pool of unlabeled units that contain both hidden positives and controls, as described in Section 3.1. Please rephrase.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation chain: ATE estimates are benchmarked against independent simulation ground truth, and the self-citations used for real-world data are not load-bearing.

full rationale

The paper's derivation chain is: (i) train a PU classifier (SPY+iSVM) on features X to label reliable controls; (ii) estimate the ATE on the treated units and the selected controls using the adjustment set Z; (iii) compare the resulting estimates to true effects that are fixed by the simulated data-generating process or by prior published analyses. No step makes the output equal to an input by construction. In the simulations, the reported ATEs (3.021 vs. true 3.000; 9.719 vs. true 9.525) are computed from the selected control sample, not read off from the fitted PU model, and the true effects are determined by the DGP independently of the method. The hide-and-seek evaluation compares recovered control labels to held-out true labels, which is an external benchmark rather than a self-derived target. Self-citations ([19] for the sowing causal graph and expected ATE, [20] for the PU idea, [4] for Sentinel-2 indices) supply inputs, data, and context, but the central claim also rests on the two simulations with known ground truth, so these citations are not load-bearing. The most serious concern, using the outcome Y inside the PU feature set X, is a selection-bias/validity threat rather than a circularity: the ATE is not fitted from Y, and the close agreement in the simulations is an empirical result aided by near-perfect control recovery. The paper itself acknowledges trimming-induced selection bias and the local-ATE interpretability issue. Therefore, no circular step meeting the quoted-evidence standard is present.

Assumptions & free parameters 4 free parameters · 8 assumptions · 0 invented entities

The method rests on standard PU assumptions (SCAR, no label noise, separability, smoothness) plus standard causal assumptions. The most fragile additions are the choice of trimming thresholds and the inclusion of the outcome in the feature set, both of which are ad hoc and affect the resulting ATE.

free parameters (4)
  • Spy fraction = 30% for simulations, 15% for real-world datasets
    The proportion of treated units hidden as spies in the unlabeled set; chosen by the authors and not varied systematically.
  • Propensity score trimming range = [0.1, 0.6] for sowing and fertilization, [0.1, 0.3] for some Z-based runs
    Ad hoc thresholds to ensure overlap after PU selection; the paper notes this changes the estimand to a local ATE.
  • SVM hyperparameters = Not specified
    The iterative SVM uses a linear kernel but parameter C or other settings are not reported.
  • Naive Bayes smoothing parameter = Not specified
    Used in the SPY first step to estimate posterior probabilities; smoothing is not described.
assumptions (8)
  • domain assumption SCAR: P(S=1|T=1,X)=P(S=1|T=1)=c, c in (0,1).
    Eq. 5. Assumes labeled positives are selected uniformly at random from treated units; the paper emulates this via hide-and-seek but real data may violate it.
  • domain assumption No label noise in treated: P(T=0|S=1)=0.
    Eq. 6. Assumes no unit labeled as treated is actually a control.
  • domain assumption Controls in unlabeled set: P(T=0|S=0)>0.
    Eq. 7. Requires at least some controls in the unlabeled pool.
  • domain assumption Separability: P(T=1|X) is much greater than 0.5 for most treated and much less than 0.5 for most controls.
    Eq. 8. Necessary for reliable control identification via thresholding; may fail when treated and control distributions overlap.
  • domain assumption Smoothness: nearby feature vectors share treatment status.
    Eq. 9. Used to justify including post-treatment and exogenous features in X.
  • domain assumption Standard causal assumptions: SUTVA, unconfoundedness, positivity, consistency.
    Eqs. 2-4. Required for ATE identification and effect estimation.
  • ad hoc to paper The causal graphs G_sim, G_sowing, G_fertilizer are correctly specified.
    The adjustment set Z is derived from these graphs; a misspecified graph would bias the ATE estimates.
  • ad hoc to paper Including the outcome variable Y in the PU feature set X does not bias the downstream ATE estimate.
    The best-performing configurations use X that contains Y; if outcome-based selection induces collider or selection bias, the estimated ATE is not trustworthy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Positive-Unlabeled Learning for Control Group Construction in Observational Causal Inference." pith.science (2026). https://pith.science/paper/XRTVNXWV

@misc{pith2026250714528,
  author       = {Pith},
  title        = {Pith review of: Positive-Unlabeled Learning for Control Group Construction in Observational Causal Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XRTVNXWV}},
  note         = {Machine review of arXiv:2507.14528}
}
read the original abstract

In causal inference, whether through randomized controlled trials or observational studies, access to both treated and control units is essential for estimating the effect of a treatment on an outcome of interest. When treatment assignment is random, the average treatment effect (ATE) can be estimated directly by comparing outcomes between groups. In non-randomized settings, various techniques are employed to adjust for confounding and approximate the counterfactual scenario to recover an unbiased ATE. A common challenge, especially in observational studies, is the absence of units clearly labeled as controls-that is, units known not to have received the treatment. To address this, we propose positive-unlabeled (PU) learning as a framework for identifying, with high confidence, control units from a pool of unlabeled ones, using only the available treated (positive) units. We evaluate this approach using both simulated and real-world data. We construct a causal graph with diverse relationships and use it to generate synthetic data under various scenarios, assessing how reliably the method recovers control groups that allow estimates of true ATE. We also apply our approach to real-world data on optimal sowing and fertilizer treatments in sustainable agriculture. Our findings show that PU learning can successfully identify control (negative) units from unlabeled data based only on treated units and, through the resulting control group, estimate an ATE that closely approximates the true value. This work has important implications for observational causal inference, especially in fields where randomized experiments are difficult or costly. In domains such as earth, environmental, and agricultural sciences, it enables a plethora of quasi-experiments by leveraging available earth observation and climate data, particularly when treated units are available but control units are lacking.

Figures

Figures reproduced from arXiv: 2507.14528 by the authors.

Figure 1
Figure 1. Overview of the use of PU learning as a preparatory step for causal estimation tasks that lack control groups. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The causal graph 𝐺𝑠𝑖𝑚 serves as the data-generating process for both linear and non-linear simulations. control for mediator 𝑀. 𝑋9 is a typical ’bad control’ because con￾trolling for it opens a colliding path and induces ’selection bias’. Finally, vertices 𝑉𝑛𝑒𝑢𝑡𝑟𝑎𝑙 = {𝑋3, 𝑋8} are neutral in terms of insert￾ing or removing bias but can be useful controls in terms of 𝐴𝑇 𝐸 precision. Thus, as expected from graph constr… view at source ↗
Figure 3
Figure 3. Propensity scores of 4 different combinations on linear experimental dataset [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Propensity scores of 4 different combinations on non-linear experimental dataset [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Propensity scores of 4 different combinations on experimental dataset regarding optimal sowing [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Propensity scores of 4 different combinations on experimental dataset regarding digestate fertilization application [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Interpretability and Model Comparison via SVM Coefficient Slope Chart for linear dataset [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Interpretability and Model Comparison via SVM Coefficient Slope Chart for non-linear dataset [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Interpretability and Model Comparison via SVM Coefficient Slope Chart for dataset regarding optimal sowing [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Interpretability and Model Comparison via SVM Coefficient Slope Chart for dataset regarding digestate fertilization [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 18 canonical work pages

  1. [1]

    Jessa Bekker and Jesse Davis. 2020. Learning from positive and unlabeled data: A survey. Machine Learning 109, 4 (2020), 719–760

    work page 2020

  2. [2]

    Carlos Cinelli, Andrew Forney, and Judea Pearl. 2024. A crash course in good and bad controls. Sociological Methods & Research 53, 3 (2024), 1071–1104

    work page 2024

  3. [3]

    Maxence Dodin, Florent Levavasseur, Antoine Savoie, Lucie Martin, Jean Foulon, and Emmanuelle Vaudour. 2023. Sentinel-2 satellite images for monitoring cattle slurry and digestate spreading on emerging wheat crop: A field spectroscopy experiment. Geocarto International 38, 1 (2023), 2245371

    work page 2023

  4. [4]

    Andreas Kalogeras, Dimitrios Bormpoudakis, Iason Tsardanidis, Dimitra A Loka, and Charalampos Kontoes. 2025. Monitoring digestate application on agricultural crops using Sentinel-2 Satellite imagery. arXiv preprint arXiv:2504.19996 (2025)

    work page arXiv 2025

  5. [5]

    Masahiro Kato, Fumiaki Kozai, and Ryo Inokuchi. 2025. PUATE: Semiparamet- ric Efficient Average Treatment Effect Estimation from Treated (Positive) and Unlabeled Units. arXiv preprint arXiv:2501.19345 (2025)

    arXiv 2025

  6. [6]

    Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. 2019. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences 116, 10 (2019), 4156–4165

    work page 2019

  7. [7]

    Milan Kuzmanovic, Tobias Hatt, and Stefan Feuerriegel. 2023. Estimating condi- tional average treatment effects with missing treatment information. In Interna- tional Conference on Artificial Intelligence and Statistics . PMLR, 746–766

    work page 2023

  8. [8]

    Tony Lancaster and Guido Imbens. 1996. Case-control studies with contaminated controls. Journal of Econometrics 71, 1-2 (1996), 145–160

    work page 1996

Show all 21 references
  1. [9]

    Bing Liu, Wee Sun Lee, Philip S Yu, and Xiaoli Li. 2002. Partially supervised classification of text documents. In ICML, Vol. 2. Sydney, NSW, 387–394

    2002

  2. [10]

    Francesca Molinari. 2010. Missing treatments. Journal of Business & Economic Statistics 28, 1 (2010), 82–95

    2010

  3. [11]

    Judea Pearl. 2009. Causality. Cambridge university press

    2009

  4. [12]

    Judea Pearl and Dana Mackenzie. 2018. The book of why: the new science of cause and effect. Basic books

    2018

  5. [13]

    Paul R Rosenbaum and Donald B Rubin. 1985. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician 39, 1 (1985), 33–38

    1985

  6. [14]

    Donald B Rubin. 2005. Causal inference using potential outcomes: Design, mod- eling, decisions. Journal of the American statistical Association 100, 469 (2005), 322–331

    2005

  7. [15]

    Jack D Saunders and Alex A Freitas. 2022. Evaluating the predictive performance of positive-unlabelled classifiers: a brief critical review and practical recommen- dations for improvement. ACM SIGKDD Explorations Newsletter 24, 2 (2022), 5–11

    2022

  8. [16]

    Somya Sharma, Swati Sharma, Licheng Liu, Rishabh Tushir, Andy Neal, Robert Ness, John Crawford, Emre Kiciman, and Ranveer Chandra. 2023. Knowledge Guided Representation Learning and Causal Structure Learning in Soil Science. arXiv preprint arXiv:2306.09302 (2023)

    2023 arXiv

  9. [17]

    Dan Steinberg and N Scott Cardell. 1992. Estimating logistic regression models when the dependent variable has no variance. Communications in Statistics- Theory and Methods 21, 2 (1992), 423–450

    1992

  10. [18]

    Elizabeth A Stuart. 2010. Matching methods for causal inference: A review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics 25, 1 (2010), 1

    2010

  11. [19]

    Ilias Tsoumas, Georgios Giannarakis, Vasileios Sitokonstantinou, Alkiviadis Koukos, Dimitra Loka, Nikolaos Bartsotas, Charalampos Kontoes, and Ioannis Athanasiadis. 2023. Evaluating digital agriculture recommendations with causal inference. In Proceedings of the AAAI Conferenc...

    2023

  12. [20]

    Ilias Tsoumas, Vasileios Sitokonstantinou, Georgios Giannarakis, Evagelia Lam- piri, Christos Athanassiou, Gustau Camps-Valls, Charalampos Kontoes, and Ioannis N Athanasiadis. 2025. Leveraging causality and explainability in digital agriculture. Environmental Data Science 4 (2...

    2025

  13. [21]

    Hwanjo Yu. 2005. Single-class classification with mapping convergence. Machine Learning 61 (2005), 49–69. KDD 2025 Workshop, August 03–07, 2025, Toronto, CA Tsoumas et al. A Real-World Datasets For the sowing case, we follow the causal graph construction pre- sented in Tsoumas...

    2005

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.