Pith. sign in

REVIEW 2 major objections 1 minor 57 references

Choosing A Headline Estimand from Matching, DID, and Hybrid Designs: A Minimax-Regret Approach

T0 review · 2 major / 1 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read When matching, DID, and DIDM estimands can be ordered with the hybrid in the middle, DIDM minimizes maximum regret and should be reported as the headline estimate.

desk verdict The paper uses minimax regret to recommend the hybrid DIDM as headline when estimands are ordered with it between matching and DID. read the letter →

arxiv 2606.20435 v1 pith:KHN4TAHC submitted 2026-06-18 econ.EM

Yechan Park , Yuya Sasaki This is my paper · ORCID
classification econ.EM
keywords difference-in-differencesmatchinghybridestimatorminimaxregretpaneldatacausalinferenceestimandselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Researchers using panel data routinely select among difference-in-differences, matching on lagged outcomes, or a hybrid of both when estimating causal effects. The paper supplies conditions under which these three estimands fall into an ordered relation, with the hybrid value lying between the other two. Under that ordering the hybrid becomes the choice that minimizes the largest possible regret across a wide family of loss functions. The authors therefore advise reporting the hybrid as the main number and treating the pure matching and pure DID numbers as bounds around it.

What carries the argument

The ordering condition that places the hybrid DIDM estimand strictly between the matching and difference-in-differences estimands, which then selects DIDM under the minimax-regret criterion.

What would settle it

A data-generating process or empirical application in which the matching estimate, the DID estimate, and the DIDM estimate violate the bracketing order with DIDM in the middle.

Watch

Extended reading notes

Core claim

We give conditions under which the corresponding estimands are ordered, with DIDM bracketed between matching and DID. This makes DIDM the minimax-regret choice among the three under a broad class of loss functions. We recommend reporting DIDM as the headline estimate, with matching and DID as bounds.

Load-bearing premise

Conditions exist under which the three estimands are ordered so that the hybrid lies between matching and difference-in-differences.

Editorial extensions

If this is right

  • DIDM should be presented as the headline estimate in applications that satisfy the ordering conditions.
  • Matching and DID estimates function as explicit bounds around the headline number.
  • The minimax-regret justification applies across a broad class of loss functions.
  • The recommendation supplies a default reporting rule when the three identifying assumptions are non-nested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Applied researchers could check whether their data satisfy the ordering conditions before adopting the hybrid as headline.
  • The same bracketing logic might be examined for other combinations of estimators that use lagged outcomes.
  • The bounds could be used to conduct a simple form of sensitivity analysis around the headline number.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper claims that under conditions where the estimands from matching (M), difference-in-differences (DID), and the hybrid DIDM are ordered with DIDM bracketed between M and DID, DIDM is the minimax-regret choice among the three for a broad class of loss functions. It recommends reporting DIDM as the headline estimate, with M and DID serving as bounds, and illustrates the approach in applications.

Significance. If the ordering conditions and minimax-regret mapping hold, the paper supplies a decision-theoretic rationale for selecting among non-nested identifying assumptions in panel-data causal inference, a frequent practical problem. The explicit recommendation to report bounds alongside the headline estimate is a concrete, usable contribution.

major comments (2)
  1. [Abstract and theoretical results] Abstract, paragraph 2 and the theoretical section stating the ordering conditions: the claim that 'we give conditions under which the corresponding estimands are ordered, with DIDM bracketed between matching and DID' is asserted without derivation of the ordering from the identifying assumptions or verification against concrete DGPs. This is load-bearing because the minimax-regret justification and the headline recommendation collapse if the ordering fails to hold.
  2. [Applications] Applications section: the manuscript does not report the realized values of the three estimands or test whether DIDM lies between M and DID in the data, leaving it impossible to confirm that the minimax-regret recommendation applies to the illustrated cases.
minor comments (1)
  1. [Notation and setup] The notation for the hybrid estimator (DIDM) is introduced without an explicit equation reference in the main text, which would aid readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. The two major comments identify places where the manuscript can be strengthened by making derivations and empirical checks more explicit. We address each point below and will incorporate the suggested changes.

read point-by-point responses

  1. Referee: [Abstract and theoretical results] Abstract, paragraph 2 and the theoretical section stating the ordering conditions: the claim that 'we give conditions under which the corresponding estimands are ordered, with DIDM bracketed between matching and DID' is asserted without derivation of the ordering from the identifying assumptions or verification against concrete DGPs. This is load-bearing because the minimax-regret justification and the headline recommendation collapse if the ordering fails to hold.

    Authors: We agree that the link between the identifying assumptions and the ordering of the three estimands should be derived explicitly rather than stated. In the revised manuscript we will add a short subsection (new Theorem 1 and proof) that starts from the standard parallel-trends and conditional-independence assumptions and shows the resulting inequality DIDM ∈ [min(M,DID), max(M,DID)]. We will also include a small Monte Carlo exercise with concrete DGPs that satisfy or violate the assumptions, confirming when the bracketed ordering holds. These additions directly address the load-bearing concern. revision: yes

  2. Referee: [Applications] Applications section: the manuscript does not report the realized values of the three estimands or test whether DIDM lies between M and DID in the data, leaving it impossible to confirm that the minimax-regret recommendation applies to the illustrated cases.

    Authors: We accept this point. The revised applications section will include a table (or inline text) that reports the three point estimates for each empirical example together with a direct check of whether DIDM lies between M and DID. When the ordering holds we will note that the minimax-regret rationale applies; when it does not we will discuss the implications for the headline recommendation. This change makes the empirical illustrations self-contained and verifiable. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; ordering conditions and minimax-regret application are derived independently

full rationale

The paper states conditions under which estimands from matching, DID, and DIDM are ordered (with DIDM between the others) and concludes that DIDM is minimax-regret optimal for a broad loss class. This is a standard theoretical derivation resting on external properties of loss functions and the stated ordering, not on any reduction of the result to fitted inputs, self-definitions, or self-citation chains. No load-bearing step in the abstract or described argument reduces by construction to the paper's own inputs. The recommendation follows directly from the derived ordering rather than being presupposed.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The paper rests on mathematical conditions for estimand ordering and properties of minimax regret; no free parameters or new entities are introduced in the abstract.

assumptions (1)
  • domain assumption Estimands from matching, DID, and DIDM can be ordered with DIDM between the other two under stated conditions
    Invoked in abstract as the premise that enables the minimax-regret conclusion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Choosing A Headline Estimand from Matching, DID, and Hybrid Designs: A Minimax-Regret Approach." pith.science (2026). https://pith.science/paper/KHN4TAHC

@misc{pith2026260620435,
  author       = {Pith},
  title        = {Pith review of: Choosing A Headline Estimand from Matching, DID, and Hybrid Designs: A Minimax-Regret Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KHN4TAHC}},
  note         = {Machine review of arXiv:2606.20435}
}
read the original abstract

Researchers using panel data to estimate causal effects routinely choose among three approaches to using past outcomes: difference-in-differences (DID), conditioning on lagged outcomes (matching, M), and a hybrid that does both (DIDM). The corresponding identifying assumptions are non-nested, leaving little guidance on which to report. We give conditions under which the corresponding estimands are ordered, with DIDM bracketed between matching and DID. This makes DIDM the minimax-regret choice among the three under a broad class of loss functions. We recommend reporting DIDM as the headline estimate, with matching and DID as bounds. We illustrate in applications.

Figures

Figures reproduced from arXiv: 2606.20435 by the authors.

Figure 1
Figure 1. Benchmark job-training evidence on double bracketing. The top panel summa [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗
Figure 2
Figure 2. Estimates and 95% confidence intervals for the M, DIDM, and DID estimands in [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗
Figure 3
Figure 3. Evidence on Assumption 4.1(i) for the NSW program, CPS (top) and PSID (bottom). Solid and dashed lines show estimates of E[Y0 | W = 0, Y−s = y] and E[Y0 | W = 1, Y−s = y], respectively. Shaded regions are 95% confidence bands. Both axes are in thousands of U.S. dollars. See Footnote 14 for estimation details. Decreasing untreated growth. Assumption 4.1(iii) requires the function Φ(y) := E[Y1 − Y0 | W = 0, Y−s = y] t… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Evidence on Assumption 4.1(ii) for the NSW program, CPS (left) and PSID (right). Solid and dotted lines show empirical CDFs of Y−s for W = 0 and W = 1, respectively, with 95% confidence bands. Horizontal axes are in thousands of U.S. dollars. both CPS and PSID, the est…
Figure 5
Figure 5. Figure 5: Evidence on Assumption 4.1(iii) for the NSW program, CPS (top) and PSID (bottom). The function Φ(y) = E[Y1 − Y0 | W = 0, Y−s = y] is estimated nonparametrically with 95% confidence bands. Both axes are in thousands of U.S. dollars. See Footnote 14 for estimation detail…
Figure 6
Figure 6. Figure 6: Evidence on Assumption 4.1(i) for the educational program. Solid and dashed lines show estimates of E[Y0 | W = 0, Y−s = y] and E[Y0 | W = 1, Y−s = y], respectively, with 95% confidence bands (barely visible due to large sample size) [PITH_FULL_IMAGE:figures/full_fig_p…
Figure 7
Figure 7. Figure 7: Evidence on Assumption 4.1(ii) for the educational program. Solid and dotted lines show empirical CDFs of Y−s for W = 0 and W = 1, respectively, with 95% confidence bands. Distributional dominance in Y−s [PITH_FULL_IMAGE:figures/full_fig_p053_7.png]
Figure 8
Figure 8. Figure 8: Evidence on Assumption 4.1(iii) for the educational program. The function Φ(y) = E[Y1 − Y0 | W = 0, Y−s = y] is estimated nonparametrically with 95% confidence bands. Decreasing untreated growth. We estimate Φ(y) = E[Y1 − Y0 | W = 0, Y−s = y] nonparametrically and plot…
Figure 9
Figure 9. Figure 9: Evidence of Assumption 4.1 (i) for the NSW program using the CPS (top) and PSID (bottom) data sets after residualizing with respect to auxiliary covariates. The solid and dashed lines represent estimates of the conditional expectation functions y 7→ E[Y0|W = 0, Y−s = y…
Figure 10
Figure 10. Figure 10: Evidence of Assumption 4.1 (iii) for the NSW program using the CPS (top) and PSID (bottom) data sets after residualizing with respect to auxiliary covariates. The con￾ditional expectation function Φ is estimated non-parametrically by the partitioning-based least squar…
Figure 11
Figure 11. Figure 11: Evidence of Assumption 4.1 (i) for the educational program after residualizing with respect to auxiliary covariates. The solid and dashed lines represent estimates of the conditional expectation functions y 7→ E[Y0|W = 0, Y−s = y] and y 7→ E[Y0|W = 1, Y−s = y], respec…
Figure 12
Figure 12. Figure 12: Evidence of Assumption 4.1 (iii) for the educational program after residualizing with respect to auxiliary covariates. The conditional expectation function Φ is estimated non-parametrically by the partitioning-based least squares regression on the residualized variabl…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 3 canonical work pages

  1. [1]

    Abadie, A. (2005). Semiparametric difference-in-differences estimators. Review of Economic Studies , 72(1):1--19

    2005

  2. [2]

    Acemoglu, D., Naidu, S., Restrepo, P., and Robinson, J. A. (2019). Democracy does cause growth. Journal of Political Economy , 127(1):47--100

    2019

  3. [3]

    and Shapiro, J

    Andrews, I. and Shapiro, J. M. (2024). Communicating scientific uncertainty via approximate posteriors. Technical report, National Bureau of Economic Research

    2024

  4. [4]

    Angrist, J., Hull, P., and Walters, C. R. (2022). Methods for measuring school effectiveness. NBER Working Paper 30803, National Bureau of Economic Research, Cambridge, MA

    2022

  5. [5]

    D., Hull, P

    Angrist, J. D., Hull, P. D., Pathak, P. A., and Walters, C. R. (2017). Leveraging lotteries for school value-added: Testing and estimation. Quarterly Journal of Economics , 132(2):871--919

    2017

  6. [6]

    D., Hull, P

    Angrist, J. D., Hull, P. D., and Walters, C. R. (2023). Methods for measuring school effectiveness. In Hanushek, E. A., Machin, S. J., and Woessmann, L., editors, Handbook of the Economics of Education, Volume 7 , pages 1--60. Elsevier

    2023

  7. [7]

    Angrist, J. D. and Pischke, J.-S. (2009). Mostly Harmless Econometrics: An Empiricist's Companion . Princeton University Press

    2009

  8. [8]

    Arranz, J. M. and Garc \'i a-Serrano, C. (2025). Assessing the impact of an increase in the minimum wage on household income and poverty. Social Science Research , 127:103143

    2025

Show all 57 references
  1. [9]

    Ashenfelter, O. (1978). Estimating the effect of training programs on earnings. Review of Economics and Statistics , 60(1):47--57

    1978

  2. [10]

    Athey, S., Chetty, R., and Imbens, G. (2025). The experimental selection correction estimator: Using experiments to remove biases in observational estimates. Technical report, National Bureau of Economic Research

    2025

  3. [11]

    Bach, L., Bozio, A., Guillouzouic, A., and Malgouyres, C. (2023). Dividend taxes and the allocation of capital: Comment. American Economic Review , 113(7):2048--2052

    2023

  4. [12]

    Bertrand, M., Duflo, E., and Mullainathan, S. (2004). How much should we trust differences-in-differences estimates? The Quarterly journal of economics , 119(1):249--275

    2004

  5. [13]

    and Matray, A

    Boissel, C. and Matray, A. (2022). Retracted by the authors: Dividend taxes and the allocation of capital. American Economic Review , 112(9):2884--2920. Retracted

    2022

  6. [14]

    and Sant'Anna, P

    Callaway, B. and Sant'Anna, P. H. (2018). Difference-in-differences with multiple time periods and an application on the minimum wage and employment. arXiv preprint arXiv:1803.09015

    2018

  7. [15]

    D., Farrell, M

    Cattaneo, M. D., Farrell, M. H., and Feng, Y. (2019). lspartition: Partitioning-based least squares regression. arXiv preprint arXiv:1906.00202

    2019

  8. [16]

    Cengiz, D., Dube, A., Lindner, A., and Zipperer, B. (2019). The effect of minimum wages on low-wage jobs. Quarterly Journal of Economics , 134(3):1405--1454

    2019

  9. [17]

    Chab \'e -Ferret, S. (2017). Should we combine difference in differences with conditioning on pre-treatment outcomes?

    2017

  10. [18]

    B., Katz, L

    Chetty, R., Diamond, R., Foster, T. B., Katz, L. F., Porter, S., Staiger, M., and Tach, L. (2026). Creating high-opportunity neighborhoods: Evidence from the hope vi program. Technical report, National Bureau of Economic Research

    2026

  11. [19]

    N., and Rockoff, J

    Chetty, R., Friedman, J. N., and Rockoff, J. E. (2014a). Measuring the impacts of teachers i: Evaluating bias in teacher value-added estimates. American Economic Review , 104(9):2593--2632

  12. [20]

    N., and Rockoff, J

    Chetty, R., Friedman, J. N., and Rockoff, J. E. (2014b). Measuring the impacts of teachers ii: Teacher value-added and student outcomes in adulthood. American economic review , 104(9):2633--2679

  13. [21]

    Couch, K. A. and Placzek, D. W. (2010). Earnings losses of displaced workers revisited. American Economic Review , 100(1):572--589

    2010

  14. [22]

    Currie, J., Kleven, H., and Zwiers, E. (2020). Technology and big data are changing economics: Mining text to track methods. AEA Papers and Proceedings , 110:42--48

    2020

  15. [23]

    Daw, J. R. and Hatfield, L. A. (2018). Matching and regression to the mean in difference-in-differences analysis. Health services research , 53(6):4138--4156

    2018

  16. [24]

    and d’Haultfoeuille, X

    De Chaisemartin, C. and d’Haultfoeuille, X. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review , 110(9):2964--2996

    2020

  17. [25]

    and d’Haultfoeuille, X

    De Chaisemartin, C. and d’Haultfoeuille, X. (2023). Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: A survey. Econometrics Journal , 26(3):C1--C30

    2023

  18. [26]

    Dehejia, R. (2005). Practical propensity score matching: a reply to smith and todd. Journal of Econometrics , 125(1-2):355--364

    2005

  19. [27]

    Dehejia, R. H. and Wahba, S. (1999). Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs. Journal of the American statistical Association , 94(448):1053--1062

    1999

  20. [28]

    Dehejia, R. H. and Wahba, S. (2002). Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics , 84(1):151--161

    2002

  21. [29]

    and Li, F

    Ding, P. and Li, F. (2019). A bracketing relationship between difference-in-differences and lagged-dependent-variable adjustment. Political Analysis , 27(4):605--615

    2019

  22. [30]

    and Imbens, G

    Doudchenko, N. and Imbens, G. W. (2016). Balancing, regression, difference-in-differences and synthetic control methods: A synthesis. Technical report, National Bureau of Economic Research

    2016

  23. [31]

    Dube, A., Girardi, D., Jorda, O., and Taylor, A. M. (2023). A local projections approach to difference-in-differences event studies. Technical report, National Bureau of Economic Research

    2023

  24. [32]

    Goldsmith-Pinkham, P. (2024). Tracking the credibility revolution across fields. arXiv preprint arXiv:2405.20604

    2024

  25. [33]

    and Lochner, B

    Hafner, L. and Lochner, B. (2022). Do minimum wages improve self-rated health? evidence from a natural experiment. Empirical Economics , 62(6):2989--3014

    2022

  26. [34]

    J., Ichimura, H., Smith, J

    Heckman, J. J., Ichimura, H., Smith, J. A., and Todd, P. E. (1998a). Characterizing selection bias using experimental data. Econometrica , 66(5):1017--1098

  27. [35]

    J., Ichimura, H., and Todd, P

    Heckman, J. J., Ichimura, H., and Todd, P. (1998b). Matching as an econometric evaluation estimator. Review of Economic Studies , 65(2):261--294

  28. [36]

    J., Ichimura, H., and Todd, P

    Heckman, J. J., Ichimura, H., and Todd, P. E. (1997). Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. The review of economic studies , 64(4):605--654

    1997

  29. [37]

    Heckman, J. J. and Smith, J. A. (1999). The pre-programme earnings dip and the determinants of participation in a social programme: Implications for simple programme evaluation strategies. The Economic Journal , 109(457):313--348

    1999

  30. [38]

    Hyslop, D. R. and Townsend, W. (2017). The longer term impacts of job displacement on labour market outcomes. Motu Working Paper 17-12, Motu Economic and Public Policy Research, Wellington, New Zealand

    2017

  31. [39]

    Hyslop, D. R. and Townsend, W. (2019). The longer-term impacts of job displacement on labour market outcomes in new zealand. Australian Economic Review , 52(2):158--177

    2019

  32. [40]

    F., and Trenkle, S

    Illing, H., Schmieder, J. F., and Trenkle, S. (2024). The gender gap in earnings losses after job displacement. Journal of the European Economic Association , 22(5):2108--2147

    2024

  33. [41]

    S., and Wang, E

    Imai, K., Kim, I. S., and Wang, E. H. (2023). Matching methods for causal inference with time-series cross-sectional data. American Journal of Political Science , 67(3):587--605

    2023

  34. [42]

    S., LaLonde, R

    Jacobson, L. S., LaLonde, R. J., and Sullivan, D. G. (1993). Earnings losses of displaced workers. American Economic Review , 83(4):685--709

    1993

  35. [43]

    and Lewandowski, P

    Kami \'n ska, A. and Lewandowski, P. (2015). The effects of minimum wage on a labour market with high temporary employment. IBS Working Paper 07/2015, Institute for Structural Research (IBS), Warsaw

    2015

  36. [44]

    Kane, T. J. and Staiger, D. O. (2008). Estimating teacher impacts on student achievement: An experimental evaluation. Working Paper 14607, National Bureau of Economic Research

    2008

  37. [45]

    Lachowska, M., Mas, A., and Woodbury, S. A. (2020). Sources of displaced workers' long-term earnings losses. American Economic Review , 110(10):3231--3266

    2020

  38. [46]

    LaLonde, R. J. (1986). Evaluating the econometric evaluations of training programs with experimental data. American Economic Review , pages 604--620

    1986

  39. [47]

    Lenhart, O. (2017a). Do higher minimum wages benefit health? evidence from the UK . Journal of Policy Analysis and Management , 36(4):828--852

  40. [48]

    Lenhart, O. (2017b). The impact of minimum wages on population health: Evidence from 24 OECD countries. European Journal of Health Economics , 18(8):1031--1039

  41. [49]

    P., Smith, J

    McCall, B. P., Smith, J. A., and Wunsch, C. (2016). Government-sponsored vocational education for adults. In Hanushek, E. A., Machin, S., and Woessmann, L., editors, Handbook of the Economics of Education , volume 5, pages 479--652. Elsevier

    2016

  42. [50]

    G., Hanushek, E

    Rivkin, S. G., Hanushek, E. A., and Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica , 73(2):417--458

    2005

  43. [51]

    H., Bilinski, A., and Poe, J

    Roth, J., Sant’Anna, P. H., Bilinski, A., and Poe, J. (2023). What’s trending in difference-in-differences? a synthesis of the recent econometrics literature. Journal of Econometrics , 235(2):2218--2244

    2023

  44. [52]

    Rothstein, J. (2010). Teacher quality in educational production: Tracking, decay, and student achievement. Quarterly Journal of Economics , 125(1):175--214

    2010

  45. [53]

    F., Von Wachter, T., and Heining, J

    Schmieder, J. F., Von Wachter, T., and Heining, J. (2023). The costs of job displacement over the business cycle and its sources: evidence from germany. American Economic Review , 113(5):1208--1254

    2023

  46. [54]

    and Shanthikumar, J

    Shaked, M. and Shanthikumar, J. G. (2007). Stochastic Orders . Springer Series in Statistics. Springer, New York

    2007

  47. [55]

    Smith, J. A. and Todd, P. E. (2005). Does matching overcome lalonde's critique of nonexperimental estimators? Journal of econometrics , 125(1-2):305--353

    2005

  48. [56]

    Song, K. (2014). Point decisions for interval--identified parameters. Econometric Theory , 30(2):334--356

    2014

  49. [57]

    Staiger, D. O. and Rockoff, J. E. (2010). Searching for effective teachers with imperfect information. Journal of Economic Perspectives , 24(3):97--118

    2010

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.