pith. sign in

arxiv: 2606.30664 · v1 · pith:WX5UWDQ7new · submitted 2026-06-17 · 📊 stat.AP · cs.AI· cs.LG

Estimating the Effect of Timing on Coupon Effectiveness

Deddy Jobson This is my paper

Pith reviewed 2026-07-01 07:35 UTC · model grok-4.3

classification 📊 stat.AP cs.AIcs.LG
keywords causal inferencecoupon effectivenesstiming effectsnatural experimentsmarketing optimizationuser onboardingretention campaignsobservational data
0
0 comments X

The pith

Variations in coupon delivery times create natural experiments that causally identify optimal send timing.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes a framework that treats observed differences in when coupons are sent as natural randomized experiments. Causal inference is then applied to these variations to estimate how timing affects coupon effectiveness, without the need for a separate A/B test or real-time triggering system. The approach is shown on an internal user onboarding campaign and replicated on a public retention dataset, producing results that support specific business choices about when to deliver incentives during the customer journey.

Core claim

The authors claim that natural variations in coupon delivery timing constitute valid experiments allowing causal identification of timing effects on effectiveness, and that applying standard causal methods to such data yields actionable estimates for marketing decisions. They demonstrate the framework first on company onboarding data and then on a public retention dataset to confirm that the same logic produces usable guidance without dedicated testing infrastructure.

What carries the argument

Natural randomized control trial experiments arising from observed variations in coupon send times, which allow causal identification when timing is ignorable conditional on observed covariates.

If this is right

  • The framework lets marketers estimate timing effects directly from historical campaign records.
  • Business decisions on coupon timing can be made without building real-time event-triggered distribution systems.
  • The same logic applies across campaign types, as shown by consistent application to both onboarding and retention settings.
  • Results from the natural experiments can replace or reduce the need for new dedicated tests when choosing when to send incentives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This approach could be applied to timing questions for other interventions such as emails or push notifications.
  • If unmeasured factors still influence both timing and outcomes, the estimates would require additional robustness checks such as sensitivity analyses.
  • The public-dataset replication suggests the framework can be tested for reproducibility on other open marketing or user-behavior datasets.
  • Extending the method to continuous time or to multiple simultaneous treatments would require only modest changes to the identification strategy.

Load-bearing premise

Observed differences in coupon delivery timing are as good as randomly assigned once measured covariates are accounted for.

What would settle it

Running a true randomized A/B test that assigns coupon send times independently and finding materially different effect estimates from the natural-experiment analysis would falsify the central claim.

Figures

Figures reproduced from arXiv: 2606.30664 by Deddy Jobson.

Figure 1
Figure 1. Figure 1: The causal model for our onboarding campaign [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Our framework for estimating the effectiveness of timing on coupons. The time-related feature of [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of actual and counterfactual BCR of treatment and control users for different registration [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The causal model for Lenta’s retention campaign [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of actual and counterfactual BCR of treatment and control users for different purchase [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
read the original abstract

The coupon incentive is one of the most common tools marketers use to court users to engage with a business at various stages of the customer life cycle. A variety of factors can affect the effectiveness of a coupon incentive on users, timing being one of them. We hypothesize that coupons can be more effective when delivered at critical times in the customer journey, right when a user is engaging with the platform. Verifying such a hypothesis would typically require real time event-triggered coupon distribution software that may be too expensive to implement. In this paper, we propose a framework in which we apply causal inference on "natural randomized control trial experiments" to measure the effectiveness of sending coupons at the right time to users without requiring a dedicated AB test. We demonstrate the usefulness of our framework in the case of a user onboarding coupon campaign held in our company and show how the results can lead to correct data-driven decisions for the business. Furthermore, in order to test the generalizability of our framework, and to make our research more reproducible, we apply our framework on a user retention campaign with a publicly available dataset.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper proposes a framework that treats observed variations in coupon delivery timing as natural randomized control trials, applies causal inference methods to estimate the effect of timing on coupon effectiveness, and demonstrates the approach on an internal user onboarding campaign plus a public user retention dataset, claiming this enables data-driven decisions without dedicated A/B tests or real-time event-triggered systems.

Significance. If the ignorability assumption holds and the framework is shown to be robust, the work could offer a practical, low-cost alternative for optimizing coupon timing in observational marketing data settings. The inclusion of a public dataset for reproducibility is a strength that supports broader applicability and verification.

major comments (1)
  1. [Framework description (abstract and methods)] The central claim rests on the assumption that timing variations constitute valid natural RCTs permitting causal identification (i.e., timing is ignorable conditional on observed covariates). No explicit identification strategy, balance tables, sensitivity analysis, or robustness checks are described to address potential confounding from user behavior that may drive both delivery timing and response outcomes.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback. The major comment concerns the need for greater explicitness around the causal identification strategy, balance checks, and robustness to confounding. We respond below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Framework description (abstract and methods)] The central claim rests on the assumption that timing variations constitute valid natural RCTs permitting causal identification (i.e., timing is ignorable conditional on observed covariates). No explicit identification strategy, balance tables, sensitivity analysis, or robustness checks are described to address potential confounding from user behavior that may drive both delivery timing and response outcomes.

    Authors: We agree that the manuscript would be strengthened by a more explicit treatment of the identification strategy. In the revision we will add a dedicated subsection to the Methods that states the core identifying assumptions (conditional ignorability of coupon timing given the observed covariates such as user demographics, prior engagement metrics, and onboarding stage) and the associated potential-outcomes notation. We will also insert balance tables that compare covariate means and distributions across the observed timing groups, together with standardized mean differences. Finally, we will report sensitivity analyses that assess robustness to unobserved confounding, including (i) re-estimation under alternative covariate sets and (ii) application of a bounding approach (e.g., Rosenbaum bounds or similar) that quantifies how large an unobserved confounder would need to be to overturn the main findings. These additions directly address the possibility that user behavior jointly influences both delivery timing and response. revision: yes

Circularity Check

0 steps flagged

No significant circularity; framework relies on external identification assumption

full rationale

The paper proposes a causal framework treating observed coupon timing variation as natural RCTs for estimating timing effects, without any equations, parameter fitting, or predictions that reduce to inputs by construction. The load-bearing step is the claim that timing is ignorable conditional on covariates, presented as an empirical assumption rather than derived from or equivalent to the framework itself. No self-citations, ansatzes, or renamings appear in the text that would create definitional or fitted circularity. The derivation is self-contained as a methodological proposal whose validity rests on external falsifiability of the ignorability assumption.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The framework rests on the domain assumption that natural timing variations provide valid causal identification; no free parameters or invented entities are mentioned in the abstract.

axioms (1)
  • domain assumption Natural experiments in coupon delivery timing act as randomized control trials permitting causal inference on timing effects.
    This is the core premise invoked to avoid dedicated AB tests.

pith-pipeline@v0.9.1-grok · 5710 in / 1125 out tokens · 26342 ms · 2026-07-01T07:35:08.316660+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 16 canonical work pages · 1 internal anchor

  1. [1]

    2015), 485–505

    Estimating Conditional Average Treatment Effects.Journal of Business & Economic Statistics33, 4 (Oct. 2015), 485–505. https://doi.org/10.1080/07350015.2014.975555 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/07350015.2014.975555. Douglas G. Altman, Berthold Lausen, Willi Sauerbrei, and Martin Schumacher

    work page doi:10.1080/07350015.2014.975555 2015

  2. [2]

    https://doi.org/10.1093/jnci/86.11.829 M

    Dangers of Using “Optimal” Cutpoints in the Evaluation of Prognostic Factors.JNCI: Journal of the National Cancer Institute86, 11 (June 1994), 829–835. https://doi.org/10.1093/jnci/86.11.829 M. Bonetti and R. D. Gelber

    work page doi:10.1093/jnci/86.11.829 1994

  3. [3]

    2000), 2595–2609

    A graphical method to assess treatment-covariate interactions using the Cox model on subsets of the data.Statistics in Medicine19, 19 (Oct. 2000), 2595–2609. https://doi.org/10.1002/1097-0258(20001015)19: 19<2595::aid-sim562>3.0.co;2-m Milos Brankovic, Isabella Kardys, Ewout W. Steyerberg, Stanley Lemeshow, Maja Markovic, Dimitris Rizopoulos, and Eric Boersma

    work page doi:10.1002/1097-0258(20001015)19: 2000

  4. [4]

    2019), e13145

    Understanding of interaction (subgroup) analysis in clinical trials.European Journal of Clinical Investigation49, 8 (Aug. 2019), e13145. https://doi.org/10.1111/eci.13145 Alexander D’Amour

    work page doi:10.1111/eci.13145 2019

  5. [5]

    On Multi-Cause Causal Inference with Unobserved Confounding: Counterexamples, Impossibility, and Alternatives

    On Multi-Cause Causal Inference with Unobserved Confounding: Counterexamples, Impossibility, and Alternatives.arXiv:1902.10286 [cs, stat](March 2019). http://arxiv.org/abs/1902.10286 arXiv: 1902.10286. Uplift Modeling Dataset

    work page internal anchor Pith review Pith/arXiv arXiv 1902

  6. [6]

    1999), 37–48

    Causal Diagrams for Epidemiologic Research.Epidemiology 10, 1 (Jan. 1999), 37–48. https://journals.lww.com/epidem/Abstract/1999/01000/Causal_Diagrams_for_Epidemiologic_ Research.8.aspx Pierre Gutierrez and Jean-Yves Gérardy

    1999

  7. [7]

    2013), 361–373

    To Whom, When, and How Much to Discount? A Constrained Optimization of Customized Temporal Discounts.Journal of Retailing89, 4 (Dec. 2013), 361–373. https://doi.org/10. 1016/j.jretai.2013.08.002 David B. Klenosky, Suzeanne B. Benet, and Petr Chadraba

    2013

  8. [8]

    https://doi.org/10.1016/0148- 2963(95)00121-2 Sören R

    Assessing Czech consumers’ reactions to western marketing practices: A conjoint approach.Journal of Business Research36, 2 (June 1996), 189–198. https://doi.org/10.1016/0148- 2963(95)00121-2 Sören R. Künzel, Jasjeet S. Sekhon, Peter J. Bickel, and Bin Yu

    work page doi:10.1016/0148- 1996

  9. [9]

    https://doi.org/10.1073/pnas.1804597116 Publisher: Proceedings of the National Academy of Sciences

    Metalearners for estimating heterogeneous treatment effects using machine learning.Proceedings of the National Academy of Sciences116, 10 (March 2019), 4156–4165. https://doi.org/10.1073/pnas.1804597116 Publisher: Proceedings of the National Academy of Sciences. Scott T. Leatherdale

    work page doi:10.1073/pnas.1804597116 2019

  10. [10]

    2019), 19–35

    Natural experiment methodology for research: a review of how different methods can support real-world research.International Journal of Social Research Methodology22, 1 (Jan. 2019), 19–35. https://doi.org/10. 1080/13645579.2018.1488449 Publisher: Routledge _eprint: https://doi.org/10.1080/13645579.2018.1488449. Ana Rita Nogueira, Andrea Pugnana, S. Ruggie...

    work page doi:10.1080/13645579.2018.1488449 2019

  11. [11]

    Discov.(2022)

    Methods and tools for causal discovery and causal inference.WIREs Data Mining Knowl. Discov.(2022). https://doi.org/10.1002/widm.1449 Judea Pearl

    work page doi:10.1002/widm.1449 2022

  12. [12]

    2009), 96–146

    Causal inference in statistics: An overview.Statistics Surveys3, none (Jan. 2009), 96–146. https: //doi.org/10.1214/09-SS057 Publisher: Amer. Statist. Assoc., the Bernoulli Soc., the Inst. Math. Statist., and the Statist. Soc. Canada. Judea Pearl and Elias Bareinboim

    work page doi:10.1214/09-ss057 2009

  13. [13]

    In 2011 IEEE 11th International Conference on Data Mining Workshops

    Transportability of Causal and Statistical Relations: A Formal Approach. In 2011 IEEE 11th International Conference on Data Mining Workshops. IEEE, Vancouver, BC, Canada, 540–547. https: //doi.org/10.1109/ICDMW.2011.169 Nicholas Radcliffe

    work page doi:10.1109/icdmw.2011.169 2011

  14. [14]

    https://www.research.ed.ac.uk/en/publications/using-control-groups-to- target-on-predicted-lift-building-and-ass Patrick Royston and Willi Sauerbrei

    Using control groups to target on predicted lift: Building and assessing uplift model.Direct Marketing Analytics Journal(2007), 14–21. https://www.research.ed.ac.uk/en/publications/using-control-groups-to- target-on-predicted-lift-building-and-ass Patrick Royston and Willi Sauerbrei

    2007

  15. [15]

    https: //doi.org/10.1002/sim.1815 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/sim.1815

    A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials.Statistics in Medicine23, 16 (2004), 2509–2525. https: //doi.org/10.1002/sim.1815 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/sim.1815. Willi Sauerbrei, Patrick Royston, and Karina Zapien

    work page doi:10.1002/sim.1815 2004

  16. [16]

    https://doi.org/10.1016/j.csda.2006.12.041 Greg Shaffer and Z

    Detecting an interaction between treatment and a continuous covariate: A comparison of two approaches.Computational Statistics & Data Analysis51, 8 (May 2007), 4054–4063. https://doi.org/10.1016/j.csda.2006.12.041 Greg Shaffer and Z. John Zhang

    work page doi:10.1016/j.csda.2006.12.041 2007

  17. [17]

    1995), 395–416

    Competitive Coupon Targeting.Marketing Science14, 4 (Nov. 1995), 395–416. https://doi.org/10.1287/mksc.14.4.395 Publisher: INFORMS. Deddy Jobson Yitao Shen, Yue Wang, Xingyu Lu, Feng Qi, Jia Yan, Yixiang Mu, Yao Yang, YiFan Peng, and Jinjie Gu

    work page doi:10.1287/mksc.14.4.395 1995

  18. [18]

    A framework for massive scale personalized promotion.arXiv:2108.12100 [cs](Aug. 2021). http://arxiv.org/abs/2108.12100 arXiv: 2108.12100. Gerard J. Tellis and Fred S. Zufryden

    work page arXiv 2021

  19. [19]

    https://econpapers.repec.org/article/inmormksc/v_3a14_3ay_ 3a1995_3ai_3a3_3ap_3a271-299.htm Publisher: INFORMS

    Tackling the Retailer Decision Maze: Which Brands to Discount, How Much, When and Why?Marketing Science14, 3 (1995), 271–299. https://econpapers.repec.org/article/inmormksc/v_3a14_3ay_ 3a1995_3ai_3a3_3ap_3a271-299.htm Publisher: INFORMS. Hirotsugu Uchida, Yuko Onozaka, Tamaki Morita, and Shunsuke Managi

    1995

  20. [20]

    2014), 68–76

    Demand for ecolabeled seafood in the Japanese market: A conjoint analysis of the impact of information and interaction with other labels.Food Policy44 (Feb. 2014), 68–76. https://doi.org/10.1016/j.foodpol.2013.10.002 Janet A. Young and Michelle D. Pain

    work page doi:10.1016/j.foodpol.2013.10.002 2014