Pith. sign in

REVIEW 4 major objections 5 minor 14 references

Upper-funnel ads that successfully create demand can lower a platform's measured performance when buyers complete purchases on trusted marketplaces, because attribution systems miss those conversions.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 01:45 UTC pith:NSE7SJAB

load-bearing objection Clean formalization of a real measurement pathology plus a practical ambient-ITT + individual-PIE design; theory is sound, evidence is simulation-only and commercially affiliated. the 4 major comments →

arxiv 2607.09608 v1 pith:NSE7SJAB submitted 2026-07-10 econ.EM

Media Measurement and the Assisted Own Goal: Attribution, Marketing-Mix Models, and Individual-Level Incrementality

Tobias Konitzer (GrowthLoop) This is my paper
classification econ.EM
keywords media measurementadvertising attributionROASincrementalitymarketing-mix modelingintent-to-treatplatform competitiondemand generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that a demand-generating ad platform can cause incremental purchases that are booked and credited elsewhere when buyers distrust its checkout and migrate to a trusted marketplace. Under attribution-based ROAS, those diverted conversions are invisible, so measured returns equal true incremental returns scaled down by an observability factor that falls with distrust and rises with recovery of off-platform signals. Budget rules that threshold on measured ROAS therefore underfund the generating platform, and can even make greater true ad effectiveness reduce its attributed revenue—the assisted own goal. Marketing-mix models see the diverted sales in total but often re-credit them to the harvesting channel whose spend tracks the arriving demand. The constructive claim is that ambient audience-level intent-to-treat holdouts, paired with individual-level prediction of experiment-identified lift, recover the true effect because contrasts are formed on channel-complete outcomes.

Core claim

Whenever distrust diverts purchases off the generating platform and recovery of those signals is incomplete, measured ROAS understates true incremental ROAS by the observability factor κ = 1 − τ(1 − ϕ). ROAS-thresholding then produces equilibrium under-investment in the generator, and greater true effectiveness can lower its attributed revenue. Randomized intent-to-treat contrasts on channel-complete outcomes are invariant to diversion, recovery, and harvester claim share, so the own goal disappears under that measurement design.

What carries the argument

The attribution wedge κ = 1 − τ(1 − ϕ), which scales measured ROAS relative to true incremental ROAS, together with ambient audience-level intent-to-treat randomization on channel-complete outcomes and an individual-level mapping from features to experiment-identified incremental effects.

Load-bearing premise

The design requires that the brand can observe every purchase—wherever it is booked—for people assigned to treatment or control, and that the people who can be caused to convert are listable before the ads run.

What would settle it

In a setting with known high off-platform diversion, run ambient holdouts and compare three quantities: channel-complete ITT lift, attributed ROAS, and an independent measure of true incremental demand. The claim requires ITT to recover true incremental demand while attributed ROAS remains near κ times truth; failure of ITT to recover the diverted sales falsifies the immunity claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Advertisers who allocate solely on attribution-based ROAS systematically underfund upper-funnel, demand-generating channels relative to first-best.
  • A platform that improves creative or targeting can lose measured revenue if the extra demand it creates is disproportionately diverted off-platform.
  • Marketing-mix models that include the harvester’s sponsored spend can re-credit diverted demand to the harvester through endogeneity, even when total sales are observed.
  • Always-on ambient experiments supply a standing library of ground-truth lifts that can be projected to audiences and campaigns without holdouts.
  • Individual-level predicted incrementality can rank users for the next audience, concentrating spend on the most persuadable people.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If brands can estimate diversion and recovery rates, they could apply an explicit κ correction to attributed ROAS as a partial fix without full experimentation.
  • The same diversion logic likely extends to digital ads that drive offline purchases at competing retailers, not only marketplace checkout.
  • Competition over checkout trust becomes a measurement contest: platforms that suppress referrer signals raise the wedge against generators.
  • When acquisition audiences cannot be enumerated before exposure, geo or budget-pulse randomization on channel-complete outcomes remains unbiased but coarser.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper formalizes the 'assisted own goal': upper-funnel demand generation on platform G can create incremental purchases that are booked on a trusted marketplace R, so attribution-based ROAS understates G's true returns by the observability factor κ(τ,φ)=1−τ(1−φ) (Proposition 1). Under ROAS-thresholding, this induces equilibrium under-investment sattr_G/strue_G=κ^{1/(1−β)} (Proposition 2), and can even make higher true effectiveness lower G's attributed revenue when diversion rises with α. MMM sees diverted demand only in aggregate and can re-credit it to R via endogenous sponsored spend; channel-complete ITT is invariant to τ, φ, and η (Proposition 3). The constructive proposal is ambient audience-level ITT via deterministic salted hashing plus an individual-level extension of PIE that maps features to experiment-identified δ(x). A simulation from the Section 2 DGP illustrates the wedge, the backfire region, and recovery by ITT versus attribution and MMM.

Significance. If the mechanism is quantitatively important, the paper supplies a clean, portable diagnosis of why attribution systematically defunds upper-funnel platforms and why MMM can re-credit diverted demand, together with a scalable operational alternative (ambient ITT + individual-level PIE) that is immune by construction when outcomes are channel-complete. Propositions 1–3 are transparent and immediately usable for advertisers and platforms. The main limitations are that the simulation is generated from the same DGP (so it cannot independently validate the theory) and that immunity and ambient design rest on channel-complete first-party Y and pre-exposure enumerability of the assignment universe—conditions the paper itself flags. Within those bounds the contribution is useful for media measurement and platform competition.

major comments (4)
  1. [Section 2.7, eqs. (7)–(8)] Section 2.7: the derivation of sattr_G and strue_G (equations 7–8) is introduced with 'Skipping some math.' For a formal proposition that underpins the under-investment claim, the first-order condition from ROAS-thresholding (marginal measured ROAS = λ) should be written out explicitly so that the inversion of the power demand function is checkable without reconstruction.
  2. [Section 3.1] Section 3.1 (effectiveness backfires): the own-goal region is generated by letting τ rise with α, τ(α)=0.5+0.25(α−1). This is an auxiliary assumption, not derived from the base model of Section 2. Either microfound why marginal buyers recruited by higher α have systematically lower trust, or reframe the backfire as a conditional possibility rather than a central implication of the assisted own goal.
  3. [Section 6] Section 6: the simulation is drawn from the Section 2 DGP and therefore reproduces κ and ITT≈truth by construction. It usefully illustrates magnitudes but does not independently validate the theory or the individual-level PIE extension. At minimum, state this limitation clearly; ideally add a sensitivity check with misspecified τ/φ or non-power demand, or a sketch of how real first-party data would be used to estimate δ(x).
  4. [Sections 5.2–5.4, Proposition 3] Sections 5.2–5.4: Proposition 3 and the ambient design require channel-complete first-party outcomes and an enumerable assignment universe before exposure. Acquisition/lookalike campaigns and platform-observed-only outcomes break both. The paper notes the constraint and the transportability assumption for acquisition, but the operational claim that the own goal 'disappears' should be scoped more tightly to list-based / retargeting settings where these conditions hold, with clearer guidance on when geo or budget-pulse designs are the fallback.
minor comments (5)
  1. [Abstract / References] Abstract and Introduction: '[CITE]' and incomplete GrowthLoop URLs in the reference list should be replaced with proper citations before publication.
  2. [Sections 2 and 6] Notation: recovery rate is written both φ and φ (and φ in the simulation text); standardize on one symbol throughout.
  3. [Table 1, Figure 1] Table 1 and Figure 1 are helpful; ensure panel labels and the own-goal region in (b) are readable in grayscale and that confidence intervals in (c) are defined in the caption.
  4. [Section 5.3] Section 5.3: the nesting claim that campaign-level Δ̂_a = E[δ̂(X_i)|i∈a] is correct but brief; a one-line aggregation identity would help readers connect individual-level PIE to Gordon et al. (2023).
  5. [Introduction / Section 4.3] JEL and keywords are appropriate; a short related-work paragraph situating ambient hashing relative to ghost ads (Johnson et al. 2017) and geo experiments would strengthen the positioning.

Circularity Check

1 steps flagged

No significant circularity: Props 1–3 are transparent algebraic consequences of the stated model; the only mild loop is the Section 6 simulation regenerating its own DGP.

specific steps
  1. self definitional [Section 6, Figure 1(a) and surrounding text]
    "Consumer journeys are drawn from the data-generating process of Section 2: spend creates intent (β=0.5), each intent converts on the marketplace with probability τ and on the generator's storefront with probability 1−τ, and the attribution layer matches on-platform purchases perfectly but recovers diverted purchases only with recovery rate φ. ... The simulated ratios (circles in panel a) mimic κ=1−τ(1−φ) to three decimal places"

    The simulation regenerates the exact DGP that defines κ, then reports that attributed/true ratios match κ. That match is forced by construction of the data, not an independent check of the wedge. The paper frames it as illustration rather than out-of-sample prediction, so the circularity is mild and does not infect Props 1–3.

full rationale

The load-bearing claims are model-internal comparative statics, not fitted predictions. Proposition 1 follows by algebra once κ is defined from (τ, φ) and attributed conversions are written as κq; Proposition 2 is the standard inversion of a power-function ROAS-thresholding rule; Proposition 3 is the textbook invariance of a randomized ITT contrast on a channel-complete outcome. None of these steps smuggles the conclusion into the premises beyond ordinary definitional setup of a theoretical model. GrowthLoop self-citations (2026a/b) document the production hashing design but are not used as uniqueness theorems or as fitted parameters that force the formal results. The only mild circularity is the simulation study, which draws journeys from the Section 2 DGP and therefore recovers κ and ITT≈truth by construction; the paper presents this as an illustration of the model rather than independent empirical validation, so it does not elevate the score above a minor flag. The derivation chain is otherwise self-contained.

Axiom & Free-Parameter Ledger

7 free parameters · 8 axioms · 4 invented entities

The central claims rest on a reduced-form demand and trust model, a ROAS-threshold budget rule, channel-complete outcomes, and ambient hashed assignment—not on fitted physical constants. Free parameters in the simulation and comparative statics are chosen by hand. Invented entities are mostly named mechanisms and productized designs rather than new physical objects; independent field evidence for the size of the own goal is not provided.

free parameters (7)
  • α (demand effectiveness scale)
    Scale of intent production in q(s_G)=α s_G^β; varied in backfire simulation, not estimated from data.
  • β (returns-to-scale exponent)
    Set to 0.5 in simulation; governs under-investment ratio exponent 1/(1−β).
  • τ (distrust / diversion share)
    Population share booking on R; grid and τ(α) path chosen for figures, not measured.
  • φ (off-platform recovery rate)
    Match probability for R bookings back to G; set to 0 or 0.1/0.3 in sims.
  • λ (ROAS hurdle)
    Brand’s measured-ROAS threshold that pins sattr_G; free policy parameter.
  • η (harvester claim share)
    Share of diverted conversions R attributes to its sponsored placements; used in reallocation narrative.
  • m (unit margin)
    Product margin entering ROAS definitions; normalized role in spend formulas.
axioms (8)
  • domain assumption Incremental intent is q(s_G)=α s_G^β with β∈(0,1).
    Section 2.2 power demand-generation function; standard but untested here.
  • domain assumption Given intent, Pr(book on R)=τ and Pr(book on G)=1−τ; total purchases equal intent.
    Section 2.3 trust-driven channel choice; heart of diversion mechanism.
  • domain assumption Attributed conversions equal κ(τ,φ)q with κ=1−τ(1−φ).
    Section 2.5 attribution measurement layer; defines the wedge.
  • domain assumption Brand invests until measured marginal ROAS equals hurdle λ.
    Section 2.7 ROAS-thresholding allocation rule driving Prop. 2.
  • domain assumption Outcome Y is channel-complete first-party purchases; Z is randomized assignment (ITT).
    Section 5.2; required for Prop. 3 immunity.
  • ad hoc to paper Deterministic salted hash yields valid independent audience-level holdouts with negligible cross-experiment interference.
    Section 5.1 ambient design as deployed by GrowthLoop; SUTVA discussed via self-cite.
  • ad hoc to paper Conditional ITT δ(x) is transportable from experimented audiences to unheld-out and (under strong assumptions) acquisition audiences.
    Section 5.3–5.4 projection and lookalike discussion; author flags common-support strain.
  • standard math Standard potential-outcomes / randomization identification of ITT.
    Used for Δ_ITT and δ(x) throughout Section 5.
invented entities (4)
  • Assisted own goal (measurement pathology) no independent evidence
    purpose: Name and organize the mechanism by which successful demand generation can lower the generator’s attributed revenue and budget.
    Framing device for Sections 1–3; not an external physical entity. Independent evidence would be field budget/ROAS series under diversion—not provided beyond simulation.
  • Observability factor κ(τ,φ) no independent evidence
    purpose: Scalar linking true and attributed ROAS under partial recovery of diverted sales.
    Definitional construct in Eq. (4); useful but not independently measured in real platforms here.
  • Ambient audience-level randomization (GrowthLoop-style salted holdouts) no independent evidence
    purpose: Make every activated audience an ITT experiment without separate experiment ops.
    Productized design in Section 5.1; described via company docs. Falsifiable in production A/A and holdout audits, but not shipped with this paper.
  • Individual-level PIE extension δ(x) no independent evidence
    purpose: Map person features to experiment-identified incremental outcomes and project beyond holdout coverage.
    Extension of Gordon et al. campaign-level PIE; no trained model or OOS metrics reported in this manuscript.

pith-pipeline@v1.1.0-grok45 · 14512 in / 4331 out tokens · 47420 ms · 2026-07-13T01:45:12.163479+00:00 · methodology

0 comments
read the original abstract

We use the assisted own goal hypothesis as a lens into media measurement. A demand-generating (upper-funnel) advertising platform such as a short-video social network can cause an incremental purchase, yet see that purchase booked on -- and credited to -- a downstream trusted marketplace, because consumers who discover a product on the platform complete the transaction elsewhere, for example because of distrust of the generating platform as a psychological mechanism. Under attribution-based return-on-ad-spend (ROAS) measurement, the diverted conversions are invisible to the originating platform. Marketing-mix models (MMMs) do not know which channel to credit with the outcome, and channel-by-week aggregation denies the audience-level granularity that budget decisions require. We develop an incrementality-based measurement model with two ingredients: ambient audience-level randomization -- each activated audience carries its own intent-to-treat (ITT) experiment -- and an individual-level extension of Predicted Incrementality by Experimentation (PIE), which learns a mapping from individual features to experiment-identified incremental outcomes. Because ITT contrasts are computed on channel-complete outcomes, the estimator is unbiased and the own goal disappears

Figures

Figures reproduced from arXiv: 2607.09608 by Tobias Konitzer (GrowthLoop).

Figure 1
Figure 1. Figure 1: The own goal in simulation. (a) Attributed-to-true ROAS ratios from 200,000 simulated [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references

  1. [1]

    Abhishek, V., Fader, P., & Hosanagar, K. (2017). Media exposure through the funnel: A model of multi-stage attribution.Working paper

    2017

  2. [2]

    Berman, R. (2018). Beyond the last touch: Attribution in online advertising.Marketing Science, 37(5), 771–792

    2018

  3. [3]

    Blake, T., Nosko, C., & Tadelis, S. (2015). Consumer heterogeneity and paid search effectiveness: A large-scale field experiment.Econometrica, 83(1), 155–174

    2015

  4. [4]

    Chan, D., & Perry, M. (2017). Challenges and opportunities in media mix modeling.Google Research Technical Report

    2017

  5. [5]

    Einav, L., Farronato, C., & Levin, J. (2016). Peer-to-peer markets.Annual Review of Economics, 8, 615–635

    2016

  6. [6]

    Acomparisonofapproachesto advertising measurement: Evidence from big field experiments at Facebook.Marketing Science, 38(2), 193–225

    Gordon, B.R., Zettelmeyer, F., Bhargava, N., &Chapsky, D.(2019). Acomparisonofapproachesto advertising measurement: Evidence from big field experiments at Facebook.Marketing Science, 38(2), 193–225

    2019

  7. [7]

    R., Moakler, R., & Zettelmeyer, F

    Gordon, B. R., Moakler, R., & Zettelmeyer, F. (2023). Predicted incrementality by experimentation (PIE) for ad measurement.arXiv preprint arXiv:2304.06828. audience-level-randomization-vs-global-control. https://a.storyblok.com/f/340788/x/3afcf1437d/audience-level-randomization-vs-global-control-exec-summary-tk-7-2026. pdf. 11 Always-on Measurement fragme...

    arXiv 2023

  8. [8]

    Jin, Y., Wang, Y., Sun, Y., Chan, D., & Koehler, J. (2017). Bayesian methods for media mix modeling with carryover and shape effects.Google Research Technical Report

    2017

  9. [9]

    Johnson, G. A. (2023). Inferno: A guide to field experiments in online display advertising.Journal of Economics & Management Strategy, 32(3), 469–490

    2023

  10. [10]

    A., Lewis, R

    Johnson, G. A., Lewis, R. A., & Nubbemeyer, E. I. (2017). Ghost ads: Improving the economics of measuring online ad effectiveness.Journal of Marketing Research, 54(6), 867–884. Künzel, S. R., Sekhon, J. S., Bickel, P. J., & Yu, B. (2019). Metalearners for estimating heteroge- neous treatment effects using machine learning.Proceedings of the National Acade...

    2017

  11. [11]

    A., & Rao, J

    Lewis, R. A., & Rao, J. M. (2015). The unfavorable economics of measuring the returns to adver- tising.Quarterly Journal of Economics, 130(4), 1941–1973

    2015

  12. [12]

    Shao, X., & Li, L. (2011). Data-driven multi-touch attribution models.Proceedings of the 17th ACM SIGKDD, 258–264

    2011

  13. [13]

    Tadelis, S. (2016). Reputation and feedback systems in online platform markets.Annual Review of Economics, 8, 321–340

    2016

  14. [14]

    Wager, S., & Athey, S. (2018). Estimation and inference of heterogeneous treatment effects using random forests.Journal of the American Statistical Association, 113(523), 1228–1242. 12

    2018