pith. sign in

arxiv: 2607.02278 · v1 · pith:UUJLPTNZnew · submitted 2026-07-02 · 💰 econ.TH

Washed Out by the Crowd? Accountability under Sequential Review

Siming Ye This is my paper

Pith reviewed 2026-07-03 01:59 UTC · model grok-4.3

classification 💰 econ.TH
keywords sequential reviewaccountabilityincentiveshidden effortperformance payorganizational designinformation productionreview process
0
0 comments X

The pith

When reviewers can overturn an initial report, paying based on the final outcome fails to motivate the analyst's hidden effort, but paying based on agreement between first and last report retains incentive value.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper models an organization that must reward an analyst's hidden effort to make an early report more accurate, even though payments can be based only on the record left after a sequence of reviewers react to it. The key decomposition splits any such record's value into two parts: the share due to effort improving the first report, and the share that still signals whether the first report was accurate. When later reviewers overturn flawed reports, the process sharpens the final decision but removes the trace of the analyst's contribution from the outcome, so rewards tied to the final result stop working while rewards for agreement between the first and final word keep their incentive effect. The choice between these two reward schemes turns on a single parameter: the probability that the review process repairs an initial mistake. This setup applies to any setting where early information production is followed by sequential oversight, such as project approvals or hiring decisions.

Core claim

When later reviewers can overturn a flawed report, review improves decisions but washes the analyst's effort out of the final outcome; therefore, rewarding the final outcome stops working, while rewarding agreement between the first and last word has incentive value instead. Which reward is better comes down to one thing: how likely the review process is to repair an initial mistake.

What carries the argument

The decomposition of any observable record's incentive value into the component attributable to the analyst's effort on the initial report and the component that still indicates the accuracy of that initial report.

If this is right

  • Rewarding the final outcome provides incentives only when reviewers tend to copy the initial report rather than overturn it.
  • Rewarding agreement between the first and final report provides incentives when reviewers tend to correct mistakes in the initial report.
  • The relative performance of the two reward rules is determined solely by the probability that review repairs an initial error.
  • Organizations must match the reward rule to the correction behavior of their specific review chain to sustain early effort.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The result implies that increasing the independence or authority of later reviewers will strengthen the case for using agreement-based rather than outcome-based pay.
  • In lab or field experiments, varying the correction rate while holding payment rules fixed should produce the predicted switch in optimal incentives.
  • The same logic extends to chains with multiple early analysts, where each stage's contribution may be diluted by subsequent corrections.
  • Transparency rules that reveal reviewer actions could reduce the washing-out effect and change which reward rule is preferred.

Load-bearing premise

The organization can pay only on the final record left by the review sequence, with the analyst's effort affecting only the accuracy of the initial report and reviewers then reacting to it.

What would settle it

Measure whether analysts choose higher effort levels when compensated for agreement with the final report than when compensated for the final report itself, across review processes that differ in how often they correct initial errors.

Figures

Figures reproduced from arXiv: 2607.02278 by Siming Ye.

Figure 1
Figure 1. Figure 1: On-path accuracy and branch probabilities. Parameters are given in [PITH_FULL_IMAGE:figures/full_fig_p020_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Incentive power of the two endpoint records. The outcome record weakens with the [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Minimum success bonuses for κ = 0.005. The dashed line is the wage cap w¯ = 0.20. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Directional contestability under asymmetric priors. The horizontal axis is the public [PITH_FULL_IMAGE:figures/full_fig_p027_4.png] view at source ↗
read the original abstract

When an early information producer is judged only after others have reviewed and revised the work, the same review that sharpens the final decision can blur the question of who deserves the credit. This paper asks how an organization can still reward careful early work once a chain of later reviewers has acted on it. In the model, an analyst's hidden effort makes an initial report more likely to be right; a sequence of reviewers then reacts to it; and the organization can pay only on the record this process leaves behind. The main result splits the value of any such record into two parts: how much effort improved the first report, and how well the final record still indicates whether the first report was accurate. When later reviewers can overturn a flawed report, review improves decisions but washes the analyst's effort out of the final outcome; therefore, rewarding the final outcome stops working, while rewarding agreement between the first and last word has incentive value instead. When the first report is simply copied by downstream reviewers, the reverse holds. Which reward is better comes down to one thing: how likely the review process is to repair an initial mistake.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper models an analyst who exerts hidden effort to raise the accuracy of an initial report, followed by a sequence of reviewers who may revise or copy it. The organization is restricted to payments based solely on the observable final record. The main result decomposes the value of any record into an effort-improvement term and an accuracy-indication term. It establishes that outcome-based rewards lose power when reviewers can overturn flawed reports (washing out the analyst's effort), while agreement-based rewards retain incentive value; the ranking reverses when reports are copied downstream. The comparison turns only on the probability that the review process repairs an initial mistake.

Significance. If the decomposition and comparative-static result hold, the paper supplies a clean, parameter-light framework for incentive design in sequential review settings. It isolates a single observable—the repair probability—as the sole determinant of whether outcome or agreement rewards are preferred, yielding a directly testable prediction. The effort-versus-indication split is a reusable conceptual device for analyzing accountability when later agents can alter earlier work.

major comments (2)
  1. [§3] §3, main decomposition: the claim that record value splits additively into effort-improvement and accuracy-indication components appears to follow from the maintained assumption that the organization observes only the final record and that reviewer reactions are deterministic functions of the initial report. However, the derivation should explicitly show that no cross-term arises when the repair probability is interior; otherwise the comparative static on reward type could be an artifact of the functional-form restriction on reviewer behavior.
  2. [Proposition 2] Proposition 2 (or equivalent comparative-static result): the statement that outcome-based rewards are dominated precisely when the repair probability exceeds a threshold relies on the maintained institutional constraint that payments cannot be conditioned on the analyst's hidden effort. If the model allows even a small probability that the organization can observe a noisy signal of effort, the threshold result may not survive; the paper should state the robustness condition explicitly.
minor comments (2)
  1. [Abstract] The abstract and introduction use the phrase 'washes the analyst's effort out of the final outcome' without a formal definition; a one-sentence gloss linking it to the effort-improvement term in the decomposition would improve readability.
  2. [§2] Notation for the repair probability (denoted p in the abstract) should be introduced with a clear mapping to the underlying state space before the main propositions are stated.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive suggestions. Below we respond point by point to the two major comments, indicating planned revisions where appropriate.

read point-by-point responses

  1. Referee: §3, main decomposition: the claim that record value splits additively into effort-improvement and accuracy-indication components appears to follow from the maintained assumption that the organization observes only the final record and that reviewer reactions are deterministic functions of the initial report. However, the derivation should explicitly show that no cross-term arises when the repair probability is interior; otherwise the comparative static on reward type could be an artifact of the functional-form restriction on reviewer behavior.

    Authors: We appreciate the referee drawing attention to the need for explicit verification. The decomposition in §3 follows from applying the law of iterated expectations to the final record, conditioning separately on the initial report's accuracy and on the (deterministic) reviewer reaction. Because the repair event is independent of effort conditional on the initial report, the expectation factors without a cross-term for any interior repair probability 0 < p < 1. We will insert an additional displayed step in the proof of the decomposition (new equation (3.4)) that isolates the two additive components and confirms the cross-term is identically zero under the maintained assumptions. This step does not rely on further functional-form restrictions beyond those already stated. revision: yes

  2. Referee: Proposition 2 (or equivalent comparative-static result): the statement that outcome-based rewards are dominated precisely when the repair probability exceeds a threshold relies on the maintained institutional constraint that payments cannot be conditioned on the analyst's hidden effort. If the model allows even a small probability that the organization can observe a noisy signal of effort, the threshold result may not survive; the paper should state the robustness condition explicitly.

    Authors: The entire analysis is conducted under the explicit institutional restriction, stated in the model section, that the organization observes only the final record and cannot condition on any signal of the analyst's effort. This is the relevant setting for the paper's question. Introducing even a small probability of a noisy effort signal would indeed change the feasible contract space and could affect the threshold, but such an extension is outside the paper's scope. We will add a brief clarifying sentence immediately after the statement of Proposition 2 noting that the result holds under the maintained observability constraint and that robustness to partial effort observability remains an open question for future research. revision: partial

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper defines a model with explicit primitives—hidden analyst effort improving initial report accuracy, sequential reviewers reacting to it, and organizational payments restricted solely to the observable final record—and derives a decomposition of record value into an effort-improvement component and an accuracy-indication component. The central claim that outcome-based rewards lose incentive power when review repairs mistakes (while agreement-based rewards gain value) follows directly as an implication of this setup and the repair probability parameter, without any reduction to fitted inputs, self-definitional quantities, or load-bearing self-citations. The derivation remains self-contained against the stated institutional constraint.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The model rests on standard hidden-action assumptions in contract theory plus the institutional restriction that pay can be based only on the observable record; no free parameters or invented entities are visible in the abstract.

axioms (3)
  • domain assumption Analyst's hidden effort makes an initial report more likely to be right
    Stated explicitly in the abstract as the setup for the model.
  • domain assumption A sequence of reviewers then reacts to the initial report
    Stated explicitly in the abstract as the setup for the model.
  • domain assumption The organization can pay only on the record this process leaves behind
    Stated explicitly in the abstract as the institutional constraint.

pith-pipeline@v0.9.1-grok · 5715 in / 1534 out tokens · 33307 ms · 2026-07-03T01:59:52.910920+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages · 1 internal anchor

  1. [1]

    Luca Anderlini, Dino Gerardi, and Roger Lagunoff

    doi: 10.1093/restud/rdr004. Luca Anderlini, Dino Gerardi, and Roger Lagunoff. Social memory, evidence, and conflict.Review of Economic Dynamics, 13(3):559–574,

    work page doi:10.1093/restud/rdr004

  2. [2]

    Marco Angrisani, Antonio Guarino, Philippe Jehiel, and Toru Kitagawa

    doi: 10.1016/j.red.2009.06.006. Marco Angrisani, Antonio Guarino, Philippe Jehiel, and Toru Kitagawa. Information redundancy neglect versus overconfidence: A social learning experiment.American Economic Journal: Microeconomics, 13(3):163–197,

    work page doi:10.1016/j.red.2009.06.006 2009

  3. [3]

    Anil Arya, Jonathan Glover, and Brian Mittendorf

    doi: 10.1257/mic.20180394. Anil Arya, Jonathan Glover, and Brian Mittendorf. Hierarchical reporting, aggregation, and information cascades.Managerial and Decision Economics, 27(5):355–362,

    work page doi:10.1257/mic.20180394

  4. [4]

    Sushil Bikhchandani, David Hirshleifer, and Ivo Welch

    doi: 10.2307/2118364. Sushil Bikhchandani, David Hirshleifer, and Ivo Welch. A theory of fads, fashion, custom, and cultural change as informational cascades.Journal of Political Economy, 100(5):992–1026,

    work page doi:10.2307/2118364

  5. [5]

    Patrick Bolton and Mathias Dewatripont

    doi: 10.1086/261849. Patrick Bolton and Mathias Dewatripont. The firm as a communication network.Quarterly Journal of Economics, 109(4):809–839,

    work page doi:10.1086/261849

  6. [6]

    Gabriel Carroll

    doi: 10.2307/2118349. Gabriel Carroll. Robust incentives for information acquisition.Journal of Economic Theory, 181: 382–420,

    work page doi:10.2307/2118349

  7. [7]

    Hector Chade and Natalia Kovrijnykh

    doi: 10.1016/j.jet.2019.03.001. Hector Chade and Natalia Kovrijnykh. Delegated information acquisition with moral hazard.Journal of Economic Theory, 162:55–92,

    work page doi:10.1016/j.jet.2019.03.001 2019

  8. [8]

    Pierre Chaigneau, Alex Edmans, and Daniel Gottlieb

    doi: 10.1016/j.jet.2015.12.004. Pierre Chaigneau, Alex Edmans, and Daniel Gottlieb. The value of informativeness for contracting. Working Paper 20542, National Bureau of Economic Research,

    work page doi:10.1016/j.jet.2015.12.004 2015

  9. [9]

    Aubrey Clark and Giovanni Reggiani

    doi: 10.1257/aer.20231394. Aubrey Clark and Giovanni Reggiani. Contracts for acquiring information. arXiv:2103.03911,

    work page doi:10.1257/aer.20231394

  10. [10]

    KimonDrakopoulos, AsumanOzdaglar, andJohnN.Tsitsiklis

    doi: 10.3982/TE3388. KimonDrakopoulos, AsumanOzdaglar, andJohnN.Tsitsiklis. Onlearningwithfinitememory.IEEE Transactions on Information Theory, 59(10):6859–6872,

    work page doi:10.3982/te3388

  11. [11]

    Luis Garicano

    doi: 10.1109/TIT.2013.2262037. Luis Garicano. Hierarchies and the organization of knowledge in production.Journal of Political Economy, 108(5):874–904,

    work page doi:10.1109/tit.2013.2262037 2013

  12. [12]

    Marina Halac, Elliot Lipnowski, and Daniel Rappoport

    doi: 10.1086/317671. Marina Halac, Elliot Lipnowski, and Daniel Rappoport. Rank uncertainty in organizations.American Economic Review, 111(3):757–786,

    work page doi:10.1086/317671

  13. [13]

    40 Bengt Holmström

    doi: 10.1257/aer.20200555. 40 Bengt Holmström. Moral hazard and observability.Bell Journal of Economics, 10(1):74–91,

    work page doi:10.1257/aer.20200555

  14. [14]

    Ian Jewitt, Ohad Kadan, and Jeroen M

    doi: 10.2307/3003320. Ian Jewitt, Ohad Kadan, and Jeroen M. Swinkels. Moral hazard with bounded payments.Journal of Economic Theory, 143(1):59–82,

    work page doi:10.2307/3003320

  15. [15]

    Navin Kartik, SangMok Lee, Tianhao Liu, and Daniel Rappoport

    doi: 10.1016/j.jet.2007.12.004. Navin Kartik, SangMok Lee, Tianhao Liu, and Daniel Rappoport. Beyond unbounded beliefs: How preferences and information interplay in social learning.Econometrica, 92(4):1033–1062,

    work page doi:10.1016/j.jet.2007.12.004 2007

  16. [16]

    Thomas Marschak and Stefan Reichelstein

    doi: 10.3982/ECTA21470. Thomas Marschak and Stefan Reichelstein. Network mechanisms, informational efficiency, and hierarchies.Journal of Economic Theory, 79(1):106–141,

    work page doi:10.3982/ecta21470

  17. [17]

    Maneesha Papireddygari and Bo Waggoner

    doi: 10.1006/jeth.1997.2375. Maneesha Papireddygari and Bo Waggoner. Contracts with information acquisition, via scoring rules. InProceedings of the 23rd ACM Conference on Economics and Computation, pages 703–704,

    work page doi:10.1006/jeth.1997.2375 1997

  18. [18]

    Andrea Prat

    doi: 10.1145/3490486.3538261. Andrea Prat. The wrong kind of transparency.American Economic Review, 95(3):862–877,

    work page doi:10.1145/3490486.3538261

  19. [19]

    Raaj Kumar Sah and Joseph E

    doi: 10.1257/0002828054201297. Raaj Kumar Sah and Joseph E. Stiglitz. The architecture of economic systems: Hierarchies and polyarchies.American Economic Review, 76(4):716–727,

    work page doi:10.1257/0002828054201297

  20. [20]

    Lones Smith, Peter Norman Sørensen, and Jianrong Tian

    doi: 10.1111/1468-0262.00113. Lones Smith, Peter Norman Sørensen, and Jianrong Tian. Informational herding, optimal ex- perimentation, and contrarianism.Review of Economic Studies, 88(5):2527–2554,

    work page doi:10.1111/1468-0262.00113

  21. [21]

    Mark Whitmeyer and Kun Zhang

    doi: 10.1093/restud/rdab001. Mark Whitmeyer and Kun Zhang. Buying opinions. arXiv:2202.05249, revised July 2023,

    work page doi:10.1093/restud/rdab001 2023

  22. [22]

    Incentivizing Information Acquisition

    Fan Wu. Incentivizing information acquisition. arXiv:2410.13978, revised May 2026,

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  23. [23]

    Expert incentives under partially contractible states

    Zizhe Xia. Expert incentives under partially contractible states. arXiv:2508.10170, revised November 2025,

    work page arXiv 2025