Pith. sign in

REVIEW 2 major objections 4 minor 13 references

When the reported object is a dynamic local-projection response, switchback persistence should be chosen to minimize that object's design risk—not a generic average effect.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 03:48 UTC pith:7T6GMUC7

load-bearing objection Clean closed-form persistence rule for LP-targeted Markov switchbacks, with honest calibration that the formula is a benchmark not a field prescription. the 2 major comments →

arxiv 2607.11694 v1 pith:7T6GMUC7 submitted 2026-07-13 stat.ME econ.EM

Calibrated Horizon-Weighted Local Projection Designs for Markov Switchbacks

Makoto Nakakita , Teruo Nakatsuma This is my paper
classification stat.ME econ.EM
keywords dynamic experimental designlocal projectionsswitchback experimentsMarkov assignmentoptimum input designdemand responsehorizon-weighted riskHAC calibration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that switchback experiments should be designed for the dynamic response curve or horizon-weighted functional that researchers plan to report, not for a contemporaneous average treatment effect. In a balanced first-order Markov class, the lagged-assignment information matrix is AR(1)-Toeplitz with a tridiagonal inverse, so the risk of a pre-specified contrast has a closed form and maps the shape of the reporting weights into a recommended persistence: iid for isolated horizons, high persistence for smooth cumulative targets, and moderate or alternating designs for rebound contrasts. Field recommendations replace that benchmark covariance with residualized, HAC, pilot, residual-bootstrap, or realized-schedule risk of the estimator actually used. A semi-synthetic Low Carbon London evaluation injects known responses into realistic half-hourly load dynamics and shows when the closed form should be replaced by calibrated selection, and when high-persistence designs need randomization-first inference because few active spells undermine normal approximations. The practical claim is that the assignment path is chosen after the reporting object is fixed.

Core claim

The horizon-weighted local-projection (HW-LP) design criterion treats the pre-specified dynamic response or contrast as the design target. Under balanced homoskedastic Markov switchbacks, risk has the closed form Rc(r)=4σ²(a+br²−2dr)/(1−r²), with interior optimum r_int_c=((a+b)−√((a+b)²−4d²))/(2d) when 0<|d|<(a+b)/2; otherwise the rule is iid or a feasible endpoint. Field use minimizes tr{W V̂(r)} over the feasible class when residual dependence or calendar residualization departs from the Toeplitz benchmark.

What carries the argument

The HW-LP risk tr{W Vπ} for a pre-specified weight matrix W on the path-adjusted local-projection response curve, specialized under balanced Markov assignment to the closed-form contrast risk Rc(r)=4σ²(a+br²−2dr)/(1−r²) because Q_H(r)=(1/4)(r^{|i−j|}) has a tridiagonal inverse.

Load-bearing premise

The response after residualizing calendar controls is a design-invariant finite-memory linear path, and the closed-form covariance treats errors as conditionally homoskedastic and serially uncorrelated given the assignment path; if carryover, compliance, or residual dependence break that structure, the closed form is only a diagnostic.

What would settle it

In a semi-synthetic or pilot setting with known response paths and realistic residual autocovariance, check whether the closed-form persistence still minimizes target risk relative to the calibrated selector over the same feasible grid; if calibrated risk for delayed or rebound targets systematically prefers a different r and the gap exceeds the paper's tolerance (for example the delayed LCL case), the benchmark rule fails as a field prescription.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies temporal assignment design for Markov switchback experiments when the reported object is a horizon-weighted local-projection (HW-LP) target. It defines design risk as tr{W V_π} for a pre-specified response curve or contrast, derives a closed-form optimal persistence for balanced homoskedastic first-order Markov switchbacks from the AR(1)-Toeplitz information matrix and its tridiagonal inverse, and maps target shapes (immediate, cumulative, delayed, rebound) into persistence recommendations. Field use is redirected to a calibrated selector that minimizes residualized, HAC, pilot, residual-bootstrap, or realized-schedule risk; omitted-carryover local-bias sensitivity, buffer horizons, near-boundary randomization-first inference, and a semi-synthetic Low Carbon London evaluation complete the implementation layer.

Significance. If the results hold, the paper gives a clean, target-specific design rule for dynamic experiments that report multi-horizon LP objects rather than scalar ATEs. The closed form Rc(r)=4σ²(a+br²−2dr)/(1−r²) and r_int_c are transparent and inherit standard Toeplitz precision facts while translating them into a binary switchback feasible class; the calibrated-selector, omitted-tail, and LCL layers make the contribution operational rather than purely formal. Strengths include careful target-invariance of path-adjusted LP (Lemma 1 vs Proposition 1), explicit proofs in Appendix D, pilot-regret and HAC uniformity statements, and a semi-synthetic evaluation that falsifies mechanical use of the closed form when residualized covariance departs from the benchmark (Table 6 delayed row). This is a useful design-stage contribution for demand-response, platform, and other high-frequency switchback settings.

major comments (2)
  1. The central closed-form claim is sound under the stated scope (Propositions 3–6; Appendix D), and the paper already treats Assumption 1 and the homoskedastic Proposition 4 covariance as a diagnostic rather than a universal prescription. The main load-bearing empirical claim—that calibrated selection should replace the closed form when residualized covariance differs—rests heavily on the delayed-reduction stress test in Table 6 (and the selector comparison in Table 7). That finding is important, but the manuscript should state more sharply which residual features (calendar residualization, load autocovariance, finite-sample Gram dispersion) drive the r=0.382→0.70 shift, and whether the same shift appears under alternative residualization menus or bandwidth multipliers beyond Table 2. Without that decomposition, the field rule “switch when calibrated gain exceeds 20%” remains somewhat prot
  2. Near-boundary inference is correctly flagged (Lemma 3, Table 4, Section 6.2), but the operational trigger (r>0.95, binding HAC cap, or low active-spell tail) is still somewhat ad hoc relative to the many-spell scaling Tp(1−p)(1−r). For cumulative targets the variance-only optimum is the upper endpoint, so the paper’s practical recommendation often lands exactly where normal/HAC approximations are weakest. A short pre-analysis simulation protocol—reporting active-spell quantiles, Fisher/Neymanian power, and coverage under the locked design before field deployment—should be elevated from diagnostic text into a required step whenever the selected design is a boundary or near-boundary design.
minor comments (4)
  1. Figure 1 and several appendix figures use scientific notation and clipped axes; a brief note in the caption on normalization (risk relative to the target-specific minimum) would help readers compare panels.
  2. Notation for the feasible set R, active share p0, and residualized Q_{H,Z}(r) is introduced in stages; a short notation table early in Section 4 would reduce cross-referencing.
  3. The multi-arm diagnostic in Appendix Figure A.3 is useful but dense; one sentence in the main text stating that directional cycling can raise alternating-target risk by ~45% under the reported δ=0.80 case would better motivate the scope limit.
  4. Minor copy-editing: a few long sentences in Sections 3.1 and 8.1 could be split; check consistency of “HW-LP” hyphenation and of r_max vs rmax in tables.

Circularity Check

0 steps flagged

No significant circularity: closed-form HW-LP risk follows from the AR(1)-Toeplitz information matrix under stated assumptions; field use and LCL evaluation do not fit persistence to recover a pre-chosen target.

full rationale

The load-bearing closed form Rc(r)=4σ²(a+br²−2dr)/(1−r²) and r_int_c are obtained by substituting the standard tridiagonal inverse of the balanced Markov AR(1)-Toeplitz matrix QH(r) into c′VH(r)c under conditional homoskedasticity (Props. 3–6). That inverse is a classical precision-matrix fact the paper explicitly inherits from optimal input design; the novel layer is the LP reporting-object map and binary Markov feasible class, not a re-derivation that smuggles the answer. Target invariance (Lemma 1) and naive-LP design dependence (Prop. 1) are algebraic consequences of the finite-memory model, not fits. Field recommendations replace the benchmark with min tr{W V̂(r)} (Props. 5, 8) when residualization or serial dependence departs; the LCL delayed-target stress test (Table 6) falsifies mechanical use of the closed form. Semi-synthetic selection uses c′V̂d c, not injected g magnitudes; historical tariffs are fixed-schedule information diagnostics only. No self-citation uniqueness theorem, fitted-input-as-prediction, or self-definitional loop is load-bearing for the strongest claim.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 1 invented entities

The closed-form claim rests on standard Markov/Toeplitz linear-algebra facts plus domain assumptions about finite-memory path responses and conditional error moments. Field claims add operational free parameters (bandwidth, switch thresholds, bias radii, conditioning caps) that are pre-specified protocol knobs rather than estimated scientific constants. No new physical entities are postulated; HW-LP is a design criterion built from existing LP and OID pieces.

free parameters (6)
  • HAC bandwidth rule b_T(r) and multipliers
    Operational formula b0_T(r)=⌈1.3 T^{1/3}(1+|r|)/(1−|r|)⌉ with caps at T/4 and 0.25 Teff; sensitivity at 0.25–4×. Chosen by hand as a conservative rule, not an optimal-bandwidth theorem.
  • Calibrated-vs-benchmark switch threshold (default 20%)
    Pre-specified relative-risk gain threshold for replacing closed form with calibrated selector; sensitivity at 2/5/10/20% reported. Protocol input affecting field recommendations.
  • Local omitted-tail radius M0 grid
    Bias-augmented criterion uses M0∈{0,0.5,1,2}× pilot tail norm (or frontier over M0). Sensitivity radius chosen by protocol, not identified from confirmatory data.
  • Numerical conditioning caps κ2(Q) > 10^10 / 10^12
    Hard/soft exclusions for near-singular designs; affect feasible R and when regularization is required.
  • Application feasible r interval (e.g. [−0.7,0.7]) and active-share budgets p0
    Operational constraints (run length, fatigue, event budget) that pin boundary optima for cumulative targets; design inputs, not estimated parameters.
  • Ridge diagnostic η for realized-schedule Q̂
    Default η=10^{−3} times λ_max(Q̂) for Moore–Penrose/ridge diagnostics on historical schedules; sensitivity at 10^{−4} and 10^{−2}.
axioms (6)
  • domain assumption Finite-memory linear projection of residualized potential outcomes on lagged assignments with design-invariant g (Assumption 1(iv)).
    Load-bearing for target invariance and the distributed-lag information matrix; omitted tails handled only via local-bias sensitivity.
  • domain assumption No anticipation and sequential design exogeneity of assignment innovations (Assumption 1(ii)–(iii)).
    Standard potential-outcome timing restrictions for time-series experiments; required for causal reading of g.
  • domain assumption Stationary two-state Markov assignment with |r|<1 and interior feasible set for mixing and nonsingular Q_H (Assumptions 3, Lemma 2).
    Defines the design class in which closed-form persistence is derived.
  • domain assumption Conditional mean-zero, homoskedastic, serially uncorrelated errors given the assignment path for V=σ²Q^{-1} (Proposition 4 benchmark).
    Gives the clean closed form; paper replaces this with HAC/long-run covariance for field use.
  • standard math Inverse of AR(1) correlation/Toeplitz matrix is tridiagonal (standard linear algebra).
    Inherited precision-matrix fact used to obtain Rc(r) and r_int.
  • standard math Uniform HAC consistency under mixing, moments, and bandwidth conditions (Assumption 2, Proposition 5).
    Supports uniform risk estimation over compact interior R for the calibrated selector.
invented entities (1)
  • Horizon-weighted local projection (HW-LP) design criterion R_W(π)=tr{W V_π} independent evidence
    purpose: Makes the pre-specified dynamic reporting object the design target for switchback persistence selection.
    Not a physical entity but a named design loss; independent content is the mapping to Markov r and calibrated implementation, not a new latent force.

pith-pipeline@v1.1.0-grok45 · 52229 in / 3999 out tokens · 38523 ms · 2026-07-14T03:48:46.102720+00:00 · methodology

0 comments
read the original abstract

We study temporal assignment design for Markov switchback experiments when the reported object is a dynamic local-projection target. We develop a calibrated selector that chooses the feasible persistence minimizing the covariance, HAC, residual-bootstrap, or realized-schedule risk of the estimator and reporting object specified before the experiment. A balanced homoskedastic Markov benchmark yields a closed form because the lagged-assignment information matrix is AR(1)-Toeplitz with a tridiagonal inverse. The benchmark maps local-projection reporting weights into persistence recommendations within a prespecified first-order Markov class. Field recommendations replace the benchmark covariance with residualized, serially dependent, pilot-calibrated, or randomization-based risk. A semi-synthetic Low Carbon London evaluation uses observed half-hourly baseline dynamics and known injected responses to assess design risk. It evaluates the covariance calculations under realistic load autocovariance and identifies when calibrated covariance selection should replace the homoskedastic Markov formula. Near-boundary designs use randomization-first inference when many-spell normal approximations are unsupported.

Figures

Figures reproduced from arXiv: 2607.11694 by Makoto Nakakita, Teruo Nakatsuma.

Figure 1
Figure 1. Figure 1: Target-specific risk and variance-minimizing persistence. Normalized HW-LP risk [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Persistence, effective sample size, and candidate-specific HAC bandwidths for [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Unknown carryover creates a bias–variance design tradeoff. The left panel shows [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Power curves expressed as 80 percent MDEs in residual-standard-deviation units. The [PITH_FULL_IMAGE:figures/full_fig_p039_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 3 canonical work pages

  1. [1]

    P. M. Aronow and C. Samii. Estimating average causal effects under general interference, with application to a social network experiment.Annals of Applied Statistics, 11(4):1912–1947,

    1912

  2. [2]

    arXiv preprint arXiv:2603.20394. K. Chaloner and I. Verdinelli. Bayesian experimental design: A review.Statistical Science, 10 (3):273–304,

    arXiv

  3. [3]

    doi:10.1145/2783258.2788583. S. R. Howard, A. Ramdas, J. McAuliffe, and J. Sekhon. Time-uniform, nonparametric, nonasymp- totic confidence sequences.Annals of Statistics, 49(2):1055–1080,

    doi:10.1145/2783258.2788583

  4. [4]

    arXiv preprint arXiv:2312.15574. Ò. Jordà. Estimation and inference of impulse responses by local projections.American Economic Review, 95(1):161–182,

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2510.22864. D. V. Lindley. On a measure of the information provided by an experiment.Annals of Mathematical Statistics, 27(4):986–1005,

    arXiv

  6. [6]

    doi:10.1016/j.erss.2018.06.008. E. Pesavento and B. Rossi. Impulse response confidence intervals for persistent data: What have we learned?Journal of Economic Dynamics and Control, 31(7):2398–2412,

    work page doi:10.1016/j.erss.2018.06.008 2018

  7. [7]

    arXiv preprint arXiv:1903.01637, revised January

    Pith/arXiv arXiv 1903

  8. [8]

    Strbac, S

    G. Strbac, S. H. Tindemans, M. Woolf, M. Bilton, R. Carmichael, and J. R. Schofield. Low Carbon London Project: Data from the Dynamic Time-of-Use Electricity Pricing Trial, 2013 [data collection]. UK Data Service, Study 7857,

    2013

  9. [9]

    [dataset], doi:10.5255/UKDA-SN-7857-2. A. Wald. Sequential tests of statistical hypotheses.Annals of Mathematical Statistics, 16(2): 117–186,

    work page doi:10.5255/ukda-sn-7857-2

  10. [10]

    arXiv preprint arXiv:2409.09577, revised February

    arXiv

  11. [11]

    arXiv preprint arXiv:2406.06768. T. Yunusov and J. Torriti. Distributional effects of Time of Use tariffs based on electricity demand and time use.Energy Policy, 156:112412,

    Pith/arXiv arXiv

  12. [12]

    doi:10.1016/j.enpol.2021.112412. M. B. Zarrop.Optimal Experiment Design for Dynamic System Identification. Springer,

    work page doi:10.1016/j.enpol.2021.112412 2021

  13. [13]

    arXiv preprint arXiv:2604.02489. 57

    Pith/arXiv arXiv