pith. sign in

arxiv: 2606.28200 · v1 · pith:QPEA647Mnew · submitted 2026-06-26 · 📊 stat.ME

Experimental Design When N Equals One

Wenxuan Guo , Tengyuan Liang This is my paper

Pith reviewed 2026-06-29 02:27 UTC · model grok-4.3

classification 📊 stat.ME
keywords N-of-1 trialsexperimental designMarkovian designimpulse-response modelOLS estimationtreatment effectsasymptotic theoryBernoulli design
0
0 comments X

The pith

Markovian designs for N-of-1 trials minimize OLS error for treatment effects under impulse-response models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a Markovian framework to design treatment assignments in single-subject experiments by governing the process with transition matrices. It optimizes these probabilities to reduce the error of ordinary least squares estimators for treatment effects, assuming outcomes follow a finite-order impulse-response model. Large-T asymptotic theory is derived for random-switch and cycle-switch designs, showing the optimum depends on the target estimand such as cumulative or lag-specific effects. A sympathetic reader would care because N-of-1 trials appear in clinical research and online platforms yet lack clear rules for handling temporal dependence in assignments.

Core claim

Under a finite-order impulse-response model, the design problem is cast as minimizing the asymptotic variance of the OLS estimator for target treatment effects. Complete large-T asymptotics are established for the optimal random-switch and cycle-switch designs, which justifies the robustness of i.i.d. Bernoulli designs and quantifies the dependence of the optimum on whether the target is a cumulative or lag-specific effect.

What carries the argument

Markovian treatment assignment governed by transition matrices that control the temporal dependence structure of the experiment.

If this is right

  • The optimal transition probabilities differ according to whether the target is cumulative treatment effects or lag-specific effects.
  • i.i.d. Bernoulli designs achieve near-optimal performance across many settings even if not exactly optimal.
  • Cycle-switch designs admit period-specific tuning that can reduce error relative to fully random switching.
  • The Markov framework allows explicit control of temporal dependence via the transition probabilities.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The asymptotic results could guide selection between random and deterministic switching when treatment effects are expected to persist for known durations.
  • The framework might extend to adaptive updating of transition matrices based on accumulating data without losing the large-T guarantees.
  • In platform experiments the same structure could inform how to set switch rates when user-level dependence varies across individuals.

Load-bearing premise

The response to treatment follows a finite-order impulse-response model whose order is known or correctly specified.

What would settle it

An N-of-1 dataset where the true impulse response has order higher than assumed and the optimized design produces larger mean squared error for the target effect than a fixed alternating assignment.

Figures

Figures reproduced from arXiv: 2606.28200 by Tengyuan Liang, Wenxuan Guo.

Figure 1
Figure 1. Figure 1: Illustration of minimax design optimization. Each row represents a candidate design and each [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Loss landscape of L𝑎𝑠𝑦 (𝜌, 𝛾; 𝑤) for 𝑤 = (1, −1, −1, −1, −1). For each 𝑇, the optimal solution (𝜌 ★ 𝑇 , 𝛾★ 𝑇 ) is highlighted in red. The limiting solution (𝜌 ★ = 𝛾 ★ ≈ 0.366 from Example 4.4) is marked by the yellow cross, and the trajectory of finite-sample solutions is shown in black. 4.2 Robust Design Optimization Next, we consider the robust design optimization problem min 𝜌,𝛾∈ [ 𝛿,1−𝛿] max ∥𝑤∥ ≤1 𝑤 ⊤… view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of the matrix eΣ under cycle-switch designs. The matrix eΣ coincides exactly with Σ in the divisible case and serves as an approximation in nondivisible cases. In fact, as shown in Section C, eΣ is the limit of Σ as 𝑇 → ∞ [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Variance curve of cycle-switch designs for [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Worst-case variances for different designs. In (a) and (b), we target the lag-specific and cumulative [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
read the original abstract

N-of-1 trials, or time-series experiments, are widely used in clinical research and online platforms. Yet the theoretically optimal design for estimating many treatment effects remains unclear. We propose a simple Markovian framework for experimental design in which the treatment assignment process is governed by possibly time-varying transition matrices. This formulation encompasses many existing N-of-1 designs and provides a principled way to control temporal dependence in treatment assignment through Markov transition probabilities. Under a finite-order impulse-response model, we formulate the design objective as minimizing the estimation error of ordinary least squares estimators for target treatment effects, and propose practical design optimization procedures. To characterize the optimal temporal structure, we focus on two structured design classes, random-switch and cycle-switch designs, and establish a complete large-$T$ asymptotic theory for the optimal designs in both classes. Our results justify the robustness of i.i.d. Bernoulli designs in N-of-1 trials and quantify how the optimal design depends on the target estimand, including cumulative and lag-specific treatment effects. Simulations demonstrate the effectiveness and robustness of the proposed designs across multiple scenarios.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper proposes a Markovian framework for N-of-1 trial design in which treatment assignments are governed by (possibly time-varying) transition matrices. Under a finite-order impulse-response outcome model, the design problem is cast as minimization of OLS estimation error for target treatment effects (cumulative or lag-specific). Complete large-T asymptotic characterizations are derived for the optimal designs within the random-switch and cycle-switch classes; these results are used to justify the robustness of i.i.d. Bernoulli designs and to show how the optimal transition probabilities depend on the target estimand.

Significance. If the asymptotic derivations hold, the work supplies a principled, model-based justification for design choices in single-subject experiments that are common in clinical research and online platforms. The explicit dependence of the optimal design on the target estimand (cumulative vs. lag-specific) and the complete large-T theory for two structured design families are concrete contributions. The finding that simple i.i.d. Bernoulli designs remain robust is practically useful when the modeling assumptions are met.

minor comments (2)
  1. Abstract and formulation section: the finite-order impulse-response assumption is stated as the setting for the OLS objective, but the manuscript would be strengthened by a brief discussion (even without new theorems) of how the optimality results and Bernoulli robustness degrade under order misspecification or non-impulse-response dynamics.
  2. The abstract claims 'complete large-T asymptotic theory' and 'simulations demonstrate effectiveness,' yet the provided text does not list the precise regularity conditions, convergence rates, or error-bar construction used in the simulations; adding these details would improve verifiability.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary, significance assessment, and recommendation of minor revision. The report accurately captures the paper's contributions on the Markovian framework, asymptotic optimality results for random-switch and cycle-switch designs, and the robustness of i.i.d. Bernoulli designs under finite-order impulse-response models.

Circularity Check

0 steps flagged

No circularity; derivation rests on explicit modeling assumption independent of data fits

full rationale

The paper assumes a finite-order impulse-response model as the data-generating process (abstract) and defines the design objective as minimizing OLS estimation error for target effects under that model. Large-T asymptotics for random-switch and cycle-switch designs are then derived from this setup. No step reduces a prediction or optimality result to a fitted parameter from the same data, a self-citation chain, or a definitional tautology; the model choice is stated upfront as an assumption rather than derived from the target estimands. This is a standard modeling framework with independent content.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the finite-order impulse-response model (domain assumption) and the large-T regime (standard_math). No free parameters or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption The outcome process follows a finite-order impulse-response model whose order is known or correctly specified.
    Invoked when the design objective is defined as minimizing OLS estimation error for target treatment effects.
  • standard math Large-T asymptotics are valid for characterizing optimal designs.
    Used to establish complete asymptotic theory for random-switch and cycle-switch classes.

pith-pipeline@v0.9.1-grok · 5708 in / 1419 out tokens · 20536 ms · 2026-06-29T02:27:00.648283+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references

  1. [1]

    Randomization inference when N equals one , volume =

    Liang, Tengyuan and Recht, Benjamin , journal =. Randomization inference when N equals one , volume =

  2. [2]

    Management Science , volume=

    Design and analysis of switchback experiments , author=. Management Science , volume=. 2023 , publisher=

    2023

  3. [3]

    arXiv:2510.22864 , year=

    Unifying regression-based and design-based causal inference in time-series experiments , author=. arXiv:2510.22864 , year=

    arXiv

  4. [4]

    2000 , publisher=

    The theory of the design of experiments , author=. 2000 , publisher=

    2000

  5. [5]

    Management Science , year=

    Efficient switchback experiments with surrogate variables: Estimation and experimental design , author=. Management Science , year=

  6. [6]

    2010 , publisher=

    Graphs and matrices , author=. 2010 , publisher=

    2010

  7. [7]

    Personalized medicine , volume=

    The n-of-1 clinical trial: the ultimate strategy for individualizing medicine? , author=. Personalized medicine , volume=. 2011 , publisher=

    2011

  8. [8]

    Trials , volume=

    A methodological review of randomised n-of-1 trials , author=. Trials , volume=. 2024 , publisher=

    2024

  9. [9]

    Journal of the Royal Society of Medicine , volume=

    The history and development of N-of-1 trials , author=. Journal of the Royal Society of Medicine , volume=. 2017 , publisher=

    2017

  10. [10]

    2025 , eprint=

    Switchback Experiments under Geometric Mixing , author=. 2025 , eprint=

    2025

  11. [11]

    Econometrica , volume=

    Investigating causal relations by econometric models and cross-spectral methods , author=. Econometrica , volume=. 1969 , publisher=

    1969

  12. [12]

    The American economic review , volume=

    Money, income, and causality , author=. The American economic review , volume=. 1972 , publisher=

    1972

  13. [13]

    Biometrika , volume=

    Minimax designs for causal effects in temporal experiments with treatment habituation , author=. Biometrika , volume=. 2023 , publisher=

    2023

  14. [14]

    Mathematical modelling , volume=

    A new approach to causal inference in mortality studies with a sustained exposure period-application to control of the healthy worker survivor effect , author=. Mathematical modelling , volume=. 1986 , publisher=

    1986

  15. [15]

    Proceedings of the Second Seattle Symposium in Biostatistics: analysis of correlated data , pages=

    Optimal structural nested models for optimal sequential decisions , author=. Proceedings of the Second Seattle Symposium in Biostatistics: analysis of correlated data , pages=. 2004 , organization=

    2004

  16. [16]

    Journal of the American Statistical Association , volume=

    Estimation of the causal effect of a time-varying exposure on the marginal mean of a repeated binary outcome , author=. Journal of the American Statistical Association , volume=. 1999 , publisher=

    1999

  17. [17]

    Epidemiology , volume=

    Marginal structural models and causal inference in epidemiology , author=. Epidemiology , volume=. 2000 , publisher=

    2000

  18. [18]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Optimal dynamic treatment regimes , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2003 , publisher=

    2003

  19. [19]

    Biometrics , volume =

    Longitudinal data analysis for discrete and continuous outcomes , author=. Biometrics , volume =. 1986 , publisher=

    1986

  20. [20]

    Journal of the American Statistical Association , volume=

    Marginal mean models for dynamic regimes , author=. Journal of the American Statistical Association , volume=. 2001 , publisher=

    2001

  21. [21]

    American Political Science Review , volume=

    How to make causal inferences with time-series cross-sectional data under selection on observables , author=. American Political Science Review , volume=. 2018 , publisher=

    2018

  22. [22]

    Causal Inference: What If , year =

    Hern. Causal Inference: What If , year =

  23. [23]

    The annals of mathematical statistics , volume=

    Optimum designs in regression problems , author=. The annals of mathematical statistics , volume=. 1959 , publisher=

    1959

  24. [24]

    Journal of the American Statistical Association , volume=

    A basis for the selection of a response surface design , author=. Journal of the American Statistical Association , volume=. 1959 , publisher=

    1959

  25. [25]

    2015 , publisher=

    Handbook of design and analysis of experiments , author=. 2015 , publisher=

    2015

  26. [26]

    Journal of the American Statistical Association , volume=

    Balancing covariates in randomized experiments with the gram--schmidt walk design , author=. Journal of the American Statistical Association , volume=. 2024 , publisher=

    2024

  27. [27]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , pages =

    Guo, Wenxuan and Liang, Tengyuan and Toulis, Panos , title =. Journal of the Royal Statistical Society Series B: Statistical Methodology , pages =

  28. [28]

    Tutorials in Operations Research: Smarter Decisions for a Better World , pages=

    Experimental design for causal inference through an optimization lens , author=. Tutorials in Operations Research: Smarter Decisions for a Better World , pages=. 2024 , publisher=

    2024

  29. [29]

    Journal of the American Statistical Association , volume=

    Time series experiments and causal estimands: exact randomization tests and trading , author=. Journal of the American Statistical Association , volume=. 2019 , publisher=

    2019

  30. [30]

    Management Science , volume=

    Near-optimal ab testing , author=. Management Science , volume=. 2020 , publisher=

    2020

  31. [31]

    Fisher, R. A. , title =. Journal of the Ministry of Agriculture of Great Britain , year =

  32. [32]

    1923 , note =

    Neyman, Jerzy , title =. 1923 , note =

    1923

  33. [33]

    , author=

    Estimating causal effects of treatments in randomized and nonrandomized studies. , author=. Journal of educational Psychology , volume=. 1974 , publisher=

    1974

  34. [34]

    arXiv preprint arXiv:2406.06768 , year=

    Data-driven switchback experiments: Theoretical tradeoffs and empirical bayes designs , author=. arXiv preprint arXiv:2406.06768 , year=

    arXiv

  35. [35]

    2006 , publisher=

    Optimal design of experiments , author=. 2006 , publisher=

    2006

  36. [36]

    The Annals of Statistics , volume =

    On the robustness and efficiency of some randomized designs , author=. The Annals of Statistics , volume =. 1981 , publisher=

    1981

  37. [37]

    2012 , publisher=

    Matrix analysis , author=. 2012 , publisher=

    2012

  38. [38]

    and Wellner, J.A

    van der Vaart, A. and Wellner, J.A. , title=. 1996 , publisher =

    1996

  39. [39]

    2012 , publisher=

    Markov chains and stochastic stability , author=. 2012 , publisher=

    2012

  40. [40]

    1998 , publisher=

    Markov chains , author=. 1998 , publisher=

    1998

  41. [41]

    A survey and some open questions , author=

    Basic properties of strong mixing conditions. A survey and some open questions , author=. 2005 , journal=

    2005

  42. [42]

    Annales de l'IHP Probabilit

    Covariance inequalities for strongly mixing processes , author=. Annales de l'IHP Probabilit

  43. [43]

    Harvard Business Review , volume=

    Avoid the Pitfalls of A/B Testing Make sure your experiments recognize customers' varying needs , author=. Harvard Business Review , volume=

  44. [44]

    Journal of Rational Mechanics and Analysis , volume=

    On the eigen-values of certain Hermitian forms , author=. Journal of Rational Mechanics and Analysis , volume=. 1953 , publisher=

    1953

  45. [45]

    ANZIAM Journal , volume=

    Explicit inverses of Toeplitz and associated matrices , author=. ANZIAM Journal , volume=