Pith. sign in

REVIEW 1 major objections 1 minor 1 cited by

Why is prompting hard? Understanding prompts on binary sequence predictors

T0 review · 1 major / 1 minor · reviewed 2026-05-23 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Optimal conditioning sequences for binary sequence predictors are often unintuitive and explained better by the pretraining distribution than by the target task.

desk verdict Binary predictors give a clean statistical account of why some prompts beat task demos, but the step to frontier models stays tentative. read the letter →

arxiv 2502.10760 v2 submitted 2025-02-15 cs.CL cs.LGstat.ML

Li Kevin Wenliang , Anian Ruoss , Jordi Grau-Moya , Marcus Hutter , Tim Genewein This is my paper · ORCID
classification cs.CLcs.LGstat.ML
keywords promptingconditioningsequencessequencepredictionpretrainingdistributionbinarypredictorsin-contextlearningfrontiermodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper treats prompting as the problem of selecting the best conditioning sequence for a near-optimal sequence predictor. In controlled experiments with binary sequence predictors whose pretraining distributions are fully known, optimal prompts frequently appear unrelated to the downstream task and outperform standard choices such as task-specific demonstrations. Even exhaustive search over conditioning sequences fails to make optimal prompt identification reliable. The same empirical setup applied to frontier models yields analogous patterns. The pretraining distribution therefore supplies the key explanatory lens for why certain prompts succeed.

What carries the argument

The framing of prompting as search for the best conditioning sequence on a near-optimal sequence predictor, tested via binary sequence predictors with known pretraining distributions.

What would settle it

A binary sequence experiment in which the empirically optimal conditioning sequence is not the one predicted by the pretraining distribution or is consistently beaten by task demonstrations.

Watch

Extended reading notes

Core claim

We view prompting as finding the best conditioning sequence on a near-optimal sequence predictor. On numerous well-controlled experiments, we show that unintuitive optimal conditioning sequences can be better understood given the pretraining distribution, which is not usually available. Even using exhaustive search, reliably identifying optimal prompts for practical neural predictors can be surprisingly difficult. Popular prompting methods, such as using demonstrations from the targeted task, can be surprisingly suboptimal. Using the same empirical framework, we analyze optimal prompts on frontier models, revealing patterns similar to the binary examples and previous findings.

Load-bearing premise

Binary sequence predictors trained to near-optimality on fully known distributions behave like the conditioning mechanisms inside practical neural predictors and frontier models.

Editorial extensions

If this is right

  • Popular methods that rely on task demonstrations can remain suboptimal even after exhaustive search over alternatives.
  • The pretraining distribution supplies a systematic way to predict which conditioning sequences will perform well.
  • Reliable identification of optimal prompts stays difficult for predictors that are only approximately optimal.
  • Patterns observed on binary predictors appear again when the same search procedure is run on frontier models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the pretraining distribution governs optimal conditioning in simple cases, prompt design for large models may benefit from explicit modeling of training-data statistics rather than task-specific heuristics.
  • The difficulty of exhaustive search suggests that future prompt-finding algorithms should incorporate distributional priors instead of treating the search as unstructured.
  • The binary-predictor setup isolates the effect of conditioning from other model behaviors, offering a test bed for theories of in-context learning that focus on sequence statistics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper claims that prompting is equivalent to finding the best conditioning sequence on a near-optimal sequence predictor. Through controlled experiments on binary sequence predictors trained to near-optimality on fully known distributions, it shows that optimal conditioning sequences are often unintuitive and explained by the pretraining distribution, that popular methods such as task demonstrations remain suboptimal even under exhaustive search, and that similar patterns hold when the same empirical framework is applied to frontier models.

Significance. If the results hold, the work supplies a useful statistical perspective on prompting that complements frontier-model studies by leveraging fully known distributions and exhaustive enumeration to identify suboptimality. The controlled binary-predictor setting and the reported replication of patterns on frontier models are explicit strengths that allow precise, falsifiable observations about pretraining-distribution effects.

major comments (1)
  1. [Abstract and introduction framing; § on frontier-model experiments] The central extension from binary predictors to frontier models rests on the assumption that the conditioning dynamics are governed by the same statistical factors. The manuscript invokes this framing in the abstract and introduction but provides no ablation or quantitative comparison (e.g., of context-length scaling or token-dependency effects) between the attention-free binary models and transformer in-context behavior, leaving the representativeness claim load-bearing yet under-supported.
minor comments (1)
  1. [Methods] Notation for conditioning sequences and pretraining distributions would benefit from an explicit running example early in the methods to improve readability for readers outside the binary-sequence setting.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive feedback and for recognizing the strengths of the controlled binary-predictor experiments. We respond to the major comment below.

read point-by-point responses

  1. Referee: [Abstract and introduction framing; § on frontier-model experiments] The central extension from binary predictors to frontier models rests on the assumption that the conditioning dynamics are governed by the same statistical factors. The manuscript invokes this framing in the abstract and introduction but provides no ablation or quantitative comparison (e.g., of context-length scaling or token-dependency effects) between the attention-free binary models and transformer in-context behavior, leaving the representativeness claim load-bearing yet under-supported.

    Authors: We agree that the current framing in the abstract and introduction could more precisely delineate the scope of the frontier-model analysis. The manuscript applies the same empirical procedure (exhaustive or targeted search over conditioning sequences) and reports qualitatively similar patterns, but does not assert that the underlying statistical factors or scaling behaviors are identical across architectures. The binary setting supplies known distributions and exhaustive enumeration; the frontier-model results are presented as an existence check that the observed phenomena are not artifacts of the simplified model class. To address the concern, we will revise the abstract, introduction, and discussion to (i) state explicitly that we observe analogous patterns without claiming mechanistic equivalence, (ii) note the architectural differences (attention-free vs. transformer attention), and (iii) acknowledge the absence of direct ablations on context-length scaling or token dependencies. These changes will clarify that the representativeness claim is limited to the recurrence of the reported qualitative phenomena. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: results from direct experiments on known distributions

full rationale

The paper frames prompting as conditioning a near-optimal sequence predictor and reports empirical results from training binary predictors on fully specified distributions, identifying optimal sequences via search, and comparing to task demonstrations. These are direct measurements, not derivations. No equations, fitted parameters renamed as predictions, or self-citation chains are invoked to force the central claims. The extension to frontier models is presented as an empirical check showing similar patterns. The work is self-contained against external benchmarks via controlled experiments; the modeling choice of binary predictors does not create a definitional loop.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Central claim rests on the domain assumption that prompting equals conditioning a near-optimal sequence predictor and that binary predictors are representative; no free parameters or invented entities are stated in the abstract.

assumptions (1)
  • domain assumption Prompting can be viewed as finding the best conditioning sequence on a near-optimal sequence predictor.
    Explicitly stated as the modeling choice in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Why is prompting hard? Understanding prompts on binary sequence predictors." pith.science (2026). https://pith.science/paper/2502.10760

@misc{pith2026250210760,
  author       = {Pith},
  title        = {Pith review of: Why is prompting hard? Understanding prompts on binary sequence predictors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2502.10760}},
  note         = {Machine review of arXiv:2502.10760}
}
read the original abstract

Frontier models can be prompted or conditioned to do many tasks, but finding good prompts is not always easy, nor is understanding some performant prompts. We view prompting as finding the best conditioning sequence on a near-optimal sequence predictor. On numerous well-controlled experiments, we show that unintuitive optimal conditioning sequences can be better understood given the pretraining distribution, which is not usually available. Even using exhaustive search, reliably identifying optimal prompts for practical neural predictors can be surprisingly difficult. Popular prompting methods, such as using demonstrations from the targeted task, can be surprisingly suboptimal. Using the same empirical framework, we analyze optimal prompts on frontier models, revealing patterns similar to the binary examples and previous findings. Taken together, this work takes an initial step towards understanding optimal prompts, from a statistical and empirical perspective that complements research on frontier models.

Figures

Figures reproduced from arXiv: 2502.10760 by the authors.

Figure 1
Figure 1. Experimental design to obtain optimal prompts under pretraining and task data generators. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Results for a pretraining DG p = BernMix(0.2, 0.7) and a task DG q = Bern(0.7). Left, the proportion correct for Bayes predictor (103 seeds per data point) and two neural predictors (30 seeds per data point). Error bar show 1 SEM. The black dotted line is the theoretical value for T = 1 (see Appendix B.1.2). Additional results are in Appendix B.1.1. Right, empirically optimal prompts at Lmax = 5 for the Bayes predic… view at source ↗
Figure 3
Figure 3. Results for p = BernMix(0.2, 0.7) and q = Bern(0.6). Left, each circle represents the heads/tails count of the theoretically optimal prompts. The orange dotted line indicates the maximum prompt length Lmax. The green dashed line marks 60% ONEs in s ∗ . Right, the proportion correct of sˆ = s ∗ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (24 more)
Figure 4
Figure 4. Figure 4: Results for p = BetaBern(1, β) with β ∈ {1, 2}, and q = Bern(τ ) with τ ∈ {0.7, 0.9}. Left, the ratio of ONEs in the theoretical s ∗ . Red dotted line shows true bias of q. Right, proportion correct of sˆ for β = 1 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Results for a random switching pretraining DG and four switching downstream DGs with different causes [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Upper, experiment design for the bandit task. Lower, 4 equivalent theoretical [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Final pretraining loss under various DGs [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: The proportion correct of the empirically optimal prompt for [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: The loss “landscape” of p = BernMix(0.2, 0.7) and q = Bern(0.7). The leftmost column shows the KL divergence of each prompt sorted in increasing order against the prompt rank (sort indices, or argsort), with lower rank meaning lower KL divergence (12). The other column…
Figure 10
Figure 10. Figure 10: The distribution of the prompt counts for [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: The proportion correct of empirically optimal prompt for [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: The distribution of the prompt counts for [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: The loss “landscape” of p = BernMix(0.2, 0.7) and q = Bern(0.6).The leftmost column shows the KL divergence of each prompt sorted in increasing order against the prompt rank (sort indices, or argsort), with lower rank meaning lower KL divergence (12). The other column…
Figure 14
Figure 14. Figure 14: extends the results in [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: The distribution of empirically optimal prompt for [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: (left) shows that the theoretical optimal prompt has a distinctively lower KL divergence compared to other prompts, wit hthe exception that T = 1 still has a very flat landscape. The other columns show a clearer optimal region, especially for τ = 0.7. These are in sta…
Figure 17
Figure 17. Figure 17: Proprotion of empirical prompts that match the theoretically [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: Proportion correct of sˆ. Same as [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]
Figure 19
Figure 19. Figure 19: Distribution of empirically optimal prompts. Here, we set the [PITH_FULL_IMAGE:figures/full_fig_p025_19.png]
Figure 20
Figure 20. Figure 20: The loss “landscape” of [PITH_FULL_IMAGE:figures/full_fig_p025_20.png]
Figure 21
Figure 21. Figure 21: Same as Fig [PITH_FULL_IMAGE:figures/full_fig_p027_21.png]
Figure 22
Figure 22. Figure 22: Same as Fig [PITH_FULL_IMAGE:figures/full_fig_p028_22.png]
Figure 23
Figure 23. Figure 23: Relationship between the return and the uniform random variable for different values of the power index [PITH_FULL_IMAGE:figures/full_fig_p029_23.png]
Figure 24
Figure 24. Figure 24: Estimated return of each prompt by Monte Carlo against the prompt rank. The prompts are sorted using estimated [PITH_FULL_IMAGE:figures/full_fig_p030_24.png]
Figure 25
Figure 25. Figure 25: The estimated rollout return for empirically optimized prompts. Errorbars show 1 SEM from 100 seeds. [PITH_FULL_IMAGE:figures/full_fig_p031_25.png]
Figure 26
Figure 26. Figure 26: The distribution of Win-Stay/Lose-Shift for all empirical prompts for [PITH_FULL_IMAGE:figures/full_fig_p032_26.png]
Figure 27
Figure 27. Figure 27: Total reward and regret of typical prompts, compared with Thompson sampling agent and the optimal prompt [PITH_FULL_IMAGE:figures/full_fig_p032_27.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrations

    cs.AI 2024-12 conditional novelty 7.0 of 10

    LMAct is an open benchmark showing that frontier AI models mostly fail to improve at interactive tasks even when given up to 512 expert demonstrations in a million-token context.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Map the binary tokens xt ∈ {0, 1} to embeddings et ∈ Rh through et = Wembxt where Wemb ∈ Rh×2, and h ∈ N+ is the hidden size

    work page

  2. [2]

    Sequentially map h1:T through some neural architecture, called the torso, such as LSTM, multi-head attention, etc, to obtain some hidden activations ut ∈ Rh

    work page

  3. [3]

    For each t, map ut through the fully connected MLP to vt ∈ Rh that is usually found after the attention layer in a transformer block (Vaswani, 2017)

    work page 2017

  4. [4]

    There is also a residual connection from step 2 to 3 and from 3 to 4

    For each t, map vt to output logits through a linear map. There is also a residual connection from step 2 to 3 and from 3 to 4. The different neural architectures differ only by the torso. This maintains a flexible enough architecture for different tasks while controlling for the model complexity between different architectures. For the torso, we use the ...

    work page

  5. [5]

    Vanilla recurrent neural networks (Elman, 1990)

    work page 1990

  6. [6]

    Long-short term memory (LSTM) (Hochreiter & Schmidhuber, 1997), reported in main text

    work page 1997

  7. [7]

    sLSTM (Beck et al., 2024)

    work page 2024

  8. [8]

    Softmax-attention transformer (Transformer) (Vaswani, 2017), reported in main text

    work page 2017

Show all 12 references
  1. [9]

    Linear transformer (Katharopoulos et al., 2020)

    2020

  2. [10]

    landscape

    Another variant of Linear transformer we refer to as Inner-product transformer (IP transformer) (Li et al., 2020; Shen et al., 2021) We found that step 3 above is crucial for transformer architectures to perform some of the tasks, although this is not essential for LSTMs to pe...

    2020

  3. [11]

    mismatch

    The first bias y1 associated with s1 may not be the first ε appearing in Definition 5.1. In other words, the “phase” of the y is unknown and y1:L. Taking λ = 3 for example, y can start with any of the following: [ε, ε, ε, 1−ε, 1−ε, 1−ε, . . . ] [ε, ε, 1−ε, 1−ε, 1−ε, ε, . . . ]...

    2021

  4. [12]

    retrieval

    Together with the constantly reward streak from the other arm, creating a posterior of Beta (1 + 7τ, 1), choosing the rewarding arm is more likely if τ is large. Essentially, a large reward gap between the two Beta distributions helps the predictor identify τ. 30 Optimal promp...

    1957

Pith tools

Reviewed May 23, 2026 · model on record in the stance chip above.