Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?

Florian Mai; Lucie Flek; Nicholas Kluge Corr\^ea; Shivam Rawat

REVIEW 3 major objections 3 minor 2 cited by

Reasoning tokens beat architecture on state-tracking tasks

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-05 01:10 UTC pith:T35E6CGL

load-bearing objection Reasoning-token augmentation dominates architectural bias in controlled state-tracking tasks; the task-structure crossover pattern is intriguing but underpowered. the 3 major comments →

arxiv 2604.21454 v2 pith:T35E6CGL submitted 2026-04-23 cs.CL cs.AI

Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?

Shivam Rawat , Lucie Flek , Florian Mai , Nicholas Kluge Corr\^ea This is my paper

classification cs.CL cs.AI

keywords reasoning primitivesstate-trackingrecallhybrid architecturestransformer modelsreasoning tokensstate-based recallinductive bias

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper examines whether the reasoning gains seen in large language models come from a single general capability or from simpler operations like recall and state-tracking. The authors compare matched transformer and hybrid architectures with and without reasoning-token augmentation across five controlled task families centered on state-based recall. They find that reasoning-token augmentation is the dominant factor for performance, often by large margins, while hybrid architectural inductive bias yields no uniform accuracy advantage once reasoning tokens are available. When architectural differences do appear, they follow task structure: hybrid Think models are more robust on strictly sequential chained updates, while transformer Think models are more robust on flat multi-hop retrieval. The authors frame this as a descriptive account of what drives performance on state-based recall tasks, suggesting that hybrid advantages are narrower, task-dependent, and potentially more about inference efficiency than overall capability.

Core claim

The paper establishes that reasoning-token augmentation is the dominant factor for performance on state-based recall tasks, while hybrid architectural inductive bias yields no uniform accuracy advantage once reasoning tokens are available. The authors propose a State over Tokens view: externalized reasoning traces help because they carry intermediate state forward in token space. Architectural differences that do appear follow task structure, with hybrid Think models more robust on sequential chained updates and transformer Think models more robust on flat multi-hop retrieval.

What carries the argument

State over Tokens view: externalized reasoning traces help because they carry intermediate state forward in token space, rather than relying on architectural inductive bias to maintain that state internally.

Load-bearing premise

The five controlled task families adequately decompose reasoning into recall and state-tracking primitives such that conclusions about architectural differences generalize to broader reasoning demands in real-world applications.

What would settle it

If reasoning-token augmentation did not outperform instruction-only variants across the task suite, or if hybrid architectures showed a uniform accuracy advantage over transformers regardless of reasoning-token availability, the paper's central claims would be undermined.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If reasoning-token augmentation is the dominant factor, then architectural choices for reasoning models may matter less than training and inference strategies that produce effective reasoning traces.
The task-dependent nature of architectural differences suggests that model selection for reasoning applications should be guided by the structure of the target task (sequential vs. flat retrieval) rather than by general architectural claims.
The State over Tokens view implies that the benefit of reasoning traces is fundamentally about externalizing intermediate computation, which connects to broader questions about chain-of-thought and scratchpad mechanisms.
If hybrid advantages are primarily about inference efficiency rather than capability, then cost-benefit analyses for deploying hybrid architectures should focus on computational savings rather than accuracy improvements.
The controlled task families used here could serve as a diagnostic suite for isolating which reasoning primitives different models and architectures actually support.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Reasoning-token augmentation dominates architectural bias in controlled state-tracking tasks; the task-structure crossover pattern is intriguing but underpowered.

read the letter

The main thing to know: this paper finds that chain-of-thought reasoning tokens matter more than hybrid vs. transformer architecture on state-based recall tasks, and that when architectural differences do appear, they track task structure (hybrid wins on sequential chained updates, transformer on flat multi-hop retrieval). That crossover pattern is the genuinely new empirical observation here — the “CoT helps” finding itself is not surprising and the authors seem to know that, framing their contribution as descriptive rather than mechanistic. Credit is due for the experimental design: matched architectures with and without reasoning augmentation, controlled task families decomposing reasoning into recall and state-tracking primitives, and they release code and data. That is solid empirical hygiene and makes the work independently checkable, which matters. The “State over Tokens” interpretation — that reasoning traces help by carrying intermediate state forward in token space — is reasonable as a framing device, though not deeply novel. The soft spots are real but proportional. The biggest one is the matching criterion. The abstract says “matched transformer and hybrid architectures” without specifying what matched means — same parameter count, same training data, same compute, same context window? If the hybrid model is effectively smaller or less trained, the null result on architecture is uninterpretable. This is load-bearing and needs to be verified in the full text. Second, five task families is thin for claiming a task-structure-dependent crossover pattern. A pattern of “hybrid better on sequential, transformer better on flat” across five families could be driven by one or two outlier tasks. Without significance testing across tasks or a principled account of why the architecture should produce these specific crossovers, the descriptive claim is underdetermined. The reader’s CONDITIONAL verdict and LOW confidence are about right for an abstract-only pass. The stress-test concern about model matching is the one that matters most; the five-task concern is real but secondary. This paper is for practitioners and researchers thinking about architecture selection for reasoning workloads, and for anyone interested in decomposing what CoT actually buys you. It deserves a serious referee who can verify the matching, check the significance testing, and assess whether the task families are representative enough to support the generalization claims. I recommend sending it out for full review.

Referee Report

3 major / 3 minor

Summary. The manuscript investigates whether architectural differences between hybrid and transformer LLMs yield advantages on state-tracking and recall tasks, and whether reasoning-token augmentation dominates over architectural inductive bias. The authors design five controlled task families centered on state-based recall, comparing matched transformer and hybrid architectures with and without reasoning augmentation. They find that reasoning-augmented variants substantially outperform instruction-only variants, while hybrid architectural bias does not yield a uniform accuracy advantage once reasoning tokens are available. Architectural differences that do appear follow task structure: hybrid Think is more robust on sequential chained updates, transformer Think on flat multi-hop retrieval. The authors frame this as a descriptive account and release code and data.

Significance. The question of whether hybrid architectures offer genuine reasoning advantages or whether reasoning-token augmentation is the dominant factor is timely and well-motivated. The controlled task-family design, matched-architecture comparison, and the with/without reasoning augmentation factorial structure are methodologically appropriate in principle. The release of codebase and data is a positive step toward reproducibility. The 'State over Tokens' interpretive framework offers a useful lens for understanding why externalized reasoning traces help. However, the significance of the findings is contingent on the matching criterion being rigorous and the task suite being representative enough to support the descriptive generalizations claimed.

major comments (3)

The central claim that reasoning-token augmentation dominates over architectural differences requires that the hybrid and transformer models are genuinely matched in capacity, training data, compute budget, and context window. The abstract states 'matched transformer and hybrid architectures' but does not specify the matching criterion. Without this specification, the null result on architecture is uninterpretable: if the hybrid model has fewer effective parameters or less training compute, the absence of a uniform advantage would be expected regardless of architectural merit. The manuscript must explicitly state the matching protocol and justify that the comparison is fair. This is load-bearing for the paper's central claim.
The claim that architectural differences 'follow task structure' (hybrid better on sequential, transformer on flat multi-hop) is based on only five task families. With five families, a crossover pattern could arise from noise or from one or two outlier tasks. The descriptive claim about task-structure-dependent advantages requires either statistical significance testing across tasks or a principled mechanistic explanation for why the task structure should produce these specific crossover patterns. Without significance testing or a larger task suite, the task-structure pattern is underdetermined. The manuscript should report per-task variance, confidence intervals, or significance tests, and should acknowledge the limitation of generalizing from five families.
The five controlled task families are described as 'centered on state-based recall,' but whether these synthetic tasks capture the distribution of reasoning demands in real-world applications is not established. If the tasks are too narrow or too clean, the descriptive conclusions about hybrid vs. transformer trade-offs may not transfer. The manuscript should discuss the scope of its claims relative to the task suite and clarify that the conclusions are bounded by the task families studied. This is load-bearing because the paper's contribution is framed as a descriptive account of 'what drives performance on state-based recall tasks,' and the generality of this account depends on task representativeness.

minor comments (3)

The abstract does not specify the model sizes, training data, or compute budgets used. Including these would strengthen the reader's ability to assess the matching claim.
The 'State over Tokens' view is introduced as an interpretation but its precise relationship to the experimental findings could be stated more explicitly — is it a falsifiable hypothesis or a post-hoc explanatory frame? Clarifying this would help the reader assess its scientific status.
The phrase 'potentially more about inference efficiency than overall capability' is hedged; if the paper has data on inference efficiency, this should be reported; if not, this should be flagged as speculation.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive report. The referee raises three major comments, all of which are substantive and addressable in revision. We summarize our planned revisions here and provide point-by-point responses below. In brief: (1) we will add an explicit matching-protocol section to the manuscript, specifying parameter counts, training data, compute budgets, and context windows for all compared models, and we agree this is load-bearing for the central claim; (2) we will add per-task variance, confidence intervals, and significance tests where sample sizes permit, and will explicitly acknowledge the limitation of generalizing from five task families; (3) we will add a scoped-claims discussion clarifying that our conclusions are bounded by the synthetic task suite and may not transfer to all real-world reasoning distributions. We view all three as legitimate requests that strengthen the paper.

read point-by-point responses

Referee: The central claim that reasoning-token augmentation dominates over architectural differences requires that the hybrid and transformer models are genuinely matched in capacity, training data, compute budget, and context window. The abstract states 'matched transformer and hybrid architectures' but does not specify the matching criterion. Without this specification, the null result on architecture is uninterpretable. The manuscript must explicitly state the matching protocol and justify that the comparison is fair.

Authors: The referee is correct that the matching criterion is load-bearing and that its absence from the current draft is a significant gap. We will address this in revision by adding a dedicated subsection specifying the matching protocol in full detail: parameter counts (total and non-embedding), training token budgets, training data composition, compute budget (FLOPs or GPU-hours), optimizer and learning-rate schedule, and context window length for each model pair. To preview: our matched pairs are drawn from the same model family and training run, differing only in whether the recurrent/global-attention component is active (hybrid) or ablated to a pure transformer configuration, with all other hyperparameters held fixed. This means the comparison is controlled at the level of architecture within a fixed training budget, not across independently trained models with different resource allocations. We agree that without this specification the null result on architecture is uninterpretable, and we will make the protocol explicit in both the abstract (briefly) and the methods section (in full). revision: yes
Referee: The claim that architectural differences 'follow task structure' (hybrid better on sequential, transformer on flat multi-hop) is based on only five task families. With five families, a crossover pattern could arise from noise or from one or two outlier tasks. The descriptive claim about task-structure-dependent advantages requires either statistical significance testing across tasks or a principled mechanistic explanation for why the task structure should produce these specific crossover patterns. The manuscript should report per-task variance, confidence intervals, or significance tests, and should acknowledge the limitation of generalizing from five families.

Authors: This is a fair concern. We will address it on two fronts. First, we will add per-task variance estimates, confidence intervals, and pairwise significance tests (bootstrap or paired t-tests as appropriate) for each task family, reporting these alongside the accuracy figures. This will make clear which crossover patterns are statistically supported and which are suggestive but underpowered. Second, we will strengthen the mechanistic explanation for the crossover pattern: the hybrid architecture's recurrent component is designed to compress sequential context into a fixed-size state, which should advantage tasks requiring chained sequential updates where intermediate state must be carried forward; conversely, flat multi-hop retrieval benefits from direct access to all prior tokens via full attention, which favors the transformer configuration. We agree that five task families is a small base for strong generalization claims, and we will explicitly acknowledge this limitation, softening the language from a definitive claim to a hypothesis supported by initial evidence. If the significance tests do not support a given crossover at conventional thresholds, we will say so rather than overclaiming. revision: yes
Referee: The five controlled task families are described as 'centered on state-based recall,' but whether these synthetic tasks capture the distribution of reasoning demands in real-world applications is not established. If the tasks are too narrow or too clean, the descriptive conclusions about hybrid vs. transformer trade-offs may not transfer. The manuscript should discuss the scope of its claims relative to the task suite and clarify that the conclusions are bounded by the task families studied.

Authors: We agree that the scope of our claims should be explicitly bounded by the task suite, and we will add a dedicated discussion subsection on this point. Our task families are designed to isolate specific reasoning primitives (recall and state-tracking) in controlled settings, which is a methodological choice that trades ecological validity for interpretability. We will state clearly that our conclusions apply to state-based recall tasks of the types studied and that transfer to real-world reasoning distributions (which may involve multi-step planning, code generation, mathematical reasoning, or open-ended question answering) is not established by this work. We will also note that the synthetic, relatively clean structure of our tasks may favor reasoning-token augmentation more strongly than messier real-world tasks would, since the intermediate-state decomposition is more straightforward in controlled settings. This is an honest limitation and we will frame it as such rather than claiming broader generalization than the evidence supports. We believe the controlled setting remains valuable for isolating the mechanisms at play, but the referee is correct that the descriptive account should not be presented as covering all reasoning demands. revision: yes

Circularity Check

0 steps flagged

No circularity: this is an empirical study with no derivation chain to be circular.

full rationale

This paper is an empirical, descriptive study comparing transformer and hybrid LLM architectures on state-based recall tasks. There is no first-principles derivation, no equation chain, and no fitted-parameter-as-prediction structure. The 'State over Tokens' view is offered as an interpretive framework for observed results, not as a derivation whose outputs are equivalent to its inputs by construction. The main empirical claims (reasoning-token augmentation dominates; hybrid advantages are task-dependent) are supported by experimental results on controlled task families, not by self-referential definitions or self-citation chains. The reader's concern about task-design confirmation bias is a standard empirical validity concern (correctness risk), not a circularity pattern. No self-definitional, fitted-input-as-prediction, or self-citation-load-bearing structure is present in the available text. The study is self-contained against its own benchmarks, and the codebase and data are released for reproduction. No circularity is found.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

No free parameters, no invented entities, and three domain assumptions extracted from the abstract. The paper is empirical/descriptive, so the axiom ledger is light. The key assumptions concern task design adequacy, architectural matching, and generalization scope. Full-text review would likely reveal additional assumptions about model training, hyperparameter selection, and statistical testing.

axioms (3)

domain assumption Recall and state-tracking are sufficient primitives to decompose reasoning performance differences between architectures
The paper's five task families are 'centered on state-based recall,' and the conclusions about reasoning primitives depend on these two operations capturing the relevant dimensions of reasoning. Stated in the abstract's framing of 'two such primitives, recall and state-tracking.'
domain assumption Matched transformer and hybrid architectures are comparable in parameter count, training data, and compute, so observed differences reflect architecture rather than scale
The abstract refers to 'matched transformer and hybrid architectures,' implying this matching is established somewhere in the full text. This is a load-bearing premise for attributing performance differences to architecture rather than confounds.
domain assumption Controlled task families generalize to broader reasoning capabilities
The descriptive conclusions about what drives performance on 'state-based recall tasks' are implicitly generalized to reasoning more broadly. The scope of generalization is not bounded in the abstract.

pith-pipeline@v1.1.0-glm · 4605 in / 2042 out tokens · 223904 ms · 2026-07-05T01:10:41.017944+00:00 · methodology

0 comments

read the original abstract

Reasoning in large language models is often discussed as a single capability, but some of its gains may stem from simpler underlying operations. We examine two such primitives, recall and state-tracking, through five controlled task families centered on state-based recall, and compare matched transformer and hybrid architectures with and without reasoning augmentation. Across the suite, reasoning-augmented variants substantially outperform instruction-only variants, often by large margins. This pattern is consistent with the State over Tokens view: externalized reasoning traces help because they carry the intermediate state forward in token space. By contrast, hybrid inductive bias does not yield a uniform advantage in accuracy once reasoning tokens are available. When architectural differences do appear, they follow task structure: the hybrid Think model is more robust on strictly sequential chained updates, whereas the transformer Think model is more robust on flat multi-hop retrieval. We therefore cast the main contribution of this study as a descriptive account of what drives performance on state-based recall tasks: reasoning-token augmentation appears to be the dominant factor, while hybrid advantages are narrower, task-dependent, and potentially more about inference efficiency than overall capability. We also release the codebase and data required to reproduce these results.

Figures

Figures reproduced from arXiv: 2604.21454 by Florian Mai, Lucie Flek, Nicholas Kluge Corr\^ea, Shivam Rawat.

**Figure 2.** Figure 2: State-based Astro Recall accuracy for instruction-tuned models as task complexity increases. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Figure 3: State-based Astro Recall accuracy for reasoning-augmented models in higher-complexity settings. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

**Figure 4.** Figure 4: Collision Simulator accuracy across difficulty levels for instruction-tuned and reasoning-augmented models. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

**Figure 5.** Figure 5: Parsed-weighted accuracy for Think models on Collision Simulator across difficulty levels. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale
cs.AI 2026-07 conditional novelty 6.0

Capability, unlike representation, converges to access-complete hybrid architectures, and a witness task separates pure from hybrid families under fixed budgets.
The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale
cs.AI 2026-07 conditional novelty 6.0

Long-context capability converges to hybrid architectures pairing O(1) state with a scalable index; pure scaling of SSMs or Transformers alone hits information, horizon, or circuit walls.