Lifted State Hypothesis in Large Language Models

Bumjin Park; Jaesik Choi

REVIEW 4 major objections 5 minor 15 references

Lifted State Hypothesis in Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large language models treat computationally equivalent inputs as sharing a latent state, so later evidence that revises one subtype also changes its parent type.

desk verdict Useful benchmark and a reproducible revision-asymmetry result, but the 'lifted state' interpretation is underdetermined and should be reframed. read the letter →

arxiv 2607.19360 v2 pith:DE6USRM3 submitted 2026-06-02 cs.AI

Bumjin Park , Jaesik Choi This is my paper · ORCID

classification cs.AI

keywords liftedstatehypothesiscomputation-relativetypesnon-monotonicrevisionfine-tuningin-contextlearningmodulararithmeticgeneralizationLLMinterpretability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes the Lifted State Hypothesis: under fixed parameters, context, and target computation, LLMs group inputs that behave identically into a computation-relative type and reuse a single latent component—the lifted state—to support that type's computation. This reuse gives type-level generalization for free, but when later evidence distinguishes a subtype, revising the shared state necessarily couples the subtype's change to preserved members of the parent type. The authors introduce the NMR-Type Dataset, which first trains a broad modulo rule and then flips a withheld subtype's label while replaying the original groups. Across full fine-tuning, LoRA, and in-context learning, models often learn the flip on the subtype yet also degrade the parent group significantly more than a disjoint same-label control, a behavioral asymmetry that would be expected if the two share a lifted state.

What carries the argument

The lifted state, a minimal filter–value decomposition s_k = (z_k, phi_k), where z_k is a shared filter that determines which episodes recruit the state and phi_k is the value that supports the target computation. Episodes within one computation-relative type (samples indistinguishable in their behavior under the selected computation, context scope, and parameters) are hypothesized to reuse the same phi_k through episode-dependent activations. Proposition 3 is the load-bearing formal mechanism: if the parent episode e and subtype episode e' keep similar activations of the same state at both stages, then the state-specific revisions for the two episodes are bounded together; only by separatin

What would settle it

A concrete falsifier would be an experiment that removes all input-feature overlap while keeping the type structure—for example, a dataset where the parent and subtype share no obvious surface feature—and shows that the G2 decline still exceeds the G5 decline. Alternatively, an intervention that suppresses the hypothesized shared component (e.g., ablating a candidate filter) while leaving input features intact should eliminate the asymmetry; if the asymmetry persists under either manipulation, the lifted-state explanation is not needed.

Watch

Extended reading notes

Core claim

The central claim is that an LLM's predictions change together for inputs it treats as computationally equivalent, because those inputs reuse the same latent filter–value component—the lifted state. Formally, a state s_k = (z_k, phi_k) is lifted when episodes within a computation-relative type activate it and reuse the same value phi_k. Two propositions carry the argument: parent learning can increase an unseen subtype's activation when the subtype representation is aligned with the filter update (Prop. 2), and if the parent and subtype continue to activate the same state with similar strength, then changes to that state cannot differ much between them (Prop. 3), forcing either rerouting, fi

Load-bearing premise

The load-bearing assumption is that the observed behavioral asymmetry (larger G2 decline than G5 decline) is caused by a shared latent lifted state, rather than by simple input-feature overlap (both G2 and G4 are even numbers) or by gradient interference between same-label examples.

Editorial extensions

If this is right

If the hypothesis is correct, any training method that updates a rule for a subtype will tend to also update predictions for the parent type, even when the parent's behavior should be preserved.
The asymmetry between the parent group (G2) and the disjoint control (G5) provides a diagnostic signature: an update that moves a subtype more than an unrelated same-label group can be used to detect when a model is reusing a shared computational state.
Sequential learning of a broad rule followed by subtype-specific data can outperform joint learning in some settings because the initial broad state serves as a scaffold; the benefit depends on the model's ability to later reorganize that state.
The framework suggests that model editing, unlearning, and continual learning should be analyzed in terms of which computation-relative types share a latent state, since edits propagate through that state's filter and value.
In-context learning shows the same coupled-revision pattern, implying that the lifted-state organization is not specific to parameter update methods but arises from how the model forms computational types in context.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the hypothesis would intervene on the hypothesized filter or value directions (e.g., by patching or steering a specific component) and check whether the parent group's accuracy change tracks the subtype's revision; the paper's evidence is behavioral and correlational, so such an intervention would make the latent state observable.
The framework predicts that making the subtype's distinguishing feature more explicit in the input (e.g., cueing that multiples of 4 are a separate kind) should reduce the shared activation and therefore shrink the G2 decline, providing a testable extension.
The same coupled-revision pattern may explain why safety fine-tuning can inadvertently weaken refusal behavior: if refusal-relevant inputs form a computation-relative type sharing a lifted state, updating a benign subtype could propagate to the whole type.
The activation analysis suggests a practical metric—relative closeness of the subtype to the retained parent versus to controls—that could be used during training as an early indicator of whether a revision will localize or spread.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my read. The paper is worth engaging for the dataset and the cross-method empirical pattern, but the theoretical package around 'lifted states' does more work than the evidence can support. You should treat it as a useful behavioral study with an over-ambitious interpretation.

The genuinely new thing is the NMR-Type benchmark, plus the systematic finding that revising the withheld subtype G4 degrades the parent G2 more than the same-label control G5 across full fine-tuning, LoRA, and ICL on a wide sweep of models. That is a reproducible, well-measured pattern. The authors also deserve credit for their honesty: they repeatedly call the evidence 'behavioral,' they put the descriptive activation analysis in the appendix, and they state the limits of identifying latent states. The formal automata/lifted-inference analogy is clean and the two propositions are correct, but they are not deep—Prop 2 is just monotonicity of the sigmoid and Prop 3 is a triangle inequality.

Now the soft spots. The core asymmetry G2>G5 is confounded by parity: G2 and G4 are even, G5 is odd, so an even-parity feature cluster is enough to produce the effect. The appendix's mixed-vs-pure G5 comparison actually confirms that surface-feature overlap increases G5 decline, which supports the low-level overlap story; the residual effect on pure G5 shows 'something shared' but not specifically a computation-relative lifted state. The activation analysis is correlational, uses diagonal-Mahalanobis distances, and is explicitly descriptive; it never tests whether a single causal component is reused across G2 and G4. Eq. (7) is assumed, not fitted. So while the paper's conclusion is phrased as 'consistent with' the hypothesis, that is genuinely all the evidence supports—the hypothesis is not differentiated from feature overlap or gradient interference on these data.

There is also no significance testing on the main 5-seed means (the figures show means and some bands, but the headline asymmetry is not quantified with confidence intervals or a test). That is a minor fix but should be asked for.

Bottom line: the paper deserves a serious referee—the benchmark and the empirical asymmetry are valuable and likely to be used by others. But the review should push for either causal/representational evidence for the lifted-state claim or a reframing that keeps the behavioral finding and drops the latent-state attribution. I would not cite it for the theory; I would cite it for the dataset and the revision-asymmetry result.

Referee Report

4 major / 5 minor

Summary. The paper introduces the Lifted State Hypothesis (LSH): under fixed model parameters, context scope, and target computation, samples that are indistinguishable in computation-relevant behavior form a computation-relative type, and the model is hypothesized to realize such types through reusable latent filter–value components ('lifted states', Eq. 7). It formalizes consequences for non-monotonic revision (Propositions 2–3), and introduces the NMR-Type Dataset, where multiples of 4 are withheld during W1 (P2→positive, P3→negative, P5→positive) and then revised to negative in W2 while G2, G3, G5 are replayed. Experiments across full fine-tuning, LoRA, and in-context learning on several LLM families show that G4 flips while G2 (the preserved parent) declines more than G5 (disjoint positive control). The authors interpret this asymmetry as behavioral evidence that revision follows computation-relative type structure. The appendix includes activation-distance analyses, which the authors describe as descriptive rather than statistical pass criteria.

Significance. The paper addresses an important question: when an LLM revises a subtype, what determines which other predictions change together. The LSH is clearly framed, connects to automata/lifted probabilistic inference, and the NMR-Type task is a simple, transparent benchmark. The formal development is sound as far as it goes, and the paper is candid that its experiments do not directly characterize latent states. However, the central empirical discriminator—the G2 versus G5 asymmetry—is confounded by low-level feature overlap: G2 and G4 are both even, while G5 is odd. The appendix's mixed-versus-pure G5 comparison shows that surface-feature overlap contributes to the asymmetry. The activation evidence is correlational and explicitly descriptive. Consequently, the current experiments do not yet distinguish the LSH from ordinary input-feature overlap or gradient interference. The confound is addressable with additional controls and interventional analyses, so the work is a promising starting point rather than a definitive demonstration.

major comments (4)

[Results; Eq. (32)] Equation (32) defines Acct(Gj) with y_{j,t} the target label at that stage, and the dataset section states W1 target G4→positive. However, the Results section says: 'Because G4 is evaluated against its final negative label, this generalization appears as low accuracy before W2.' If G4 accuracy is scored against the final negative label throughout, then the low W1 G4 curve is a measurement convention, not an empirical demonstration of subtype generalization. The metric must be defined consistently, and the main curves should be reported under both stage-specific and final-label conventions.
[Results, G2 vs G5 asymmetry; Figs. 7–10] The central claim that revision is concentrated in G2 rather than G5 is confounded by parity/feature overlap: G2 and G4 are even, while G5 is odd. A low-level parity or embedding-feature overlap is sufficient to produce the observed asymmetry. The appendix's mixed-versus-pure G5 comparison (Figs. 7–10) explicitly shows that numerical overlap increases G5 decline, confirming that surface-feature overlap contributes to the effect. A matched control that shares the low-level feature but is not in a parent–subtype relation is needed, along with a quantitative estimate of the G2–G5 asymmetry and its uncertainty. The current main figures report 5-seed means without error bars or significance tests.
[Eq. (7), Hypothesis 1; Eqs. (16), (26)] Propositions 2 and 3 are analytic consequences of the stated assumptions, but those assumptions—the filter–value decomposition (Eq. 7), initial alignment (Eq. 16), and persistent sharing (Eq. 26)—are exactly what requires empirical support. The paper does not fit or validate Eq. (7), and the activation analysis in the appendix uses aggregate diagonal-Mahalanobis distances (Eqs. 67–71) that are explicitly 'descriptive rather than statistical pass criteria.' These analyses do not test whether a single latent component is reused across G2 and G4 or whether that component causally mediates the output change. Without patching/ablation or another causal intervention on a candidate state, the behavioral asymmetry remains equally explained by feature overlap or gradient interference.
[Limitations and Future Work; Discussion] The manuscript itself states that the experiments 'provide behavioral evidence but do not directly characterize how lifted states are represented or realized within the model.' This is an appropriate limitation, but it means the strongest claim in the Results—that prediction changes are concentrated within a 'computation-relative type' implicated by the exception—goes beyond the evidence. The paper should either soften the central claim to 'behavioral pattern consistent with' the hypothesis, or add the causal/interventional analyses needed to support the latent-state interpretation.

minor comments (5)

[Eq. (32); Evaluation] The definition of accuracy counts outputs without a valid label as incorrect. This conflates format errors with semantic errors; reporting a format-error rate separately would make the accuracy curves easier to interpret.
[Figure 2] The 'Sensitivity' panel is defined as |Δaccuracy| per context example, but the normalization is not stated in the text. Please define it explicitly in the caption or methods.
[Appendix, Activation analysis] The use of a diagonal precision matrix with shrinkage is reasonable, but the choice of the baseline set and the shrinkage amount should be reported in a table so the descriptive statistics are reproducible.
[Notation] The word 'type' is used both for the logical benchmark groups (G2, G3, G4, G5) and for 'computation-relative type.' This is potentially confusing; consider a distinct notation for the theoretical construct versus the dataset groups.
[Table 2] The table has duplicated header rows and irregular spacing. A single header with clear column grouping would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; the paper explicitly frames its results as behavioral evidence, and no fitted parameter or self-citation chain is load-bearing.

full rationale

The paper's central derivations are conditional formalizations, not predictions derived from fitted quantities. Eq. (7) is explicitly an 'idealized local decomposition' and Hypothesis 1 is stated as a hypothesis; neither is fitted to the experimental accuracy data. Propositions 2 and 3 are algebraic identities under their stated assumptions (e.g., persistent activation bounds), so they do not smuggle the experimental outcome into the premises. The behavioral result — a larger G2 decline than G5 decline — is an empirical observation, and the paper repeatedly limits its interpretation: 'Our experiments provide behavioral evidence but do not directly characterize how lifted states are represented or realized within the model' (Limitations), and the activation analysis is 'descriptive rather than statistical pass criteria' (Appendix). The alternative parity/feature-overlap explanation is acknowledged: 'Parity is one possible explanation, because pure Type 5 and Type 3 examples are odd while Type 2 examples are even, but the results do not uniquely establish it.' That is underdetermination, not circularity. No parameter is fitted to the target asymmetry, no external result is replaced by a self-citation, and no uniqueness theorem from the authors' prior work is invoked. The main-text claim that the results 'distinguish type-related revision from broader same-label interference' is stronger than the evidence supports, but overclaiming is a validity concern, not a circularity concern.

Assumptions & free parameters 0 free parameters · 6 assumptions · 1 invented entities

The central claim rests less on fitted constants than on assumed latent structure: the filter–value decomposition (Eq. 7), the default generalization bias, the initial alignment condition (Eq. 16), and the persistent-sharing bound (Eq. 26) are posited rather than measured. No free parameters are fitted to data; the experimental hyperparameters are standard.

assumptions (6)

ad hoc to paper The filter–value decomposition (Eq. 7): h_i = Σ a_k(e_i) φ_k + r_i, with a_k(e_i) = σ(z_kᵀ q(e_i)), is a valid functional approximation of LLM internal computation.
Introduced in §'Lifted State Hypothesis in Language Models' without empirical identification; the authors call it 'a functional approximation rather than a unique factorization.'
domain assumption Default generalization bias: absent distinguishing evidence, learning favors the coarser extension of the observed parent outcome.
Assumed in §'Subtype Generalization from Incomplete Evidence' to justify that G4 inherits the P2 rule in W1; no learning-theoretic derivation.
ad hoc to paper Initial alignment condition (Eq. 16): the unseen subtype activates the parent state positively but less strongly than observed parent episodes before training.
Needed for Prop. 2's transfer story; not measured.
ad hoc to paper Persistent sharing assumption (Eq. 26): activation discrepancy between parent and subtype remains bounded before and after subtype learning.
Assumed to derive Prop. 3; the paper does not verify this bound empirically.
standard math Myhill–Nerode theorem and lifted probabilistic inference results are correct and applicable as precedents.
Used in the appendix to motivate equivalence-based reuse; standard results.
domain assumption LLM training on integer-label pairs with the given prompts induces learning relevant to the target computation rather than format artifacts.
The evaluation assumes deterministic decoding and label parsing reflect intended rule learning; not guaranteed.

invented entities (1)

Lifted state s_k = (z_k, φ_k)
purpose: A reusable latent filter–value component said to support a computation-relative type and to explain coupled non-monotonic revision.
The paper provides only behavioral and correlational activation evidence; no direct measurement, ablation, or falsifiable quantitative prediction. The authors state that 'identifying the corresponding latent states requires representational and causal analysis.'

how reviews work

0 comments

Cite this review

Pith. "Pith review of Lifted State Hypothesis in Large Language Models." pith.science (2026). https://pith.science/paper/DE6USRM3

@misc{pith2026260719360,
  author       = {Pith},
  title        = {Pith review of: Lifted State Hypothesis in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DE6USRM3}},
  note         = {Machine review of arXiv:2607.19360}
}

read the original abstract

Large language models (LLMs) adapt rapidly through fine-tuning and in-context learning, yet it remains unclear which inputs they treat as the same case and why their predictions change together. We introduce the Lifted State Hypothesis. Under fixed model parameters, context scope, and target computation, samples indistinguishable in their observed computation-relevant behavior form a computation-relative type. We hypothesize that compatible episodes activate a reusable latent component---a lifted state---that supports the target computation. State reuse enables type-level generalization but creates a non-monotonic revision problem. When later evidence distinguishes a subtype, revising a state still shared with its parent may affect members whose predictions should remain unchanged. The model must separate the subtype through rerouting, a new state, or input-specific compensation. We formalize this relation between generalization and revision. We introduce the NMR-Type Dataset to evaluate LLMs. The dataset first supports a broad modulo rule. It then provides conflicting supervision for a withheld subtype while replaying earlier examples. Across full fine-tuning, LoRA, and in-context learning, models often generalize the broad rule to the subtype but fail to localize its later revision. These results provide behavioral evidence consistent with the hypothesis and motivate further study of lifted-state formation and revision.

Figures

Figures reproduced from arXiv: 2607.19360 by the authors.

**Figure 1.** Label-wise test accuracy during sequential [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

**Figure 2.** Accuracy on four datasets and overall accuracy under parameter updates. Specific [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Figures from the paper (13 more)

**Figure 3.** Figure 3: In-context exception learning. Accuracy on the negative exception region, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]

**Figure 3.** Figure 3: Functional correspondence between lifted probabilistic modeling and the Lifted State Hypothesis under sequential [PITH_FULL_IMAGE:figures/full_fig_p014_3.png]

**Figure 4.** Figure 4: Model-wise accuracy trajectories under ordered parameter updates. In the general-to [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]

**Figure 5.** Figure 5: Average in-context learning accuracy across models. In the general-to-specific order, [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]

**Figure 5.** Figure 5: Final mean accuracy over all types across model [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

**Figure 6.** Figure 6: Model-wise and region-wise in-context learning trajectories. Exception examples often [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]

**Figure 7.** Figure 7: Accuracy trajectories averaged over Types 2, 3, 4, and 5 for mixed and pure [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

**Figure 8.** Figure 8: Model-wise accuracy trajectories comparing mixed and pure [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

**Figure 9.** Figure 9: Mean accuracy over all models by Type 4 demonstrations for mixed and pure 5 id ith 2/3/4 5 l [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

**Figure 10.** Figure 10: Model-wise accuracy by Type 4 demonstrations for mixed and pure [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]

**Figure 11.** Figure 11: The blue curve shows the baseline-relative convergence gain of Type 2–Type 4, while the gray dashed curve shows the [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

**Figure 12.** Figure 12: The y-axis reports the proportion of the four cross-control distances [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]

**Figure 13.** Figure 13: Layerwise change in Type 2–Type 4 relative closeness from update [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 2 linked inside Pith

[1]

By analogy, an LLM may realize such computational states through neu- rons, features, circuits, or distributed combinations of components

Existence of computational states.Automata show that distinct inputs can share a state that retains the informa- tion required for a target computation. By analogy, an LLM may realize such computational states through neu- rons, features, circuits, or distributed combinations of components
[2]

Computational states are thereforenotintrinsiccategoriesofinputs,buttask-relative organizations formed according to which differences mat- ter for producing the target output

Target-relative organization of states.The distinctions preserved by a state are determined by the target compu- tation and its output behavior. Computational states are thereforenotintrinsiccategoriesofinputs,buttask-relative organizations formed according to which differences mat- ter for producing the target output
[3]

Unlike in automata, these relationsarecontinuous,distributed,andimplicitinLLMs

Revision through input–state reorganization.When a subset requires different behavior, revision must change its routing or state membership. Unlike in automata, these relationsarecontinuous,distributed,andimplicitinLLMs. The correspondence is functional rather than structural. Automata partition inputs into exact finite states. LLMs may realize comparable...
[4]

Evaluate their shared computation collectively rather than processing each grounding separately

Lift repeated computation.Identify ground factors that are interchangeable under the current evidence and query. Evaluate their shared computation collectively rather than processing each grounding separately
[5]

A probabilistic model is conventionally written using generic random-variable names such asP(X, Y, Z)

Refine when necessary.If new evidence distinguishes membersofaliftedgroup,splitorshatterthegroupbefore continuing inference. A probabilistic model is conventionally written using generic random-variable names such asP(X, Y, Z). To dis- tinguish the roles played by variables in the computation, we instead write P(X, C, F), where X describes the instance,C ...

2008
[7]

Construct the probabilistic model.Define random vari- ablesandlocalfactorswhoseproductrepresentstheirjoint distribution
[8]

Form the ground-level computation.Replace each logi- calvariablewithaconcretedomainindividual.Theresult- ing object-specific variables and factors are calledground variables and factors (e.g., replacingB with penguin givesCanFly(penguin))
[9]

Compute the target probability.Condition on the ob- served evidence and marginalize variables that are not part of the query

Show all 15 references

[12]

Model scale and checkpoints.Does the generalization– revision pattern persist across model sizes?
[13]

Sequential revision versus joint learning.Is it more effectivetofirstlearnthebroad W1 ruleandreviseitunder W2, or to learn allW2 cases jointly from the start? We compare sequential W1 →W 2 learning with W2-only learning using the same number ofW2 updates
[14]

Specificity under overlapping and non-overlapping controls.Does the revision pattern involvingG5 persist when G5 isconstructedwithorwithoutnumericaloverlap with the other groups?
[15]

We compareLlama,Gemma-3,andQwen3atmultiplescalesand evaluate five checkpoints of OLMo-2-7B and OLMo-3-7B under full and LoRA finetuning

Activation and causal analysis.DoesW1 form shared MLP activation structure betweenG2 and G4, and do changestothesesharedcomponentsexplainthesubsequent G2 decline through routing or output-value revision? Model Scale and Checkpoints We examine whether the generalization–revisio...
[2022]

In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8493–8502

Knowledge Neurons in Pretrained Transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8493–8502. Association for Computational Linguistics. Darwiche, A.; and Pearl, J. 1997. On the logic of iterated be...

1997 arXiv
[2024]

InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 10480– 10496

Belief revision: The adaptability of large language models reasoning. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 10480– 10496. Yang, A.; et al. 2025a. Qwen3 Technical Report. arXiv:2505.09388. Yang, W.; et al. 2025b. Coarse-to-Fin...

2024 arXiv
[5426]

Rudinger, R.; Shwartz, V.; Hwang, J

Association for Computational Linguistics. Rudinger, R.; Shwartz, V.; Hwang, J. D.; Bhagavatula, C.; Forbes, M.; Le Bras, R.; Smith, N. A.; and Choi, Y. 2020. Thinking like a skeptic: Defeasible inference in natural lan- guage. InFindings of the Association for Computational L...

2020

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

By analogy, an LLM may realize such computational states through neu- rons, features, circuits, or distributed combinations of components

Existence of computational states.Automata show that distinct inputs can share a state that retains the informa- tion required for a target computation. By analogy, an LLM may realize such computational states through neu- rons, features, circuits, or distributed combinations of components

[2] [2]

Computational states are thereforenotintrinsiccategoriesofinputs,buttask-relative organizations formed according to which differences mat- ter for producing the target output

Target-relative organization of states.The distinctions preserved by a state are determined by the target compu- tation and its output behavior. Computational states are thereforenotintrinsiccategoriesofinputs,buttask-relative organizations formed according to which differences mat- ter for producing the target output

[3] [3]

Unlike in automata, these relationsarecontinuous,distributed,andimplicitinLLMs

Revision through input–state reorganization.When a subset requires different behavior, revision must change its routing or state membership. Unlike in automata, these relationsarecontinuous,distributed,andimplicitinLLMs. The correspondence is functional rather than structural. Automata partition inputs into exact finite states. LLMs may realize comparable...

[4] [4]

Evaluate their shared computation collectively rather than processing each grounding separately

Lift repeated computation.Identify ground factors that are interchangeable under the current evidence and query. Evaluate their shared computation collectively rather than processing each grounding separately

[5] [5]

A probabilistic model is conventionally written using generic random-variable names such asP(X, Y, Z)

Refine when necessary.If new evidence distinguishes membersofaliftedgroup,splitorshatterthegroupbefore continuing inference. A probabilistic model is conventionally written using generic random-variable names such asP(X, Y, Z). To dis- tinguish the roles played by variables in the computation, we instead write P(X, C, F), where X describes the instance,C ...

2008

[6] [7]

Construct the probabilistic model.Define random vari- ablesandlocalfactorswhoseproductrepresentstheirjoint distribution

[7] [8]

Form the ground-level computation.Replace each logi- calvariablewithaconcretedomainindividual.Theresult- ing object-specific variables and factors are calledground variables and factors (e.g., replacingB with penguin givesCanFly(penguin))

[8] [9]

Compute the target probability.Condition on the ob- served evidence and marginalize variables that are not part of the query