Pith. sign in

REVIEW 3 major objections 6 minor 57 references

Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that ChatGPT-like AIs undergo a temperature-driven population inversion—output entropy peaks then falls as decoder temperature rises—and that a hidden internal coordinate predicts and steers repetition.

desk verdict The hidden-coordinate steering result is real and worth refereeing, but the 'entropy maximum and population inversion' is a tautology of the paper's own binary-entropy definition. read the letter →

arxiv 2608.00939 v1 pith:UPSJSY4K submitted 2026-08-02 physics.soc-ph cond-mat.dis-nncs.AInlin.AOphysics.app-ph

Neil F. Johnson , Frank Yingjie Huo , Bella Xinrui Li This is my paper · ORCID
classification physics.soc-phcond-mat.dis-nncs.AInlin.AOphysics.app-ph
keywords largelanguagemodelsdecodertemperaturepopulationinversionoutputentropyautoregressivefeedbackhiddenstatecoordinatenonlineardynamicstextrepetition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that ChatGPT-like AIs are not stochastic parrots but controllable nonlinear systems, by showing that raising the decoder temperature drives the long-time output population through an entropy maximum and into population inversion. Across 12,000 continuations from 11 AIs, the complex/noisy sector crosses equal occupation with the recurrent sector at a mean temperature $T_d^*=0.517$, so the coarse two-sector entropy first rises and then falls. The paper also identifies a hidden coordinate $x_n$, built from internal hidden states on training data, that predicts repetition on 140 separate test trajectories with rank correlation 0.933, and shows that steering the model along this coordinate changes repetition. If these claims are right, temperature is not a simple randomness dial, and LLM output has a measurable, steerable internal state.

What carries the argument

The load-bearing object is the hidden coordinate $x_n$, a scalar obtained by centering the transformer's hidden state and projecting onto a fixed direction learned from the average hidden states of the most- and least-repetitive training trajectories. The paper shows this coordinate is genuinely dynamical rather than a post-hoc label: including the previous value $x_{n-1}$ improves held-out prediction at all 14 temperatures by 1.1–8.0%, and four matched orthogonal perturbations affect behavior only through the later change in $x_n$. The coordinate is embedded in an effective stochastic map $x_{n+1}\simeq a+b x_n - c x_n^3 + \kappa x_{n-1} + \eta_n$, whose cubic term is the leading saturation

What would settle it

Regenerate the 12,000-continuation survey with an externally validated output-diversity metric (for example, self-BLEU or a compression-based complexity score) instead of the six-class rule set, and test whether the high-diversity fraction still crosses 0.5 near $T_d=0.517$ and continues to rise to $T_d=1$; if it does not, the claimed population inversion is an artifact of the classifier.

Watch

Extended reading notes

Core claim

Decoder temperature $T_d$ controls the whole autoregressive output, not just single tokens: although higher $T_d$ broadens each next-token distribution, the long-time text population passes through a coarse two-sector entropy maximum and inverts. All 11 AIs cross equal occupation between recurrent and complex/noisy sectors (crossings $T_d=0.425$–$0.562$; mean $T_d^*=0.517$), and the complex/noisy occupation reaches $0.932$ at $T_d=1$—a bounded-spectrum inversion in the occupancy-defined $\beta_{\text{eff}}\Delta$. For Llama-3.1-70B, a hidden coordinate $x_n$ (a projection of hidden states onto a training-defined direction) predicts repetition on 140 unseen trajectories with rank correlation

Load-bearing premise

Everything in the population-inversion measurement—the entropy peak, the crossing at $T_d^*=0.517$, and the occupancy-defined inverse temperature—rests on a hand-built sentence-recurrence classifier and a fixed two-sector pooling; a different rule set or pooling could shift or remove the inversion.

Editorial extensions

If this is right

  • Above the mean crossing $T_d^*=0.517$, further raising the decoder temperature reduces the coarse two-sector entropy: users who increase temperature to get more varied text can instead get output concentrated in the complex/noisy sector.
  • The hidden coordinate $x_n$, measured from internal hidden states before each token is chosen, predicts repetition on held-out trajectories (rank correlation 0.933 overall, 0.565 within fixed temperature), so repetition has an internal precursor available during generation.
  • Steering activations along the hidden direction shifts repetition monotonically, with the largest positive steering at $T_d=1$ increasing repetition by 0.255, meaning internal-state interventions can push output across the recurrent/complex boundary.
  • The effective map $x_{n+1}\simeq a + b x_n - c x_n^3 + \kappa x_{n-1} + \eta_n$ implies the transition is organized by fixed points, cycles, intermittency and noise-induced ordering, giving a nonlinear-dynamics classification of output regimes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper, if the hidden-coordinate construction generalizes to other prompts and model families, repetition could be monitored in real time and used for closed-loop control of sampling parameters—something the paper does not demonstrate.
  • The inversion uses an occupancy-defined $\beta_{\text{eff}}\Delta$; a true negative-absolute-temperature reading would require a Hamiltonian over the output distribution and a bounded spectrum, which the paper does not establish.
  • A natural testable extension is to re-run the 12,000-continuation survey with a model-free complexity metric (e.g., compression ratio) to check whether the entropy maximum and crossing are properties of the language output rather than of the six-class rule set.
  • If the cubic map is generic, varying prompt bias ($a$) or feedback gain ($b$) near $T_d^*$ should produce hysteresis or critical slowing down, measurable through autocorrelation of $x_n$.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript studies temperature-dependent text generation in GPT-2, Pythia, and Llama-3.1-70B models. It classifies continuations into six sentence-level recurrence regimes, pools them into a 'recurrent' and a 'complex/noisy' sector, and tracks the complex/noisy fraction q as a function of decoder temperature T_d. It reports that q crosses 0.5 at a mean temperature T_d* = 0.517, and interprets this as an entropy maximum and a population inversion, with the effective inverse-temperature parameter β_effΔ changing sign. The paper then analyzes hidden states of Llama-3.1-70B, defines a one-dimensional hidden coordinate x_n from training data, and reports that its trajectory average predicts repetition on 140 disjoint test trajectories (rank correlation 0.933; within-temperature rank correlation 0.565). Perturbing the hidden state along this direction changes repetition, and the authors fit a cubic map with memory to the x_n dynamics. The paper argues that LLMs are not 'stochastic parrots' but controllable nonlinear physical systems.

Significance. The hidden-coordinate results are potentially significant: the train/test split is properly disjoint, the steering experiment includes zero-strength controls, and the within-temperature correlation suggests that the coordinate captures more than a temperature proxy. The provision of code, data, and a SHA-256 checksum is a strength. If the hidden coordinate is reproducible and steerable, it would be a practically useful diagnostic for repetition in LLMs. However, the paper's headline 'entropy maximum and population inversion' is not an independent finding: Eq. (1) defines S_2 as the binary entropy of the two-sector split, so S_2 is maximal at q=0.5 by construction, and β_effΔ = ln((1-q)/q) changes sign at q=0.5 by construction. The empirical content reduces to a classifier-dependent occupancy crossover. This weakens the claimed statistical-mechanical novelty, even though the underlying q(T_d) crossover and the hidden-coordinate analysis may remain of interest.

major comments (3)
  1. [Temperature drives population inversion, Eq. (1), Fig. 2] The central claim that autoregressive feedback drives output 'through an entropy maximum and into population inversion' is not supported as an independent measurement. Eq. (1) defines S_2 = -q ln q - (1-q) ln(1-q) and β_effΔ = ln((1-q)/q). For any empirical q, S_2 is maximized at q=1/2 and β_effΔ changes sign at q=1/2. Thus the 'entropy maximum' and 'inversion' in the abstract and Fig. 2 are mathematical restatements of the fact that q(T_d) crosses 0.5. The paper does later state that 'the robust transfer ... through equal occupation is the central empirical result', but the abstract and surrounding text present the entropy/inversion as a physical discovery. The manuscript should be reframed to report the occupancy crossover as the empirical result, and either avoid the entropy/inversion terminology or support it with an independent measure of output entropy or level populations with sep
  2. [Methods, 'Cross-AI survey'] The entire q(T_d) curve rests on a hand-built sentence-recurrence classifier. The main text says only that 'a fixed rule set classified frozen, sparse, periodic, intermittent, complex and noisy regimes' and that the two-sector pooling is 'effectively' valid, with no thresholds or validation in the main text. Because the existence and location of the crossing at T_d* = 0.517 could depend on these thresholds and on the pooling choice, this is not a purely presentational issue. The authors should provide the full classifier rules and a sensitivity analysis (e.g., varying the thresholds or the pooling) to show that the occupancy crossover is robust.
  3. [Hidden coordinate predicts and steers output, Fig. 4] The overall rank correlation of 0.933 is dominated by between-temperature variation, since repetition collapses sharply with T_d. The within-temperature correlation of 0.565 is more convincing and is correctly reported, but the abstract's phrase 'strongly predicts output repetition' should be qualified. More importantly, the hidden direction is constructed from the upper and lower quartiles of trajectories ranked by repetition, so the high between-temperature correlation is partly built in. The steering result partially addresses causality, but the paper should state more explicitly how much of the 0.933 is a temperature effect and how much is per-temperature discrimination.
minor comments (6)
  1. [Methods] The main text should state the number of continuations per model/temperature/prompt to reconcile the claimed 12,000 continuations with the reported seeds and conditions.
  2. [Fig. 1] Figure 1 is described as illustrative, but the color coding or symbols used for coarse-grained sentences are not defined; a small legend or caption detail would help.
  3. [Fig. 4b] The figure caption says color denotes decoder temperature, but no colorbar or temperature legend is visible; please add one.
  4. [Introduction] The characterization of the 'stochastic parrots' metaphor as claiming machines are 'systems that recombine statistical patterns ... without grounded understanding' is accurate, but the paper uses it as a straw man against 'apparent randomness'. The original critique is about understanding and accountability, not about whether LLM output is random or deterministic; please engage with that distinction.
  5. [Eq. (2) and Supplementary Note 6] The derivation of the cubic map from 'projecting the transformer operations' is heuristic. The empirical comparison of candidate maps is useful, but the paper should more clearly label Eq. (2) as a phenomenological model rather than a derived equation of motion.
  6. [Conclusions] The claim that LLMs are 'a new class of controllable nonlinear physical systems' is too broad given that the hidden-coordinate analysis is performed on a single model (Llama-3.1-70B). Please temper the generalization or state the scope explicitly.

Circularity Check

2 steps flagged · score 6.0 of 10

Eq. (1) makes the entropy maximum and population-inversion sign change algebraic consequences of q crossing 0.5; the hidden-coordinate prediction/steering is a genuine out-of-sample result.

  1. self definitional [Section 'Temperature drives population inversion', Eq. (1) and Fig. 2a]
    "m= 1−2q, S2 =−qlnq−(1−q) ln(1−q), βeff∆ = ln((1−q)/q). ... Only the dimensionless product βeff∆ is determined by q. ... The entropy therefore rises to its maximum and then falls as the complex/noisy sector becomes overpopulated."

    S2 is defined to be the binary entropy of the two-sector fraction q, so S2 is maximal at q=0.5 by construction; βeff∆ is defined as the log-odds of q, so it is zero and changes sign exactly at q=0.5. The observed peak near Td*=0.517 and the 'inversion' are therefore algebraic rewrites of the measured crossing q(Td)=0.5. The only empirical content is the crossing itself; the entropy maximum and sign change are not independent measurements.

  2. self definitional [Section 'Temperature drives population inversion', text following Fig. 2c]
    "Beyond equal occupation, raising the positive decoder temperature reduces the coarse mixing entropy even though ... it increases the entropy of the distribution for token n + 1 when Cn ... is held fixed. ... the dimensionless inverse-temperature parameter βeff∆ changes sign and becomes negative. The coarse output population therefore displays a bounded-spectrum inversion."

    The decrease in 'coarse mixing entropy' and the negativity of βeff∆ both follow from q>0.5 under Eq. (1), once the complex/noisy sector is labelled the 'higher' level. Calling this a bounded-spectrum inversion renames the two-sector occupancy imbalance; no additional dynamical information is measured beyond the q(Td) curve.

full rationale

The macro-level signature is partially circular: S2 and βeff∆ in Eq. (1) are explicit functions of q alone, so the 'entropy maximum' and 'population inversion' are definitional consequences of q crossing 0.5, not new measurements. The paper itself states that the robust transfer through equal occupation is the central empirical result and the entropy/sign change are its signatures, but the abstract presents these signatures as the discovery. The hidden-coordinate analysis is not circular: the direction is defined from training trajectories only and tested on 140 disjoint test trajectories, with rank correlation 0.933, within-temperature correlation 0.565, and steering interventions compared to exact α=0 controls. No load-bearing self-citation appears: refs [27,28] are the authors' own but are cited only for the standard softmax/Boltzmann temperature relationship alongside refs [25,26]. The q-crossing itself depends on a hand-built TF-IDF recurrence classifier whose thresholds are not given, but that is a measurement-validity issue, not circularity. Score 6 reflects that the central statistical-mechanical claim reduces by construction while a substantial independent result (hidden coordinate) remains.

Assumptions & free parameters 5 free parameters · 6 assumptions · 1 invented entities

The central inversion claim rests on hand-chosen classifier and pooling choices, not on a direct entropy measurement. The hidden-coordinate result depends on a supervised construction and fitted map coefficients, but it is validated on held-out trajectories.

free parameters (5)
  • sentence-recurrence classifier thresholds = not disclosed in main text; 'fixed rule set' in Supplementary Note 2
    Defines six output classes and therefore q; the central crossing at Td*=0.517 is relative to these thresholds.
  • two-sector pooling of six classes = frozen/sparse/periodic/intermittent -> recurrent; complex/noisy -> complex/noisy
    S2 and beta_eff delta are defined on this pooling; a different pooling would move or remove the inversion.
  • hidden direction construction (upper/lower quartile split) = weighted difference of average normalized hidden states of high/low repetition training trajectories
    The coordinate xn is this projection; the quartile split and layer choice are chosen on training data, though validated on test data.
  • effective map coefficients a, b, c, kappa, noise = not reported in main text; fitted to training trajectories
    Eq. (2) is a fitted phenomenological map; memory term improves test prediction by median 4.6%.
  • initial settling period (16 steps) = 16
    Hidden-state measurement begins at step 17; results could shift with a different settling window.
assumptions (6)
  • standard math Softmax temperature Td acts as a Boltzmann factor with effective energies Ei=-zi
    Standard softmax sampling; stated in intro.
  • domain assumption Autoregressive sampling with context-dependent logits is a stochastic dynamical system
    Defines generation as a map Cn -> Cn+1; core modeling framing.
  • domain assumption Sentence-level TF-IDF recurrence classes capture the relevant slow output structure
    Used to define q and all subsequent thermodynamic signatures; no external validation.
  • ad hoc to paper Six classes can be reduced to two sectors
    Pooling is the paper's choice; S2 depends on it.
  • ad hoc to paper Projection of transformer updates onto one hidden direction yields cubic map Eq. (2)
    Derivation deferred to Supplementary Note 6; terms a, b, c, kappa, eta asserted.
  • domain assumption Hidden direction learned from training trajectories transfers to disjoint test trajectories and to perturbed runs
    Supported by held-out correlations but assumed for the steering interpretation.
invented entities (1)
  • hidden coordinate xn independent evidence
    purpose: Scalar effective state variable for LLM dynamics; predicts and steers repetition
    Has out-of-sample predictive handle (rank corr 0.933 on 140 test trajectories) and perturbation response; though constructed from training repetition labels rather than discovered independently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs." pith.science (2026). https://pith.science/paper/UPSJSY4K

@misc{pith2026260800939,
  author       = {Pith},
  title        = {Pith review of: Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UPSJSY4K}},
  note         = {Machine review of arXiv:2608.00939}
}
read the original abstract

Increasing the temperature of an ordinary many-state system increases access to a wider range of states and hence increases its entropy. We find the opposite in ChatGPT-like AIs, even though raising the decoder temperature likewise increases access to a wider range of states (next-token choices). Across 12,000 continuations from 11 AIs, autoregressive feedback drives the long-time output population through an entropy maximum and into population inversion. The transition features frozen states, cycles, intermittency and noise-induced ordering. We present evidence of a hidden coordinate that acts as the state variable of an effective nonlinear map. Its trajectory average strongly predicts output repetition in separate test trajectories. ChatGPT-like AIs therefore behave not as `stochastic parrots', but as a new class of controllable nonlinear physical systems whose internal dynamics can be measured and perturbed.

Figures

Figures reproduced from arXiv: 2608.00939 by the authors.

Figure 1
Figure 1. Illustrative GPT-2 output for the prompt ‘AI is . . . ’. As Td rises, generation moves from a single repetitive attractor-like pattern through additional periodicities and switching, and eventually to regularity and novelty coexisting in human-like output. For convenience, we coarse-grain sentences to symbols [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Maximum entropy marks ChatGPT-like AIs’ output inversion. a, Normal￾ized mixing entropy for the two output sectors. Its maximum is near equal occupation. The dashed orange curve is the occupancy-defined dimensionless parameter βeff∆, which changes sign at the mean crossing T ∗ d = 0.517; the grey band spans the 11 individual-AI crossings. b, Occupancy of the complex/noisy sector for the 11 AIs (GPT-2, blue; Pythia, … view at source ↗
Figure 3
Figure 3. Different structured routes through the same population inversion. P1, P4 and P5 denote prompts 1, 4 and 5 in the fixed five-prompt set listed in Supplementary Note 2. a, Frozen output is the most common regime for Llama-3.1-70B at low and intermediate temperatures, before stronger noise destroys that ordering. At each Td, the marker position and background colour identify the most common regime among ten runs; mark… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: A hidden coordinate defined from training data predicts repetition in separate test trajectories. a, Repetition falls sharply between Td = 0.7 and 0.8 in separate training and test runs. Lines show medians across ten runs at each temperature; shading spans the interqua…
Figure 5
Figure 5. Figure 5: Steering the AI’s hidden state changes repetition. a, Change in repetition as the hidden state is steered with different strengths along the hidden direction. b, Resulting change in the hidden coordinate xn at later next-token generation steps, measured immediately bef…
Figure 6
Figure 6. Figure 6: Behavioural effects align with the later change in the hidden coordinate xn across perturbations. a, Behavioural change closely tracks the later change in xn after autoregressive evolution. b, It is only weakly related to the immediate change in xn. Each point represen…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 29 canonical work pages

  1. [1]

    Large language models encode clinical knowledge.Nature, 620:172–180, 2023

    Karan Singhal, Shekoofeh Azizi, Tao Tu, et al. Large language models encode clinical knowledge.Nature, 620:172–180, 2023. doi: 10.1038/s41586-023-06291-2

    doi:10.1038/s41586-023-06291-2 2023

  2. [2]

    Can ChatGPT forecast stock price movements? return predictability and large language models.Journal of Financial Economics, 184: 104335, 2026

    Alejandro Lopez-Lira and Yuehua Tang. Can ChatGPT forecast stock price movements? return predictability and large language models.Journal of Financial Economics, 184: 104335, 2026. doi: 10.1016/j.jfineco.2026.104335

    work page arXiv 2026

  3. [3]

    Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond. Generative AI at work.The Quarterly Journal of Economics, 140(2):889–942, 2025. doi: 10.1093/qje/qjae044

    doi:10.1093/qje/qjae044 2025

  4. [4]

    Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo

    Daniel M. Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo. GPT-4 passes the bar exam.Philosophical Transactions of the Royal Society A, 382(2270): 20230254, 2024. doi: 10.1098/rsta.2023.0254. 11

    arXiv 2024

  5. [5]

    Scientific discovery in the age of artificial intelligence.Nature, 620:47–60, 2023

    Hanchen Wang, Tianfan Fu, Yuanqi Du, et al. Scientific discovery in the age of artificial intelligence.Nature, 620:47–60, 2023. doi: 10.1038/s41586-023-06221-2

    doi:10.1038/s41586-023-06221-2 2023

  6. [6]

    Summary of NATO’s revised artifi- cial intelligence strategy

    North Atlantic Treaty Organization. Summary of NATO’s revised artifi- cial intelligence strategy. Technical report, NATO, 2024. URL https: //www.nato.int/en/about-us/official-texts-and-resources/official-texts/ 2024/07/10/summary-of-natos-revised-artificial-intelligence-ai-strategy . Accessed 31 July 2026

    work page 2024

  7. [7]

    Hudson, Ehsan Adeli, et al

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021. doi: 10.48550/arXiv.2108. 07258

    arXiv doi:10.48550/arxiv.2108.07258 2021

  8. [8]

    The curious case of neural text degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. InInternational Conference on Learning Representations,

    work page

Show all 57 references
  1. [9]

    A theoretical analysis of the repetition problem in text generation

    Zihao Fu, Wai Lam, Anthony Man-Cho So, and Bei Shi. A theoretical analysis of the repetition problem in text generation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12848–12856, 2021. doi: 10.1609/aaai.v35i14.17520

    2021 doi

  2. [10]

    When less is more: The LLM scaling paradox in context compression.arXiv preprint arXiv:2602.09789, 2026

    Ruishan Guo, Yibing Liu, Guoxin Ma, Yan Wang, Yueyang Zhang, Long Xia, Kecheng Chen, Zhiyuan Sun, and Daiting Shi. When less is more: The LLM scaling paradox in context compression.arXiv preprint arXiv:2602.09789, 2026. doi: 10.48550/arXiv.2602. 09789

    2026 doi arXiv

  3. [11]

    From condensation to rank collapse: A two-stage analysis of transformer training dynamics

    Zheng-An Chen and Tao Luo. From condensation to rank collapse: A two-stage analysis of transformer training dynamics. InAdvances in Neural Information Processing Systems, volume 38, 2025. URL https://proceedings.neurips.cc/paper_files/paper/2025/ hash/3b576711b12ab036b45130fc8...

    2025

  4. [12]

    The curse of depth in large language models

    Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin, Yefeng Zheng, and Shiwei Liu. The curse of depth in large language models. InAdvances in Neural Information Processing Systems, volume 38, 2025. URL https://proceedings.neurips.cc/paper_files/ paper/2025/hash/eeb57fdf745eb31a3c7...

    2025

  5. [13]

    Attention is not all you need: Pure attention loses rank doubly exponentially with depth

    Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. InProceedings of the 38th International Conference on Machine Learning, volume 139 ofProceedings of Machine Learning Research, pag...

    2021

  6. [14]

    Shadows in the attention: Contextual perturbation and representation drift in the dynamics of hallucination in LLMs.arXiv preprint arXiv:2505.16894, 2025

    Zeyu Wei, Shuo Wang, Xiaohui Rong, Xuemin Liu, and He Li. Shadows in the attention: Contextual perturbation and representation drift in the dynamics of hallucination in LLMs.arXiv preprint arXiv:2505.16894, 2025. doi: 10.48550/arXiv.2505.16894

    2025 doi arXiv

  7. [15]

    HalluLens: LLM hallucination benchmark

    Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, and Pascale Fung. HalluLens: LLM hallucination benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 24128–24156, 2025. doi...

    2025 doi

  8. [16]

    Large language models hallucination: A compre- hensive survey.arXiv preprint arXiv:2510.06265, 2025

    Aisha Alansari and Hamzah Luqman. Large language models hallucination: A compre- hensive survey.arXiv preprint arXiv:2510.06265, 2025. doi: 10.48550/arXiv.2510.06265

    2025 doi

  9. [17]

    Progress measures for grokking via mechanistic interpretability

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. InInternational Con- ference on Learning Representations, 2023. URL https://openreview.net/forum?id= 9XFSbDPmdW

    2023

  10. [18]

    A mechanistic interpretability analysis of grokking, 2022

    Neel Nanda and Tom Lieberum. A mechanistic interpretability analysis of grokking, 2022. URL https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/ a-mechanistic-interpretability-analysis-of-grokking. Accessed 7 May 2024

    2022

  11. [19]

    Paper replication walkthrough: Reverse-engineering modular ad- dition, 2023

    Neel Nanda. Paper replication walkthrough: Reverse-engineering modular ad- dition, 2023. URL https://www.neelnanda.io/mechanistic-interpretability/ modular-addition-walkthrough. Accessed 7 May 2024

    2023

  12. [20]

    Sparse autoencoders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. InInternational Conference on Learning Representations, 2024. URL https://openreview.net/forum? id=F76bwRSLeK

    2024

  13. [21]

    Circuit tracing: Revealing computational graphs in language models, 2025

    Emmanuel Ameisen, Jack Lindsey, Adam Pearce, et al. Circuit tracing: Revealing computational graphs in language models, 2025. URL https://transformer-circuits. pub/2025/attribution-graphs/methods.html. Accessed 28 March 2025

    2025

  14. [22]

    HeadVis: An interactive tool for investigating attention heads, 2026

    Rodrigo Luger, Harish Kamath, Doug Finkbeiner, Purvi Goel, Adam Jermyn, Sam Zimmerman, Joshua Batson, and Tom Conerly. HeadVis: An interactive tool for investigating attention heads, 2026. URL https://transformer-circuits.pub/2026/ headvis/index.html. Transformer Circuits Thre...

    2026

  15. [23]

    Ver- balizable representations form a global workspace in language models.arXiv preprint arXiv:2607.15495, 2026

    Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, et al. Ver- balizable representations form a global workspace in language models.arXiv preprint arXiv:2607.15495, 2026. doi: 10.48550/arXiv.2607.15495. URL https: //transformer-circuits.pub/2026/workspace/index.html

    2026 doi arXiv

  16. [24]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610–623,

    2021

  17. [25]

    Edwin T. Jaynes. Information theory and statistical mechanics.Physical Review, 106 (4):620–630, 1957. doi: 10.1103/PhysRev.106.620

    1957 doi

  18. [26]

    Hinton, Oriol Vinyals, and Jeff Dean

    Geoffrey E. Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015. doi: 10.48550/arXiv.1503.02531

    2015 doi arXiv

  19. [27]

    Frank Yingjie Huo and Neil F. Johnson. Physics of generative ai’s atom: Repetition, bias, and beyond.AIP Advances, 16(3):035305, 2026. doi: 10.1063/5.0296911

    2026 doi

  20. [28]

    Johnson and Frank Yingjie Huo

    Neil F. Johnson and Frank Yingjie Huo. Simple picture of how output from chatgpt-like ai shifts from good to bad.PNAS Nexus, 5(5):pgag148, 2026. doi: 10.1093/pnasnexus/ pgag148. 13

    2026 doi

  21. [29]

    Strogatz.Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering

    Steven H. Strogatz.Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Chapman and Hall/CRC, 3 edition, 2024. doi: 10.1201/9780429398490

    2024 doi

  22. [30]

    Motter and R´ eka Albert

    Adilson E. Motter and R´ eka Albert. Networks in motion.Physics Today, 65(4):43–48,

  23. [31]

    Adilson E. Motter. Networkcontrology.Chaos, 25(9):097621, 2015. doi: 10.1063/1. 4931570

    2015 doi

  24. [32]

    MIT Press, Cambridge, MA, 2010

    Olaf Sporns.Networks of the Brain. MIT Press, Cambridge, MA, 2010

    2010

  25. [33]

    LLMs as high-dimensional nonlinear autoregressive models with attention: Training, alignment and inference.arXiv preprint arXiv:2602.00426, 2026

    Vikram Krishnamurthy. LLMs as high-dimensional nonlinear autoregressive models with attention: Training, alignment and inference.arXiv preprint arXiv:2602.00426, 2026. doi: 10.48550/arXiv.2602.00426

    2026 doi

  26. [34]

    Phase transitions in the output distribution of large language models.arXiv preprint arXiv:2405.17088,

    Julian Arnold, Flemming Holtorf, Frank Sch¨ afer, and Niels L¨ orch. Phase transitions in the output distribution of large language models.arXiv preprint arXiv:2405.17088,

    arXiv

  27. [35]

    Phase transition in large language models and the criticality of natural languages.arXiv preprint arXiv:2406.05335,

    Kai Nakaishi, Yoshihiko Nishikawa, and Koji Hukushima. Phase transition in large language models and the criticality of natural languages.arXiv preprint arXiv:2406.05335,

    arXiv

  28. [36]

    States of llm-generated texts and phase transitions between them

    Nikolay Mikhaylovskiy. States of llm-generated texts and phase transitions between them. InMathAI 2025, 2025. URLhttps://arxiv.org/abs/2503.06330

    2025 arXiv

  29. [37]

    Phase transitions in large language models and the O(N) model.arXiv preprint arXiv:2501.16241, 2025

    Youran Sun and Babak Haghighat. Phase transitions in large language models and the O(N) model.arXiv preprint arXiv:2501.16241, 2025. doi: 10.48550/arXiv.2501.16241

    2025 doi arXiv

  30. [38]

    Generative criticality in large language model temperature scaling.arXiv preprint arXiv:2606.06238, 2026

    Huajian Ruan, Jinyang Li, Xingyu Guo, and Lingxiao Wang. Generative criticality in large language model temperature scaling.arXiv preprint arXiv:2606.06238, 2026. doi: 10.48550/arXiv.2606.06238. Contributed to PAI 2026

    2026 doi arXiv

  31. [39]

    Unveiling attractor cycles in large language models: A dynamical systems view of successive paraphrasing

    Zhilin Wang, Yafu Li, Jianhao Yan, Yu Cheng, and Yue Zhang. Unveiling attractor cycles in large language models: A dynamical systems view of successive paraphrasing. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape...

  32. [40]

    version 3, revised 2026

    doi: 10.48550/arXiv.2406.05335. version 3, revised 2026

    2026 doi arXiv

  33. [41]

    Tatsuya Hiraoka and Kentaro Inui. Repetition neurons: How do language models produce repetitions? InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers),...

    2025

  34. [42]

    Language models are unsupervised multitask learners

    Alec Radford et al. Language models are unsupervised multitask learners. Technical report, OpenAI, 2019. 14

    2019

  35. [43]

    Pythia: A suite for analyzing large language models across training and scaling

    Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, Usvsn Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. Pythia: A suite for analyzing large langu...

    2023

  36. [44]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Aaron Grattafiori et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    arXiv

  37. [45]

    Purcell and Robert V

    Edward M. Purcell and Robert V. Pound. A nuclear spin system at negative temperature. Physical Review, 81(2):279–280, 1951. doi: 10.1103/PhysRev.81.279

    1951 doi

  38. [46]

    Chaos in reason: How chain-of- thought LLMs can look for an answer.arXiv preprint arXiv:2607.27805, 2026

    Gregorio Jaca, Krist´ of Benedek, and J´ anos T¨ or¨ ok. Chaos in reason: How chain-of- thought LLMs can look for an answer.arXiv preprint arXiv:2607.27805, 2026. doi: 10.48550/arXiv.2607.27805

    2026 doi arXiv

  39. [47]

    Hodgman, Tim Rom, Immanuel Bloch, and Ulrich Schneider

    Simon Braun, Jan Philipp Ronzheimer, Michael Schreiber, Sean S. Hodgman, Tim Rom, Immanuel Bloch, and Ulrich Schneider. Negative absolute temperature for motional degrees of freedom.Science, 339(6115):52–55, 2013. doi: 10.1126/science.1227831

    2013 doi

  40. [48]

    doi: 10.18653/v1/2025.naacl-short.41

    2025 doi

  41. [49]

    Springer-Verlag, Berlin, Heidelberg, 1984

    Werner Horsthemke and Ren´ e Lefever.Noise-Induced Transitions: Theory and Applica- tions in Physics, Chemistry, and Biology, volume 15 ofSpringer Series in Synergetics. Springer-Verlag, Berlin, Heidelberg, 1984. doi: 10.1007/3-540-36852-3. 15

    1984 doi

  42. [52]

    doi: 10.48550/arXiv.2407.21783

    doi arXiv

  43. [54]

    Norman F. Ramsey. Thermodynamics and statistical mechanics at negative absolute temperatures.Physical Review, 103(1):20–28, 1956. doi: 10.1103/PhysRev.103.20

    1956 doi

  44. [56]

    Physics of negative absolute temperatures.Physical Review E, 95(1):012125, 2017

    Eitan Abraham and Oliver Penrose. Physics of negative absolute temperatures.Physical Review E, 95(1):012125, 2017. doi: 10.1103/PhysRevE.95.012125

    2017 doi

  45. [2012]

    doi: 10.1063/PT.3.1518

    doi

  46. [2020]

    URLhttps://arxiv.org/abs/1904.09751

    1904 arXiv

  47. [2021]

    doi: 10.1145/3442188.3445922

  48. [2024]

    doi: 10.48550/arXiv.2405.17088

    doi arXiv

  49. [2025]

    doi: 10.18653/v1/2025.acl-long.624

    2025 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.