Thermodynamic Limits of Physical Intelligence

Koichi Takahashi; Yusuke Hayashi

REVIEW 3 major objections 4 minor 1 cited by

Thermodynamic Limits of Physical Intelligence

T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a fundamental thermodynamic ceiling on how efficiently a closed-cycle learner can convert dissipated heat into stored structural information about its environment.

desk verdict A careful bits-per-joule accounting paper whose central limit is a correctly-stated corollary of Goldt-Seifert; the real value is the boundary-closure checklist, not new thermodynamics. read the letter →

arxiv 2602.05463 v2 pith:ALC5AZ4E submitted 2026-02-05 cs.LG cs.AIcs.ITmath.IT

Koichi Takahashi , Yusuke Hayashi This is my paper · ORCID

classification cs.LGcs.AIcs.ITmath.IT

keywords thermodynamicepiplexitybitsperjouleLandauer'sprincipleempowermentstochasticthermodynamicslearningefficiencyclosed-cyclebenchmarkminimumdescriptionlength

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to put a physical number on how efficiently intelligence can be bought with energy. It splits intelligence into two axes—recognition (structure learned about the world) and control (influence over future observations)—and measures each in bits per joule. Its central result is a closed-cycle bound: under steady-state conditions, the bits of new environment structure an agent stores can be at most Q_diss/(k_B T ln 2), so the dissipation-normalized learning efficiency cannot exceed 1/(k_B T ln 2) bits per joule—about 3.5×10^20 bits/J at room temperature. It also shows that this ceiling is only meaningful when the accounting boundary includes the cost of fresh low-entropy memory; without that, information gain and dissipated heat can be decoupled arbitrarily. If correct, the bound gives a physical benchmark any closed-cycle learner—biological or artificial—must respect, while current AI systems sit many orders of magnitude above the corresponding energy cost per bit.

What carries the argument

The load-bearing object is thermodynamic epiplexity, defined as the conditional mutual information ΔI = I(W_post; Z | W_pre)—new bits about an environment-instance variable Z stored in the agent state W after an episode. The proof mechanism is the chain: data-processing inequality (ΔI ≤ I(W_post; X | W_pre)) followed by a thermodynamic-learning inequality (I(W_post; X | W_pre) ≤ (ΔS_sys + Q_diss/T)/(k_B ln 2)) for bipartite Markov learning dynamics satisfying local detailed balance. Together they yield the Landauer-scale ceiling; boundary closure—counting the preparation of fresh low-entropy memory—is what makes the ceiling physically binding in repeated operation.

What would settle it

Calorimetrically run a small repeated-learning circuit—say, a one-bit belief state updated from a biased coin and reset each cycle—measuring heat dissipated Q_diss and the mutual information ΔI about the coin's bias. If the measured bits-per-joule (ΔI/Q_diss) exceeds 1/(k_B T ln 2) within experimental error, Corollary 1 is wrong. If a reversible circuit writing n bits with a supposedly free zeroed register fails to show growing bits-per-joule as n increases, Proposition 1's accounting decoupling is not physically realizable.

Watch

Extended reading notes

Core claim

The central claim is Corollary 1: for an isothermal, closed-cycle learning process with a reusable memory, the acquired epiplexity ΔI = I(W_post; Z | W_pre) is bounded above by (ΔS_sys + Q_diss/T)/(k_B ln 2). When the system returns to the same physical entropy each cycle (ΔS_sys = 0), this becomes ΔI ≤ Q_diss/(k_B T ln 2), and the dissipation-normalized efficiency η̃_E = ΔI/Q_diss ≤ 1/(k_B T ln 2) bits/J. The proof chains two inequalities: the data-processing inequality says structure information about a latent environment variable cannot exceed information gained about the data stream, and a thermodynamic-learning inequality says that data information cannot exceed the entropy produced by

Load-bearing premise

The whole bound rests on treating a learning run as a bipartite Markov process with local detailed balance, so that Q_diss is well defined, and on identifying measured energy E_cons with dissipated heat Q_diss; if either fails, the Landauer-scale ceiling does not constrain actual measured bits per joule.

Editorial extensions

If this is right

Under the paper's closed-cycle assumptions, any reusable-memory learner must dissipate at least k_B T ln 2 joules per bit of newly stored structure; at room temperature that is ~3.5×10^20 bits per joule as an upper bound.
Bits-per-joule numbers are only comparable when the accounting boundary, reset protocol, coarse-graining, and horizon are specified; absent those conventions, the paper's Proposition 1 shows the metric can be made arbitrarily large.
The empowerment-per-joule axis gives a matching control-side efficiency, and the two axes together imply a fixed dissipation budget must be divided between learning and control in closed-loop agents.
When the latent structure variable Z is unavailable, compute-bounded MDL epiplexity/compression-gain surrogates are the recommended operational companions for reporting.
Under power-law scaling of test loss with compute, the marginal compression gain per unit training energy decays as C^{-(α+1)}, making diminishing returns an explicit energy-efficiency statement rather than only an empirical trend.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A corollary the paper leaves implicit: if training efficiency is bounded by this Landauer scale, then continued scaling of large models will require either algorithmic gains that increase bits per joule or a hard energy floor per unit of learned structure; the paper's framework gives a way to measure which is happening.
The decoupling proposition suggests a testable prediction: systems that rely on large external retrieval stores (fresh memory from outside the training boundary) will appear to have anomalously high bits-per-joule unless the cost of building and maintaining the store is included; auditors of AI energy claims should check where the memory comes from.
The bound applies to dissipation, not measured wall-plug energy; extending it to real data centers requires measuring the non-dissipative terms in the paper's energy balance, so a practical next step is to build an instrumented closed-cycle benchmark that tracks both E_cons and Q_diss.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this is a metrics-and-conventions paper, not a new thermodynamic result. Corollary 1 is a two-line combination of the Goldt-Seifert learning inequality and the data processing inequality. That is fine, but the contribution is the packaging: two complementary bits-per-joule axes (recognition vs control), a Landauer-scale benchmark for closed cycles, and a sensible reporting checklist. The paper does a genuinely good job of being explicit about where the numbers come from and why open-boundary comparisons are meaningless. I would not referee it as a claim about new physical limits; I would referee it as a proposal for standard reporting practice.

What is good: the definitions are careful. They distinguish measured energy from thermodynamic dissipation, they define acquired epiplexity as conditional mutual information to handle forgetting, they recommend MDL surrogates when the latent Z is unavailable, and they flag the free-memory loophole in Proposition 1. The proof of the main corollary is correct given its premise. The checklist in Section 5 is practical and could help clean up bits-per-joule comparisons in the AI energy literature. The stress-test worry about Proposition 1 is mostly addressed by the authors' own boundary-closure language; the real soft spot is the unproven import of Lemma 1.

Where I part company with the authors: the central bound is only as good as Lemma 1, which is imported with a hand-wavy 'exact form depends on subsystem choice.' A referee should insist that the exact form be stated and proved, or at least quoted with page and equation, and that the Q_diss in Lemma 1 be connected to the accounting boundary used for Econs. The paper says Econs ≈ Qdiss is a convention, but then uses the Landauer scale as if it constrained measured bits-per-joule. That is a real gap. Section 4's claim about a Landauer-scale cost for control information is asserted, not derived. The paper also has no worked example with a fully specified Z and estimators, so it is hard to see how the framework is actually operated. These are fixable.

My bottom line: send it to peer review. It deserves referee time, mostly because the reporting checklist and the explicit boundary conventions could become a useful standard. I would ask for one concrete instantiation and for the control-side bound to be either proved or labeled conjectural.

Referee Report

3 major / 4 minor

Summary. The paper proposes two complementary bits-per-joule metrics for physical intelligence: thermodynamic epiplexity per joule (ηE), measuring acquired structural information about an environment variable per unit energy, and empowerment per joule (ηC), measuring sensorimotor control capacity per unit energy. The central theoretical claim is Corollary 1 (Eq. 14), which states that under a closed-cycle/steady-state regime with ΔS_sys = 0, acquired epiplexity satisfies ΔI ≤ Q_diss/(k_B T ln 2), giving a Landauer-scale bound on dissipation-normalized learning efficiency η̃_E. The bound is derived by combining a thermodynamic learning inequality imported from Goldt & Seifert [4] with the data processing inequality. The paper also includes Proposition 1, an open-boundary decoupling construction showing that without charging for externally prepared low-entropy memory, bits-per-joule ratios can be made arbitrarily large, motivating boundary closure. A reporting checklist and an operational MDL/compression surrogate framework are provided.

Significance. If the central bound is accepted, the paper offers a principled physical ceiling for energy-efficient learning, connecting thermodynamic dissipation to structural information acquisition. The work is careful about accounting conventions and explicitly distinguishes Econs from Qdiss, which is a strength. The derivation of Corollary 1 is straightforward once Lemma 1 is granted, and the paper correctly identifies the open-boundary loophole. The proposed two-axis framework and reporting checklist are useful contributions to reproducible bits-per-joule benchmarking. However, the central theoretical result rests entirely on an imported lemma whose precise conditions are not fully specified, and the step from Qdiss to measured energy is treated as a convention rather than a theorem.

major comments (3)

[§2, Lemma 1 and Corollary 1] The central bound Eq. (14) depends entirely on Lemma 1, which is quoted from Goldt & Seifert [4] with the caveat that 'the exact form depends on the subsystem choice.' The lemma is not proved in the manuscript, and the precise meaning of Q_diss (heat of the W-subsystem only, or total heat within the accounting boundary) is ambiguous. Since Corollary 1 is the paper's main theoretical contribution, the authors should either provide a self-contained proof or a precise statement of the conditions (subsystem decomposition, local detailed balance, definition of Q_diss) under which the inequality holds, and justify that these conditions apply to the closed-cycle benchmark.
[§1.1 and Remark 1] The paper correctly distinguishes Qdiss from Econs, but the abstract and conclusion state the Landauer-scale benchmark without always carrying this caveat. Corollary 1 bounds η̃_E = ΔI/Qdiss, not ηE = ΔI/Econs unless Econs ≈ Qdiss is explicitly justified. The abstract should state that the Landauer-scale limit applies to dissipation-normalized efficiency under closed-cycle accounting, not to measured bits-per-joule on conventional hardware, to avoid overclaiming.
[§2 and §4, boundary closure] The assertion that closing the accounting boundary restores Landauer-scaled bounds is supported only by a citation to Sagawa & Ueda [10] and a qualitative argument. Proposition 1 convincingly shows the open-boundary loophole, but the paper does not provide a concrete derivation showing that in a closed cycle with bounded reusable memory and total Q_diss including initialization/erasure, Lemma 1 yields Eq. (14). This is load-bearing for the central benchmark; a short model or explicit argument is needed.

minor comments (4)

[§2, Proposition 1] Proposition 1 defines ΔI as a difference of marginal mutual informations, whereas Eq. (3) defines acquired epiplexity as a conditional mutual information. For a deterministic initial register these coincide, but the paper should explicitly note the equivalence to avoid ambiguity.
[§5, checklist] In the reporting checklist, the terms 'total vs. incremental energy' for ηC are described, but it would be helpful to give a concrete formula for incremental ηC (e.g., (I(A;O) - I0)/(E - E0)) to prevent ambiguity.
[References] There are several typographical issues in the references: 'Reykjavic' should be 'Reykjavík', 'Y usuke' should be 'Yusuke', and the DOI for [5] appears incomplete. These should be corrected.
[General notation] The symbols ηE and η̃_E are introduced close together (Eq. 8 and the paragraph before Lemma 1) but used somewhat interchangeably later. Please make the distinction consistently clear, especially in Section 4.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Corollary 1 imports an external thermodynamic-learning inequality and applies DPI; the bound is not true by construction and no load-bearing self-citation appears in the derivation.

full rationale

The central derivation is self-contained as a derivation chain even though it imports its key physical inequality. Lemma 1 is explicitly presented as a restatement of Goldt & Seifert [4], an external published result, not as a result proven inside this paper. Corollary 1 then combines Lemma 1 with the conditional data processing inequality under the stated Markov relation Z -> X -> Wpost; this is a standard mathematical step, not a definitional tautology. Neither epiplexity Delta-I nor the efficiencies eta_E and tilde-eta_E are defined so that the bound holds by construction: the bound depends on substantive assumptions (bipartite Markov process, local detailed balance, well-defined Q_diss, closed-cycle Delta-S_sys = 0), all of which the paper states. The paper also openly flags the non-triviality of translating Q_diss into measured E_cons: Section 1.1 calls E_cons = Q_diss a 'reporting convention' rather than a theorem, and Proposition 1 explicitly demonstrates that without boundary closure Delta-I and Q_diss can decouple. Thus the Landauer-scale claim is conditional, not circular. The self-citations [5] and [12] appear in related-work or speculative contexts and are not used to prove Eq. (14), so they do not constitute load-bearing self-citation. There are no fitted parameters disguised as predictions and no known result merely renamed. The main risk is correctness/validity of the imported Lemma 1 under real training conditions, not circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 1 invented entities

No data fitting or ad hoc parameters; but the framework introduces a latent benchmark variable and relies on external thermodynamic-learning inequalities.

assumptions (5)

domain assumption The learning dynamics satisfies the Goldt-Seifert thermodynamic learning inequality: isothermal, bipartite Markov process with local detailed balance, so ΔI_W←X ≤ (ΔS_sys + Q_diss/T)/(k_B ln 2).
Invoked in Lemma 1 as an external result [4]; no proof in this paper. All subsequent benchmark statements depend on it.
standard math Data processing inequality holds for Z → X → W_post conditioned on W_pre.
Standard information-theoretic fact; assumes the learning update uses only episode data X.
domain assumption Closed-cycle/steady-state regime with ΔS_sys = 0 and reusable bounded memory.
Required for the Landauer-scale form of Corollary 1 (Eq. 14). For many learning processes the internal state entropy changes.
domain assumption Logically reversible gates can be implemented quasistatically with arbitrarily small dissipation.
Used in Proposition 1 to show decoupling; standard in reversible computing (Bennett [1]) but an idealization.
domain assumption For empowerment, an energetic cost model c(a0:τ−1) and stated resolution/noise model make I(A;Oτ)/E[c] well-posed.
Needed for Eq. (16); not derived, and free-control artifacts require explicit cost conventions.

invented entities (1)

Z, the environment-instance latent variable
purpose: Normative target in the definition of epiplexity I(W;Z); benchmark-specific hidden structure that generates agent experience.
Explicitly a theoretical variable, not observable; the choice of Z is part of benchmark specification. Metric values depend on this choice, so it is a convention rather than an independently measurable entity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Thermodynamic Limits of Physical Intelligence." pith.science (2026). https://pith.science/paper/ALC5AZ4E

@misc{pith2026260205463,
  author       = {Pith},
  title        = {Pith review of: Thermodynamic Limits of Physical Intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ALC5AZ4E}},
  note         = {Machine review of arXiv:2602.05463}
}

read the original abstract

Modern AI systems achieve remarkable capabilities at the cost of substantial energy consumption. To connect intelligence to physical efficiency, we propose two complementary bits-per-joule metrics under explicit accounting conventions: (1) Thermodynamic Epiplexity per Joule, new bits of structure about a specified environment-instance variable encoded in an agent's state per unit energy, and (2) Empowerment per Joule, sensorimotor channel capacity per expected energetic cost over a fixed horizon. These give two axes of physical intelligence, recognition versus control, but the resulting numbers are benchmark-relative rather than universal. Drawing on stochastic thermodynamics, we formulate a Landauer-scale closed-cycle benchmark for epiplexity acquisition by combining a thermodynamic-learning inequality with data processing, and clarify why boundary closure is required; conversely, a decoupling construction shows that without such assumptions information gain and in-boundary dissipation need not be tightly linked. For empirical settings where the latent structure variable is unavailable, we recommend compute-bounded MDL epiplexity / compression-gain surrogates. Finally, we propose a unified efficiency framework with a minimal checklist of conventions for relative bits-per-joule comparisons, and give a compact language-model reporting example.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Thermodynamics of Learning: A Typed Four-Component Accounting of Memory, Fit, and Value
cond-mat.stat-mech 2026-08 conditional novelty 7.0 of 10

A typed accounting separates record correlation from operational capital value in finite learning devices, with separation, capitalization-efficiency, and value-retention theorems.

Reference graph

Works this paper leans on

15 extracted references · 3 canonical work pages · cited by 1 Pith paper

[4]

Thermodynamic efﬁciency of learning a rule in neural net- works

Sebastian Goldt and Udo Seifert. Thermodynamic efﬁciency of learning a rule in neural net- works. Physical Review Letters, 118:010601, 2017

2017
[10]

Role of mutual information in entropy production under information exchanges

Takahiro Sagawa and Masahito Ueda. Role of mutual information in entropy production under information exchanges. New Journal of Physics , 15:125012, 2013

2013
[1]

Charles H. Bennett. The thermodynamics of computation—a review. International Journal of Theoretical Physics, 21(12), 1982

1982
[2]

Zico Kolter, and Andrew Gordon Wilson

Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, and Andrew Gordon Wilson. From entropy to epiplexity: Rethinking information for computationally bounded intelligence, 2026

2026
[3]

The free-energy principle: a uniﬁed brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010

Karl Friston. The free-energy principle: a uniﬁed brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010

2010
[5]

Universal ai maximizes variational empowerment

Y usuke Hayashi and Koichi Takahashi. Universal ai maximizes variational empowerment. In Matthew Iklé, Anton Kolonin, and Michael Bennett, editors, Artiﬁcial General Intelligence: 18th International Conference, AGI 2025, Reykjavic, Iceland, August 10–13, 2025, Proceed- ings, Part I , volume 16057 of Lecture Notes in Computer Science , pages 250–262. Sprin...

work page doi:10.1007/978-3-032-00686-8_23 2025
[6]

Klyubin, Daniel Polani, and Chrystopher L

Alexander S. Klyubin, Daniel Polani, and Chrystopher L. Nehaniv. Empowerment: A univer- sal agent-centric measure of control. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), 2005

2005
[7]

Irreversibility and heat generation in the computing process

Rolf Landauer. Irreversibility and heat generation in the computing process. IBM Journal of Research and Development, 5(3):183–191, 1961

1961

Show all 15 references

[8]

Universal intelligence: A deﬁnition of machine intelligence

Shane Legg and Marcus Hutter. Universal intelligence: A deﬁnition of machine intelligence. Minds and Machines, 17(4):391–444, 2007

2007
[9]

Watts-per-intelligence: Part i (energy efﬁciency)

Elija Perrier. Watts-per-intelligence: Part i (energy efﬁciency). In Matthew Iklé, Anton Kolonin, and Michael Bennett, editors, Artiﬁcial General Intelligence: 18th International Conference, AGI 2025, Reykjavic, Iceland, August 10–13, 2025, Proceedings, Part II , vol- ume 1605...

2025 doi
[11]

Sivak, Anthony J

Susanne Still, David A. Sivak, Anthony J. Bell, and Gavin E. Crooks. Thermodynamics of prediction. Physical Review Letters, 109:120604, 2012. 12

2012
[12]

Scenarios and branch points to future machine intelligence

Koichi Takahashi. Scenarios and branch points to future machine intelligence. arXiv:2302.14478, 2023. Presented at AAAI PSS 2025

2023 arXiv
[13]

On channel capacity per unit cost

Sergio V erdú. On channel capacity per unit cost. IEEE Transactions on Information Theory , 36(5):1019–1030, 1990

1990
[14]

Large language models as computable approximations to solomonoff induction, 2025

Jun Wan and Lingrui Mei. Large language models as computable approximations to solomonoff induction, 2025

2025
[15]

Transformers as approximations of solomonoff induc- tion

Nathan Y oung and Michael Witbrock. Transformers as approximations of solomonoff induc- tion. In Neural Information Processing, volume 15286 of Lecture Notes in Computer Science , pages 16–25. Springer, 2025. doi: 10.1007/978-981-96-6576-1_2. 13

2025 doi

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [4]

Thermodynamic efﬁciency of learning a rule in neural net- works

Sebastian Goldt and Udo Seifert. Thermodynamic efﬁciency of learning a rule in neural net- works. Physical Review Letters, 118:010601, 2017

2017

[2] [10]

Role of mutual information in entropy production under information exchanges

Takahiro Sagawa and Masahito Ueda. Role of mutual information in entropy production under information exchanges. New Journal of Physics , 15:125012, 2013

2013

[3] [1]

Charles H. Bennett. The thermodynamics of computation—a review. International Journal of Theoretical Physics, 21(12), 1982

1982

[4] [2]

Zico Kolter, and Andrew Gordon Wilson

Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, and Andrew Gordon Wilson. From entropy to epiplexity: Rethinking information for computationally bounded intelligence, 2026

2026

[5] [3]

The free-energy principle: a uniﬁed brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010

Karl Friston. The free-energy principle: a uniﬁed brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010

2010

[6] [5]

Universal ai maximizes variational empowerment

Y usuke Hayashi and Koichi Takahashi. Universal ai maximizes variational empowerment. In Matthew Iklé, Anton Kolonin, and Michael Bennett, editors, Artiﬁcial General Intelligence: 18th International Conference, AGI 2025, Reykjavic, Iceland, August 10–13, 2025, Proceed- ings, Part I , volume 16057 of Lecture Notes in Computer Science , pages 250–262. Sprin...

work page doi:10.1007/978-3-032-00686-8_23 2025

[7] [6]

Klyubin, Daniel Polani, and Chrystopher L

Alexander S. Klyubin, Daniel Polani, and Chrystopher L. Nehaniv. Empowerment: A univer- sal agent-centric measure of control. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), 2005

2005

[8] [7]

Irreversibility and heat generation in the computing process

Rolf Landauer. Irreversibility and heat generation in the computing process. IBM Journal of Research and Development, 5(3):183–191, 1961

1961