Acquiring Human-Like Data-Efficient Mechanics Prediction from Deep Reinforcement Learning

Jingruo Peng; Shuze Zhu

REVIEW 3 major objections 5 minor 39 references

Acquiring Human-Like Data-Efficient Mechanics Prediction from Deep Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A reinforcement-learning agent that alternates between two or three nearby task parameters during training can predict mechanical solutions over parameter ranges far wider than the training data, achieving coefficients of determination abov

desk verdict Plausible empirical trick, but the causal role of episodic switching is not established and the theory adds little; worth a referee only to demand proper controls. read the letter →

arxiv 2601.21881 v2 pith:YL6PL734 submitted 2026-01-29 physics.comp-ph

Jingruo Peng , Shuze Zhu This is my paper · ORCID

classification physics.comp-ph

keywords reinforcementlearningmechanicsintuitionepisodicswitchingBellmanoptimalitygeneralizationbrachistochronelarge-deformationplate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a reinforcement-learning agent can develop 'mechanics intuition' from only two or three similar examples, provided the physical parameters of the task are encoded into the agent's state and training alternates between those neighboring parameter values. On the brachistochrone curve and on a large-deformation elastic plate, a frozen policy trained this way predicts solutions over parameter ranges far wider than the union of what single-task training covers, with R² ≥ 0.90 across much of the test grid. The proposed explanation is that episodic switching enforces cross-parameter Bellman consistency, making the Bellman residual approximately stationary with respect to parameter changes and biasing the learned value function onto a low-dimensional solution manifold. If correct, this gives a principled, data-efficient route to physics-based reinforcement learning and a possible computational analogy for how people generalize from sparse mechanical observations.

What carries the argument

The central mechanism is episodic observation-switching: at episode boundaries the environment's physical observation parameter ζ is switched to a neighboring value, while ζ is always included as extra state channels (endpoint coordinates, load magnitude, plate size, orientation). The Bellman residual δQθ = Qθ − BζQθ measures how far the learned action-value function is from satisfying the Bellman optimality equation for parameter ζ. Because the replay buffer mixes transitions from nearby tasks, each gradient update reduces Bellman residuals at several ζ values simultaneously. The paper's theoretical core is the stationarity condition dδQθ(ζ)/dζ ≈ 0, whose discrete finite-difference form (δQ

What would settle it

Train a network on the same three training observations without switching, using the same architecture, total number of gradient steps, and amount of data (e.g., uniformly sampling transitions from all three tasks in every minibatch), then freeze and evaluate on the same test grid. If the generalization region collapses to roughly the union of the single-task regions, the switching protocol is necessary; if it stays as large, switching is not the mechanism. A second check would measure the finite difference of Bellman residuals on unseen ζ values and test whether it is close to zero, as the st

Watch

Extended reading notes

Core claim

The central discovery is that training on a small local cluster of neighboring task parameters, with the parameter vector supplied as extra state channels and with episodes switching between cluster members, turns a dueling deep Q-network into a parameter-conditioned policy whose frozen forward pass solves unseen instances far outside the cluster. The authors demonstrate this for the brachistochrone endpoint problem and for clamped square plates under varying load, size, and orientation, reporting mean coefficients of determination around 0.92–0.93 at distant test points. They argue that the switching protocol makes the value function's Bellman residual approximately parameter-independent, s

Load-bearing premise

The load-bearing premise is that the observed generalization gain is caused by the episodic switching protocol itself rather than by simply training the network on more observations, and the paper does not include a control that matches total training data without switching.

Editorial extensions

If this is right

With two or three neighboring observations and episodic switching, the frozen policy yields R² ≥ 0.90 over a substantially enlarged region of parameter space, exceeding the union of single-observation generalization regions.
The train–freeze–execute pipeline means solutions to new instances are produced by a single forward pass of the learned policy, with no iterative solver or fine-tuning at test time.
The same protocol works across qualitatively different mechanics problems: brachistochrone endpoint optimization and large-deformation clamped-plate displacement prediction under varying load, size, and orientation.
Because the protocol only needs a known scalar functional to define rewards, the authors argue it extends to any physics optimization problem with such a functional, including quantum ground-state or control problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The most direct next experiment would be a control that trains one network on the same two or three observations without the switching protocol, holding total data and gradient steps fixed; if that control generalizes equally far, the switching mechanism itself would not be the cause of the expansion.
The stationarity condition suggests an explicit regularizer on the finite difference of Bellman residuals across neighboring ζ values might reproduce or even strengthen the switching effect with fewer episodes; this is a testable implementation-level prediction.
If the mechanism is as general as claimed, the protocol should transfer to other parameterized functional-optimization families, such as heat conduction paths, drag minimization, or the quantum harmonic oscillator mentioned in the abstract; a cheap transfer test would train on two or three neighboring instances and evaluate extrapolation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the empirical story is plausible: a parameter-conditioned DQN trained on two or three nearby tasks can freeze and then generalize over wide parameter ranges on the brachistochrone and a large-deformation plate, with reported R² ≥ 0.90. Second, the paper does not actually show that episodic switching is doing the work. Every multi-task condition confounds switching with simply training on more tasks, and the theory is mostly tautological at the optimum.

What is actually new: the concrete demonstration of a train–freeze–execute RL pipeline for mechanics problems with very few observations. The reward design—using the physical functional directly—is sensible. If the reported R² values hold up, this would be a useful subfield contribution, especially for reducing data needs in physics-based RL. The generalization regions shown in the figures appear impressive, though the figures themselves are not reproduced in the text I saw.

The soft spots are real and load-bearing. The central attribution to episodic switching is unsupported: Single/Double/Triple conditions vary both the number of observations and the switching protocol. There is no control that trains on two or three observations with plain i.i.d. replay and no switching, nor is total training data matched. So the expanded R² ≥ 0.90 region could come entirely from more tasks or more data. No error bars, seeds, hyperparameters, or code are reported, so the empirical claims are hard to check. Also, the arXiv abstract promises a quantum harmonic oscillator demonstration that does not appear in the full text.

The theory does not rescue it. Eq. (10) is the derivative of the Bellman optimality condition, which holds identically for Q*. The paper admits this. The step to Eq. (12) for a learned Qθ is asserted, not proved, and would apply to any multi-task Bellman-residual minimization, not specifically to episodic switching. This is a heuristic, not a derivation.

Who it is for: researchers in physics-informed ML and RL for engineering design who want to see whether a simple curriculum can reduce data requirements. It is a conference-quality finding, not a settled result. I would send it to peer review, but with the expectation of major revision: the authors need a no-switching multi-task control matched for data, per-seed error bars, and either code or a serious downgrade of the theoretical claims.

Referee Report

3 major / 5 minor

Summary. The paper proposes a deep RL framework for acquiring 'mechanics intuition' from very few training observations. Physical parameters (e.g., endpoint coordinates, load, plate size, orientation) are encoded as extra state channels, and training alternates between a small cluster of nearby parameter values under an 'episodic observation-switching' protocol. After training, the policy is frozen and used in a single forward rollout to predict brachistochrone trajectories and large-deformation plate displacement fields. The authors report that training on two or three neighboring observations yields generalization regions (R² ≥ 0.90) that substantially exceed the union of single-task training regions. They explain this via Bellman-consistency reasoning: episodic switching enforces stationarity of the Bellman residual with respect to the physical parameter ζ, Eq. (12), which is claimed to bias the learned Q-function toward a smooth low-dimensional solution manifold.

Significance. If the central claim were established, the paper would provide a practically valuable demonstration that a single frozen DQN policy can generalize across a continuous family of physics-based optimal-control tasks after seeing only two or three task instances. The strengths are the physically grounded reward (signed improvement of travel time or potential energy), the parameter-conditioned state representation, and the train–freeze–execute evaluation protocol, which is a clean way to test generalization. However, the causal attribution to episodic switching is not currently identified: the experiments confound the switching protocol with simply training on more tasks and more data. Moreover, the theoretical explanation in Sec. 5.2 is tautological at the optimum and the non-trivial finite-difference claim is asserted rather than derived. These are load-bearing issues for the paper's main message, so the manuscript requires substantial revision before the claims can be accepted.

major comments (3)

[Sec. 2.6, Figs. 2(c–e), 3(l), Sec. 5.4] The central claim that 'episodic switching' is what enables the large generalization regions is not supported by the experimental design. Single, Double, and Triple conditions vary two factors simultaneously: the number of training observations and the use of episode-level switching. There is no Double or Triple condition trained on the same observations with ordinary i.i.d. multi-task replay (no switching), and no matching of total gradient steps or total transitions across conditions. Consequently, the expanded R²≥0.90 region could be entirely due to more training tasks or more data. The authors should add control conditions: (i) multi-task replay with the same observations but no episodic switching; (ii) single-task training with the same total number of gradient steps/transitions as the multi-task conditions; and (iii) ideally a matched-data condition where the same total transitions
[Sec. 5.2, Eqs. (9)–(12)] The theoretical derivation is tautological at the point where it matters. Eq. (10), dδQ(ζ)/dζ = 0, is obtained by differentiating the Bellman optimality equation for Q*, and the text itself states that 'for the optimal solution Q*, the Bellman residual vanishes identically for all ζ, and Eq. (10) holds trivially.' The step to a learned Qθ is then asserted: Eq. (11) is called a necessary condition, and Eq. (12) is claimed to be 'implicitly enforced' by alternating Bellman updates. But minimizing Bellman residuals at two nearby parameters does not, by itself, imply that the residual difference is approximately zero; any multi-task Bellman-residual minimization would exhibit a similar finite-difference expression, so this does not distinguish episodic switching from other multi-task training. The assumption that encoding ζ as a state channel makes ΔP ≡ 0 and permits same-action differentiat
[Sec. 3, Figs. 2–3] All quantitative generalization claims rely on single-run R² values without error bars, confidence intervals, or repeated-seed statistics. The paper reports 'substantially enlarged' generalization intervals and 'more-than-additive expansion,' but the figures do not provide exact numerical sizes of the generalized regions along the parameter axes, and no variance across seeds is given. Since the central empirical message is quantitative (R² ≥ 0.90 over wide parameter ranges), the authors should report mean and standard deviation over multiple random seeds, and include axis values and numerical interval lengths in Figs. 2 and 3. Without this, the reader cannot assess whether the reported expansion is robust or due to a single favorable run.

minor comments (5)

[Abstract] The lead abstract states that the method is 'demonstrated on the brachistochrone, a large-deformation elastic plate, and the quantum harmonic oscillator,' but the full text contains no quantum harmonic oscillator experiment; Sec. 5.5 only mentions it as a speculative future direction. Remove the quantum claim from the abstract or include the experiment.
[Sec. 2.6 / Appendix] The paper does not specify DQN hyperparameters, number of training episodes, replay buffer size, learning rate, target-network update frequency, exact training parameter values, or the discretization/action bounds. These details are needed for reproducibility and to assess whether the comparison conditions are truly matched. A table in an appendix would suffice.
[Sec. 4.2, Eq. (2)] The Lipschitz bound in Eq. (2) is stated as following from a 'standard contraction mapping argument,' but the assumptions are not fully spelled out (e.g., the domain of ζ, the regularity of r, and the fact that the state space itself depends on ζ). A short proof or a precise statement of the assumptions would improve rigor.
[Sec. 4.1, Eq. (1)] The paper defines the problem as a deterministic MDP but writes Eq. (1) with an expectation and transition probabilities. This is not a substantive issue, but it is confusing; either define P as a Dirac distribution or consistently present the stochastic formulation.
[Figure captions, Sec. 3] Figures 2(c–e) and 3(l) are described as 'quantitative summary' but appear qualitative. Add axis labels with numerical values, and in Fig. 3(l) specify what is plotted on each axis (e.g., parameter range vs. model variant).

Circularity Check

2 steps flagged · score 6.0 of 10

The 'unified theory' reduces to a tautology: Eq. (10) is the derivative of an identity for Q*, and Eq. (12) is asserted as an algorithmic consequence rather than derived.

self definitional [Sec. 5.2, Eqs. (5)-(11)]
"For the optimal solution Q*, the Bellman residual vanishes identically for all ζ, and Eq. (10) holds trivially. However, during learning we instead optimize a parameterized approximation Qθ ... In this case, Eq. (10) implies that a necessary condition for accurately approximating the optimal solution over a neighborhood of ζ is that the Bellman residual varies smoothly with respect to the physical observation parameter: dδQθ(ζ)/dζ ≈ 0. (11)"

Eq. (10), the 'derived' stationarity of the Bellman residual, is obtained by differentiating the Bellman optimality equation that Q* satisfies identically; the paper itself concedes it is trivial. The transfer to a learned Qθ in Eq. (11) is presented as an implication, but the smooth-stationarity property is exactly the explanatory conclusion the theory is supposed to establish. The mechanism is thus the desired premise restated in derivative form, not a consequence derived from episodic switching.
other [Sec. 5.2, Eq. (12); Sec. 5.3]
"For two similar parameters ζi and ζj, this procedure implicitly enforces the finite-difference constraint [δQθ(ζi) − δQθ(ζj)]/[ζi − ζj] ≈ 0, ... In episodic observation-switching training, the network parameters θ are alternately optimized to minimize the Bellman residual at multiple nearby environments ζi. This procedure not only enforces δQθ at each sampled environment, but also implicitly minimizes the finite-difference approximation of the residual derivative in Eq. (12)."

The finite-difference stationarity (12) is the load-bearing premise used in Sec. 5.4 to explain the non-additive generalization gain. The paper does not derive this constraint from the switching protocol; it asserts that alternating Bellman-residual minimization 'implicitly enforces' it. Minimizing residuals at ζi and ζj makes their difference small only insofar as both residuals are small, which is standard multi-task Bellman minimization and does not single out episodic switching as the mechanism. The desired conclusion is assumed as an algorithmic property.

full rationale

The empirical demonstrations on the brachistochrone and elastic plate are genuine measurements: generalization is evaluated by frozen-policy rollouts against analytic cycloids or Abaqus references, so the reported R² regions are not constructed from the training inputs. However, the paper's theoretical explanation of these results is substantially circular. Eq. (10) is the derivative of the Bellman optimality identity for Q*, and the paper explicitly admits it holds trivially; the step to Eq. (11) for a learned Qθ is an assertion of the smoothness/stationarity property that the theory is supposed to explain. Eq. (12), the finite-difference constraint that Sec. 5.4 relies on to predict the non-additive expansion of the generalization region, is likewise asserted to be 'implicitly enforced' by alternating updates; no derivation is given, and the property would hold for any multi-task residual minimization, not specifically for episodic switching. Separately, the comparison in Sec. 2.6 and Figs. 2(c-e), 3(l) confounds the switching protocol with the number of training observations and total training data, so the causal attribution to episodic switching is not isolated; this is a validity concern rather than a circularity by construction. Reference [11] is a self-citation by the same authors, but it is not load-bearing in the derivation, so no self-citation circularity is scored. Overall, the empirical core is independent, but the central theoretical 'unified view' reduces to a tautology plus an unproved assertion, warranting a partial-circularity score of 6.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The paper's empirical results rest on several hand-selected training points and thresholds; its theory uses a smoothness prior and an unproved identification between switching and residual stationarity. No fitted physics constants are introduced, but the methodology depends on many unreported choices. No new physical entities are postulated.

free parameters (5)

Training observation clusters (ζ1, ζ2, ζ3) per task = Not reported quantitatively (shown in figures)
The exact brachistochrone endpoints and plate load/size/orientation triples are hand-selected to be close; the size of the generalization region is sensitive to this choice and to the definition of 'local cluster' (Sec. 2.6).
Generalization threshold R² = 0.90
A test point is declared generalized if R² ≥ 0.90; changing this threshold changes all reported generalization intervals (Sec. 2.8).
DQN hyperparameters
Learning rate, batch size, replay buffer capacity, discount factor, layer sizes, and exploration schedule are not given; labeled 'same hyperparameter set' but never specified (Sec. 2.6).
Episodic switching schedule
Episode length, switching order, and termination rules are not specified; this is the central protocol and should be fully described (Sec. 2.5).
Discretization grid and action bounds
Collocation points/step sizes and action magnitude bounds are not reported; they affect training and rollout quality (Secs. 2.3, 2.7).

assumptions (6)

standard math Bounded rewards and γ∈(0,1) ensure the Bellman operators are contractions
Invoked in Sec. 4.1 to guarantee unique fixed points; standard.
domain assumption Reward r_ζ is Lipschitz continuous in ζ
Used to derive Eq. (2); the actual sign reward in the experiments is piecewise constant in ζ and is not Lipschitz, so the assumption is not satisfied by the implemented reward.
domain assumption Optimal Q* and optimal policy vary smoothly with ζ; solutions lie on a low-dimensional submanifold
Stated as the 'core assumption' in Sec. 2.6 and Sec. 5.1; not proven, and it is the property the method is designed to exploit.
domain assumption The maximizing action is unchanged under small perturbations of ζ (same-action differentiation)
Assumed in Sec. 5.2 to differentiate max_a' Q*; fails when there are ties or switches between discrete actions, which is common in DQN.
ad hoc to paper Encoding ζ as an input channel makes the transition law ζ-independent (ΔP≡0)
Used to drop the distributional term in Eq. (8); since the transition function operates on the full state that includes ζ, the transition law is not independent of the parameter—only the dependence is moved into the state, so the derivation is not formally justified.
ad hoc to paper Episodic switching of Bellman updates implies the finite-difference constraint Eq. (12)
The paper asserts in Sec. 5.3 that mixed minibatches force residual consistency; no proof is given, and minimizing residuals at two points does not generally bound the residual difference between them.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Acquiring Human-Like Data-Efficient Mechanics Prediction from Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/YL6PL734

@misc{pith2026260121881,
  author       = {Pith},
  title        = {Pith review of: Acquiring Human-Like Data-Efficient Mechanics Prediction from Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YL6PL734}},
  note         = {Machine review of arXiv:2601.21881}
}

read the original abstract

Humans can infer mechanical outcomes by learning from a few observations. This capacity for mechanics intuition is acquired in a data-efficient manner. Here, we propose a reinforcement learning framework to mimic this process, in which an agent encodes continuous physical observation parameters into its state and is trained via episodic switching across closely related observations. With merely two or three similar observations, the agent acquires robust mechanics intuition that generalizes over wide parameter ranges beyond the training data. Our method is demonstrated on the brachistochrone, a large-deformation elastic plate, and the quantum harmonic oscillator. We explain this generalization through a unified theoretical view: it is associated with cross-parameter Bellman consistency encouraged by episodic switching across neighboring task parameters, promoting approximate stationarity of the Bellman residual with respect to physical variations. This is consistent with a smooth policy that tracks a low-dimensional solution manifold underlying the continuum of tasks. Our work identifies episodic switching as a practically effective and theoretically motivated route to artificial mechanics intuition and suggests a computational analogy to data-efficient generalization in biological learners.

Figures

Figures reproduced from arXiv: 2601.21881 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 2 linked inside Pith

[1]

intuitive reasoning

Introduction Biological organisms acquire intelligent behavior through continual interaction with their environment . Reinforcement learning (RL) algorithms emulate such biological learning by optimizing actions through environmental interaction [1–3]. In humans, this ability to project the consequences of actions without direct trial -and-error is often ...
[2]

train–freeze–execute

Formulating Mechanics Problems as Reinforcement Learning Tasks This method provides a complete, self‑contained pipeline for converting mechanics problems into RL tasks and obtaining generalizable solutions through structured representation and episodic switching. The final solution is produced in a single forward pass of the learned policy, without iterat...
[3]

train –freeze–execute

Computational Results 3.1 Brachistochrone We validate our "train –freeze–execute" protocol on a canonical mechanics benchmark: the brachistochrone problem. In this task the agent predicts the time-optimal descent curve for a point particle subject to gravity that travels from the fixed start (0,0) to a parametrized end point (𝑋̃, 𝐻̃) and the analytic time...
[4]

This implies that small perturbations in a parameter 𝜁 should not lead to abrupt, discontinuous shifts in the optimal value function or policy

Perspective from Bellman Optimality on Intuitive Learning in DRL Mechanics intuition, in the context of reinforcement learning, can be formalized as the expectation that the optimal control strategy for a physical system varies smoothly and predictably with changes in continuous physical observation parameters. This implies that small perturbations in a p...
[5]

Unified Theory of Generalization for Mechanics Intuition via Reinforcement Learning Building upon the Bellman consistency framework established in Section 4, this section presents a unified theoretical explanation for the strong generalization observed in our experiments. We interpret the continuum of physical tasks as a smooth manifold and show that our ...
[6]

Conclusions By encoding physical parameters into the state and switching between related tasks during training, our method enables the agent to extract and reuse shared structures, thus acquiring broad mechanics intuition from few examples. Theoretically, this is achieved by enforcing stationarity of the Bellman residual across parameters, which biases th...
[7]

M. K. Eckstein, L. Wilbrecht, and A. G. Collins, What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience, Value Based Decis.-Mak. 41, 128 (2021)

2021
[8]

T. H. Muller, J. L. Butler, S. Veselic, B. Miranda, J. D. Wallis, P. Dayan, T. E. J. Behrens, Z. Kurth-Nelson, and S. W. Kennerley, Distributional reinforcement learning in prefrontal cortex, Nat. Neurosci. 27, 403 (2024)

2024

Show all 39 references

[10]

L. S. Piloto, A. Weinstein, P. Battaglia, and M. Botvinick, Intuitive physics learning in a deep -learning model inspired by developmental psychology, Nat. Hum. Behav. 6, 1257 (2022)

2022
[11]

Tiedemann, Y

H. Tiedemann, Y . Morgenstern, F. Schmidt, and R. W. Fleming, One-shot generalization in humans revealed through a drawing task, eLife 11, e75485 (2022)

2022
[12]

K. M. Collins et al., Building machines that learn and think with people, Nat. Hum. Behav. 8, 1851 (2024)

2024
[13]

J. B. Tenenbaum, C. Kemp, T. L. Griffiths, and N. D. Goodman, How to Grow a Mind: Statistics, Structure, and Abstraction, Science 331, 1279 (2011)

2011
[14]

Malaviya, I

M. Malaviya, I. Sucholutsky, K. Oktar, and T. L. Griffiths, Can Humans Do Less-Than-One-Shot Learning?, in (2022)

2022
[15]

Bhoopchand et al., Learning few-shot imitation as cultural transmission, Nat

A. Bhoopchand et al., Learning few-shot imitation as cultural transmission, Nat. Commun. 14, 7536 (2023)

2023
[16]

B. M. Lake and M. Baroni, Human-like systematic generalization through a meta-learning neural network, Nature 623, 115 (2023)

2023
[17]

Peng and S

J. Peng and S. Zhu, Developing artificial mechanics intuitions from extremely small data, Extreme Mech. Lett. 78, (2025)

2025
[18]

Zhao et al., A framework for the general design and computation of hybrid neural networks, Nat

R. Zhao et al., A framework for the general design and computation of hybrid neural networks, Nat. Commun. 13, 3427 (2022)

2022
[19]

Casper et al., Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback, arXiv:2307.15217

S. Casper et al., Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback, arXiv:2307.15217

arXiv
[21]

T. Wise, K. Emery, and A. Radulescu, Naturalistic reinforcement learning, Trends Cogn. Sci. 28, 144 (2024)

2024
[22]

Gupta, S

A. Gupta, S. Savarese, S. Ganguli, and L. Fei-Fei, Embodied intelligence via learning and evolution, Nat. Commun. 12, 5721 (2021)

2021
[23]

Y . Meng, Z. Bing, X. Yao, K. Chen, K. Huang, Y . Gao, F. Sun, and A. Knoll, Preserving and combining knowledge in robotic lifelong reinforcement learning, Nat. Mach. Intell. 7, 256 (2025)

2025
[24]

Ouyang et al., Training Language Models to Follow Instructions with Human Feedback, in Advances in Neural Information Processing Systems, edited by S

L. Ouyang et al., Training Language Models to Follow Instructions with Human Feedback, in Advances in Neural Information Processing Systems, edited by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, V ol. 35 (Curran Associates, Inc., 2022), pp. 27730–27744

2022
[25]

J. Oh, G. Farquhar, I. Kemaev, D. A. Calian, M. Hessel, L. Zintgraf, S. Singh, H. van Hasselt, and D. Silver, Discovering state-of-the-art reinforcement learning algorithms, Nature 648, 312 (2025)

2025
[26]

Guo et al., DeepSeek -R1 incentivizes reasoning in LLMs through reinforcement learni ng, Nature 645, 633 (2025)

D. Guo et al., DeepSeek -R1 incentivizes reasoning in LLMs through reinforcement learni ng, Nature 645, 633 (2025)

2025
[27]

H. Yu, I. Mineyev, L. R. Varshney, and J. A. Evans, Learning from one and only one shot, Npj Artif. Intell. 1, 13 (2025)

2025
[28]

Steyvers, H

M. Steyvers, H. Tejeda, A. Kumar, C. Belem, S. Karny, X. Hu, L. W. Mayer, and P. Smyth, What large language models know and what people think they know, Nat. Mach. Intell. 7, 221 (2025)

2025
[29]

Hattori, N

R. Hattori, N. G. Hedrick, A. Jain, S. Chen, H. You, M. Hattori, J.-H. Choi, B. K. Lim, R. Yasuda, and T. Komiyama, Meta-reinforcement learning via orbitofrontal cortex, Nat. Neurosci. 26, 2182 (2023)

2023
[30]

Pedamonti, S

D. Pedamonti, S. Mohinta, M. V . Dimitrov, H. Malagon-Vina, S. Ciocchi, and R. P . Costa, Hippocampus supports multi-task reinforcement learning under partial observability, Nat. Commun. 16, 9619 (2025)

2025
[31]

W. Song, S. Jiang, G. Camps -Valls, M. Williams, L. Zhang, M. Reichstein, H. Vereecken, L. He, X. Hu, and L. Shi, Towards data-driven discovery of governing equations in geosciences, Commun. Earth Environ. 5, 589 (2024)

2024
[32]

H. Xu, Y . Chen, R. Cao, T. Tang, M. Du, J. Li, A. H. Callaghan, and D. Zhang, Generative discovery of partial differential equations by learning from math handbooks, Nat. Commun. 16, 10255 (2025)

2025
[33]

Z. Chen, Y . Liu, and H. Sun, Physics-informed learning of governing equations from scarce data, Nat. Commun. 12, 6136 (2021)

2021
[34]

F. Wang, Z. Zhai, Z. Zhao, Y . Di, and X. Chen, Physics -informed neural network for lithium -ion battery degradation stable modeling and prognosis, Nat. Commun. 15, 4332 (2024)

2024
[35]

Z. Liu, Y . Liu, X. Yan, W. Liu, H. Nie, S. Guo, and C. Zhang, Automatic network structure discovery of physics informed neural networks via knowledge distillation, Nat. Commun. 16, 9558 (2025)

2025
[36]

K. Luo, J. Zhao, Y . Wang, J. Li, J. Wen, J. Liang, H. Soekmadji, and S. Liao, Physics-informed neural networks for PDE problems: a comprehensive review, Artif. Intell. Rev. 58, 323 (2025)

2025
[37]

G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Physics-informed machine learning, Nat. Rev. Phys. 3, 422 (2021)

2021
[38]

Cornelio, S

C. Cornelio, S. Dash, V . Austel, T. R. Josephson, J. Goncalves, K. L. Clarkson, N. Megiddo, B. El Khadir, and L. Horesh, Combining data and theory for derivable scientific discovery with AI-Descartes, Nat. Commun. 14, 1777 (2023)

2023
[39]

J. Hu, J. Cui, and B. Yang, Learning interpretable network dynamics via universal neural symbolic regression, Nat. Commun. 16, 6226 (2025)

2025
[40]

Liu and M

Z. Liu and M. Tegmark, Machine Learning Conservation Laws from Trajectories, Phys. Rev. Lett. 126, 180604 (2021)

2021
[41]

X. Fu, Z. Liu, and M. Tegmark, Do Two AI Scientists Agree?, arXiv:2504.02822

arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

intuitive reasoning

Introduction Biological organisms acquire intelligent behavior through continual interaction with their environment . Reinforcement learning (RL) algorithms emulate such biological learning by optimizing actions through environmental interaction [1–3]. In humans, this ability to project the consequences of actions without direct trial -and-error is often ...

[2] [2]

train–freeze–execute

Formulating Mechanics Problems as Reinforcement Learning Tasks This method provides a complete, self‑contained pipeline for converting mechanics problems into RL tasks and obtaining generalizable solutions through structured representation and episodic switching. The final solution is produced in a single forward pass of the learned policy, without iterat...

[3] [3]

train –freeze–execute

Computational Results 3.1 Brachistochrone We validate our "train –freeze–execute" protocol on a canonical mechanics benchmark: the brachistochrone problem. In this task the agent predicts the time-optimal descent curve for a point particle subject to gravity that travels from the fixed start (0,0) to a parametrized end point (𝑋̃, 𝐻̃) and the analytic time...

[4] [4]

This implies that small perturbations in a parameter 𝜁 should not lead to abrupt, discontinuous shifts in the optimal value function or policy

Perspective from Bellman Optimality on Intuitive Learning in DRL Mechanics intuition, in the context of reinforcement learning, can be formalized as the expectation that the optimal control strategy for a physical system varies smoothly and predictably with changes in continuous physical observation parameters. This implies that small perturbations in a p...

[5] [5]

Unified Theory of Generalization for Mechanics Intuition via Reinforcement Learning Building upon the Bellman consistency framework established in Section 4, this section presents a unified theoretical explanation for the strong generalization observed in our experiments. We interpret the continuum of physical tasks as a smooth manifold and show that our ...

[6] [6]

Conclusions By encoding physical parameters into the state and switching between related tasks during training, our method enables the agent to extract and reuse shared structures, thus acquiring broad mechanics intuition from few examples. Theoretically, this is achieved by enforcing stationarity of the Bellman residual across parameters, which biases th...

[7] [7]

M. K. Eckstein, L. Wilbrecht, and A. G. Collins, What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience, Value Based Decis.-Mak. 41, 128 (2021)

2021

[8] [8]

T. H. Muller, J. L. Butler, S. Veselic, B. Miranda, J. D. Wallis, P. Dayan, T. E. J. Behrens, Z. Kurth-Nelson, and S. W. Kennerley, Distributional reinforcement learning in prefrontal cortex, Nat. Neurosci. 27, 403 (2024)

2024