First-Passage Approach to Optimizing Perturbations for Improved Training of Machine Learning Models

Barak Hirshberg; Sagi Meir; Shlomi Reuveni; Tommer D. Keidar

REVIEW 3 major objections 4 minor 66 references

First-Passage Approach to Optimizing Perturbations for Improved Training of Machine Learning Models

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that if unperturbed neural-network training reaches a quasi-steady state, the residual time to target accuracy after a perturbation becomes independent of when it is applied, so a single measurement predicts mean…

desk verdict A solid, honestly-reported prediction protocol for perturbation training; the main caveats (marginal QSS test, P=3 extrapolation) are real but acknowledged and do not sink the central idea. read the letter →

arxiv 2502.04121 v3 pith:CQUL3DZQ submitted 2025-02-06 cs.LG cond-mat.dis-nnphysics.chem-ph

Sagi Meir , Tommer D. Keidar , Shlomi Reuveni , Barak Hirshberg This is my paper · ORCID

classification cs.LGcond-mat.dis-nnphysics.chem-ph

keywords first-passageprocessesquasi-steadystateneuralnetworktrainingstochasticresettingshrinkandperturbperturbationoptimizationCIFAR-10ResNet-18

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper treats training a neural network up to a target test accuracy as a first-passage process, and asks how periodic perturbations—shrink-and-perturb, stochastic resetting, partial re-initialization—change the mean time to reach that target. Its central claim is that if the unperturbed training dynamics reach a quasi-steady state, then the average residual time after a perturbation becomes independent of when the perturbation is applied. That makes one measurement at a single perturbation interval $P^*$ sufficient to predict the mean first-passage time for every interval in a wide range. On CIFAR-10 with ResNet-18 the paper uses this to predict that shrink-and-perturb gives about a 16-fold speedup at $P=3$, close to the roughly 21-fold speedup found by brute force, and to select shrink-and-perturb over partial resetting and full stochastic resetting. The same recipe is shown to transfer to CIFAR-100, MNIST, momentum SGD, Adam, and a regression task.

What carries the argument

The load-bearing object is the quasi-steady-state (QSS) factorization of the training propagator, $G(\theta,t)\simeq\phi(\theta)\Psi_T(t)$, which says that after a relaxation time the relative population of network states freezes while the overall fraction of surviving models decays. Its role is to make the network state sampled at the perturbation time $P$ independent of $P$, converting the residual time $\bar\tau_P$ into a single number $\bar\tau$ that can be measured in one run. Eq. (9) is the resulting identity that turns that single measurement into a prediction curve for every perturbation frequency in the valid range.

What would settle it

On a small network where all weights can be tracked, verify the full-state quasi-steady-state directly; then apply a perturbation that changes weights but leaves test accuracy nearly unchanged. If the residual time $\bar\tau_P$ varies with $P$ while the accuracy-marginal Kolmogorov–Smirnov test still says quasi-steady-state holds, the central claim is false.

Watch

Extended reading notes

Core claim

The central discovery is that under the quasi-steady-state factorization $G(\theta,t)\simeq\phi(\theta)\Psi_T(t)$ for $t>t_r$, the conditional mean residual time after a perturbation becomes $\bar\tau_P\simeq\bar\tau$, independent of the perturbation interval $P$, as long as $P$ is larger than both the relaxation time and the residual time itself. Substituting this into the first-passage response identity gives Eq. (9), $\mathbb{E}[T_P]=\sum_{t=0}^{P-1}\Psi_T(t)+\Psi_T(P)\bar\tau$, which is a parameter-free prediction: sample training once with the perturbation applied at a single large $P^*$, read off the survival function and the residual time, and predict the mean first-passage time for all $P$ in $[\max(t_r,\bar\tau),P^*]$. The authors verify the quasi-steady-state proxy on test-accuracy marginals with Kolmogorov–Smirnov tests, show experimentally that $\bar\tau_P$ is flat over nearly two decades of $P$, and use the predicted speedup curves to rank perturbation protocols and identify an efficient interval.

Load-bearing premise

The whole prediction relies on the full network state, not just test accuracy, reaching a quasi-steady state; the paper only checks the accuracy marginal, so if weight configurations keep evolving while accuracy looks stationary, P-independence can fail.

Editorial extensions

If this is right

If correct, protocol selection reduces to comparing residual times $\bar\tau$ measured at one large $P^*$: the perturbation with the smallest residual time is the preferred one across the whole valid frequency range.
For stochastic resetting, Eq. (5) gives the speedup directly from the unperturbed survival function, so any candidate perturbation can be screened against a resetting baseline without extra training runs.
The predicted speedup curve lets a practitioner choose not only which perturbation to use but also the perturbation interval that maximizes acceleration, without brute-force sweeps.
The experimental observation that $\bar\tau_P$ stays flat even for $P$ below the relaxation time suggests the practical range of validity may be wider than the formal bound $P>\max(t_r,\bar\tau_P)$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same one-shot response measurement could generalize to random or non-periodic perturbation schedules, such as exponentially distributed waiting times, by using the response-theory form in its full generality rather than fixed $P$.
If the quasi-steady-state assumption holds for other collective variables, the recipe could be used to select among data augmentation policies, learning-rate schedules, or architecture modifications, each treated as a perturbation with a measurable residual time.
For large-scale training where an ensemble is already run in parallel with early stopping, the survival function is available for free, so the only added cost of the method is one perturbed run at $P^*$.
The gap between the predicted speedup (about 16) and the brute-force speedup (about 21) indicates a weak residual dependence on $P$ even in the valid range; quantifying that dependence could sharpen the prediction or bound its error.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is a legitimate, well-executed application of first-passage response theory to neural network training. The authors show that if unperturbed training reaches a quasi-steady state, the mean residual time after a perturbation is roughly independent of when you apply it, so one measurement at a single P* plus the unperturbed survival curve predicts speedups across a wide range of perturbation periods. They verify this on CIFAR-10 with ResNet-18 and show transferability to CIFAR-100, MNIST, different optimizers, and a regression task. The code and data are public.

What's actually new is the QSS reduction plus the single-P* prediction protocol. Eq. (3) itself is a law-of-total-expectation identity and is correctly derived in the SI; the predictive step is not circular because bar_tau is measured, not fitted. The empirical constancy of tau_P over P=10-100 in Fig. 5 is direct evidence for the practical claim, and the 1000-run ensemble gives the survival curves solid footing.

The soft spots are in proportion. Most importantly, the full-state QSS factorization (Eq. 6) is only tested through the marginal distribution of test accuracy, which is a proxy. The perturbation acts on weights, so in principle residual times could depend on the weight configuration that the marginal doesn't capture. The constancy of tau_P in Fig. 5 is reassuring, but it's measured at four P values for two perturbations. Second, the headline S&P speedup at P=3 is outside the formal validity condition P > max(t_r, tau_bar), since t_r ~ 28. The paper honestly reports the mismatch (predicted 16 vs brute-force 21) and the prediction lines are only plotted down to P = tau_bar, so the P=3 claim is an extrapolation. That said, the authors flag this themselves, and the discrepancy is a quantitative gap, not a qualitative failure.

This paper is for anyone working on perturbation training protocols—shrink-and-perturb, warm restarts, stochastic resetting—and for people who like statistical-physics-style reductions of training dynamics. It deserves a serious referee; the main revisions are to tighten the discussion of when the prediction is trustworthy at small P and to be clearer that the QSS is empirically supported but not proven at full state dimension. I'd engage with it.

Referee Report

3 major / 4 minor

Summary. The paper proposes a first-passage framework for rationally choosing periodic perturbation protocols (shrink-and-perturb, partial stochastic resetting, and full stochastic resetting) in neural-network training. It defines a survival probability Psi_T(t) for unperturbed training to a target test accuracy, derives an exact expression for the mean first-passage time under perturbations, Eq. (3), and then argues that if the full network-state propagator reaches a quasi-steady state (QSS), Eq. (6), and perturbations are rare, the mean residual time bar_tau_P becomes P-independent, yielding the predictive formula Eq. (9). The authors test the QSS assumption on CIFAR-10 ResNet-18 via marginal test-accuracy distributions and Kolmogorov-Smirnov tests, measure bar_tau_P for S&P and partial SR over a range of P, and use Eq. (9) to predict speedups, reporting that S&P at P=3 gives a predicted speedup of about 16 versus a brute-force value of about 21. Transferability experiments across CIFAR-100, momentum, and a regression task appear in the supporting information.

Significance. If the central claim holds, the method is valuable: it promises a cheap, single-ensemble protocol for ranking perturbation schemes and choosing perturbation periods, based on a physically motivated QSS assumption. The paper has notable strengths: 1000 training trajectories for the main experiment, a publicly available code/data repository, a clean derivation of Eq. (3), and an explicit parameter-free speedup prediction for full stochastic resetting via Eq. (5). The transferability experiments, though less detailed, broaden the scope. However, the load-bearing QSS assumption is verified only through a marginal collective variable, and the headline P=3 prediction lies outside the formal validity range of Eq. (9). These issues are fixable but require additional evidence or a reframing of the claims.

major comments (3)

[Section III A, Eq. (6)] The derivation of P-independent residual times rests entirely on the full-state quasi-steady-state factorization G(theta,t) ≈ phi(theta) Psi_T(t). The verification in Section III B (Eqs. (10)-(11), Fig. 3D) concerns only the marginal distribution of test accuracy A, which the authors explicitly call a proxy. Since S&P and partial SR act directly on theta (shrinking or reinitializing weights), the residual time E[tau(theta)] in Eq. (8) averages over weight configurations, and a marginal QSS in A does not imply the required factorization of the full state distribution. The constancy of bar_tau_P in Fig. 5 is supportive evidence, but it is established only for the specific perturbations and P values tested. Please either provide a more direct test of the full-state QSS (e.g., stationarity of layer-wise weight norms or gradient norms conditioned on survival) or clearly label Eq. (9) as an assumption-based prediction whose range of validity is itself part of the empirical claim.
[Section III D, Fig. 6] The headline S&P prediction at P=3 violates the stated validity condition P > max(t_r, bar_tau_P) given after Eq. (8) and repeated in the methodology paragraph: for the 72% target, t_r=28 and bar_tau≈3, so P=3 is smaller than t_r. The predicted speedup curves in Fig. 6 are plotted down to P=bar_tau_P*, not down to max(t_r, bar_tau_P*). The reported gap between the predicted speedup (~16) and the brute-force value (~21) is consistent with this extrapolation. The authors should either restrict their predictions and recommendations to P in [max(t_r, bar_tau_P*), P*], or provide an explicit empirical justification for extending the range to P<t_r.
[Section III B and SI B, Eqs. (S10)-(S11)] The Kolmogorov-Smirnov test used to establish the QSS compares each epoch's CDF F_t(A) to the average CDF over epochs 20-100, and for t in that window the average includes F_t itself. This reference distribution is not independent of the tested sample, which biases the test toward accepting the QSS hypothesis and may lead to underestimated relaxation times t_r. A cleaner procedure would compare each F_t to an average over a disjoint hold-out set of epochs, or use pairwise tests between non-overlapping time windows.

minor comments (4)

[Introduction] The phrase "designed ad hocby intuition" is missing a space and should read "ad hoc by intuition."
[Fig. 3D] The caption and axis label would benefit from stating explicitly that the y-axis is on a logarithmic scale and how p-values equal to machine precision are handled.
[Section II, Eq. (3)] The derivation of Eq. (3) in the SI is clear, but the main text could note more explicitly that the exact identity holds for arbitrary perturbation protocols and all P, with the approximation entering only later through the QSS assumption.
[Section V] The repository link would be more useful with a version or commit identifier and a license statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central prediction uses an independently measured residual time and is benchmarked against separate brute-force runs.

full rationale

The paper's derivation chain is self-contained at every load-bearing step, and the apparent self-citations are not circular. Equation (2), attributed to the authors' prior work [38], is a law-of-total-expectation decomposition of the perturbed first-passage time; the paper rederives the resulting identity, Eq. (3), in its own Supporting Information (Appendix A), so the reliance on [38] is not load-bearing. Equation (9) is an approximation that replaces the P-dependent residual time bar-tau_P in Eq. (3) with a P-independent constant bar-tau, justified by the quasi-steady-state factorization Eq. (6). The constant bar-tau is measured directly at P* = 100 and is not fitted to the target speedup curve; the predicted E[T_P] values at other P are then compared with separate brute-force training runs (Fig. 6). Thus the prediction is not equivalent to its inputs by construction. The quasi-steady-state factorization Eq. (6) is indeed an unverified full-state assumption: the paper tests it only through the marginal distribution of test accuracy, Eqs. (10)-(11), explicitly calling this a proxy, and the headline S&P prediction at P = 3 lies outside the stated validity condition P > max(t_r, bar-tau) with t_r = 28. Those are correctness and extrapolation concerns, not circularity. The reported mismatch (predicted speedup about 16 vs brute-force about 21) further indicates that the prediction is an empirical extrapolation rather than a fitted restatement. No circular step could be exhibited. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical entities, particles, or mediators are introduced. The ledger is dominated by modeling assumptions about quasi-steady state and time homogeneity, plus perturbation hyperparameters inherited from prior work.

free parameters (3)

Shrink and perturb coefficients lambda and gamma = lambda=0.4, gamma=0.1
Taken from ref. [25], not optimized here. They set the magnitude of the perturbation and therefore shape the measured residual time bar_tau and the speedup.
Partial SR re-initialization fraction = 30% of smallest absolute value weights
Chosen by hand; the fraction and selection rule affect residual times and the predicted optimal P.
Measured residual time bar_tau at the single perturbation epoch = roughly 3 to 5 epochs for S&P and partial SR at the 72% target
This is an empirical input to Eq. (9), not fitted to the speedup curve. It is listed because the entire prediction depends on this one measured number and on its assumed P-independence.

assumptions (5)

domain assumption Training is a time-homogeneous stochastic process with a well-defined first-passage time to a target test accuracy.
Section II defines the propagator G(theta,t) and the absorbing target; this frames the whole analysis.
domain assumption Full-state quasi-steady state: G(theta,t) approximately equal to phi(theta) times Psi_T(t) for t larger than t_r.
Eq. (6) is the formal premise for Eq. (7). The paper tests only the marginal version.
ad hoc to paper The marginal QSS of test accuracy A is a sufficient proxy for full-state QSS.
Section III.B introduces A as a collective variable and states it will be used as a proxy; no proof is given that A captures the state dependence of residual times.
domain assumption Perturbations are rare: P larger than max(t_r, bar_tau_P), so a trajectory is perturbed at most once on average before completion.
Eq. (8) uses P larger than bar_tau_P to replace tau_P(theta) by tau(theta).
domain assumption Post-perturbation dynamics are independent of the absolute epoch P, i.e., time homogeneous.
Computational details use constant learning rate 0.02 and fixed hyperparameters; with a learning-rate schedule the residual time could depend on P through the schedule.

how reviews work

0 comments

Cite this review

Pith. "Pith review of First-Passage Approach to Optimizing Perturbations for Improved Training of Machine Learning Models." pith.science (2026). https://pith.science/paper/CQUL3DZQ

@misc{pith2026250204121,
  author       = {Pith},
  title        = {Pith review of: First-Passage Approach to Optimizing Perturbations for Improved Training of Machine Learning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CQUL3DZQ}},
  note         = {Machine review of arXiv:2502.04121}
}

read the original abstract

Machine learning models have become indispensable tools in applications across the physical sciences. Their training is often time-consuming, vastly exceeding the inference timescales. Several protocols have been developed to perturb the learning process and improve the training, such as shrink and perturb, warm restarts, and stochastic resetting. For classifiers, these perturbations have been shown to result in enhanced speedups or improved generalization. However, the design of such perturbations is usually done ad hoc by intuition and trial and error. To rationally optimize training protocols, we frame them as first-passage processes and consider their response to perturbations. We show that if the unperturbed learning process reaches a quasi-steady state, the response at a single perturbation frequency can predict the behavior at a wide range of frequencies. We employ this approach to a CIFAR-10 classifier using the ResNet-18 model and identify a useful perturbation and frequency among several possibilities. We demonstrate the transferability of the approach to other datasets, architectures, optimizers and even tasks (regression instead of classification). Our work allows optimization of perturbations for improving the training of machine learning models using a first-passage approach.

Figures

Figures reproduced from arXiv: 2502.04121 by the authors.

**Figure 2.** An illustration of a QSS of a propagator [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Experimental evidence for a quasi-steady-state. Panels A and B are violin plots of the density distributions of [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The mean trajectory of the test accuracy and its top and bottom deciles (shaded areas), for models that did not [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: The mean residual time τ¯P versus the perturbation time. The values of τ¯P for S&P (orange circles) and partial SR (green triangles) are approximately constant when P > τ¯P (outside the yellow area). Dashed dotted lines are averages of all τ¯P < P [PITH_FULL_IMAGE:fig…

**Figure 6.** Figure 6: The speedup gained by using different perturbation protocols versus the perturbation time. Orange and green [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 56 canonical work pages

[1]

Jumper, R

J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko,et al., Highly accurate protein structure prediction with alphafold, Nature596, 583 (2021)

work page 2021
[2]

A. Nir, E. Sela, R. Beck, and Y. Bar-Sinai, Machine-learning iterative calculation of entropy for physical systems, Proceedings of the National Academy of Sciences117, 30234 (2020)

work page 2020
[3]

Behler and M

J. Behler and M. Parrinello, Generalized neural-network representation of high-dimensional potential-energy surfaces, Physical review letters98, 146401 (2007)

work page 2007
[4]

Carleo and M

G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science355, 602 (2017)

2017
[5]

F. Noé, S. Olsson, J. Köhler, and H. Wu, Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning, Science365, eaaw1147 (2019)

2019
[6]

B. Máté, F. Fleuret, and T. Bereau, Neural thermodynamic integration: Free energies from energy-based diffusion models, The Journal of Physical Chemistry Letters15, 11395 (2024)

work page 2024
[7]

Anelli, H

A. Anelli, H. Dietrich, P. Ectors, F. Stowasser, T. Bereau, M. Neumann, and J. van den Ende, Robust and efficient reranking in crystal structure prediction: a data driven method for real-life molecules, CrystEngComm26, 5845 (2024)

work page 2024
[8]

Lagemann, K

C. Lagemann, K. Lagemann, S. Mukherjee, and W. Schröder, Deep recurrent optical flow learning for particle image velocimetry data, Nature Machine Intelligence3, 641 (2021)

work page 2021

Show all 66 references

[9]

Ravuri, K

S. Ravuri, K. Lenc, M. Willson, D. Kangin, R. Lam, P. Mirowski, M. Fitzsimons, M. Athanassiadou, S. Kashem, S. Madge,et al., Skilful precipitation nowcasting using deep generative models of radar, Nature597, 672 (2021)

2021
[10]

Y.Wang, J.M.L.Ribeiro,andP.Tiwary,Machinelearningapproachesforanalyzingandenhancingmoleculardynamics simulations, Current opinion in structural biology61, 139 (2020)

2020
[11]

M. Rupp, A. Tkatchenko, K.-R. Müller, and O. A. Von Lilienfeld, Fast and accurate modeling of molecular atomization energies with machine learning, Physical review letters108, 058301 (2012)

2012
[12]

Tsai, E.-J

S.-T. Tsai, E.-J. Kuo, and P. Tiwary, Learning molecular dynamics with simple language model built upon long short- term memory neural network, Nature communications11, 5115 (2020)

2020
[13]

Geiger and C

P. Geiger and C. Dellago, Neural networks for local structure detection in polymorphic systems, The Journal of chemical physics139(2013)

2013
[14]

Fukumizu and S.-i

K. Fukumizu and S.-i. Amari, Local minima and plateaus in hierarchical structures of multilayer perceptrons, Neural networks13, 317 (2000)

2000
[15]

Loshchilov and F

I. Loshchilov and F. Hutter, Sgdr: Stochastic gradient descent with warm restarts, arXiv preprint arXiv:1608.03983 (2016)

2016 arXiv
[16]

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representations by back-propagating errors, nature323, 533 (1986)

1986
[17]

X. Peng, L. Li, and F.-Y. Wang, Accelerating minibatch stochastic gradient descent using typicality sampling, IEEE transactions on neural networks and learning systems31, 4649 (2019)

2019
[18]

Duchi, E

J. Duchi, E. Hazan, and Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization., Journal of machine learning research12(2011)

2011
[19]

M. D. Zeiler, Adadelta: an adaptive learning rate method, arXiv preprint arXiv:1212.5701 (2012)

2012 arXiv
[20]

D. P. Kingma, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)

2014 arXiv
[21]

Nguyen, S

G. Nguyen, S. Dlugolinsky, M. Bobák, V. Tran, Á. López García, I. Heredia, P. Malík, and L. Hluch` y, Machine learning and deep learning frameworks and libraries for large-scale data mining: a survey, Artificial Intelligence Review52, 77 (2019)

2019
[22]

Sutskever, J

I. Sutskever, J. Martens, G. Dahl, and G. Hinton, On the importance of initialization and momentum in deep learning, inInternational conference on machine learning(PMLR, 2013) pp. 1139–1147

2013
[23]

Leimkuhler, C

B. Leimkuhler, C. Matthews, and T. Vlaar, Partitioned integrators for thermodynamic parameterization of neural networks, Foundations of Data Science1, 457 (2019)

2019
[24]

Ash and R

J. Ash and R. P. Adams, On warm-starting neural network training, Advances in neural information processing systems 33, 3884 (2020)

2020
[25]

Zaidi, T

S. Zaidi, T. Berariu, H. Kim, J. Bornschein, C. Clopath, Y. W. Teh, and R. Pascanu, When does re-initialization work?, inProceedings on(PMLR, 2023) pp. 12–26

2023
[26]

Y. Bae, Y. Song, and H. Jeong, Stochastic resetting mitigates latent gradient bias of sgd from label noise, Machine Learning: Science and Technology6, 015062 (2025)

2025
[27]

Dohare, J

S. Dohare, J. F. Hernandez-Garcia, Q. Lan, P. Rahman, A. R. Mahmood, and R. S. Sutton, Loss of plasticity in deep continual learning, Nature632, 768 (2024)

2024
[28]

Feng and Y

Y. Feng and Y. Tu, The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima, Proceedings of the National Academy of Sciences118, e2015617118 (2021). 11

2021
[29]

Cheng, D

X. Cheng, D. Yin, P. Bartlett, and M. Jordan, Stochastic gradient and langevin processes, inInternational Conference on Machine Learning(PMLR, 2020) pp. 1810–1819

2020
[30]

Jules, G

T. Jules, G. Brener, T. Kachman, N. Levi, and Y. Bar-Sinai, Charting the topography of the neural network landscape with thermal-like noise, arXiv preprint arXiv:2304.01335 (2023)

2023 arXiv
[31]

A. A. Alemi and I. Fischer, Therml: Thermodynamics of machine learning, arXiv preprint arXiv:1807.04162 (2018)

2018 arXiv
[32]

H. S. Seung, H. Sompolinsky, and N. Tishby, Statistical mechanics of learning from examples, Physical review A45, 6056 (1992)

1992
[33]

Choromanska, M

A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun, The loss surfaces of multilayer networks, in Artificial intelligence and statistics(PMLR, 2015) pp. 192–204

2015
[34]

Zdeborová, Understanding deep learning is also a job for physicists, Nature Physics16, 602 (2020)

L. Zdeborová, Understanding deep learning is also a job for physicists, Nature Physics16, 602 (2020)

2020
[35]

G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Physics-informed machine learning, Nature Reviews Physics3, 422 (2021)

2021
[36]

Carleo, I

G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborová, Machine learning and the physical sciences, Reviews of Modern Physics91, 045002 (2019)

2019
[37]

Stephan, M

M. Stephan, M. D. Hoffman, and D. M. Blei, Stochastic gradient descent as approximate bayesian inference, Journal of Machine Learning Research18, 1 (2017)

2017
[38]

T. D. Keidar and S. Reuveni, Universal linear response of the mean first-passage time, arXiv preprint arXiv:2410.16129 (2024)

2024 arXiv
[39]

Redner,A Guide to First-Passage Processes(Cambridge University Press, 2001)

S. Redner,A Guide to First-Passage Processes(Cambridge University Press, 2001)

2001
[40]

Metzler, S

R. Metzler, S. Redner, and G. Oshanin,First-passage phenomena and their applications, Vol. 35 (World Scientific, 2014)

2014
[41]

A. J. Bray, Satya N. Majumdar, and G. Schehr, Persistence and first-passage properties in nonequilibrium systems, Advances in Physics62, 225 (2013)

2013
[42]

Krizhevsky, G

A. Krizhevsky, G. Hinton,et al.,Learning multiple layers of features from tiny images, Master’s thesis, Toronto, ON, Canada (2009)

2009
[43]

Deng, The mnist database of handwritten digit images for machine learning research, IEEE Signal Processing Magazine29, 141 (2012)

L. Deng, The mnist database of handwritten digit images for machine learning research, IEEE Signal Processing Magazine29, 141 (2012)

2012
[44]

K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, inProceedings of the IEEE conference on computer vision and pattern recognition(2016) pp. 770–778

2016
[45]

Reuveni, Optimal stochastic restart renders fluctuations in first passage times universal, Physical review letters116, 170601 (2016)

S. Reuveni, Optimal stochastic restart renders fluctuations in first passage times universal, Physical review letters116, 170601 (2016)

2016
[46]

Pal and S

A. Pal and S. Reuveni, First passage under restart, Physical review letters118, 030603 (2017)

2017
[47]

M. R. Evans, S. N. Majumdar, and G. Schehr, Stochastic resetting and applications, Journal of Physics A: Mathematical and Theoretical53, 193001 (2020)

2020
[48]

Blumer, S

O. Blumer, S. Reuveni, and B. Hirshberg, Stochastic resetting for enhanced sampling, The journal of physical chemistry letters13, 11230 (2022)

2022
[49]

Blumer, S

O. Blumer, S. Reuveni, and B. Hirshberg, Combining stochastic resetting with metadynamics to speed-up molecular dynamics simulations, Nature Communications15, 240 (2024)

2024
[50]

Eliazar and S

I. Eliazar and S. Reuveni, Mean-performance of sharp restart i: statistical roadmap, Journal of Physics A: Mathematical and Theoretical53, 405004 (2020)

2020
[51]

Nitzan,Chemical dynamics in condensed phases: relaxation, transfer, and reactions in condensed molecular systems (Oxford university press, 2024)

A. Nitzan,Chemical dynamics in condensed phases: relaxation, transfer, and reactions in condensed molecular systems (Oxford university press, 2024)

2024
[52]

W. Ji, W. Qiu, Z. Shi, S. Pan, and S. Deng, Stiff-pinn: Physics-informed neural network for stiff chemical kinetics, The Journal of Physical Chemistry A125, 8098 (2021)

2021
[53]

H. A. Kramers, Brownian motion in a field of force and the diffusion model of chemical reactions, physica7, 284 (1940)

1940
[54]

Hanggi, Escape from a metastable state, Journal of Statistical Physics42, 105 (1986)

P. Hanggi, Escape from a metastable state, Journal of Statistical Physics42, 105 (1986)

1986
[55]

Hänggi, P

P. Hänggi, P. Talkner, and M. Borkovec, Reaction-rate theory: fifty years after kramers, Reviews of modern physics 62, 251 (1990)

1990
[56]

Barducci, M

A. Barducci, M. Bonomi, and M. Parrinello, Metadynamics, Wiley Interdisciplinary Reviews: Computational Molecular Science1, 826 (2011)

2011
[57]

Valsson and M

O. Valsson and M. Parrinello, Variational approach to enhanced sampling and free energy calculations, Physical review letters113, 090601 (2014)

2014
[58]

Bussi and A

G. Bussi and A. Laio, Using metadynamics to explore complex free-energy landscapes, Nature Reviews Physics2, 200 (2020)

2020
[59]

F. J. Massey Jr, The kolmogorov-smirnov test for goodness of fit, Journal of the American statistical Association46, 68 (1951)

1951
[60]

Tiwary, J

P. Tiwary, J. Mondal, J. A. Morrone, and B. Berne, Role of water and steric constraints in the kinetics of cavity–ligand unbinding, Proceedings of the National Academy of Sciences112, 12015 (2015)

2015
[61]

Blumer, S

O. Blumer, S. Reuveni, and B. Hirshberg, Short-time infrequent metadynamics for improved kinetics inference, Journal of Chemical Theory and Computation20, 3484 (2024). 12

2024
[62]

Dekel and L

S. Dekel and L. Gugel, Pr-dad: Phase retrieval using deep auto-decoders, in2022 7th International Conference on Frontiers of Signal Processing (ICFSP)(IEEE, 2022) pp. 165–172

2022
[63]

Leclerc, A

G. Leclerc, A. Ilyas, L. Engstrom, S. M. Park, H. Salman, and A. Mądry, Ffcv: Accelerating training by removing data bottlenecks, inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(2023) pp. 12011–12020. 13 SUPPOR TING INFORMA TION: FIRST-P ASS...

2023
[64]

S4, we show that the training process on the CIFAR-100 dataset also reaches a QSS

CIFAR-100 Classification In Fig. S4, we show that the training process on the CIFAR-100 dataset also reaches a QSS. We used the same optimizer, architecture, and learning rate as was used for the CIFAR-10 dataset. 0 20 40 60 80 100 Epoch 10 15 20 25 30 35 40 45Accuracy [%] 20 ...
[65]

S7, we show that the training process with a momentum term, on the CIFAR-100 dataset reaches a QSS

CIFAR-100 classification - training with momentum In Fig. S7, we show that the training process with a momentum term, on the CIFAR-100 dataset reaches a QSS. We used the same architecture and learning rate as was used for the CIFAR-10 dataset. We used a momentum value of 0.9. ...
[66]

test accuracy

Regression task - phase retrieval of images To demonstrate a QSS for a regression task, we trained a fully-connected NN to reconstruct images from their Fourier-transformed magnitude, i.e., the absolute value of their Fourier transform. We used a similar problem setting as in ...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Jumper, R

J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko,et al., Highly accurate protein structure prediction with alphafold, Nature596, 583 (2021)

work page 2021

[2] [2]

A. Nir, E. Sela, R. Beck, and Y. Bar-Sinai, Machine-learning iterative calculation of entropy for physical systems, Proceedings of the National Academy of Sciences117, 30234 (2020)

work page 2020

[3] [3]

Behler and M

J. Behler and M. Parrinello, Generalized neural-network representation of high-dimensional potential-energy surfaces, Physical review letters98, 146401 (2007)

work page 2007

[4] [4]

Carleo and M

G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science355, 602 (2017)

2017

[5] [5]

F. Noé, S. Olsson, J. Köhler, and H. Wu, Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning, Science365, eaaw1147 (2019)

2019

[6] [6]

B. Máté, F. Fleuret, and T. Bereau, Neural thermodynamic integration: Free energies from energy-based diffusion models, The Journal of Physical Chemistry Letters15, 11395 (2024)

work page 2024

[7] [7]

Anelli, H

A. Anelli, H. Dietrich, P. Ectors, F. Stowasser, T. Bereau, M. Neumann, and J. van den Ende, Robust and efficient reranking in crystal structure prediction: a data driven method for real-life molecules, CrystEngComm26, 5845 (2024)

work page 2024

[8] [8]

Lagemann, K

C. Lagemann, K. Lagemann, S. Mukherjee, and W. Schröder, Deep recurrent optical flow learning for particle image velocimetry data, Nature Machine Intelligence3, 641 (2021)

work page 2021