Pith. sign in

REVIEW 3 major objections 6 minor 61 references

A theoretical framework for overfitting in energy-based modeling

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For Gaussian energy-based models, finite-data overfitting is a predictable spectral effect: weak covariance modes are learned last, and their underestimated eigenvalues set an optimal early-stopping time that random matrix theory locates…

desk verdict The Gaussian-EBM/RMT/GCV core is a genuinely clean result that deserves a serious referee; the Section 7 NTK extension has a factor-of-2 inconsistency that must be fixed or demoted before the abstract's 'derived' claim can stand. read the letter →

arxiv 2501.19158 v3 pith:2QMUBPXZ submitted 2025-01-31 cs.LG cond-mat.dis-nncond-mat.stat-mech

Giovanni Catania , Aurélien Decelle , Cyril Furtlehner , Beatriz Seoane This is my paper · ORCID
classification cs.LGcond-mat.dis-nncond-mat.stat-mech MSC 60B2062H2568T05
keywords energy-basedmodelsoverfittingGaussianmodelrandommatrixtheoryspectralbiasearlystoppinggeneralizedcross-validationinverseIsing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that overfitting in energy-based models is not a diffuse pathology but a predictable spectral phenomenon: with finite data, the learned coupling matrix relaxes mode by mode along the eigenvectors of the empirical covariance, strong modes converge quickly and weak modes slowly, and the slow modes are exactly the ones whose empirical eigenvalues are biased downward. In the Gaussian testbed this yields an analytic training trajectory in Lambert-W form, a closed-form relation between train and test energy under spectral L1 regularization, and random-matrix-theory formulas that locate the early-stopping optimum. If correct, this gives practitioners a principled way to set early-stopping times or shrinkage corrections for pairwise energy-based models, and it extends qualitatively to Boltzmann machines for inverse Ising problems.

What carries the argument

The load-bearing object is the eigendecomposition of the empirical covariance matrix $\hat{C}_M$. Projecting gradient ascent onto that basis decouples the training dynamics into independent equations per mode; the exact solution uses the Lambert $W_0$ function for the Gaussian model, and the asymptotic form of the empirical spectrum is supplied by random matrix theory in the proportional limit $M/N = \rho$. The relation $E_{\text{test}} = E_{\text{train}}/(1 - E_{\text{train}}/\rho)$ is derived by a leave-one-out argument and serves as the EBM analogue of generalized cross-validation. For Boltzmann machines the same decomposition is used with the mean-field correlation approximation $C = (I - J)^{-1}$, giving an approximate eigenvalue evolution with the same timescale separation.

What would settle it

Train a Gaussian energy-based model on data drawn from a population covariance with a degenerate or near-degenerate block of weak eigenvalues, or at aspect ratio $\rho$ close to 1, and compare the measured non-monotonic reconstruction error and optimal stopping time against the random-matrix-theory prediction; a systematic shift or disappearance of the overfitting bump would show that eigenvector misalignment, not eigenvalue distortion alone, sets the overfitting timescale.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that finite-sample overfitting in a Gaussian energy-based model is driven by the spectrum of the empirical covariance matrix, not by eigenvector corruption: each coupling eigenvalue evolves independently as $J_\alpha(t) = \frac{1}{\hat{c}_\alpha} + \frac{1}{\hat{c}_\alpha} W_0\left[B_\alpha e^{-(\hat{c}_\alpha)^2 t/\tau}\right]$ with relaxation time proportional to $(\hat{c}_\alpha)^{-2}$, so stronger data modes are learned first and weaker, noisier modes later. Because finite $M$ underestimates the weak eigenvalues, the limiting couplings $1/\hat{c}_\alpha$ overshoot the true $1/c^*_\alpha$; early in training the eigenvalues cross their ground-truth values, producing a non-monotonic reconstruction error and a well-defined optimal stopping time $t_{\min}(\rho)$ that matches asymptotic random matrix theory. For spectral-L1 regularization the same mechanism yields $E_{\text{test}} = E_{\text{train}} / (1 - E_{\text{train}}/\rho)$, a generalized-cross-validation analogue for EBMs. The paper further claims the same timescale structure appears in binary Boltzmann machines under a mean-field approximation, and sketches a score-matching neural-tangent-kernel route through which the eigenvalue picture should extend to general energy-based models.

Load-bearing premise

The whole analysis treats overfitting as caused by distortions in the eigenvalues of the empirical covariance, assuming its eigenvectors stay close enough to the true ones that eigenvector noise can be ignored; if modes are too close together or the sample size is too small, that assumption gives way.

Editorial extensions

If this is right

  • Early stopping can be selected from the data alone: random-matrix-theory formulas give $t_{\min}(\rho)$ and match finite-size training, although this time does not coincide with the peak of the test log-likelihood.
  • Shrinkage corrections based on eigenvalue cleaning, including a simple downsampling polynomial extrapolation, reduce the overfitting bump without requiring ground truth; the polynomial fit also works for the inverse Ising Boltzmann machine where rotationally invariant shrinkage is not applicable.
  • Generation quality stabilizes before the reconstruction-error minimum, so metrics based on generated samples can miss ongoing degradation of the inferred couplings.
  • The train-test energy relation for spectral-L1 regularization offers a way to estimate test log-likelihood without a test set, in the same spirit as generalized cross-validation for ridge regression.
  • The score-matching neural-tangent-kernel formulation predicts that generic energy-based models in the kernel or lazy regime follow a linear empirical-RKHS dynamics of the same form, making the finite-$M$ spectral mechanism the default explanation of overfitting there as well.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: monitoring the bulk edge of the empirical spectral density could serve as a practical early-warning signal for when weak-mode fitting begins in any pairwise energy-based model.
  • Beyond the paper: the train-test energy relation could be tested as a model-selection score on non-Gaussian pairwise models; if it holds approximately, it would give a held-out-free estimate of test log-likelihood for inverse problems.
  • Beyond the paper: a stress test with degenerate or nearly degenerate population eigenvalues would separate the eigenvalue-distortion mechanism from eigenvector-noise effects, since the idealized val-pop experiment removes only the eigenvalue distortion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies overfitting in Gaussian energy-based models (GEBMs) by diagonalizing the training dynamics in the eigenbasis of the empirical covariance. It derives an explicit Lambert-W solution for eigenvalue dynamics, uses random matrix theory to obtain asymptotic train/test energies and coupling errors, derives a GCV-type relation for spectral-L1 regularization, and extends the analysis to Boltzmann machines. The core GEBM analysis is exact and is supported by extensive numerical tests. The final section attempts to generalize the framework to arbitrary energy-based models via neural tangent kernel (NTK) dynamics of score matching, and this extension is the main source of concern.

Significance. If the core results stand, which the numerical evidence strongly supports, the paper is a valuable contribution: the Lambert-W trajectory (Eq. 6) is verified exactly, the RMT predictions are parameter-free once the population spectrum and aspect ratio are fixed, the GCV-type relation (Eq. 9) is derived rather than fitted, and the experiments cover realistic covariance spectra as well as finite-N convergence. The Boltzmann-machine section is honestly presented as qualitative and provides useful insight. The NTK extension in Section 7 is not sound as written, but the paper's central GEBM/RMT/GCV contribution is independent of it and remains publishable after the extension is corrected or explicitly demoted to a conjecture.

major comments (3)
  1. [Section 7, Eq. (14)] The claimed derivation of j_t(x) = (1-e^{-xt})/x is incorrect. For a commuting initial condition, the scalarized version of dJ/dt = I - (\hat CJ + J\hat C) is dj/dt = 1 - 2xj, whose solution is (1-e^{-2xt})/(2x), not Eq. (14). The factor of two is not a harmless convention: it propagates into Eqs. (17)-(21) and into the closing statement that 'in the GEBM case we recover (14)'. Note also that Eq. (14) describes score-matching dynamics, which is a different training algorithm from the likelihood gradient ascent used in Sections 2-5; if it is intended only as a proxy, this must be stated explicitly. This issue is load-bearing for the abstract's final claim of 'deriving the neural tangent kernel dynamics'.
  2. [Section 7, Eqs. (17)-(21)] The transition from the RKHS dynamics of the score function to the parameter update (21) is asserted rather than derived. The application of the function j_t to the empirical kernel matrix \hat K in Eqs. (17) and (19) requires a spectral definition that is not given, and the 'parameter-sample duality' that produces Eq. (21) is not spelled out. As written, the section has the status of a heuristic proposal, not a derivation. The manuscript itself uses the word 'postulate' in Section 7 and the Discussion calls for further experimental investigation, so the authors should either supply the missing derivation or explicitly label this section as conjecture.
  3. [Abstract vs. Section 7/Discussion] The abstract's final sentence ('deriving the neural tangent kernel dynamics of the score function') is stronger than what the body supports: Section 7 states that the score-matching approach is postulated, and Section 8 says the NTK extension 'deserves further experimental investigations'. Given the factor-of-two error in Eq. (14), the abstract overstates the result. The title-level claim of a framework for arbitrary EBMs should be scaled back to the pairwise/Gaussian cases, with the NTK part presented as a conjectural extension.
minor comments (6)
  1. [Section 2, Eq. (5)] The notation \tau v_\alpha dv_\beta/dt in Eq. (5) is ambiguous; it should be written as \tau v_\alpha^\top dv_\beta/dt to make clear that the left-hand side is the scalar projection of the eigenvector rotation.
  2. [Appendix A, Eq. (28)] The derivation assumes non-symmetric perturbations of J, whereas the numerical training uses the symmetrized gradient (28). The authors note in Fig. 7 that the difference is small, but a short sentence in the main text indicating that this is a controlled approximation would help readers who only consult the main text.
  3. [Section 3, Eq. (10)] The indicator notation 11_{(a,b)}^x is nonstandard and easy to misread; please use \mathbf{1}_{(a,b)}(x).
  4. [Appendix E] The term 'Wasserstrein distance' is misspelled and should be 'Wasserstein distance'.
  5. [Appendix H.1.1] There is a typo: 'reguavlarization' should be 'regularization'.
  6. [Section 4 / Appendix B] The treatment of eigenvector fluctuations is empirically convincing, but the conditions under which eigenvectors of \hat C_M stay close to population eigenvectors (e.g., absence of level repulsion or eigenvalue crossings) are not stated. For trace-level observables this is not a blocker, yet a precise remark would strengthen the asymptotic claims at low \rho.

Circularity Check

0 steps flagged · score 1.0 of 10

No load-bearing circularity: GEBM dynamics, RMT asymptotics, and the GCV-type relation are derived from stated assumptions and validated against numerics without fitted targets; the only self-citation is non-load-bearing, and the Section 7 NTK factor error is a non-circular correctness issue.

full rationale

The core derivation chain is self-contained and non-circular. Section 2 starts from the exact log-likelihood gradient, projects it onto the eigenbasis of the coupling matrix, and integrates the resulting eigenvalue ODE to obtain the Lambert-W solution (Eq. 6); no fitted parameter is fed back into the prediction. The RMT comparisons in Section 4 (Fig. 4) are parameter-free once the population spectrum and aspect ratio are fixed, and they are compared against finite-size simulations rather than calibrated to them. The GCV-type relation (Eq. 9) is derived in Appendix G.2 by a leave-one-out concentration argument, not by fitting test energies, so it is a genuine derived relation. Early-stopping times and optimal regularization strengths use the true model only as an evaluation oracle, not as an input to the analytic curves. The Boltzmann Machine analysis (Section 6) is explicitly approximate, and the citation to Agoritsas et al. (2023) for the spectral-dynamics picture is a self-citation that is not load-bearing because the same derivation is reproduced in Appendix I. The one notable defect is in Section 7, where the paper states that dJ/dt = -(C_hat J + J C_hat) + I leads to j_t(x) = (1 - e^{-xt})/x (Eq. 14); for an initial condition commuting with C_hat, this ODE actually gives (1 - e^{-2xt})/(2x). That is an internal-consistency and correctness problem in the proposed generic-EBM extension, not a circularity: Eq. (14) is not used as an input to fit the quantities it predicts. The section itself is framed as a postulate and calls for experimental investigation, so the abstract's word 'deriving' overstates the result, but the GEBM/RMT/GCV core is unaffected. Score 1 reflects the minor non-load-bearing self-citation and this non-circular residue; no load-bearing circular step was found.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard RMT results, the Gaussian data model with ρ>1, the eigenbasis-alignment assumption for J(t), the equivalence of non-symmetric and symmetric gradient dynamics, and the mean-field diagonal-constraint removal for the BM. Free parameters are testbed choices (spectrum shape) and the heuristic fit exponent, not fitted quantities in the analytic predictions.

free parameters (2)
  • Population spectrum parameters (r, β, γ, x1, x2) of Eq. (10) = r=0.9, β=0.9, γ=1.1, x1=0.1, x2=10 in main figures; r=0.5, β=1.0, γ=0.5 in Appendix B(d)
    Chosen by hand to mimic realistic covariance spectra; inputs to the synthetic testbed, not fitted to the theory's outputs. The paper claims robustness to these choices (Appendix F).
  • Eigenmode-fit exponent ν in the empirical shrinkage protocol (Eq. 55) = ν=1 for GEBM, ν=1/2 for BM
    Selected per model to give the best reconstruction; this is a heuristic data-cleaning protocol, not part of the core RMT predictions.
assumptions (6)
  • domain assumption Data are i.i.d. zero-mean Gaussian with invertible empirical covariance (ρ = M/N > 1).
    The GEBM ML estimator is Ĉ^{-1} and the RMT analysis is carried out in the under-parameterized regime ρ>1 (Section 2, Appendix G).
  • domain assumption J(t) commutes with Ĉ_M throughout training; eigenvector rotation is fast and negligible.
    Reduces coupled matrix dynamics to independent scalar ODEs (Eqs. 5-6); numerically demonstrated in Appendices C-D, not rigorously proven.
  • domain assumption Non-symmetric perturbation gradient (Eq. 3) is dynamically equivalent to the symmetric gradient (Eq. 28).
    The exact diagonalization (Eq. 5) uses the non-symmetric form; Appendix A shows near-coincident numerics.
  • ad hoc to paper For the Boltzmann machine, the mean-field linear response C = (I - J)^{-1} holds and the diagonal constraint can be dropped.
    Appendix I (Eqs. 63-67); required for independent eigenmode dynamics; the paper acknowledges this reduces quantitative accuracy.
  • ad hoc to paper In the RMT computation, the norm fluctuations of the Gaussian samples are neglected, σ(τ)=δ(τ-1).
    Appendix G states the fluctuation of z is assumed negligible; the data are Gaussian so ∥z∥² is χ²_N, and the delta simplification is an approximation validated only by matching numerics.
  • ad hoc to paper Score-matching in the kernel (lazy) regime is a valid proxy for overfitting in generic EBMs.
    Section 7 explicitly postulates this; no experimental validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A theoretical framework for overfitting in energy-based modeling." pith.science (2026). https://pith.science/paper/2QMUBPXZ

@misc{pith2026250119158,
  author       = {Pith},
  title        = {Pith review of: A theoretical framework for overfitting in energy-based modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2QMUBPXZ}},
  note         = {Machine review of arXiv:2501.19158}
}
read the original abstract

We investigate the impact of limited data on training pairwise energy-based models for inverse problems aimed at identifying interaction networks. Utilizing the Gaussian model as testbed, we dissect training trajectories across the eigenbasis of the coupling matrix, exploiting the independent evolution of eigenmodes and revealing that the learning timescales are tied to the spectral decomposition of the empirical covariance matrix. We see that optimal points for early stopping arise from the interplay between these timescales and the initial conditions of training. Moreover, we show that finite data corrections can be accurately modeled through asymptotic random matrix theory calculations and provide the counterpart of generalized cross-validation in the energy based model context. Our analytical framework extends to binary-variable maximum-entropy pairwise models with minimal variations. These findings offer strategies to control overfitting in discrete-variable models through empirical shrinkage corrections, improving the management of overfitting in energy-based generative models. Finally, we propose a generalization to arbitrary energy-based models by deriving the neural tangent kernel dynamics of the score function under the score-matching algorithm.

Figures

Figures reproduced from arXiv: 2501.19158 by the authors.

Figure 1
Figure 1. (a): Eigenvalue spectra of the empirical covariance ma￾trices for MNIST dataset (Deng, 2012). Black lines show spectra using the full dataset size (M∗ ), while scatter colored points rep￾resent subsets (M <M∗ ). (b): Black line shows a synthetic pop￾ulation eigenvalue spectrum based on (10) for N = 100, r = 0.9, β = 0.9, γ = 1.1, x1 = 10−1 , x2 = 10; colored points show the eigenvalues from CbM calculated by samplin… view at source ↗
Figure 2
Figure 2. Training dynamics of the GEBM from a population matrix C ∗ (in (a), with system size and parameters matching those in [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. (a)-(c): for ρ = M/N = 1.5 we plot the reconstruction error during training (in (a), vs t) and the final reconstruction obtained using a L2-norm regularization (in (c), vs λ). (b): train￾ing time achieving optimal reconstruction error (points) and time of maximum test LL (squares), plotted vs ρ. (d): optimal value of regularization prior λ vs ρ, again selecting the optimum w.r.t. reconstruction error and w.r.t. the … view at source ↗
Figures from the paper (15 more)
Figure 5
Figure 5. Figure 5: Effect of data-correction protocols on the training a GEBM (in (a), for ρ = 2.8) and on the final model’s quality as a function of ρ (in (b)): comparison of the reconstruction error EJ between training from an empirical covariance matrix CbM (blue), optimal L2-regulari…
Figure 7
Figure 7. Figure 7: Difference in the eigenvalues’ evolution in the training of a GEBM when imposing symmetry or allowing asymmetry in the perturbation of Jij . The points correspond to numerical results obtained by enforcing symmetry on Jij after each update during training (i.e. using E…
Figure 8
Figure 8. Figure 8: (a)-(b)-(c): Eigenvalue spectra of the empirical covariance matrix of real datasets, respectively CIFAR-10 (in (a)) , Human Genome Dataset (in (b)), and a dataset made of equilibirum configurations of a 2-d Ising model of size N = 162 at β = 0.1 (in (c)) . Black lines …
Figure 9
Figure 9. Figure 9: Finite-M fluctuations of eigenvectors in the covariance matrix of datasets. The four panels show the norm of the matrix product between the n-th projection operator P n , containing the first n eigenvectors of the population matrix C ∗ (for a real dataset, we just take…
Figure 10
Figure 10. Figure 10: Training dynamics of the GEBM from a population matrix C ∗ . The system size and the parameters defining C ∗ are the same as in [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Results on the training dynamics GEBM at finite amount of data by varying the initial conditions. Panels (a)’s (top row) show the eigenvalues’ evolution (according to Eq. (12)), while panels (b)’s (bottom row) show the corresponding reconstruction error EJ w.r.t the g…
Figure 12
Figure 12. Figure 12: We compare the results shown in [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Same plots as in [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Same plots as in [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Same plots as in [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: Effect of the regularization priors on the inferred model’s quality. (a): the plot shows the reconstruction error EJ computed between the ground truth and the inferred model in the presence of a regularization prior with strength λ. Solid lines refer to the L2 norm, w…
Figure 17
Figure 17. Figure 17: Effect of different L2-norm regularization strengths λ on the GEBM’s learning dynamics. Panel (a) shows the reconstruction error vs time. The dotted blue line corresponds to the standard training over CbM. All the other full lines correspond to a training with a certa…
Figure 18
Figure 18. Figure 18: Examples of eigenmode fitting procedures for the GEBM and the Ising-BM. Each panel illustrates the procedure used to fit the eigenmodes of the covariance matrix CbM by downsampling to m < M samples, in order to extrapolate their behavior as m → ∞, following Eq. (55). …
Figure 19
Figure 19. Figure 19: Supplementary results on the Boltzmann Machine for the inverse Ising problem. The model, dataset and training setting are the same as in [PITH_FULL_IMAGE:figures/full_fig_p029_19.png]
Figure 20
Figure 20. Figure 20: Supplementary results on the Boltzmann Machine for the inverse Ising problem. The model, dataset and training setting are the same as in [PITH_FULL_IMAGE:figures/full_fig_p029_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 43 canonical work pages

  1. [1]

    H., Hinton, G

    Ackley, D. H., Hinton, G. E., and Sejnowski, T. J. A learning algorithm for B oltzmann machines. Cognitive science, 9 0 (1): 0 147--169, 1985

    work page 1985

  2. [2]

    S., Saxe, A

    Advani, M. S., Saxe, A. M., and Sompolinsky, H. High-dimensional dynamics of generalization error in neural networks. Neural Networks, 132: 0 428--446, 2020

    2020

  3. [3]

    Explaining the effects of non-convergent MCMC in the training of energy-based models

    Agoritsas, E., Catania, G., Decelle, A., and Seoane, B. Explaining the effects of non-convergent MCMC in the training of energy-based models. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Researc...

    work page 2023

  4. [4]

    From high-dimensional & mean-field dynamics to dimensionless odes: A unifying approach to sgd in two-layers networks

    Arnaboldi, L., Stephan, L., Krzakala, F., and Loureiro, B. From high-dimensional & mean-field dynamics to dimensionless odes: A unifying approach to sgd in two-layers networks. In The Thirty Sixth Annual Conference on Learning Theory, pp.\ 1199--1227. PMLR, 2023

    work page 2023

  5. [5]

    Scaling and renormalization in high-dimensional regression

    Atanasov, A., Zavatone-Veth, J., and Pehlevan, C. Scaling and renormalization in high-dimensional regression. arXiv preprint arXiv:2405.00592, 2024

    arXiv 2024

  6. [6]

    and Silverstein, J

    Baik, J. and Silverstein, J. W. Eigenvalues of large sample covariance matrices of spiked population models. Journal of multivariate analysis, 97 0 (6): 0 1382--1408, 2006

    work page 2006

  7. [7]

    Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation

    Belkin, M. Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation. Acta Numerica, 30: 0 203--248, 2021

    work page 2021

  8. [8]

    To understand deep learning we need to understand kernel learning

    Belkin, M., Ma, S., and Mandal, S. To understand deep learning we need to understand kernel learning. In proc. of ICML, pp.\ 541--549. PMLR, 2018

    work page 2018

Show all 61 references
  1. [9]

    K., Houkpati, Y., Irungu, J., and Oladunni, T

    Bengesi, S., El-Sayed, H., Sarker, M. K., Houkpati, Y., Irungu, J., and Oladunni, T. Advancements in generative ai: A comprehensive review of gans, gpt, autoencoders, diffusion model, and transformers. IEEE Access, 2024

    2024

  2. [10]

    Learning a restricted B oltzmann machine using biased monte carlo sampling

    B \'e reux, N., Decelle, A., Furtlehner, C., and Seoane, B. Learning a restricted B oltzmann machine using biased monte carlo sampling. arXiv preprint arXiv:2206.01310, 2022

    2022 arXiv

  3. [11]

    Fast, accurate training and sampling of restricted B oltzmann machines

    B \'e reux, N., Decelle, A., Furtlehner, C., Rosset, L., and Seoane, B. Fast, accurate training and sampling of restricted B oltzmann machines. arXiv preprint arXiv:2405.15376, 2024

    2024

  4. [12]

    Cleaning large correlation matrices: Tools from random matrix theory

    Bun, J., Bouchaud, J.-P., and Potters, M. Cleaning large correlation matrices: Tools from random matrix theory. Physics Reports, 666: 0 1--109, 2017. ISSN 0370-1573. doi:https://doi.org/10.1016/j.physrep.2016.10.005. URL https://www.sciencedirect.com/science/article/pii/S03701...

    2017 doi

  5. [13]

    Overlaps between eigenvectors of correlated random matrices

    Bun, J., Bouchaud, J.-P., and Potters, M. Overlaps between eigenvectors of correlated random matrices. Phys. Rev. E, 98: 0 052145, Nov 2018. doi:10.1103/PhysRevE.98.052145. URL https://link.aps.org/doi/10.1103/PhysRevE.98.052145

    2018 doi

  6. [14]

    On lazy training in differentiable programming

    Chizat, L., Oyallon, E., and Bach, F. On lazy training in differentiable programming. In proc. of NeurIPS, 32, 2019

    2019

  7. [15]

    Inverse statistical physics of protein sequences: a key issues review

    Cocco, S., Feinauer, C., Figliuzzi, M., Monasson, R., and Weigt, M. Inverse statistical physics of protein sequences: a key issues review. Reports on Progress in Physics, 81 0 (3): 0 032601, 2018

    2018

  8. [16]

    Consortium, . G. P. et al. A global reference for human genetic variation. Nature, 526 0 (7571): 0 68, 2015

    2015

  9. [17]

    Thermodynamics of restricted B oltzmann machines and related learning dynamics

    Decelle, A., Fissore, G., and Furtlehner, C. Thermodynamics of restricted B oltzmann machines and related learning dynamics. Journal of Statistical Physics, 172 0 (6): 0 1576--1608, 2018. doi:https://doi.org/10.1007/s10955-018-2105-y

    2018 doi

  10. [18]

    Unsupervised hierarchical clustering using the learning dynamics of restricted B oltzmann machines

    Decelle, A., Seoane, B., and Rosset, L. Unsupervised hierarchical clustering using the learning dynamics of restricted B oltzmann machines. Physical Review E, 108 0 (1): 0 014110, 2023

    2023

  11. [19]

    Decelle, A., Furtlehner, C., Gómez, A. D. J. N., and Seoane, B. Inferring effective couplings with restricted B oltzmann machines . SciPost Phys., 16: 0 095, 2024. doi:10.21468/SciPostPhys.16.4.095. URL https://scipost.org/10.21468/SciPostPhys.16.4.095

    2024 doi

  12. [20]

    Inferring high-order couplings with neural networks

    Decelle, A., de Jesús Navas Gómez, A., and Seoane, B. Inferring high-order couplings with neural networks. 2025. URL https://arxiv.org/abs/2501.06108

    2025

  13. [21]

    Gromov–wasserstein distances between gaussian distributions

    Delon, J., Desolneux, A., and Salmona, A. Gromov–wasserstein distances between gaussian distributions. Journal of Applied Probability, 59 0 (4): 0 1178–1198, 2022. doi:10.1017/jpr.2022.16

    2022 doi

  14. [22]

    The mnist database of handwritten digit images for machine learning research

    Deng, L. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29 0 (6): 0 141--142, 2012

    2012

  15. [23]

    and Mordatch, I

    Du, Y. and Mordatch, I. Implicit generation and modeling with energy based models. Advances in Neural Information Processing Systems, 32, 2019

    2019

  16. [24]

    Optimal regularizations for data generation with probabilistic graphical models

    Fanthomme, A., Rizzato, F., Cocco, S., and Monasson, R. Optimal regularizations for data generation with probabilistic graphical models. Journal of Statistical Mechanics: Theory and Experiment, 2022 0 (5): 0 053502, may 2022. doi:10.1088/1742-5468/ac650c. URL https://dx.doi.or...

    2022 doi

  17. [25]

    and Lucibello, C

    Feinauer, C. and Lucibello, C. Reconstruction of pairwise interactions using energy-based models. In Mathematical and Scientific Machine Learning, pp.\ 291--313. PMLR, 2022

    2022

  18. [26]

    Interpretable pairwise distillations for generative protein sequence models

    Feinauer, C., Meynard-Piganeau, B., and Lucibello, C. Interpretable pairwise distillations for generative protein sequence models. PLOS Computational Biology, 18 0 (6): 0 e1010219, 2022

    2022

  19. [27]

    Free dynamics of feature learning processes

    Furtlehner, C. Free dynamics of feature learning processes. J.Stat.Phys, 190 0 (3): 0 51, 2023

    2023

  20. [28]

    Generalized cross-validation as a method for choosing a good ridge parameter

    Golub, G., Heath, M., and Wahba, G. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 21 0 (2): 0 215--223, 1979

    1979

  21. [29]

    Deterministic equivalents for certain functionals of large random matrices

    Hachem, W., Loubaton, P., and Najim, J. Deterministic equivalents for certain functionals of large random matrices. 2007

    2007

  22. [30]

    Surprises in high-dimensional ridgeless least squares interpolation

    Hastie, T., Montanari, A., Rosset, S., and Tibshirani, R. Surprises in high-dimensional ridgeless least squares interpolation. The Annals of Statistics, 50 0 (2): 0 949--986, 2022

    2022

  23. [31]

    and Dayan, P

    Hyv \"a rinen, A. and Dayan, P. Estimation of non-normalized statistical models by score matching. JMLR, 6 0 (4), 2005

    2005

  24. [32]

    Neural tangent kernel: Convergence and generalization in neural networks

    Jacot, A., Gabriel, F., and Hongler, C. Neural tangent kernel: Convergence and generalization in neural networks. In In proc. of NeurIPS, volume 31, 2018

    2018

  25. [33]

    Kappen, H. J. and Rodríguez, F. B. Efficient Learning in B oltzmann Machines Using Linear Response Theory . Neural Computation, 10 0 (5): 0 1137--1156, 07 1998. ISSN 0899-7667. doi:10.1162/089976698300017386. URL https://doi.org/10.1162/089976698300017386

    1998 doi

  26. [34]

    Estimation of quenched random fields in the inverse ising problem using a diagonal matching method

    Kiwata, H. Estimation of quenched random fields in the inverse ising problem using a diagonal matching method. Phys. Rev. E, 89: 0 062135, Jun 2014. doi:10.1103/PhysRevE.89.062135. URL https://link.aps.org/doi/10.1103/PhysRevE.89.062135

    2014 doi

  27. [35]

    Learning multiple layers of features from tiny images

    Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009

    2009

  28. [36]

    and P \'e ch \'e , S

    Ledoit, O. and P \'e ch \'e , S. Eigenvectors of some large sample covariance matrix ensembles. Probability Theory and Related Fields, 151 0 (1): 0 233--264, 2011

    2011

  29. [37]

    and Wolf, M

    Ledoit, O. and Wolf, M. A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88 0 (2): 0 365--411, 2004. ISSN 0047-259X. doi:https://doi.org/10.1016/S0047-259X(03)00096-4. URL https://www.sciencedirect.com/science/article/pi...

    2004 doi

  30. [38]

    and Wolf, M

    Ledoit, O. and Wolf, M. Analytical nonlinear shrinkage of large-dimensional covariance matrices . The Annals of Statistics, 48 0 (5): 0 3043 -- 3065, 2020. doi:10.1214/19-AOS1921. URL https://doi.org/10.1214/19-AOS1921

    2020 doi

  31. [39]

    Restoring balance: principled under/oversampling of data for optimal classification

    Loffredo, E., Pastore, M., Cocco, S., and Monasson, R. Restoring balance: principled under/oversampling of data for optimal classification. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024

    2024

  32. [40]

    MacKay, D. J. C. Information Theory, Inference, and Learning Algorithms. Copyright Cambridge University Press, 2003

    2003

  33. [41]

    Magnus, J. R. and Neudecker, H. Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley, second edition, 1999. ISBN 0471986321 9780471986324 047198633X 9780471986331

    1999

  34. [42]

    A large scale analysis of logistic regression: Asymptotic performance and new insights

    Mai, X., Liao, Z., and Couillet, R. A large scale analysis of logistic regression: Asymptotic performance and new insights. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 3357--3361. IEEE, 2019

    2019

  35. [43]

    and Pastur, L

    Mar c enko, V. and Pastur, L. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR -Sbornik , 1 0 (4): 0 457--483, 1967

    1967

  36. [44]

    A mean field view of the landscape of two-layer neural networks

    Mei, S., Montanari, A., and Nguyen, P.-M. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences, 115 0 (33): 0 E7665--E7671, 2018

    2018

  37. [45]

    S., Sander, C., Zecchina, R., Onuchic, J

    Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., and Weigt, M. Direct-coupling analysis of residue coevolution captures native contacts across many protein families. Proceedings of the National Academy of Scienc...

    2011

  38. [46]

    C., Zecchina, R., and Berg, J

    Nguyen, H. C., Zecchina, R., and Berg, J. Inverse statistical problems: from the inverse I sing problem to data science. Advances in Physics, 66 0 (3): 0 197--261, 2017. doi:10.1080/00018732.2017.1341604. URL https://doi.org/10.1080/00018732.2017.1341604

    2017

  39. [47]

    Failures and successes of cross-validation for early-stopped gradient descent

    Patil, P., Wu, Y., and Tibshirani, R. Failures and successes of cross-validation for early-stopped gradient descent. In International Conference on Artificial Intelligence and Statistics, pp.\ 2260--2268. PMLR, 2024

    2024

  40. [48]

    and Bouchaud, J.-P

    Potters, M. and Bouchaud, J.-P. A first course in random matrix theory: for physicists, engineers and data scientists. Cambridge University Press, 2020

    2020

  41. [49]

    On the spectral bias of neural networks

    Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., and Courville, A. On the spectral bias of neural networks. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Pr...

  42. [50]

    The bethe approximation for solving the inverse I sing problem: a comparison with other inference methods

    Ricci-Tersenghi, F. The bethe approximation for solving the inverse I sing problem: a comparison with other inference methods. Journal of Statistical Mechanics: Theory and Experiment, 2012 0 (08): 0 P08015, aug 2012. doi:10.1088/1742-5468/2012/08/P08015. URL https://dx.doi.org...

    2012 doi

  43. [51]

    Roudi, Y., Aurell, E., and Hertz, J. A. Statistical physics of pairwise probability models. Frontiers in computational neuroscience, 3: 0 652, 2009

    2009

  44. [52]

    and Solla, S

    Saad, D. and Solla, S. Dynamics of on-line gradient descent learning for multilayer neural networks. Advances in neural information processing systems, 8, 1995

    1995

  45. [53]

    M., McClelland, J

    Saxe, A. M., McClelland, J. L., and Ganguli, S. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In Bengio, Y. and LeCun, Y. (eds.), ICLR, 2014. URL http://dblp.uni-trier.de/db/conf/iclr/iclr2014.html#SaxeMG13

    2014

  46. [54]

    and Kubo, R

    Suzuki, M. and Kubo, R. Dynamics of the I sing model near the critical point. i. Journal of the Physical Society of Japan, 24 0 (1): 0 51--60, 1968. doi:10.1143/JPSJ.24.51

    1968 doi

  47. [55]

    M., Sclocchi, A., and Wyart, M

    Tomasini, U. M., Sclocchi, A., and Wyart, M. Failure and success of the spectral bias prediction for L aplace kernel ridge regression: the case of low-dimensional data. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the ...

    2022

  48. [56]

    Learning protein constitutive motifs from sequence data

    Tubiana, J., Cocco, S., and Monasson, R. Learning protein constitutive motifs from sequence data. Elife, 8: 0 e39397, 2019

    2019

  49. [57]

    More than a toy: Random matrix models predict how real-world neural representations generalize

    Wei, A., Hu, W., and Steinhardt, J. More than a toy: Random matrix models predict how real-world neural representations generalize. In International Conference on Machine Learning, pp.\ 23549--23588. PMLR, 2022

    2022

  50. [58]

    E., Arnold, F

    Wu, Z., Johnston, K. E., Arnold, F. H., and Yang, K. K. Protein sequence design with deep generative models. Current opinion in chemical biology, 65: 0 18--27, 2021

    2021

  51. [59]

    and Tanaka, K

    Yasuda, M. and Tanaka, K. Susceptibility propagation by using diagonal consistency. Phys. Rev. E, 87: 0 012134, Jan 2013. doi:10.1103/PhysRevE.87.012134. URL https://link.aps.org/doi/10.1103/PhysRevE.87.012134

    2013 doi

  52. [60]

    Creating artificial human genomes using generative neural networks

    Yelmen, B., Decelle, A., Ongaro, L., Marnetto, D., Tallec, C., Montinaro, F., Furtlehner, C., Pagani, L., and Jay, F. Creating artificial human genomes using generative neural networks. PLoS genetics, 17 0 (2): 0 e1009303, 2021

    2021

  53. [61]

    L., Szatkownik, A., Furtlehner, C., Charpiat, G., and Jay, F

    Yelmen, B., Decelle, A., Boulos, L. L., Szatkownik, A., Furtlehner, C., Charpiat, G., and Jay, F. Deep convolutional and conditional neural networks for large-scale genomic data generation. PLoS Computational Biology, 19 0 (10): 0 e1011584, 2023

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.