Pith. sign in

REVIEW 3 major objections 3 minor 56 references

Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm

T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In a high-dimensional Gaussian mixture, ℓ2-regularized maximum likelihood with a task-tuned regularizer achieves near-Bayes-optimal estimation and prediction when unlabeled data are abundant.

desk verdict Solid AMP/SE analysis of the labeled-unlabeled GMM whose near-optimality claim for RMLE is real but hinges on oracle-tuned regularization. read the letter →

arxiv 2411.19553 v2 pith:2REOBWGS submitted 2024-11-29 cs.LG stat.ML

Xiaosi Gu , Tomoyuki Obuchi This is my paper · ORCID
classification cs.LGstat.ML MSC 62H3062F1582B2668T05
keywords semi-supervisedlearningGaussianmixturemodelapproximatemessagepassingstateevolutionphasetransitionreplicasymmetrybreakingregularizedmaximumlikelihoodhigh-dimensionalclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks when and why semi-supervised learning helps for high-dimensional binary classification. Modeling data as a two-class Gaussian mixture with known noise, it compares the Bayes-optimal posterior mean, the best possible benchmark, with ℓ2-regularized maximum likelihood, a practical convex estimator that treats unlabeled labels as hidden variables. Using message-passing equations and their state-evolution reduction, the authors map out phase diagrams showing when estimation fails and when the algorithm is unstable. Their central quantitative finding is that with the regularizer tuned to minimize the gap, RMLE almost matches the Bayes-optimal estimator: within about 0.8% in mean-squared error and 0.007% in generalization error, with the gap shrinking as unlabeled data grow. If correct, this means a convex method can serve as a near-oracle surrogate in this setting, and unlabeled data are what make the optimal regularizer finite.

What carries the argument

The unifying object is the β-posterior $p_\beta(w|D) \propto p(D|w)^\beta p(w)^\beta$, with $\beta=1$ recovering the Bayesian posterior and $\beta\to\infty$ recovering the ℓ2-regularized maximum-likelihood point estimate. On the factor graph of this posterior, belief propagation is approximated to obtain approximate message passing (AMP) updates, whose macroscopic behavior in the proportional limit is exactly captured by scalar state-evolution recursions for the overlap $k$ and variance $v$ of the estimator with the truth. A $\lambda\leftrightarrow\chi$ correspondence fixes the susceptibility $\chi$ to control convergence and recovers the matching regularizer $\lambda$, and a linear stability analysis of AMP identifies the replica-symmetry-broken region. Fixed points are converted into closed-form mean-squared error and generalization error formulas for the comparison between RMLE and the Bayes-optimal estimate.

What would settle it

Fix a parameter point inside the detected phase, run finite-N AMP and an independent gradient-based minimization of the RMLE objective for increasing N, and compare their limit to the state-evolution fixed point; if the limiting $(k^*, v^*)$ differ from the state-evolution prediction, or if the relative mean-squared-error gap to the Bayes-optimal posterior at the paper's oracle $\lambda^*$ exceeds about 0.8% when $\alpha_u$ is large, the near-optimality claim fails. A cheaper check is to test numerically whether the $\lambda^*$ that minimizes the gap can be recovered by cross-validation on labels; the paper says this is highly nontrivial for MSE, so a demonstration that no data-driven rule recovers it would falsify the practical version of the claim.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that in the proportional high-dimensional limit, the state evolution of RMLE has fixed points reproducing the full phase structure of the Gaussian labeled-unlabeled mixture: an undetected phase with zero estimate, a random phase orthogonal to the truth, a detected phase with useful overlap, and an unstable replica-symmetry-broken region. Comparing RMLE at its oracle-tuned regularizer with the Bayes-optimal posterior mean shows that the relative gaps in mean-squared error and generalization error are small across a wide range of signal-to-noise ratios and become smaller as the unlabeled ratio αu grows or the label balance ρ departs from 1/2. The optimal inverse regularizer 1/λ* is finite once unlabeled data are present, unlike the fully supervised case where the optimal regularizer diverges.

Load-bearing premise

The near-optimality claim rests on choosing the regularization strength with knowledge of the Bayes-optimal benchmark it is measured against, and the paper provides no data-driven rule for that choice in the mean-squared-error case.

Editorial extensions

If this is right

  • In semi-supervised settings with abundant unlabeled data, RMLE at its tuned regularizer can stand in for Bayes-optimal inference, avoiding intractable posterior integration.
  • Because the optimal inverse regularizer stays finite under SSL, the ridge penalty need not diverge, and cross-validation is a plausible selection rule for generalization error.
  • Finite labeled data or label imbalance removes the undetected phase and shrinks the unstable replica-symmetry-broken region, so even a few labeled points qualitatively stabilize estimation.
  • The gap to the Bayes-optimal estimate decreases as the unlabeled ratio αu increases and as the label balance ρ moves away from 1/2, quantifying the value of unlabeled data.
  • In the replica-symmetry-broken region AMP is unstable, so the detected-to-RSB boundary marks where the message-passing approach must be replaced by a more robust algorithm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: the oracle-tuning of λ* makes the near-optimality claim partly by construction; the paper derives no data-driven rule for choosing the MSE-optimal regularizer, so the strongest practical claim is the generalization-error one, where cross-validation could approximate λ*.
  • Our inference: the finiteness of the optimal regularizer suggests unlabeled data provide enough curvature that overfitting is controlled without an infinite ridge, and this may carry over to other convex losses such as logistic or hinge loss in high-dimensional SSL.
  • Our inference: the replica-symmetry-broken region marks a hard algorithmic boundary for AMP; a damping, annealing, or spectral-initialization scheme might extend near-Bayes-optimal performance into that region, which the paper does not explore.
  • Our inference: the paper's comparison shows that MSE and generalization error are optimized by different finite regularizer values, implying that a single regularization cannot simultaneously minimize both; practitioners should choose the metric before tuning.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper analyzes a two-class high-dimensional Gaussian mixture model with both labeled and unlabeled data in the proportional asymptotics regime. It derives approximate message-passing (AMP) algorithms for both ℓ2-regularized maximum likelihood estimation (RMLE) and the Bayesian posterior, and it provides state evolution (SE) recursions for the macroscopic order parameters. These recursions are used to construct phase diagrams, study replica-symmetry-breaking (RSB) instabilities, and compare the estimation error (MSE) and generalization error (GE) of RMLE against the Bayes-optimal (BO) estimator. The central claim is that, under an appropriately tuned regularizer, RMLE nearly matches BO performance, especially when the amount of unlabeled data is large.

Significance. If the central claim is supported, the paper would provide a useful asymptotic characterization of a practical convex estimator in a semi-supervised setting, extending earlier high-dimensional GMM analyses from the fully supervised and purely unsupervised cases to the labeled-unlabeled regime. The detailed derivations of ABP/AMP and SE, the explicit phase diagrams, and the numerical consistency checks between AMP and SE and between AMP and gradient descent at N=8000 are strengths and give confidence in the technical core. However, the headline near-optimality claim is conditional on an oracle-tuned regularizer, and the paper does not currently quantify how the reported small gaps degrade under a realistic or data-driven choice of the regularization parameter. This makes the practical interpretation of the claim weaker than the abstract suggests, but the underlying derivation remains sound and the issue is addressable within the manuscript's scope.

major comments (3)
  1. [Sec. 4.3 (Figs. 10-12)] The optimal regularizer λ* is defined as the value minimizing the RMLE-to-BO gap, so the small reported gaps in Fig. 11 (relative differences of about 0.008 for MSE and 0.00007 for GE) are partly a consequence of the definition rather than evidence about a practical tuning rule. The paper itself states that estimating the optimal λ for MSE is "highly nontrivial" and only suggests cross-validation for GE without analyzing the resulting error. Please add a sensitivity analysis showing how the gaps in Fig. 11 grow when λ deviates from λ* (for example, over a plausible grid or for a data-driven rule), and either analyze cross-validation for GE quantitatively or soften the abstract and conclusion to present the result as an existence statement about some regularizer rather than a practical near-optimality claim.
  2. [Secs. 4.1.1 and 4.3 (eq. (49), Figs. 10-11)] The SE predictions used for the RMLE-versus-BO comparison are known to be inaccurate in the RSB region, which the paper identifies via eq. (49). It is not shown whether the parameter combinations used in Fig. 11 and the optimal-λ curves in Fig. 12 lie outside the RSB region for every reported αu and ρ. Because the near-optimality claim relies on SE being reliable, please report the RSB boundary on the same parameter plane as the comparison, or restrict the near-optimality claim to the RSB-free region.
  3. [Sec. 4.3 (Fig. 11)] The comparison optimizes λ* separately for MSE and GE, so the statement that RMLE "results nearly match those of the BO" for both errors is not demonstrated for any single regularization parameter. In practice a user must choose one λ. Please report the cross-performance at a common λ (for example, λ*_GE evaluated for MSE, and vice versa) so the reader can see the trade-off and the extent to which the two near-optimality claims can hold simultaneously.
minor comments (3)
  1. [Eq. (35c)] The Onsager correction term in the Bayesian AMP update contains \(\hat w_i^{(t)}\) rather than \(\hat w_{i,B}^{(t)}\), which is inconsistent with the notation used for the other Bayesian variables; the corresponding equation in the appendix, eq. (90c), has the correct subscript.
  2. [Figs. 3-7] The phase diagrams use multiple colored lines (solid blue, red, green, pink, yellow) whose identities are given only in the text; in grayscale printing these are difficult to distinguish, and the captions do not reproduce the legend. Repeating the line conventions in the captions or using distinct line styles would improve readability.
  3. [Sec. 4.3] The reported optimal values of \(1/\lambda^*\) (approximately 0.70 and 0.42 in Fig. 10) are given without describing the numerical minimization procedure or its precision; a short description of how λ* is computed from the SE curves would improve reproducibility.

Circularity Check

1 steps flagged · score 5.0 of 10

Headline near-optimality claim rests on an oracle-tuned λ* defined as the minimizer of the RMLE-to-BO gap; the SE derivation itself is self-contained.

  1. fitted input called prediction [Sec. 4.3 (Comparison of RMLE and the BO estimate), around Fig. 10 and the 'Overall' paragraph]
    "The λ value giving the minimum difference in the respective errors (MSE or GE) of RMLE and the BO estimate is called optimal and is denoted as λ∗. ... We call RMLE at λ = λ∗ the optimal RMLE. ... Overall, whether using MSE or GE as the metric, the ratio of the difference between the optimal RMLE and the BO estimate is quite small, particularly with large αu. Thus, by optimally tuning λ, we can use RMLE as a good approximation of the BO estimate when handling substantial amounts of unlabeled data."

    λ∗ is not selected by an independent, data-driven rule; it is defined as the minimizer of the very gap to the BO benchmark that the paper then reports as small. Hence the comparison in Fig. 11 evaluates the tuned objective: gap(RMLE(λ∗), BO) = min_λ gap(RMLE(λ), BO) by construction. The numerical value of this minimum is not forced by the definition, but the headline claim 'RMLE can achieve near-optimal performance' is an existence claim conditional on an oracle that knows the BO result. The paper concedes that estimating the optimal λ for MSE is 'highly nontrivial' and only suggests cross-validation for GE without analyzing the resulting gap, so the practical near-optimality claim is not demonstrated independently of the tuning oracle.

full rationale

The AMP/SE derivation is not circular: the order parameters are obtained from explicit recursions (Eqs. 36 and 41) under standard Gaussian assumptions, and the phase boundaries follow from linear stability analyses of those recursions; numerical consistency with AMP and with gradient descent is checked independently. No load-bearing self-citation chain is present: the comparison with Tanaka's earlier RS solution [8] is used only as a validation check of the Bayesian AMP, not as the source of the main results. The one genuinely circular element is the Sec. 4.3 performance comparison. There, λ* is defined as the value minimizing the RMLE-to-BO difference, so the subsequently reported small relative gaps (about 0.008 for MSE and 0.00007 for GE) are the values of the minimized objective rather than predictions made under a practical tuning rule. The paper explicitly states that choosing the optimal λ for MSE is highly nontrivial, and for GE it merely suggests cross-validation without quantifying the selection error or sensitivity to mis-specified λ. The abstract's claim that 'with appropriate regularizations, RMLE can achieve near-optimal performance' is therefore supported only as an oracle-tuned existence statement. This is partial circularity in the headline comparison, while the technical derivation remains self-contained; hence a score of 5 rather than a higher score reserved for fully fitted or self-citation-forced results.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

No new physical or algorithmic entities are postulated. The central claim rests on standard AMP/SE exactness assumptions, the known-noise and known-prior setting, and the oracle-tuned regularizer lambda*. The free parameters are the tuned regularizer and incidental curve-fit parameters in the numerical validation.

free parameters (2)
  • lambda* (optimal regularizer for RMLE) = 1/lambda* ~ 0.70 for MSE and ~ 0.42 for GE at alpha_l=0.5, alpha_u=2.5, SNR=1 (Fig.
    Defined as the lambda minimizing the difference between RMLE and the BO benchmark. The headline near-optimality claim is conditional on this tuned value.
  • Power-law fit parameters for GD-AMP consistency (Delta0, a, d) = (Delta0, a, d) ~ (1.0e-5, 1.0, 0.49), (1.6e-5, 0.88, 0.49), (1.0e-5, 0.23, 0.50) for rho = 0.5, 0.4, 0.1 in Fig. 16
    Used only in the numerical validation of AMP against gradient descent; not part of the central claim.
assumptions (6)
  • domain assumption AMP and state evolution are exact in the proportional limit N, Ml, Mu tending to infinity
    The whole analysis, including eqs. (34)-(36) and the phase diagrams, relies on this standard but unproved-in-paper property of AMP/SE. It is validated numerically at finite N in App. A.6 rather than proven.
  • standard math Messages can be approximated as Gaussian via the central limit theorem
    Used in Sec. 3.3 and App. A.1-A.2 to reduce belief propagation to scalar recursions. This is standard in AMP derivations.
  • domain assumption The noise variance is known and equal to the generative value, sigma^2 = sigma_0^2
    Stated in Sec. 3.1: 'we assume that the model parameter sigma_0^2 is known and sigma^2 = sigma_0^2 holds throughout the paper.'
  • domain assumption The signal variance satisfies v_s = 1/lambda_0 in the large-N limit
    Eq. (37) and Sec. 3.5: 'Henceforth, we assume v_s = 1/lambda_0.'
  • domain assumption The chi-to-lambda correspondence is one-to-one
    Sec. 3.6 fixes chi and solves eq. (36a) for lambda. The authors state 'our SE-based result confirms this reasonable expectation' from numerics, not from a proof.
  • domain assumption Inside the RSB region, the distinction between random and mixed phases is neglected
    Sec. 4.1.1 states that because SE is inaccurate under RSB, the mixed and random phases are treated as a single RSB region. This simplification affects the phase diagrams but not the central BO comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm." pith.science (2026). https://pith.science/paper/2REOBWGS

@misc{pith2026241119553,
  author       = {Pith},
  title        = {Pith review of: Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2REOBWGS}},
  note         = {Machine review of arXiv:2411.19553}
}
abstract

Semi-supervised learning (SSL) is a machine learning methodology that leverages unlabeled data in conjunction with a limited amount of labeled data. Although SSL has been applied in various applications and its effectiveness has been empirically demonstrated, it is still not fully understood when and why SSL performs well. Some existing theoretical studies have attempted to address this issue by modeling classification problems using the so-called Gaussian Mixture Model (GMM). These studies provide notable and insightful interpretations. However, their analyses are focused on specific purposes, and a thorough investigation of the properties of GMM in the context of SSL has been lacking. In this paper, we conduct such a detailed analysis of the properties of the high-dimensional GMM for binary classification in the SSL setting. To this end, we employ the approximate message passing and state evolution methods, which are widely used in high-dimensional settings and originate from statistical mechanics. We deal with two estimation approaches: the Bayesian one and the $\ell_2$-regularized maximum likelihood estimation (RMLE). We conduct a comprehensive comparison between these two approaches, examining aspects such as the global phase diagram, estimation error for the parameters, and prediction error for the labels. A specific comparison is made between the Bayes-optimal (BO) estimator and RMLE, as the BO setting provides optimal estimation performance and is ideal as a benchmark. Our analysis shows that with appropriate regularizations, RMLE can achieve near-optimal performance in terms of both the estimation error and prediction error, especially when there is a large amount of unlabeled data. These results demonstrate that the $\ell_2$ regularization term plays an effective role in estimation and prediction in SSL approaches.

Figures

Figures reproduced from arXiv: 2411.19553 by the authors.

Figure 1
Figure 1. (Left) Factor graph corresponding to eq. (11) (Right) Visualization of messages [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. The relation between χ and 1/λ for RMLE (left) and the Bayesian approach (right) at ρ = 1/2, αl = 0, λ0 = 1, σ2 = 1. The straight black lines depict the relation 1/λ = χ, which is the correct trivial relation at αu = αl = 0. We also compare the consistency between SE and AMP, as well as between AMP and the gradient-based algorithm. The results are presented in app. A.6. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Phase diagram of RMLE at ρ = 1/2, αl = 0 with λ0 = σ 2 = 1. Summarizing the analysis so far, we can draw the phase diagram in the χ–αu plane as shown in [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Impact of labeled data αl , label imbalance ρ on the phase diagram The undetected phase disappears by introducing finite αl . This is natural because the labeled data can break the symmetry between the two label values. It is also observed 19 [PITH_FULL_IMAGE:figures/…
Figure 5
Figure 5. Figure 5: Impact of SNR 1/(λ0σ 2 ) on the phase diagram As a further investigation, in [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: Phase diagram of the Bayesian approach at [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Impact of labeled data αl , label imbalance ρ, and SNR on the phase diagram. The left and middle panels are counterparts of [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: The MSE visual representations of the BO ( [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Specific λ values (̸= λ0) for the model mismatch and their performance. The upper panels show the curves for different λ values in [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Comparison of RMLE and the BO estimate in terms of MSE and GE. In the [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: Comparison of the optimal RMLE and the BO estimate in terms of MSE [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 12
Figure 12. Figure 12: The optimal values of λ ∗ under MSE and GE. The upper panels show the relation between 1/λ∗ and SNR when using MSE, while the bottom panels show the coun￾terpart for GE. The left panels plot 1/λ∗ against SNR with various αu values, while the right panels plot the same…
Figure 13
Figure 13. Figure 13: Consistency check between AMP and SE in the RMLE case. The upper and [PITH_FULL_IMAGE:figures/full_fig_p039_13.png]
Figure 14
Figure 14. Figure 14: Consistency check between AMP and SE in the Bayesian case. This is the [PITH_FULL_IMAGE:figures/full_fig_p040_14.png]
Figure 15
Figure 15. Figure 15: Dependence of ∆GD,AMP on the threshold value (εGD) and learning rate (η). In the left panel, ∆GD,AMP is plotted against εGD with fixed η = 0.1. Each colored curve with joined markers and error bars represents different system sizes N. In the right panel, ∆GD,AMP is pl…
Figure 16
Figure 16. Figure 16: Consistency check between AMP and GD in large [PITH_FULL_IMAGE:figures/full_fig_p042_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 48 canonical work pages

  1. [1]

    Semi-supervised learning literature survey

    Xiaojin Jerry Zhu. Semi-supervised learning literature survey. 2005

    2005

  2. [2]

    Semi-Supervised Learning

    Olivier Chapelle, Bernhard Sch¨ olkopf, and Alexander Zien, editors. Semi-Supervised Learning. The MIT Press, 2006

    work page 2006

  3. [3]

    Billion-scale semi-supervised learning for image classification

    I Zeki Yalniz, Herv´ e J´ egou, Kan Chen, Manohar Paluri, and Dhruv Maha- jan. Billion-scale semi-supervised learning for image classification. arXiv preprint arXiv:1905.00546, 2019

    arXiv 1905

  4. [4]

    Pushing the limits of semi-supervised learning for automatic speech recognition

    Yu Zhang, James Qin, Daniel S Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V Le, and Yonghui Wu. Pushing the limits of semi-supervised learning for automatic speech recognition. arXiv preprint arXiv:2010.10504 , 2020

    arXiv 2010

  5. [5]

    Mixmatch: A holistic approach to semi-supervised learning

    David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural information processing systems , 32, 2019. 43

    work page 2019

  6. [6]

    Asymptotic Bayes risk for Gaussian mixture in a semi- supervised setting

    Marc Lelarge and L´ eo Miolane. Asymptotic Bayes risk for Gaussian mixture in a semi- supervised setting. In 2019 IEEE 8th International Workshop on Computational Ad- vances in Multi-Sensor Adaptive Processing (CAMSAP) , pages 639–643. IEEE, 2019

    work page 2019

  7. [7]

    Pattern recognition and machine learning, volume 4

    Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006

    2006

  8. [8]

    Statistical-mechanics analysis of Gaussian labeled-unlabeled classi- fication problems

    Toshiyuki Tanaka. Statistical-mechanics analysis of Gaussian labeled-unlabeled classi- fication problems. In Journal of Physics: Conference Series , volume 473, page 012001. IOP Publishing, 2013

    work page 2013

Show all 56 references
  1. [9]

    A study of Gaussian mixture models of color and texture features for image classification and segmentation

    Haim Permuter, Joseph Francos, and Ian Jermyn. A study of Gaussian mixture models of color and texture features for image classification and segmentation. Pattern recognition, 39(4):695–706, 2006

    2006

  2. [10]

    High dimensional classification via empirical risk mini- mization: Improvements and optimality

    Xiaoyi Mai and Zhenyu Liao. High dimensional classification via empirical risk mini- mization: Improvements and optimality. arXiv preprint arXiv:1905.13742 , 2019

    1905 arXiv

  3. [11]

    The role of regularization in classification of high-dimensional noisy Gaussian mixture

    Francesca Mignacco, Florent Krzakala, Yue Lu, Pierfrancesco Urbani, and Lenka Zde- borova. The role of regularization in classification of high-dimensional noisy Gaussian mixture. In International conference on machine learning , pages 6874–6883. PMLR, 2020

    2020

  4. [12]

    A modern maximum-likelihood theory for high- dimensional logistic regression

    Pragya Sur and Emmanuel J Cand` es. A modern maximum-likelihood theory for high- dimensional logistic regression. Proceedings of the National Academy of Sciences , 116(29):14516–14525, 2019

    2019

  5. [13]

    Realistic evaluation of deep semi-supervised learning algorithms

    Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Good- fellow. Realistic evaluation of deep semi-supervised learning algorithms. Advances in neural information processing systems , 31, 2018

    2018

  6. [14]

    Machine learning applications in genetics and genomics

    Maxwell W Libbrecht and William Stafford Noble. Machine learning applications in genetics and genomics. Nature Reviews Genetics , 16(6):321–332, 2015

    2015

  7. [15]

    Reconciling modern machine-learning practice and the classical bias–variance trade-off

    Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences , 116(32):15849–15854, 2019

    2019

  8. [16]

    The generalization error of random features regres- sion: Precise asymptotics and the double descent curve

    Song Mei and Andrea Montanari. The generalization error of random features regres- sion: Precise asymptotics and the double descent curve. Communications on Pure and Applied Mathematics , 75(4):667–766, 2022

    2022

  9. [17]

    Gaussian universality of linear classifiers with random labels in high- dimension

    Federica Gerace, Florent Krzakala, Bruno Loureiro, Ludovic Stephan, and Lenka Zdeborov´ a. Gaussian universality of linear classifiers with random labels in high- dimension. arXiv preprint arXiv:2205.13303 , 2022. 44

    2022 arXiv

  10. [18]

    Universality of empirical risk minimization

    Andrea Montanari and Basil N Saeed. Universality of empirical risk minimization. In Conference on Learning Theory, pages 4310–4312. PMLR, 2022

    2022

  11. [19]

    Non-adaptive pooling strategies for detection of rare faulty items

    Pan Zhang, Florent Krzakala, Marc M´ ezard, and Lenka Zdeborov´ a. Non-adaptive pooling strategies for detection of rare faulty items. In 2013 IEEE International Conference on Communications Workshops (ICC) , pages 1409–1414. IEEE, 2013

    2013

  12. [20]

    Clustering from sparse pairwise measurements

    Alaa Saade, Marc Lelarge, Florent Krzakala, and Lenka Zdeborov´ a. Clustering from sparse pairwise measurements. In2016 IEEE International Symposium on Information Theory (ISIT), pages 780–784. IEEE, 2016

    2016

  13. [21]

    Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex opti- mization

    Benjamin Aubin, Florent Krzakala, Yue Lu, and Lenka Zdeborov´ a. Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex opti- mization. Advances in Neural Information Processing Systems, 33:12199–12210, 2020

    2020

  14. [22]

    Optimal M-estimation in high-dimensional regression

    Derek Bean, Peter J Bickel, Noureddine El Karoui, and Bin Yu. Optimal M-estimation in high-dimensional regression. Proceedings of the National Academy of Sciences , 110(36):14563–14568, 2013

    2013

  15. [23]

    High dimensional robust M-estimation: Asymptotic variance via approximate message passing

    David Donoho and Andrea Montanari. High dimensional robust M-estimation: Asymptotic variance via approximate message passing. Probability Theory and Re- lated Fields, 166:935–969, 2016

    2016

  16. [24]

    An equivalence between high dimensional Bayes optimal inference and M-estimation

    Madhu Advani and Surya Ganguli. An equivalence between high dimensional Bayes optimal inference and M-estimation. Advances in Neural Information Processing Sys- tems, 29, 2016

    2016

  17. [25]

    Precise error analysis of regularized M-estimators in high dimensions

    Christos Thrampoulidis, Ehsan Abbasi, and Babak Hassibi. Precise error analysis of regularized M-estimators in high dimensions. IEEE Transactions on Information Theory, 64(8):5592–5628, 2018

    2018

  18. [26]

    Graphical models concepts in com- pressed sensing

    Andrea Montanari, YC Eldar, and G Kutyniok. Graphical models concepts in com- pressed sensing. Compressed Sensing, pages 394–438, 2012

    2012

  19. [27]

    Message-passing algo- rithms for compressed sensing

    David L Donoho, Arian Maleki, and Andrea Montanari. Message-passing algo- rithms for compressed sensing. Proceedings of the National Academy of Sciences , 106(45):18914–18919, 2009

    2009

  20. [28]

    Message passing algorithms for compressed sensing: I I

    David L Donoho, Arian Maleki, and Andrea Montanari. Message passing algorithms for compressed sensing: I I. Analysis and validation. In2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo) , pages 1–5. IEEE, 2010

    2010

  21. [29]

    The dynamics of message passing on dense graphs, with applications to compressed sensing

    Mohsen Bayati and Andrea Montanari. The dynamics of message passing on dense graphs, with applications to compressed sensing. IEEE Transactions on Information Theory, 57(2):764–785, 2011. 45

    2011

  22. [30]

    Low-rank matrix reconstruction and clus- tering via approximate message passing

    Ryosuke Matsushita and Toshiyuki Tanaka. Low-rank matrix reconstruction and clus- tering via approximate message passing. Advances in Neural Information Processing Systems, 26, 2013

    2013

  23. [31]

    Information-theoretically optimal sparse PCA

    Yash Deshpande and Andrea Montanari. Information-theoretically optimal sparse PCA. In 2014 IEEE International Symposium on Information Theory , pages 2197–

    2014

  24. [32]

    The estimation error of general first order methods

    Michael Celentano, Andrea Montanari, and Yuchen Wu. The estimation error of general first order methods. In Conference on Learning Theory , pages 1078–1141. PMLR, 2020

    2020

  25. [33]

    Understanding belief prop- agation and its generalizations

    Jonathan S Yedidia, William T Freeman, Yair Weiss, et al. Understanding belief prop- agation and its generalizations. Exploring artificial intelligence in the new millennium , 8(236-239):0018–9448, 2003

    2003

  26. [34]

    A CDMA multiuser detection algorithm on the basis of belief propagation

    Yoshiyuki Kabashima. A CDMA multiuser detection algorithm on the basis of belief propagation. Journal of Physics A: Mathematical and General , 36(43):11111, 2003

    2003

  27. [35]

    Approximate belief propagation, density evo- lution, and statistical neurodynamics for CDMA multiuser detection

    Toshiyuki Tanaka and Masato Okada. Approximate belief propagation, density evo- lution, and statistical neurodynamics for CDMA multiuser detection. IEEE Transac- tions on Information Theory , 51(2):700–706, 2005

    2005

  28. [36]

    P. W. Anderson D. J. Thouless and R. G. Palmer. Solution of ‘Solvable model of a spin glass’. The Philosophical Magazine: A Journal of Theoretical Experimental and Applied Physics, 35(3):593–601, 1977

    1977

  29. [37]

    Spin glass theory and be- yond: An Introduction to the Replica Method and Its Applications , volume 9

    Marc M´ ezard, Giorgio Parisi, and Miguel Angel Virasoro. Spin glass theory and be- yond: An Introduction to the Replica Method and Its Applications , volume 9. World Scientific Publishing Company, 1987

    1987

  30. [38]

    Design of capacity-approaching irregular low-density parity-check codes

    Thomas J Richardson, Mohammad Amin Shokrollahi, and R¨ udiger L Urbanke. Design of capacity-approaching irregular low-density parity-check codes. IEEE transactions on information theory , 47(2):619–637, 2001

    2001

  31. [39]

    Statistical mechanics of the maximum-likelihood density estimation

    N Barkai and Haim Sompolinsky. Statistical mechanics of the maximum-likelihood density estimation. Physical Review E , 50(3):1766, 1994

    1994

  32. [40]

    Phase transitions and optimal algorithms in high-dimensional Gaus- sian mixture clustering

    Thibault Lesieur, Caterina De Bacco, Jess Banks, Florent Krzakala, Cris Moore, and Lenka Zdeborov´ a. Phase transitions and optimal algorithms in high-dimensional Gaus- sian mixture clustering. In 2016 54th Annual Allerton Conference on Communication, Control, and Computing (A...

    2016

  33. [41]

    Theoretical in- sights into multiclass classification: A high-dimensional asymptotic view

    Christos Thrampoulidis, Samet Oymak, and Mahdi Soltanolkotabi. Theoretical in- sights into multiclass classification: A high-dimensional asymptotic view. Advances in Neural Information Processing Systems , 33:8907–8920, 2020. 46

    2020

  34. [42]

    Learning Gaussian mixtures with generalized linear mod- els: Precise asymptotics in high-dimensions

    Bruno Loureiro, Gabriele Sicuro, C´ edric Gerbelot, Alessandro Pacco, Florent Krza- kala, and Lenka Zdeborov´ a. Learning Gaussian mixtures with generalized linear mod- els: Precise asymptotics in high-dimensions. Advances in Neural Information Process- ing Systems , 34:10144–...

    2021

  35. [43]

    The role of pseudo-labels in self-training linear classifiers on high- dimensional Gaussian mixture data

    Takashi Takahashi. The role of pseudo-labels in self-training linear classifiers on high- dimensional Gaussian mixture data. arXiv preprint arXiv:2205.07739 , 2024

    2024 arXiv

  36. [44]

    An introduction to variational methods for graphical models

    Michael I Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul. An introduction to variational methods for graphical models. Machine learning, 37:183– 233, 1999

    1999

  37. [45]

    Graphical models, exponential families, and variational inference

    Martin J Wainwright, Michael I Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1(1–2):1– 305, 2008

    2008

  38. [46]

    Factor graphs and the sum-product algorithm

    Frank R Kschischang, Brendan J Frey, and H-A Loeliger. Factor graphs and the sum-product algorithm. IEEE Transactions on information theory , 47(2):498–519, 2001

    2001

  39. [47]

    Statistical mechanics and phase transitions in clustering

    Kenneth Rose, Eitan Gurewitz, and Geoffrey C Fox. Statistical mechanics and phase transitions in clustering. Physical review letters , 65(8):945, 1990

    1990

  40. [48]

    Statistical mechanics of unsupervised learning

    M Biehl and A Mietzner. Statistical mechanics of unsupervised learning. Europhysics Letters, 24(5):421, 1993

    1993

  41. [49]

    Scaling laws in learning of classification tasks

    N Barkai, Hyunjune Sebastian Seung, and Haim Sompolinsky. Scaling laws in learning of classification tasks. Physical review letters , 70(20):3167, 1993

    1993

  42. [50]

    Optimal unsupervised learning

    TLH Watkin and J-P Nadal. Optimal unsupervised learning. Journal of Physics A: Mathematical and General , 27(6):1899, 1994

    1994

  43. [51]

    The Nishimori line and Bayesian statistics

    Yukito Iba. The Nishimori line and Bayesian statistics. Journal of Physics A: Math- ematical and General , 32(21):3875, 1999

    1999

  44. [52]

    Statistical physics of spin glasses and information processing: an introduction

    Hidetoshi Nishimori. Statistical physics of spin glasses and information processing: an introduction. Number 111. Clarendon Press, 2001

    2001

  45. [53]

    A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors

    Toshiyuki Tanaka. A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors. IEEE Transactions on Information theory , 48(11):2888–2910, 2002

    2002

  46. [54]

    Generalized approximate message passing for estimation with ran- dom linear mixing

    Sundeep Rangan. Generalized approximate message passing for estimation with ran- dom linear mixing. In 2011 IEEE International Symposium on Information Theory Proceedings, pages 2168–2172. IEEE, 2011. 47

    2011

  47. [55]

    Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis

    Ayaka Sakata and Yingying Xu. Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis. Journal of Statistical Mechanics: Theory and Experiment , 2018(3):033404, 2018

    2018

  48. [56]

    Macroscopic analysis of vector ap- proximate message passing in a model-mismatched setting

    Takashi Takahashi and Yoshiyuki Kabashima. Macroscopic analysis of vector ap- proximate message passing in a model-mismatched setting. IEEE Transactions on Information Theory, 68(8):5579–5600, 2022. 48

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.