In the proportional high-dimensional regime, stronger backdoor training triggers improve clean accuracy and make attack success non-monotonic for regularized GLMs on Gaussian mixtures, with closed-form proofs for squared loss and fixed-point extensions to convex losses.
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
abstract
Generalized linear models (GLMs) arise in high-dimensional machine learning, statistics, communications and signal processing. In this paper we analyze GLMs when the data matrix is random, as relevant in problems such as compressed sensing, error-correcting codes or benchmark models in neural networks. We evaluate the mutual information (or "free entropy") from which we deduce the Bayes-optimal estimation and generalization errors. Our analysis applies to the high-dimensional limit where both the number of samples and the dimension are large and their ratio is fixed. Non-rigorous predictions for the optimal errors existed for special cases of GLMs, e.g. for the perceptron, in the field of statistical physics based on the so-called replica method. Our present paper rigorously establishes those decades old conjectures and brings forward their algorithmic interpretation in terms of performance of the generalized approximate message-passing algorithm. Furthermore, we tightly characterize, for many learning problems, regions of parameters for which this algorithm achieves the optimal performance, and locate the associated sharp phase transitions separating learnable and non-learnable regions. We believe that this random version of GLMs can serve as a challenging benchmark for multi-purpose algorithms. This paper is divided in two parts that can be read independently: The first part (main part) presents the model and main results, discusses some applications and sketches the main ideas of the proof. The second part (supplementary informations) is much more detailed and provides more examples as well as all the proofs.
verdicts
UNVERDICTED 3representative citing papers
Upper bounds on ultrametric OGPs at levels 1 and 2 for symmetric binary perceptrons are approximately 1.6578 and 1.6219, closely matching the 3rd and 4th lifting-level parametric RDT estimates, supporting conjectures that the algorithmic threshold equals the infinite-level limits of both frameworks.
Tutorial on the standard linear model with an outline of the authors' proof that replica-symmetric formulas for its phase transitions in mutual information and MMSE are exact.
citing papers explorer
-
When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks
In the proportional high-dimensional regime, stronger backdoor training triggers improve clean accuracy and make attack success non-monotonic for regularized GLMs on Gaussian mixtures, with closed-form proofs for squared loss and fixed-point extensions to convex losses.
-
Ultrametric OGP - parametric RDT \emph{symmetric} binary perceptron connection
Upper bounds on ultrametric OGPs at levels 1 and 2 for symmetric binary perceptrons are approximately 1.6578 and 1.6219, closely matching the 3rd and 4th lifting-level parametric RDT estimates, supporting conjectures that the algorithmic threshold equals the infinite-level limits of both frameworks.
-
Understanding Phase Transitions via Mutual Information and MMSE
Tutorial on the standard linear model with an outline of the authors' proof that replica-symmetric formulas for its phase transitions in mutual information and MMSE are exact.