REVIEW 2 cited by
A Markov Chain Theory Approach to Characterizing the Minimax Optimality of Stochastic Gradient Descent (for Least Squares)
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This work provides a simplified proof of the statistical minimax optimality of (iterate averaged) stochastic gradient descent (SGD), for the special case of least squares. This result is obtained by analyzing SGD as a stochastic process and by sharply characterizing the stationary covariance matrix of this process. The finite rate optimality characterization captures the constant factors and addresses model mis-specification.
Forward citations
Cited by 2 Pith papers
-
Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
When a cosine schedule would halve the learning rate, Seesaw cuts it by √2 and doubles the batch, matching loss curves with ~36% fewer serial steps.
-
On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks
Excess risk of SGD and ridge regression on GNNs is characterized through graph spectra, showing graph shape decides which algorithm generalizes better and deeper networks amplify the difference.
Discussion (0). Sign in to comment.