REVIEW 2 cited by
AdaOja: Adaptive Learning Rates for Streaming PCA
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Oja's algorithm has been the cornerstone of streaming methods in Principal Component Analysis (PCA) since it was first proposed in 1982. However, Oja's algorithm does not have a standardized choice of learning rate (step size) that both performs well in practice and truly conforms to the online streaming setting. In this paper, we propose a new learning rate scheme for Oja's method called AdaOja. This new algorithm requires only a single pass over the data and does not depend on knowing properties of the data set a priori. AdaOja is a novel variation of the Adagrad algorithm to Oja's algorithm in the single eigenvector case and extended to the multiple eigenvector case. We demonstrate for dense synthetic data, sparse real-world data and dense real-world data that AdaOja outperforms common learning rate choices for Oja's method. We also show that AdaOja performs comparably to state-of-the-art algorithms (History PCA and Streaming Power Method) in the same streaming PCA setting.
Forward citations
Cited by 2 Pith papers
-
The Phase Transition in Online PCA Depends on $n/d\log(d)$, not $n/d$
Oja's algorithm undergoes a sharp phase transition at n ≈ d log d / [δ(2θ²−δ)]: below the threshold the overlap with the planted direction vanishes; above it, it tends to sqrt((θ²−δ/2)/(θ²(1+δ/2))), and exactly at thr...
-
Beyond Sin-Squared Error: Linear-Time Entrywise Uncertainty Quantification for Streaming PCA
Entrywise concentration bounds, a central limit theorem, and a median-of-means variance estimator give linear-time coordinate-wise uncertainty quantification for streaming PCA with Oja's algorithm.
Discussion (0). Continue with ORCID to comment.