REVIEW 1 cited by
Provable Tempered Overfitting of Minimal Nets and Typical Nets
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We study the overfitting behavior of fully connected deep Neural Networks (NNs) with binary weights fitted to perfectly classify a noisy training set. We consider interpolation using both the smallest NN (having the minimal number of weights) and a random interpolating NN. For both learning rules, we prove overfitting is tempered. Our analysis rests on a new bound on the size of a threshold circuit consistent with a partial function. To the best of our knowledge, ours are the first theoretical results on benign or tempered overfitting that: (1) apply to deep NNs, and (2) do not require a very high or very low input dimension.
Forward citations
Cited by 1 Pith paper
-
A Classical View on Benign Overfitting: The Role of Sample Size
The paper proves high-probability, non-asymptotic bounds showing that kernel ridge regression and two-layer ReLU networks in the NTK regime can achieve both arbitrarily small training and test error without assuming t...
Discussion (0). Continue with ORCID to comment.