How many samples are needed to train a deep neural network?
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:HNBTOLCFrecord.jsonopen to challenge →
read the original abstract
Neural networks have become standard tools in many areas, yet many important statistical questions remain open. This paper studies the question of how much data are needed to train a ReLU feed-forward neural network. Our theoretical and empirical results suggest that the generalization error of ReLU feed-forward neural networks scales at the rate $1/\sqrt{n}$ in the sample size $n$ rather than the usual "parametric rate" $1/n$. Thus, broadly speaking, our results underpin the common belief that neural networks need "many" training samples.
This paper has not been read by Pith yet.
Forward citations
Cited by 2 Pith papers
-
Enhancing Spectral Embedding through Robust and Flexible Knowledge Transfer in Electronic Health Records
A two-step spectral embedding procedure that removes irrelevant components from a knowledge matrix then projects to recover shared and heterogeneous signals for rare-disease clinical concept and patient embeddings.
-
How Many Training Samples Are Needed for the Inverse Kinematics Solutions by Artificial Neural Networks
Empirical study on a robotic manipulator concludes that training sets larger than 125 samples yield no further gains in accuracy or efficiency for feedforward neural network inverse kinematics solvers.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.