{"work":{"id":"ea86ca50-c5fc-4322-bee8-da03ff0e6422","openalex_id":null,"doi":null,"arxiv_id":"1901.06523","raw_key":null,"title":"Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks","authors":null,"authors_text":"Xu, Z","year":2019,"venue":"cs.LG","abstract":"We study the training process of Deep Neural Networks (DNNs) from the Fourier analysis perspective. We demonstrate a very universal Frequency Principle (F-Principle) -- DNNs often fit target functions from low to high frequencies -- on high-dimensional benchmark datasets such as MNIST/CIFAR10 and deep neural networks such as VGG16. This F-Principle of DNNs is opposite to the behavior of most conventional iterative numerical schemes (e.g., Jacobi method), which exhibit faster convergence for higher frequencies for various scientific computing problems. With a simple theory, we illustrate that this F-Principle results from the regularity of the commonly used activation functions. The F-Principle implies an implicit bias that DNNs tend to fit training data by a low-frequency function. This understanding provides an explanation of good generalization of DNNs on most real datasets and bad generalization of DNNs on parity function or randomized dataset.","external_url":"https://arxiv.org/abs/1901.06523","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-07-10T23:47:46.694405+00:00","pith_arxiv_id":"1901.06523","created_at":"2026-05-11T10:51:04.339243+00:00","updated_at":"2026-07-10T23:47:46.694405+00:00","title_quality_ok":true,"display_title":"J., Zhang, Y., Luo, T., Xiao, Y., and Ma, Z","render_title":"J., Zhang, Y., Luo, T., Xiao, Y., and Ma, Z"},"hub":{"state":{"work_id":"ea86ca50-c5fc-4322-bee8-da03ff0e6422","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":24,"external_cited_by_count":null,"distinct_field_count":10,"first_pith_cited_at":"2019-06-21T16:46:04+00:00","last_pith_cited_at":"2026-07-07T18:00:17+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-22T15:49:39.380954+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":5}],"polarity_counts":[{"context_polarity":"background","n":5}],"runs":{},"summary":{},"graph":{},"authors":[]}}