Pith. sign in

REVIEW 3 cited by

Why Shallow Networks Struggle to Approximate and Learn High Frequencies

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.17301 v3 pith:R62ZAFG4 submitted 2023-06-29 cs.LG cs.NAmath.NAstat.ML

Shijun Zhang , Hongkai Zhao , Yimin Zhong , Haomin Zhou This is my paper · ORCID
classification cs.LGcs.NAmath.NAstat.ML
keywords computationalnumericalanalysiscostfrequencieshighlearningprecision
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this work, we present a comprehensive study combining mathematical and computational analysis to explain why a two-layer neural network struggles to handle high frequencies in both approximation and learning, especially when machine precision, numerical noise, and computational cost are significant factors in practice. Specifically, we investigate the following fundamental computational issues: (1) the minimal numerical error achievable under finite precision, (2) the computational cost required to attain a given accuracy, and (3) the stability of the method with respect to perturbations. The core of our analysis lies in the conditioning of the representation and its learning dynamics. Explicit answers to these questions are provided, along with supporting numerical evidence.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Second-Order Path Kernel Interpolation Formulas in Machine Learning

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Derives second-order path-kernel interpolation formulas for gradient descent, SGD, and momentum training, adding curvature terms and a concentration estimate around the expected prediction.

  2. Orthogonal greedy algorithm for linear operator learning with shallow neural network

    math.NA 2025-01 conditional novelty 6.0 of 10

    Orthogonal greedy training of shallow ReLU networks is adapted to kernel estimation for linear operators, with stated convergence rates and large accuracy gains over neural operator baselines.

  3. Structured First-Layer Initialization Pre-Training Techniques to Accelerate Training Process Based on $\varepsilon$-Rank

    math.NA 2025-07 conditional novelty 5.0 of 10

    A structured first-layer initialization raises the ε-rank of neuron functions and accelerates training of PINNs and function approximators.

Pith tools