{"id":"17f1b4b8-b491-4169-b633-faeb39cca3b1","arxiv_id":"2504.18207","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Gradient descent in a 1D shallow network acts like a shrinkage operator on the Jacobian's singular values, so the learning rate and number of iterations explicitly set the spectral bandwidth for monotonic activations.","lead":"This paper studies how gradient descent controls which frequency components a one-layer neural network learns, by viewing training as a shrinking of the network's singular values. It proposes a formula linking learning rate and iteration count to the number of frequency components kept, and shows which activation functions respond to this kind of control.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The q–K relation in Proposition 4.2 is derived from the gradient-flow mask (Eq. 15), not the exact GD mask (Eq. 29); the two disagree substantially for small K, so the headline iteration-count formula is an unquantified approximation.","rationale":"Read in good faith, the exact GD derivation in Appendix B (Eq. 29) is standard and correct: for a linear least-squares problem with w0=0, w_q = V diag((1-(1-αs_i^2)^q)/s_i) U^T y, so GD does act as a singular-value mask. That part is solid. The load-bearing step is the identification of this mask with the exponential gradient-flow mask in Proposition 4.1 and the q–K formula derived from it. Because the exponential form is not the GD dynamics but a limit, the explicit relationship in Proposition 4.2 is not a proposition in the exact sense. The quantitative check shows the discrepancy can be large when s_K is not much smaller than s_max, i.e., precisely when only a few principal components are retained. This is a correctness and rigor defect in the central claim, but it does not destroy the qualitative picture: for large q and small s_K^2/s_max^2, the masks nearly coincide, and the empirical curves may be reproducible. The paper would be acceptable conditionally, after replacing Eq. 17 with the exact expression (or a bounded approximation) and rerunning and adjusting the experiments and the Fig. 6 caption.","tokens_in":20435,"tokens_out":9053,"duration_ms":84820,"concrete_test":"Re-run the Fig. 6(a) reconstruction using exact GD with α=s_max^{-2} and q_exact(K)=-1/log(1-s_K^2/s_max^2) for each K, and compare PSNR against the DST baseline and against the published curves using Eq. 17. Also tabulate q_exact versus Eq. 17 for K=1,2,5,10,50. If all PSNR differences stay below 1 dB and q ratios remain within 10%, the approximation is harmless; if not, the claimed explicit q–K law does not describe actual GD in the tested regime.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix B derives the exact GD mask m_exact(s;α,q)=1-(1-αs^2)^q (Eq. 29), then switches to the gradient-flow mask m_gf=1-exp(-αqs^2) (Eq. 15) and calls it a 'near perfect approximation.' Proposition 4.1 states Eq. 15 as the GD shrinkage operator, and Proposition 4.2 uses it to obtain q=s_max^2 s_K^{-2} (Eq. 17). The exact condition for the mask to equal 1-e^{-1} at s=s_K is q_exact=-1/log(1-s_K^2/s_max^2), not Eq. 17. For s_K/s_max=0.8, q_exact=0.956 while Eq. 17 gives 1.5625 (≈63% error); for 0.5, q_exact=3.48 vs 4 (13% error). The approximation is good only when s_K^2<<s_max^2 (large K), and no bound is given for the small-K regime where the paper's Fig. 6 actually sweeps. The text and caption of Fig. 6(a) further write q=s_max s_K^{-1}, inconsistent with Eq. 17, compounding uncertainty about what was run. Thus the central explicit relationship between hyperparameters and bandwidth is approximate exactly where the paper needs it to be exact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript studies spectral bias in one-dimensional shallow networks f(x) = Σ_m w_m η(x − b_m) trained with full-batch gradient descent (GD) on a least-squares objective. Three main claims are made. First (Proposition 3.1), minimizing an r-th order gradient penalty as in spline smoothing is asserted to be equivalent to ridge regression on the weights when the activation is the truncated power function η(x) = x^{r−1}[x > 0]. Second (Propositions 4.1 and 4.2), the implicit regularization of GD is reinterpreted as an explicit shrinkage operator on the singular values of the design matrix A, with masking function m(s; α, q) = 1 − exp(−α q s²) and an explicit relationship q = s_max² s_K^{−2} between the iteration count q, the learning rate α = s_max^{−2}, and the number K of retained principal components. Third, experiments on a row of the peppers image (N = 256, M = 1024) show that tuning q controls bandwidth for monotonic activations (tanh, ReLU) in agreement with a discrete sine transform (DST) baseline, whereas non-monotonic activations (sinc, Gaussian) are insensitive to q and are instead controlled by their scaling σ, consistent with Shannon–Whittaker sampling. The paper concludes that GD-based spectral-bias control works only for monotonic activations and promotes sinc as an iteration-efficient surrogate.","tokens_in":20680,"tokens_out":25153,"duration_ms":229340,"significance":"The paper's strengths are its transparent SVD-based setup, the correct derivation of the exact GD iterate in Appendix B, the falsifiable experimental design that compares activation families against a DST baseline, and a genuinely useful qualitative dichotomy: monotonic activations have a steep initial singular-value falloff and respond to (α, q) tuning, whereas sinc/Gaussian have a flat spectrum up to a scale-dependent cutoff and are regularized by σ. The validation is not circular, since the masking function is derived rather than fitted, and the sinc/bandwidth connection (Eq. 19) is a nice conceptual bridge. If the quantitative claims are repaired, the paper would provide practical guidance for bandwidth selection in coordinate MLPs. However, the headline relation in Proposition 4.2 is derived from a gradient-flow approximation rather than from the exact GD mask, no error bound is given, and the experiments run a different iteration schedule from the one derived; as stated, the explicit q–K formula is therefore not established. These are local, fixable defects in an otherwise defensible central argument, so revision is appropriate.","major_comments":[{"comment":"Proposition 4.1 (Eq. 15) states the masking function as m_gd(s; α, q) = 1 − exp(−α q s²), but the exact GD iterate derived in Appendix B, Eq. 29, yields m_exact(s; α, q) = 1 − (1 − α s²)^q. Eq. 15 is the gradient-flow mask, not the GD mask, and the assertion in Appendix B that gradient flow is a 'near perfect approximation' is unquantified and fails at the top of the spectrum: with α = s_max^{−2}, αs² = 1 at the largest singular value, where the exact mask reaches 1 after one iteration while the exponential gives 1 − e^{−q}. Because Proposition 4.2 is built on Eq. 15, the headline relation q = s_max² s_K^{−2} (Eq. 17) is itself approximate. The exact threshold condition m_exact(s_K) = 1 − e^{−1} gives q = −1/log(1 − s_K²/s_max²), which differs from Eq. 17 by about 15% at s_K/s_max = 0.5 (3.48 vs 4) and about 60% at 0.8 (0.98 vs 1.56), where a sub-unit q shows that the threshold interpretation breaks down in the small-K regime that Figure 6 actually sweeps. The approximation is accurate only when s_K² ≪ s_max², and no bound or numerical comparison is supplied. Since this is the paper's central explicit contribution, Proposition 4.2 should be re-derived from the exact mask or accompanied by a quantified error estimate over the relevant range of K; the qualitative q-monotonic control of bandwidth in Figure 6 would survive either way, but the stated formula would not.","section":"§4 (Prop. 4.1–4.2); Appendix B"},{"comment":"Section 6 and the caption of Figure 6(a) state that the experiments set q = s_max·s_K^{−1}, whereas Proposition 4.2/Eq. 17 states q = s_max²·s_K^{−2}; these schedules differ by a square and cannot both describe what was run. The manuscript must state which schedule was used in the reconstruction experiments and reconcile the two formulas; as it stands, the claim that Figure 6(a) validates Proposition 4.2 is not verifiable. Relatedly, the statement in Section 4 that the active-window width satisfies ρ = s_max·s_K^{−1} = √q is consistent with Eq. 17 but not with the Figure 6(a) schedule, under which the same quantity would equal q rather than √q.","section":"§6, Fig. 6(a)"},{"comment":"Proposition 3.1 asserts an equivalence between the spline objective in Eq. 3 and the ridge-penalized basis expansion in Eq. 4, and the text promises that 'a full derivation ... can be found in Appendix A.' Appendix A in fact contains only an empirical visualization of the columns of ∇^{−r}; no derivation of the equivalence is given. The statement is plausible and can be proved via the Green's function identity d^r/dx^r (x − t)_+^{r−1} = (r−1)!·δ(x − t), but the manuscript does not provide this argument, does not account for the (r−1)! constant that is silently absorbed into λ, and does not discuss how the boundary conditions of the discrete operator ∇ (the 'same' convolution) affect the claimed equivalence. The appendix description should be corrected and the proof supplied, or the proposition should be explicitly downgraded to an empirically supported observation.","section":"§3 (Prop. 3.1); Appendix A"}],"minor_comments":[{"comment":"In the first paragraph of Section 6 the SVD spectrum is attributed to Figure 4, but the spectrum figure is Figure 3; the principal-component figures are Figures 4 and 5.","section":"§6"},{"comment":"In the same section, the falloff rate is stated for 'η(x) = x^{r−1}·[x > 1]', which should read '[x > 0]' to agree with Proposition 3.1.","section":"§6"},{"comment":"In the Key Insights list, the summary relation q ≈ α^{−2}·s²_B is dimensionally inconsistent and disagrees with Eq. 17; from Eq. 17 and α = s_max^{−2} the correct form is q ≈ α^{−1}·s_B^{−2}.","section":"§7"},{"comment":"Equation (36) gives ρ(ε) = √(q/(−log ε)) with ρ defined as s(ε)/s_max, but from Eq. 35 the correct expression is ρ(ε) = √(−log ε / q); the claimed ρ ∝ √q follows only under the main-text definition ρ = s_max/s(ε), so the two definitions should be aligned.","section":"Appendix C"},{"comment":"There are several naming and reference slips: 'Sitzman' should be 'Sitzmann', 'Whitaker' should be 'Whittaker', 'adhoc' should be 'ad hoc', 'explicitly effect spectral bias' should be 'explicitly affect spectral bias', reference [13] lacks a year, and reference [26] lacks its publication venue.","section":"§1, References"},{"comment":"The caption of Figure 2 says α = 1 'ensures the upper singular value of the mask is normalized at unity', but m(1; 1, q) = 1 − exp(−q), which equals unity only as q → ∞; the caption should be reworded.","section":"§4"},{"comment":"Proposition 4.2 is stated with the specific threshold ε = exp(−1), but any ε ∈ (0,1) is allowed in Eq. 16, so the 'explicit' q–K relation is determined only up to the constant factor −log ε; since the experiments use a different q schedule, they cannot pin down this constant, and the agreement with the DST baseline in Figure 6 should be described as qualitative.","section":"§4"},{"comment":"The derivation in Appendix B restates standard results on implicit regularization of GD and early stopping for least squares; the paper should cite that literature (e.g., Ali, Kolter, and Tibshirani (2019) and Yao, Rosasco, and Caponnetto (2007)) and soften the claim of being 'the first, to our knowledge, to make explicit such a relationship' accordingly.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the manuscript overlaps substantially with the early-stopping/implicit-regularization literature for least squares, where both the exact GD iterate (I − (I − αA^T A)^q)(A^T A)^{−1}A^T y and its gradient-flow approximation are standard; the claim to be 'the first ... to make explicit such a relationship' should be re-scoped after citing that work. The genuinely new elements are the activation-specific spectral analysis, the monotonic/non-monotonic dichotomy, and the sinc/bandwidth connection. The central issues are technical and fixable: the q–K formula should be re-derived from the exact mask, the Figure 6(a) schedule must be reconciled with the stated formula, and the missing derivation behind Proposition 3.1 should be supplied. No other concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take on arXiv:2504.18207. The paper has a genuinely useful idea: viewing GD's implicit regularization as a shrinkage mask on the Jacobian's singular values, with an explicit formula q = s_max^2 / s_K^2 linking iterations to active components. That formula will get cited, and the distinction between monotonic activations (where iterations/learning rate control bandwidth) and non-monotonic ones (where scaling does) is a real insight. The generalization of the spline-activation connection to arbitrary order r is also new, as far as I know.\n\nThe problems are concentrated in the derivation-proposition match. Appendix B correctly derives the exact GD mask as m_exact = 1 - (1 - alpha s^2)^q, but Proposition 4.1 states the gradient-flow mask m_gf = 1 - exp(-alpha q s^2) as “the implicit regularization of GD.” The paper calls the latter a “near perfect approximation” without a bound. It isn't near-perfect in the regime they actually use: for s_K/s_max = 0.8, the exact q needed is ~0.979, while Eq. 17 gives 1.56—over 50% error. For 0.5 it's 3.48 vs 4, about 15%. So the headline formula is an unquantified approximation just where the experiments live. That needs a bound or a switch to the exact formula.\n\nAlso, Fig. 6(a) writes q = s_max * s_K^{-1} in the caption, which is inconsistent with Eq. 17 (q = s_max^2 * s_K^{-2}). This is either a typo or the experiments used a different relation; either way it needs fixing. And Proposition 3.1 is called a proposition but Appendix A only validates it empirically, no proof. That's an overstatement.\n\nThe experiments are thin—one row of an image, no error bars, no code—but they're enough to illustrate the claimed effect. The spectral plots of activations are useful. The citation pattern is fine; the self-citations are to the coordinate-MLP work this builds on.\n\nOverall: the core insight is plausible and worth engaging. I'd send this to review, with instructions to fix the GD-vs-gradient-flow labeling, provide a quantitative comparison or the exact formula, resolve the Fig. 6 inconsistency, and soften Proposition 3.1. Conditional accept, not a reject.","headline":"A useful heuristic—GD as a singular-value mask with an explicit q–K formula—but the headline formula comes from gradient flow, not exact GD, and the difference is unquantified where it matters.","tokens_in":21236,"tokens_out":3343,"would_cite":true,"duration_ms":30335,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Gradient descent implicitly performs singular-value shrinkage whose learning rate and iteration count directly dial the spectral bandwidth of a shallow network.","keywords":["spectral bias","gradient descent","shrinkage operator","singular value masking","shallow neural networks","activation functions","spline smoothing","bandwidth selection"],"falsifier":"Run exact discrete gradient descent on the paper's peppers reconstruction (shallow net, $\\alpha=s_{\\max}^{-2}$, $\\sigma=1024$) and sweep $q$; compare the $q$ at which the PSNR curve crosses the discrete-sine-transform baseline for $K$ components against $q=s_{\\max}^{2}s_{K}^{-2}$ from the paper. If the crossing point is systematically off by about a factor of two or more in $q$, the exponential-mask relation fails as a quantitative predictor; the exact discrete-GD mask $1-(1-\\alpha s^2)^q$ should be used to compute the predicted crossing instead.","tokens_in":20138,"feed_emoji":"📉","tokens_out":10123,"duration_ms":88962,"temperature":0.7,"pith_summary":"This paper argues that when a 1D shallow network is trained with gradient descent on least squares, the optimization itself quietly regularizes the solution: gradient descent acts like a shrinkage operator that masks the singular values of the network's Jacobian. The paper derives an explicit mask, $m_{\\mathrm{gd}}(s;\\alpha,q)=1-\\exp(-\\alpha q s^2)$, and with it an explicit formula $q=s_{\\max}^{2}s_{K}^{-2}$ linking the number of iterations to the number $K$ of active principal components, i.e., to the bandwidth of the learned function. It then connects activation functions to spline smoothing, showing that $\\eta(x)=x^{r-1}[x>0]$ corresponds to penalizing the $r$-th derivative; monotonic activations inherit fast singular-value drop-offs, their principal components approximate the discrete sine transform, and GD regularization is effective. Non-monotonic activations such as sinc and Gaussian have a different spectral profile and must be regularized by scaling rather than by iteration count. If correct, this gives practitioners a principled way to choose learning rate and iteration count to set the spectral bias of a shallow network.","feed_headline":"A simple formula ties gradient descent runs to frequency cutoffs","feed_subtitle":"Learning rate and iteration count act as a singular-value mask that sets which frequencies a shallow net learns.","key_machinery":"The load-bearing object is the masking function $m_{\\mathrm{gd}}(s;\\alpha,q)=1-\\exp(-\\alpha q s^2)$, applied to the singular values $s$ of the design matrix $A=[\\eta(\\hat{x}_i-b_j)]$, alongside its companion relation $q=s_{\\max}^{2}s_{K}^{-2}$. The mask converts gradient descent from an iterative procedure into a closed-form shrinkage operator: singular values above roughly $s_K$ pass nearly unchanged, values below are suppressed, and the active window has width $\\rho\\approx\\sqrt{q}$. The second piece is the SVD of $A$ for different activations: for monotonic activations, the principal components approximate the discrete sine transform and the singular-value drop-off rate is set by $r$ in $\\eta(x)=x^{r-1}[x>0]$, which the paper derives from spline smoothing; for non-monotonic activations the spectrum is controlled by the scale $\\sigma$, with sinc's singular values staying near unity up to $k=\\sigma$.","core_discovery":"The central discovery is that the implicit regularization of gradient descent for least squares can be rewritten as an explicit shrinkage of the singular values $s$ of the design matrix $A$ (the Jacobian of the shallow network). For learning rate $\\alpha$ and $q$ iterations, the effective inverse singular values are $\\hat{s}^{-1}=[1-\\exp(-\\alpha q s^2)]s^{-1}$, so the mask $m_{\\mathrm{gd}}$ is a smooth low-pass filter whose active window widens like $\\sqrt{q}$. Setting $\\alpha=s_{\\max}^{-2}$ and reading the mask at threshold $\\epsilon=e^{-1}$ gives $\\kappa=\\sqrt{-(\\log\\epsilon)/(\\alpha q)}$ and hence $q=s_{\\max}^{2}s_{K}^{-2}$: the number of iterations needed to keep the first $K$ principal components, which for monotonic activations are approximately discrete-sine-transform frequency bases, so $K$ is essentially the bandwidth. The paper further claims that activation choice determines how effective this mechanism is: monotonic activations ($\\eta(x)=x^{r-1}[x>0]$) correspond to spline smoothing of order $r$, their singular spectra drop steeply, and GD can select components, whereas non-monotonic activations such as sinc and Gaussian have a flat-then-steep spectrum controlled by scaling $\\sigma$, so scaling rather than iterations sets their bandwidth.","pith_inferences":["[Editorial inference] The relation $q=s_{\\max}^{2}s_{K}^{-2}$ can be inverted into a bandwidth curriculum: increasing $q$ over training would let a monotonic network fit low frequencies first and then admit higher frequencies, a training schedule the paper does not discuss.","[Editorial inference] The same singular-value-mask reasoning could be applied layer by layer in deep networks; measuring per-layer Jacobian spectra during training would test whether the shallow-network picture of spectral bias survives depth.","[Editorial inference] The paper's 'grey zone' -- the slow transition of the GD mask -- predicts leakage of higher-frequency content when singular values around $s_K$ are closely spaced; the exact discrete-GD mask $1-(1-\\alpha s^2)^q$ could turn that leakage into a quantitative bound and a sharper predictor than the exponential approximation.","[Editorial inference] For non-monotonic activations, the paper's $\\sigma=K$ rule suggests that adapting $\\sigma$ per sample or per epoch could act as a data-dependent low-pass filter in coordinate networks, a testable design not explored in the paper."],"forward_implications":["For monotonic activations, choosing $q=s_{\\max}^{2}s_{K}^{-2}$ (with $\\alpha=s_{\\max}^{-2}$) selects $K$ active principal components in advance, making spectral bias a tunable design choice rather than an accident of training.","The spline connection sets the order of implicit smoothing by activation: Heaviside gives first-order, ReLU second-order, and ReLU$^2$ third-order derivative penalties, with higher $r$ making GD regularization increasingly iteration-expensive.","GD regularizes monotonic activations well and non-monotonic activations poorly: for sinc and Gaussian, learning rate and iteration count barely affect bandwidth, and the activation scale $\\sigma$ (with $\\sigma\\approx K$) is the controlling hyperparameter.","A suitably regularized monotonic network matches a discrete-sine-transform band-limited reconstruction, and a sinc network with $\\sigma=K$ and a modest fixed $q$ matches that baseline almost exactly, making sinc an iteration-efficient surrogate for spectral bias.","For $r=1$ activations (Heaviside, tanh) the iteration counts required to keep a given number of components are orders of magnitude smaller than for ReLU-family activations, directly affecting training cost."],"supporting_citations":[{"why":"Supplies the neural tangent kernel / gradient-flow approximation used to derive the exponential masking function in Proposition 4.1.","marker":"[10]"},{"why":"Establishes the ReLU-as-spline-smoothing connection (second-order case) that Proposition 3.1 generalizes to arbitrary order r.","marker":"[24]"},{"why":"Independently connects ReLU implicit regularization to spline smoothing in the 1-D two-layer case, the prior result Proposition 3.1 extends.","marker":"[8]"},{"why":"Provides the general bridge between parameter regularization and function smoothing that motivates the spline reformulation, and links Gaussian scaling to bandwidth.","marker":"[25]"},{"why":"Supplies the Shannon-Whittaker sampling theorem framing, making sinc the band-limited interpolation baseline and justifying sigma=K as bandwidth.","marker":"[17]"},{"why":"Defines PCA regression / hard-threshold shrinkage on principal components, the comparison benchmark for judging GD's masking behavior.","marker":"[11]"},{"why":"Supplies the sampling-theory perspective on sinc activations for implicit neural representations, the empirical baseline the paper positions sinc against.","marker":"[16]"}],"fun_headline_variants":["GD masks singular values to set frequency cutoff","Gradient descent acts as low-pass filter on neural Jacobian","Iteration count sets the bandwidth via singular-value shrinkage","Learning rate and epochs act as spectral mask for shallow nets","Non-monotonic activations make iterations tune bandwidth faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's iteration-to-bandwidth formula is derived from the continuous-time (gradient-flow) approximation of gradient descent, whose mask is $1-\\exp(-\\alpha q s^2)$; discrete gradient descent actually has mask $1-(1-\\alpha s^2)^q$, and if the 'near perfect' approximation fails at the learning rates and iteration counts used in practice, then the predicted iteration counts will not deliver the claimed number of active frequency components.","fun_headline_variants_meta":{"raw":{"variants":["GD masks singular values to set frequency cutoff","Gradient descent acts as low-pass filter on neural Jacobian","Iteration count sets the bandwidth via singular-value shrinkage","Learning rate and epochs act as spectral mask for shallow nets","Non-monotonic activations make iterations tune bandwidth faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001222,"raw_usage":{"total_tokens":5031,"prompt_tokens":954,"completion_tokens":4077,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":3999}},"tokens_in":570,"tokens_out":4077,"duration_ms":27097,"temperature":1.0,"reasoning_tokens":3999,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:21:15.072941+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run exact discrete gradient descent on the paper's peppers reconstruction (shallow net, $\\alpha=s_{\\max}^{-2}$, $\\sigma=1024$) and sweep $q$; compare the $q$ at which the PSNR curve crosses the discrete-sine-transform baseline for $K$ components against $q=s_{\\max}^{2}s_{K}^{-2}$ from the paper. If the crossing point is systematically off by about a factor of two or more in $q$, the exponential-mask relation fails as a quantitative predictor; the exact discrete-GD mask $1-(1-\\alpha s^2)^q$ should be used to compute the predicted crossing instead.","supporting_citations":[{"cited_title":"Neu- ral tangent kernel: Convergence and generalization in neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the neural tangent kernel / gradient-flow approximation used to derive the exponential masking function in Proposition 4.1."},{"cited_title":"Gradient dynamics of shallow univariate relu networks","cited_arxiv_id":null,"evidence_quote":"Establishes the ReLU-as-spline-smoothing connection (second-order case) that Proposition 3.1 generalizes to arbitrary order r."},{"cited_title":"A mathematical analysis of the motion coherence theory","cited_arxiv_id":null,"evidence_quote":"Provides the general bridge between parameter regularization and function smoothing that motivates the spline reformulation, and links Gaussian scaling to bandwidth."},{"cited_title":"Communication in the presence of noise","cited_arxiv_id":null,"evidence_quote":"Supplies the Shannon-Whittaker sampling theorem framing, making sinc the band-limited interpolation baseline and justifying sigma=K as bandwidth."},{"cited_title":"A note on the use of principal components in regression","cited_arxiv_id":null,"evidence_quote":"Defines PCA regression / hard-threshold shrinkage on principal components, the comparison benchmark for judging GD's masking behavior."},{"cited_title":"A sam- pling theory perspective on activations for implicit neural representations","cited_arxiv_id":null,"evidence_quote":"Supplies the sampling-theory perspective on sinc activations for implicit neural representations, the empirical baseline the paper positions sinc against."}],"review_version":1}