{"id":"82e45b05-3ac0-4c8e-a5b8-3ffe9f8d1966","arxiv_id":"1908.11140","paper_version":3,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"For regression functions that are locally low-dimensional, sparse neural network estimates achieve the rate n^{-2p/(2p+d*)}, with the exponent depending only on the local dimension d* and not the input dimension d.","lead":"The paper proves that sparse deep neural networks can estimate regression functions at a rate determined by the local dimension of the function, not the full input dimension. This gives a theoretical explanation for why deep networks can escape the curse of dimensionality when the true relationship is locally simple.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Definition 3's P_X-border c5/n makes the 'smooth transitions' motivation vacuous for fixed functions: any positive-width transition region forces boundary mass bounded below, so Theorem 1 excludes the paper's own motivating examples.","rationale":"The reader correctly identifies the P_X-border/boundary-mass assumption in Definition 3 as the weakest point. I agree that this assumption is load-bearing, but I sharpen the concern: it does not merely limit the applicable distributions; it also contradicts the paper's stated goal of allowing smooth transitions between local low-dimensional regimes. For any fixed function with a transition layer of positive width and a distribution with density there, the boundary mass is bounded below by a positive constant, so the theorem's condition fails for large n. Therefore the class of functions for which the curse of dimensionality is circumvented is materially narrower than the introduction and Figure 1 suggest. The theorem's proof is detailed and the rate calculation is coherent; I find no algebraic error. However, because the paper's advertised scope overstates what the formal assumption delivers, the appropriate verdict is a conditional acceptance: the mathematical result appears correct, but the presentation should be revised to state explicitly that the P_X-border condition requires transition regions to have vanishing P_X-measure, and the 'smooth transitions' motivation should be tempered accordingly. This does not undermine the core theorem, but it does affect the central claim's breadth, which is why I recommend moving from an unconditional accept to a conditional accept.","tokens_in":55,"tokens_out":30438,"duration_ms":480066,"concrete_test":"Let X be uniform on [-2,2]^2 and take the Figure 1 regression function with a smooth transition of fixed width w>0 between the two regimes, e.g., f(x)=sin(4x_1) for x_2<-w, f(x)=exp(x_2) for x_2>w, and a fixed smooth interpolation inside |x_2|≤w. Verify analytically that any polytope representation satisfying the squeeze in Definition 3 must have the strip |x_2|≤w contained in ∪_k (P_k)^{δ_k}\\(P_k)_{δ_k}. Consequently P_X of the boundary union is at least c·w>0, so for n>c5/(c w) the assumption P_X-border c5/n fails and Theorem 1 does not apply. Repeating the computation with transition width w_n ~ n^{-α} shows the boundary term scales as n^{-α}; for α<2p/(2p+d*) the stated rate no longer holds, confirming that the boundary-mass condition is the bottleneck.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The rate in Theorem 1 is contingent on the P_X-border condition in Definition 3: P_X(∪_k (P_k)^{δ_k}\\(P_k)_{δ_k}) ≤ c5/n. This is not a mild technicality; it determines which functions are covered. Take a fixed function f that transitions smoothly over a strip of width w > 0 between two distinct d*-dimensional regimes, with X having density bounded below near the strip. Any polytopes P_k and shifts δ_k satisfying the squeeze in Definition 3 must place the entire transition strip inside the union of boundary bands (P_k)^{δ_k}\\(P_k)_{δ_k}; otherwise at points in the strip, f takes intermediate values that no lower/upper combination of the f_k indicators can bound. Hence P_X(boundary) ≥ c·w > 0, contradicting P_X(boundary) ≤ c5/n for all n > c5/(c w). Thus Theorem 1 applies only to functions whose transition sets are P_X-null (or whose mass shrinks with n), not to the fixed-width smooth transitions advertised in Section 1.4 and Figure 1. The proof in Supplement D appears internally sound; the issue is scope: the formal assumption does not deliver the 'smooth transition' flexibility promised in the motivation. A secondary note: the boundary contribution in Supplement D is O(P_X(boundary)), so the proof would also go through with the weaker condition P_X(boundary) ≤ C n^{-2p/(2p+d*)}; the stated c5/n is stronger than necessary, but the gap between motivation and assumption remains for any fixed positive-mass transition region.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies least-squares regression with sparse feedforward neural networks using a smooth squashing activation function. It introduces a notion of \"local dimensionality\" (Definition 3) based on squeezing the regression function between two sums of the form sum_k f_k(x_J_k) 1_{P_k}(x), where the sets P_k are polytopes and the squeeze is performed on shrunken and expanded versions of the polytopes, with a control on the P_X-measure of the intervening boundary bands. Theorem 1 states that, under this assumption with P_X-border c5/n and borders δ_i,k bounded below by c6/n^{c7}, the L2 error of the sparse neural network estimate is bounded by c8 (log n)^3 2^{K1} K2 n^{-2p/(2p+d*)}, so the rate is independent of the input dimension d. The proof in the supplement proceeds through an oracle inequality for sparse networks relative to a MARS/B-spline basis (Theorem 2), approximation of polytope indicators by truncated linear functions, approximation of smooth f_k by tensor-product B-splines, and control of the boundary bands by the probability condition. The paper also contains simulations on locally low-dimensional functions in d=10 and a bike-sharing data experiment, comparing the estimate with fully connected networks, a hierarchical-interaction network, nearest neighbor, RBF, and MARS.","tokens_in":38015,"tokens_out":19726,"duration_ms":208974,"significance":"If the main theorem is correct, the paper makes a useful contribution to the statistical theory of deep networks: it identifies a structural condition on the regression function, rather than on the covariate distribution alone, under which the curse of dimensionality is circumvented and the d*-dimensional rate is achieved. The supplement is a genuine strength: it contains complete proofs of the oracle inequality, covering number bounds, and approximation lemmas for B-splines and MARS basis functions, and the final rate is benchmarked against Stone's minimax rate. The main caveats are that the formal P_X-border condition substantially narrows the \"smooth transition\" class advertised in the introduction, and that one step in the proof of Theorem 2 needs a more careful justification. These issues are addressable without changing the central mathematical claim, but they affect the paper's interpretation and the completeness of the proof as written.","major_comments":[{"comment":"The application of Lemma 8 with \"J = d\" is not justified as written. Lemma 8 approximates linear combinations of products of B-splines over a fixed set of J coordinates, but the basis functions B_i in B^*_{n,M,K1} may have different subsets J1 of coordinates. A single B-spline is not identically 1 on [-A,A], and rewriting a B_i with |J1| < d as a product over d coordinates via the partition-of-unity property would multiply the number of terms by (K+M)^{d-|J1|}, which would destroy the needed bound when d > d*. The intended componentwise argument -- approximate each B_i separately with J=|J1_i| and then embed the resulting networks into the larger class F(L,r,α_n) -- should be stated explicitly. With that fix, the error bound I max|w_i|/n^3 remains valid. This point is load-bearing because Theorem 2 underlies the oracle inequality used to prove Theorem 1.","section":"Supplement C, proof of Theorem 2"},{"comment":"The paper presents Definition 3 as allowing \"smooth transitions\" between local regimes, and Figure 1 advertises a globally smooth function with smooth transitions, but Theorem 1's P_X-border condition excludes fixed-width smooth transitions under a continuous design. Outside the union of boundary bands ∪_k ((P_k)^{δ_k} \\(P_k)_{δ_k}), the squeeze in Definition 3 forces m(x)=∑_k f_k(x_{J_k}) 1_{(P_k)_{δ_k}}(x). Therefore the entire transition zone of a smooth f must be contained in the boundary bands. If a transition strip has width w>0 and P_X has density bounded below on that strip, then P_X(∪_k boundary bands) ≥ c w > 0, contradicting the assumption P_X(∪_k boundary bands) ≤ c5/n for all n > c5/(c w). Thus the theorem covers only functions whose transition zones have P_X-mass O(1/n), not the fixed-width smooth transitions promised in the introduction. The motivation, Figure 1, and the real-data discussion should be qualified accordingly. The proof in Supplement D does show that the boundary contribution is O(P_X(boundary)), so the weaker condition P_X(boundary) ≤ C n^{-2p/(2p+d*)} would suffice for the stated rate, but this does not remove the gap between the formal assumption and the advertised examples.","section":"Section 1.4, Definition 3, and Supplement D"}],"minor_comments":[{"comment":"The displayed formula for γ_{k,1} appears to assume that the first coordinate of a_k is nonzero; for a general a_k with zero first coordinate one should choose any coordinate with a nonzero component, and the resulting γ's still lie in the required polynomial bounds. The text should be corrected to avoid this implicit assumption.","section":"Supplement D, Step 1"},{"comment":"The notation J is used both for the number of B-spline factors in Lemma 8 and for the index subset J1 in the definition of B^*_{n,M,K1}; this makes the phrase \"Lemma 8 with J = d\" very difficult to parse. Please separate these notations and clarify which parameter is being fixed.","section":"Equation (20) and proof of Theorem 2"},{"comment":"The formula for L is printed as \"L = 3K1 + d · (M + 2)−1\", which can be misread. It should be parenthesized as L = 3K1 + d(M+2) − 1, matching Lemma 8.","section":"Theorem 1, architecture formula"},{"comment":"The data set name \"Capital Bike Sharing\" is misspelled as \"Captial Bike Sharing\" in two places. The values in Table 3 would also benefit from a sentence explaining the sample splitting into the 500 training/testing points and the remaining evaluation points.","section":"Section 1.4 / Table 3"}],"recommendation":"major_revision","confidential_remarks":"The central theorem appears sound and the supplement is thorough, but the two issues above -- the unclear application of Lemma 8 and the mismatch between the announced smooth-transition motivation and the P_X-border assumption -- need to be addressed before the paper is ready for acceptance. Neither issue appears to be a fatal technical error, but the proof gap in Theorem 2 must be fixed in writing, and the scope of the main theorem should be stated more honestly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper proves a real rate-of-convergence result for sparse DNNs under a locally low-dimensional structure, with a careful self-contained proof. But the advertised 'smooth transitions' are not actually covered by the theorem's assumptions: the P_X-border condition in Definition 3 effectively forces the transition set to have mass O(1/n), which excludes any fixed f with a positive-width smooth transition.\n\nThe new content is genuine. Definition 3 formalizes local dimensionality via polytopes and squeezed lower/upper sums; Theorem 1 gives n^{-2p/(2p+d*)} (up to logs) independent of the ambient dimension. The proof is substantial: oracle inequalities, covering-number bounds, and approximation lemmas for indicators of polytopes and B-splines, all in the supplement. The DNN-MARS connection for smooth activation functions and a generalized spline basis is also a real extension of Eckle and Schmidt-Hieber's ReLU result. The simulations are consistent with the theory and don't overclaim.\n\nThe soft spot is the one the stress-tester flagged, and I think it is a genuine gap between motivation and theorem. The squeeze condition says that outside the band ∪(P_k)^{δ_k}\\(P_k)_{δ_k}, f must equal the lower/upper representation exactly. So any transition between the values of two pieces has to happen entirely inside the band. If the transition has width w and X has a density bounded below near the boundary, then P_X(band) ≥ c w. Definition 3 requires P_X(band) ≤ c5/n. For any fixed w>0, this fails for n large. In other words, the theorem only covers functions whose transition regions are P_X-small (or shrink with n), not the fixed smooth transitions shown in Figure 1. The proof itself would still work with the weaker condition P_X(band) ≤ C n^{-2p/(2p+d*)}; but that does not rescue the motivating examples, since P_X(band) still has to go to zero.\n\nA secondary issue: the assumption P_X(border) ≤ c5/n is n-dependent, so the function class changes with sample size. That is unusual for a minimax-style statement and deserves a caveat in the paper.\n\nMinor: the real-data experiment has no error bars and no code release; those are light weights, not problems with the theory.\n\nWho for: anyone working on when DNNs beat the curse of dimensionality. It is a legitimate contribution even if the 'smooth transition' framing oversells the domain. I would send it to referees, and would ask them to push on whether the definition can be reworked so the theorem covers a fixed function class rather than an n-dependent one.\n\nRecommendation: deserves a serious referee. Accept with revisions if the authors fix the framing and clarify the boundary condition.","headline":"Solid new rate result for locally low-dimensional DNN regression, but the 'smooth transitions' in the motivation are not actually covered by the theorem's boundary-mass condition.","tokens_in":38514,"tokens_out":7468,"would_cite":true,"duration_ms":77609,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves that least-squares sparse deep neural network estimators can converge at a rate set by the local dimensionality of the regression function, not by the ambient input dimension, when the function is low-dimensional on each…","keywords":["low local dimensionality","curse of dimensionality","deep neural networks","nonparametric regression","rate of convergence","MARS","B-splines","sparse neural networks"],"falsifier":"Take $d=2$ and let $X$ be concentrated in a strip of width $n^{-1/2}$ around the diagonal of $[0,1]^2$, with $m(x)=x_1$ on one side of the diagonal and $m(x)=x_2$ on the other. Then the boundary mass is of order $n^{-1/2}$, not $c_5/n$, so the error term from misclassifying boundary points in the proof becomes $n^{-1/2}$; a simulation measuring the $L_2$ error versus $n$ would show the stated $n^{-2p/(2p+d^*)}$ rate fails for this distribution.","tokens_in":37496,"feed_emoji":"🧠","tokens_out":8081,"duration_ms":79617,"temperature":0.7,"pith_summary":"This paper proves a theorem about when deep neural networks escape the curse of dimensionality. It considers regression functions that are not low-dimensional globally but are low-dimensional on each piece of a partition: on each of finitely many polytope regions the function depends on at most $d^*$ coordinates, with smooth transitions allowed between regions. The main result, Theorem 1, shows that a least-squares sparse neural network estimator achieves expected $L_2$ error bounded by $c_8 (\\log n)^3 2^{K_1} K_2 n^{-2p/(2p+d^*)}$, a rate whose exponent depends on the local dimension $d^*$ and the smoothness $p$, and not on the input dimension $d$. If true, this means deep networks can automatically exploit locally low-dimensional structure and converge much faster than classical nonparametric methods in high-dimensional input spaces. The paper also gives simulations and a bike-sharing data experiment consistent with the theory.","feed_headline":"Deep nets converge at a rate set by local dimension, not input dimension","feed_subtitle":"When a regression is low-dimensional on each region, its error decays like $n^{-2p/(2p+d^*)}$, ignoring the ambient dimension.","key_machinery":"The load-bearing construction is a sparse neural network class: sums of $M^*$ small fully connected sigmoidal networks, with weights bounded by $n^{c_2}$ and the number $M^*$ chosen by sample splitting. The proof works through an oracle inequality (Theorem 2 in the Supplement): up to $(\\log n)^3$, the DNN error is bounded by the best error any linear combination of basis functions from $B^*_{n,M,K_1}$ could achieve, where each basis function is a product of a MARS-style truncated linear power (the hinge functions used in multivariate adaptive regression splines) and a tensor-product B-spline. This basis can exactly represent polytope indicators as differences of truncated linear functions and approximates smooth local pieces by B-splines, so the theorem reduces to a sparse-approximation problem in local dimension $d^*$. Approximation lemmas show each such basis product is representable by a DNN to accuracy $1/n^3$, and a covering-number bound controls the complexity cost $I/n$.","core_discovery":"The central discovery is that low local dimensionality—not just global low intrinsic dimensionality—is enough for deep networks to avoid the curse of dimensionality. Under Definition 3, a function has local dimensionality $d^*$ if it is sandwiched between two sums of the form $\\sum_k f_k(x_{J_k})$ times indicators of polytopes, where each $f_k$ is $(p,C)$-smooth and depends on at most $d^*$ coordinates, and the $P_X$-mass of the boundary strips between the outer and inner polytopes is at most $c_5/n$. Theorem 1 states that the sparse neural network least-squares estimator reaches the $d^*$-dimensional rate $n^{-2p/(2p+d^*)}$ up to logarithmic and constant factors, with an error bound independent of $d$. This extends earlier dimension-reduction results for hierarchical composition models and for data lying on low-dimensional manifolds to a setting where the input distribution itself need not be concentrated on any low-dimensional manifold.","pith_inferences":["Editorial inference: the boundary-mass condition is likely the real price of smoothness; on distributions where transitions carry mass $n^{-a}$ with $a<1$, the theorem's proof suggests the error would degrade, and one can test this by simulating a boundary strip of width $n^{-1/2}$.","Editorial inference: the oracle inequality suggests that data-driven selection of the number of network blocks $M^*$ should adapt to the unknown local structure, so a formal adaptive-rate theorem for unknown $d^*$ is a natural next step.","Editorial inference: because the construction approximates B-splines and truncated hinges, the same dimension-reduction argument should transfer to ReLU activations and to other spline bases, connecting the result to practical architectures."],"forward_implications":["The convergence rate for regression functions with local dimensionality $d^*$ is $n^{-2p/(2p+d^*)}$ up to log factors, so when $d^* \\ll d$ the estimator circumvents the curse of dimensionality.","The class covered by Theorem 1 contains every $(p,C)$-smooth function that depends on at most $d^*$ input components, so the rate is minimax-optimal up to logarithmic factors in that subclass by the classical lower bound.","The assumption does not require the covariate distribution to live on a low-dimensional set; the function itself may be globally smooth or piecewise smooth with polytope regions and thin boundary strips.","The same sparse-network estimates also satisfy an oracle version of the MARS error bound, meaning the DNN inherits any future improvement in approximating locally low-dimensional functions by truncated-power/B-spline products.","Numerical experiments and the bike-sharing data experiment show error reductions against MARS and other nonparametric estimators, consistent with the theoretical rate advantage."],"supporting_citations":[{"why":"Supplies the minimax lower bound that the $d^*$-dimensional rate $n^{-2p/(2p+d^*)}$ is optimal up to log factors.","marker":"Stone (1982)"},{"why":"Introduced the MARS truncated power basis that the paper's basis $B^*$ generalizes.","marker":"Friedman (1991)"},{"why":"Provides the oracle inequalities and B-spline approximation results used in the proof of Theorem 2.","marker":"Györfi et al. (2002)"},{"why":"Supplies key lemmas on covering numbers and oracle bounds for neural network regression estimates.","marker":"Bauer and Kohler (2019)"},{"why":"Establishes hierarchical composition rates for ReLU networks, providing the comparison point the paper extends.","marker":"Schmidt-Hieber (2020)"},{"why":"Gives the earlier connection between ReLU deep networks and MARS that the paper extends to smooth activation functions and a richer spline basis.","marker":"Eckle and Schmidt-Hieber (2019)"},{"why":"Provides the hierarchical interaction model rates that are contained as a special case of the low-local-dimensionality class.","marker":"Kohler and Krzyżak (2017)"}],"fun_headline_variants":["Local dimensionality beats curse of dimensionality for deep nets","Deep nets dodge curse via local low-dim structure","Neural nets: error rate depends on local dimension, not input size","When data is locally low-dim, deep nets escape the curse","Deep learning rate set by local dimension, not ambient dimension"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that the probability mass of the transition strips between local regions is at most $c_5/n$, and that the strips shrink at least as fast as $c_6/n^{c_7}$, is the load-bearing premise; if those boundaries carry more probability, the theorem's stated rate does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Local dimensionality beats curse of dimensionality for deep nets","Deep nets dodge curse via local low-dim structure","Neural nets: error rate depends on local dimension, not input size","When data is locally low-dim, deep nets escape the curse","Deep learning rate set by local dimension, not ambient dimension"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1371,"prompt_tokens":955,"completion_tokens":416,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":334}},"tokens_in":571,"tokens_out":416,"duration_ms":4792,"temperature":1.0,"reasoning_tokens":334,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:23:12.546592+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take $d=2$ and let $X$ be concentrated in a strip of width $n^{-1/2}$ around the diagonal of $[0,1]^2$, with $m(x)=x_1$ on one side of the diagonal and $m(x)=x_2$ on the other. Then the boundary mass is of order $n^{-1/2}$, not $c_5/n$, so the error term from misclassifying boundary points in the proof becomes $n^{-1/2}$; a simulation measuring the $L_2$ error versus $n$ would show the stated $n^{-2p/(2p+d^*)}$ rate fails for this distribution.","supporting_citations":[],"review_version":1}