{"id":"0f255e94-930e-4e60-b7c4-f37a393bce52","arxiv_id":"2508.19672","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"C^1 approximation of Hölder functions by rational functions and rational neural networks is proven with rate N^{-(β-1)}, via a new rational approximation of ReQU.","lead":"This paper proves that smooth functions can be approximated together with their first derivative by rational functions and rational neural networks at explicit rates. The key technical step is a new rational approximation of the ReQU activation that is exponentially accurate in value and arbitrarily algebraically accurate in derivative.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 11's width claim is contradicted by its own construction: the network width is Θ(N^d), not O(N), so the stated C^1 rate as a function of width is unsupported.","rationale":"The reader's conditional verdict is broadly confirmed, but the most load-bearing concern is not the tensor-product spline bound (16), which is standard, nor the ReQU approximation itself, which appears internally sound. The decisive issue is the mismatch between Theorem 11's stated width O(N) and the proof's explicit width Θ(N^d). This mismatch affects the central claim about approximation rates with respect to width and also propagates to Corollary 12. The theorem can likely be repaired by restating the width as O(N^d) and the rate as O(N^{-(β-1)/d}) in terms of width, or by finding a genuinely O(N)-width construction, but as written the main theorem is not established. The reader flagged this contradiction in the rationale; the weakest_assumption field pointed instead to equation (16), so my agreement is partial. A conditional verdict is appropriate: the core ReQU result and the spline-transfer strategy are promising, but the width/rate statement must be corrected before acceptance.","tokens_in":19321,"tokens_out":23876,"duration_ms":263005,"concrete_test":"Fix d=2 and β=3 in Theorem 11 and trace the proof's construction. Count the actual number of nodes: Layer 1 has (N+1)^2, Layer 2 has (N+5)^2, Layer 3 has (N+3)^2, and the final layer sums over (⌊β⌋+N)^2 coefficients. If the width is W∼N^2, re-express the final bound (31) as O(W^{-(β-1)/d}) and compare with [2, Eq. (13)], which for d=2 gives O(W^{-(β-1)/2}). Equivalently, check whether the construction can be compressed to width O(N) for a generic f in H^β; the full tensor grid of B-spline products appears necessary. This single count settles whether Theorem 11's width/rate statement is internally supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central theorem, Theorem 11, states that rational neural networks of width O(N), constant depth, and rational activation degree O(N^ε) approximate every f ∈ H^β_p in C^1 with error O(N^{-(β-1)}). The proof, however, builds the network layer-by-layer with widths (N+1)^d, (N+5)^d, (N+3)^d, etc., and the output layer sums over (⌊β⌋+N)^d tensor-product B-spline coefficients. Thus the constructed width is Θ(N^d), not Θ(N). This is not a cosmetic discrepancy: the proof only establishes the stated rate with respect to the per-dimension knot parameter N. If N is interpreted as the network width (as in the quoted [2] bound (13), which has exponent (l−β)/d), the correct approximation error is O(N^{-(β-1)/d}); if N is the per-dimension knot count, the width is O(N^d). The stated combination of O(N) width and O(N^{-(β-1)}) error would circumvent the d-dimensional curse of dimensionality and is not derived from the tensor-product spline construction. Corollary 12 inherits the same issue in the degree count N^{d+ε}. The ReQU approximation lemma (Corollary 6) appears sound; the failure is in the transfer step and in the interpretation of the rate as a function of width.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies C^1-approximation by rational functions and rational neural networks. In Section 2, it constructs Newman-polynomial-based rational approximations of the ReQU activation: Corollary 6 states rational functions of type (n+1,n-1) approximating ReQU in C^1([-1,1]) with error O(e^{-√n}) in the function value and O(n^{-(K-2)/2}) in the derivative, for every fixed K≥3. In Section 3, the paper combines this ReQU approximation with the tensor-product-spline/ReQU-network results of Belomestny et al. [2]. Theorem 11 claims rational neural networks of width O(N), constant depth, and activation degree O(N^ε) approximate every f∈H^β_p([0,1]^d) in C^1 with error O(N^{-(β-1)}). Corollary 12 transfers this to a rational-function approximation result with degree O(N^{d+ε}). Finally, Lemma 16 and Corollary 18 derive pointwise C^1-approximation statements for the ParFam and EQL^÷-type symbolic-regression architectures by composing rational approximations of the inverse activations.","tokens_in":19659,"tokens_out":11212,"duration_ms":126941,"significance":"The genuinely new technical contribution is Lemma 5 / Corollary 6: a careful, constructive proof that ReQU can be approximated in C^1 by rational functions with exponential uniform error and arbitrarily high polynomial derivative rate. This part appears sound and is a useful addition to the rational-approximation literature. If Theorem 11 were correct as stated, the paper would also show a dimension-robust width-rate trade-off for rational neural networks. Unfortunately, the proof establishes only a different statement: the constructed width is Θ(N^d), so the claimed O(N) width with O(N^{-(β-1)}) error is unsupported. With an honest restatement, the paper still contains a valid C^1-approximation transfer from ReQU spline networks to rational networks, but with a dimension-dependent rate in terms of width.","major_comments":[{"comment":"The theorem states that the rational neural network has width of order N and error O(N^{-(β-1)}). But the construction in the proof is layer-by-layer with widths (N+1)^d, (N+5)^d, (N+3)^d, and (N+m+1)^d, and the output layer sums over (⌊β⌋+N)^d tensor-product coefficients. The paper explicitly assigns the network width (N+max(4,⌊β⌋)+1)^d. Hence the width is Θ(N^d), not O(N). The N in the error estimate is a per-dimension knot parameter in the spline bound (16), not the network width. If W denotes the network width, the proof yields, after substituting W=N^d, an error O(W^{-(β-1)/d}) up to the N^ε factor, not O(N^{-(β-1)}). This is a load-bearing quantitative discrepancy and the main claim of Theorem 11 is therefore not proven as stated.","section":"Theorem 11 and its proof (§3.1, design of rational neural network)"},{"comment":"The quoted result of [2] is stated in (13) as error cN^{(l-β)/d}, so for l=1 it gives C^1 error cN^{-(β-1)/d}. The proof then uses the spline bound (16) with cN^{-(β-1)}, silently changing the interpretation of N from a width parameter to a per-dimension knot count. If (16) is meant as a per-dimension spline bound, that N is not the network width, and the subsequent layer widths must be expressed in terms of N^d. If, instead, N is meant to remain the width, then (16) should contain the factor 1/d. The two roles of N must be separated; otherwise the rates in Theorem 11 and in the derived Corollary 12 are not justified.","section":"Equations (13)–(16), proof of Theorem 11"},{"comment":"The stated type (n+1,n-1) appears inconsistent with the construction. For example, for n=2, P_2(x)+P_2(-x)=2(x^2+ξ) and P_2(x)-P_2(-x)=2(1+ξ)x, so R_2(x)=x^2 r_2(x) has numerator degree 3 and denominator degree 2, i.e. type (3,2), not (3,1). More generally, the parity of n changes the degree of P_n(x)+P_n(-x). Since the remainder of the paper only uses the order n of the degree, this does not affect the main rates, but the type statement in Lemma 5 and Corollary 6 should be corrected.","section":"Lemma 5 / Corollary 6 (type count)"}],"minor_comments":[{"comment":"The abstract and introduction describe 'C^1-approximation results' for EQL^÷ and ParFam, but Lemma 16 only establishes pointwise convergence, not uniform approximation, and no rates are given. Moreover, the proof cancels the activation functions by approximating their inverses; this is acknowledged in Remark 19. The claims in the abstract should be phrased more cautiously.","section":"Section 3.2 / Lemma 16"},{"comment":"There are several typographical infelicities: 'with with respect to' in the Introduction, and 'We no consider' at the start of Section 3.1. In Definition 10 the indexing of the coefficients is difficult to parse and could be simplified.","section":"Introduction / Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The width/rate mismatch in Theorem 11 is the central obstacle. It is likely fixable by restating the theorem honestly: either let N be the per-dimension knot count and state the width as Θ(N^d), or state the rate in terms of the actual width W as O(W^{-(β-1)/d}). The ReQU approximation lemma appears sound and is the most valuable part of the paper. The symbolic regression section is weaker than the abstract suggests and should be reframed as a pointwise existence result."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: Corollary 6, the C^1 rational approximation of ReQU, is real and clean. But Theorem 11 as stated has a width/rate mismatch that undermines the headline claim. The construction uses width Θ(N^d), not O(N), so the rate O(N^{-(β-1)}) is with respect to per-dimension knots, not network width. That should be fixed before publication.\n\nCorollary 6 is the genuinely new piece: rational functions of type (n+1,n-1) approximate ReQU in C^1 with exponential uniform error and derivative error O(n^{-(K-2)/2}) for any fixed K. The proof via Newman polynomials is detailed and, as far as I can tell, correct. That alone is worth publishing.\n\nThe transfer to general functions is where things get shaky. Theorem 11 claims rational networks of width O(N) achieve C^1 error O(N^{-(β-1)}). But the proof builds the network layer by layer with widths (N+1)^d, (N+5)^d, etc., and the output sums over (⌊β⌋+N)^d coefficients. So the width is Θ(N^d). The stated rate is really in terms of the per-dimension knot count N. In terms of width W, the best the proof gives is O(W^{-(β-1)/d}). That is a serious discrepancy, not a typo. The same issue carries into Corollary 12: degree is N^{d+ε}, not N^{1+ε}. The authors need to either restate the theorem in terms of N^d or show a different construction that actually has width O(N).\n\nThere are also smaller issues: the introduction claims the result is 'better than under polynomial approximation,' but for fixed d the rational rate is O(W^{-(β-1)/d}) and the polynomial baseline is similar; no comparison is proved. The symbolic regression lemma is pointwise and needs inverse-regularity assumptions; the authors admit this. I would not weight that heavily.\n\nOn balance: the ReQU approximation is solid, the spline transfer is a natural idea, but the main theorem's stated width dependence is not supported by the proof. This is a load-bearing flaw that needs fixing. I would send it to peer review, because the core technical contribution is genuine and the fix is likely manageable.","headline":"Genuinely useful ReQU approximation result, but the main theorem misstates width dependence: width is Θ(N^d), not O(N).","tokens_in":20103,"tokens_out":2185,"would_cite":true,"duration_ms":21192,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["33F05","41A20","41A25","26C15"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper establishes a new C^1 approximation theorem: rational functions and rational neural networks of moderate degree can approximate a smooth function together with its first derivatives at rates that match or improve on polynomial app","keywords":["rational functions","rational neural networks","C^1 approximation","higher-order approximation","ReQU activation","Newman polynomials","EQL^div architecture","physical law learning"],"falsifier":"Take n=10^4 and K=3, compute R_n from Lemma 5 explicitly, and evaluate max_x |ReQU′(x)−R_n′(x)| on a fine grid covering [−1,1]. If the maximum does not shrink like n^{−1/2} while the value error remains O(e^{−√n}), the core derivative estimate fails. Alternatively, for a fixed smooth f with known spline error, construct the network of Theorem 11 with M=N^ε and check whether the C^1 error decays as N^{−(β−1)}; any slower exponent would show the transfer step, not merely the ReQU lemma, is wrong.","tokens_in":19242,"feed_emoji":"➗","tokens_out":9659,"duration_ms":100137,"temperature":0.7,"pith_summary":"This paper shows that functions with Hölder smoothness β>2 on [0,1]^d can be approximated, together with their first derivatives, by rational functions and by rational neural networks. The rational approximants have degree roughly N^{d+ε} and achieve C^1 error O(N^{−(β−1)}); the networks have width O(N), constant depth, and rational activation degree O(N^ε) at the same rate. The engine is a new rational approximation of the ReQU activation: Newman-polynomial-based rational functions of type (n+1,n−1) approximate ReQU with value error O(e^{−√n}) and derivative error O(n^{−(K−2)/2}) for any chosen K≥3. Because the underlying spline/ReQU network already attains higher-order approximation rates, replacing each ReQU activation by such rational functions transfers the rate. The result also yields C^1 approximation for the EQL^div and ParFam symbolic-regression architectures used in physical law learning.","feed_headline":"Rational functions now match functions and derivatives at once","feed_subtitle":"A degree-N rational model reproduces a β-smooth target and its gradient to order N^{-(β−1)} on [0,1]^d.","key_machinery":"Newman-polynomial rational approximants. The paper constructs R_n(x)=x^2(P_n(x)−P_n(−x))/(P_n(x)+P_n(−x)) using Newman polynomials P_n(x)=∏_{i=0}^{n−1}(x+e^{−i/√n}). The exponentially spaced roots make the ratio (P_n(x)−P_n(−x))/(P_n(x)+P_n(−x)) approximate the sign function, so x^2 times it approximates x^2 sgn(x), and ReQU is recovered as half the sum of x^2 and x^2 sgn(x). The derivative estimate O(n^{−(K−2)/2}) follows by controlling x^2 r_n′(x) through the sum over the Newman roots, using a density lemma for those roots. Inserting this rational C^1 approximant into the B-spline representation of the ReQU network from [2] transfers the higher-order approximation rate to rational function","core_discovery":"Corollary 6 is the key new technical result: the rectified quadratic unit ReQU(x)=max(x,0)^2 is approximated in C^1([−1,1]) by rational functions of type (n+1,n−1) with error O(e^{−√n}) for values and O(n^{−(K−2)/2}) for derivatives, for any prescribed K≥3. The approximants are R_n(x)=x^2(P_n(x)−P_n(−x))/(P_n(x)+P_n(−x)), with Newman polynomials P_n(x)=∏_{i=0}^{n−1}(x+e^{−i/√n}); this rational sign-function approximates x^2 sgn(x), and ReQU is half of x^2 plus x^2 sgn(x). The paper then replaces ReQU activations in the spline-based ReQU networks of [2], yielding constant-depth rational neural networks of width O(N) and rational degree O(N^ε) that approximate every f∈H^β([0,1]^d), β>2, in C^1","pith_inferences":["The construction suggests a general transfer principle: any higher-order universal approximator whose activation can be C^1-approximated by rationals likely inherits rational C^1 approximation at the same rate; the authors note this for other Sobolev and ReLU/ReQU results, and extending it to W^{m,p} norms would be a natural next step.","If the same Newman-polynomial mechanism can control higher derivatives of x^2 r_n(x), the argument would extend to C^k approximation for k≥2, which the paper explicitly leaves open.","The symbolic-regression result in Lemma 16 currently proves pointwise convergence by effectively using one rational layer to cancel the activations; a rate-preserving version would need to exploit the non-rational activations directly, and the paper's Remark 19 flags this as the weak point of the proof strategy."],"forward_implications":["For every f in the Hölder class H^β([0,1]^d) with β>2, rational functions of degree O(N^{d+ε}) approximate f and its first derivatives uniformly with error O(N^{−(β−1)}), so rational approximation preserves first-order smoothness at a rate comparable to spline approximation.","Rational neural networks with width O(N), constant depth, and rational activation degree O(N^ε) achieve the same C^1 rate, showing that using rational activations does not sacrifice first-order accuracy.","For ReQU itself, the rational approximants attain derivative error of arbitrarily high polynomial order, whereas polynomial C^1 approximants are limited to rate n^{−1}; this gap is what makes the rational transfer possible.","The EQL^div and ParFam symbolic-regression architectures inherit C^1 approximation: under mild regularity on the activation functions and their inverses, they can match both function values and gradients, which is directly relevant to physical law learning.","If the activation inverses satisfy Hölder regularity with β>2, the pointwise C^1 approximation holds; under global Lipschitz assumptions, the approximation is uniform on [0,1]^d."],"supporting_citations":[{"why":"Supplies Newman polynomials and the O(e^{−√n}) rational approximation of |x| that the ReQU approximants in Lemma 5 and Corollary 6 are built on.","marker":"[19]"},{"why":"Provides the higher-order ReQU network and tensor-product spline bound (equation (16)) whose C^1 approximation rate Theorem 11 transfers to rational networks.","marker":"[2]"},{"why":"Defines the EQL^div architecture; the paper derives its C^1 approximation as a corollary of Corollary 12.","marker":"[21]"},{"why":"Defines the ParFam architecture; the paper derives its C^1 approximation as a corollary of Corollary 12.","marker":"[22]"}],"fun_headline_variants":["Rational functions nail C^1 value and gradient approximation","C^1 rates for rational functions and networks: value and derivative","Exponential C^1 approximation: rational functions and nets","Rational networks: C^1 fit for function and its gradient","C^1 approximation: rational nets match value and gradient"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The proof inherits the tensor-product spline approximation bound from [2] (equation (16)): every function in the Hölder class H^β with β>2 is within c N^{−(β−1)} of a spline in C^1, with bounded spline coefficients; if that bound is false or requires stronger smoothness, the main rates do not follow.","fun_headline_variants_meta":{"raw":{"variants":["Rational functions nail C^1 value and gradient approximation","C^1 rates for rational functions and networks: value and derivative","Exponential C^1 approximation: rational functions and nets","Rational networks: C^1 fit for function and its gradient","C^1 approximation: rational nets match value and gradient"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001387,"raw_usage":{"total_tokens":5430,"prompt_tokens":703,"completion_tokens":4727,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":4644}},"tokens_in":447,"tokens_out":4727,"duration_ms":32413,"temperature":1.0,"reasoning_tokens":4644,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:35:07.441454+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take n=10^4 and K=3, compute R_n from Lemma 5 explicitly, and evaluate max_x |ReQU′(x)−R_n′(x)| on a fine grid covering [−1,1]. If the maximum does not shrink like n^{−1/2} while the value error remains O(e^{−√n}), the core derivative estimate fails. Alternatively, for a fixed smooth f with known spline error, construct the network of Theorem 11 with M=N^ε and check whether the C^1 error decays as N^{−(β−1)}; any slower exponent would show the transfer step, not merely the ReQU lemma, is wrong.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Newman polynomials and the O(e^{−√n}) rational approximation of |x| that the ReQU approximants in Lemma 5 and Corollary 6 are built on."},{"cited_title":"Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations","cited_arxiv_id":null,"evidence_quote":"Provides the higher-order ReQU network and tensor-product spline bound (equation (16)) whose C^1 approximation rate Theorem 11 transfers to rational networks."},{"cited_title":"Learning equa- tions for extrapolation and control","cited_arxiv_id":null,"evidence_quote":"Defines the EQL^div architecture; the paper derives its C^1 approximation as a corollary of Corollary 12."},{"cited_title":"ParFam -- (Neural Guided) Symbolic Regression Based on Continuous Global Optimization","cited_arxiv_id":"2310.05537","evidence_quote":"Defines the ParFam architecture; the paper derives its C^1 approximation as a corollary of Corollary 12."}],"review_version":1}