{"id":"57aa417c-07b0-46bc-a68b-fee00fab022e","arxiv_id":"2607.04597","paper_version":2,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"With inner width one, residual networks need block width exactly max(dx, dy) for L^p universal approximation and at most min(dx+dy, max(2dx+1, dy)) for uniform approximation.","lead":"Residual nets with one-neuron residual branches still universally approximate if the ambient block width is large enough. The paper pins the exact minimum block width for L^p approximation and tight bounds for uniform approximation.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The paper's central theorems are self-contained, activation-independent lower bounds are elementary, and the uniform upper bound, while longer, re-uses only previously established diffeomorphism and MLP approximation results that the residual architecture can emulate once affine and piecewise-linear maps are available (Theorems 1–2). No free parameters, no circular citations, and the numerical self-intersection example is merely illustrative. The reader's ACCEPT / high-confidence assessment is therefore left unchanged.","tokens_in":24240,"tokens_out":421,"duration_ms":4918,"concrete_test":"Independently re-derive the scalar residual construction of Theorem 2 for a single LeakyReLU_k block (Appendix B.3) and verify that the same affine coefficients lift coordinate-wise to R^w without enlarging the ambient dimension; if the lift fails for any coordinate, Proposition 8 collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest-assumption flag (the length of the Proposition 8 chain) is real but does not break the central claim. Proposition 8 rests on three published ingredients already used for MLP width theory: (i) every C^{2}-diffeomorphism is a uniform limit of invertible neural networks built from compactly supported single-coordinate maps (Teshima et al.), (ii) those maps are approximable by width-w LeakyReLU MLPs (Hwang / Yang–Zhou–Zhou), and (iii) the paper's own Theorems 1–2 show that residual blocks of inner width 1 can reproduce both affine maps and the needed coordinate-wise LeakyReLU/ABS maps. The chain is long but each link is elementary and standard; no hidden dimensional restriction or non-uniformity appears. The L^p exact-width result (Theorem 5) is independent of this chain and rests only on the elementary lower bound of Theorem 4 plus a direct residual approximation of known MLP constructions. Consequently the strongest claim survives scrutiny.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper determines the minimum ambient (block) width of residual networks whose residual branches have inner width one, for universal approximation on compact domains. For LeakyReLU, ReLU and a large class of ReLU-like activations, the exact minimum block width for L^p approximation (1 ≤ p < ∞) is max{d_x, d_y} (Theorem 5). For uniform approximation the same lower bound holds and an upper bound min{d_x + d_y, max{2 d_x + 1, d_y}} is proved (Theorem 7). Independently of the activation family and of the inner width, block width strictly less than max{d_x, d_y} is shown to be insufficient in both topologies (Theorem 4). The arguments combine elementary rank/image-dimension lower bounds with residual-block approximations of affine maps and coordinate-wise piecewise-linear maps, plus an embedding-plus-diffeomorphism route for the uniform upper bound.","tokens_in":24422,"tokens_out":942,"duration_ms":26154,"significance":"The work gives the first exact minimum-block-width result for residual networks under the most restrictive inner-width constraint, removes the dimensional restriction d_x ≥ d_y present in earlier ResNet/ODENet analyses, and improves the best previously published uniform upper bounds for a broad activation class. The activation-independent lower bound of Theorem 4 is elementary and sharp. The L^p characterization (Theorem 5) is clean and load-bearing; the uniform bounds, while not sharp, already improve Tabuada–Gharesifard and Aizawa et al. in several regimes. The proofs are self-contained once standard MLP and diffeomorphism-approximation lemmas are granted, and the numerical illustration that a width-3 ResNet can approximate a map obstructing width-3 MLPs is a useful geometric observation. These contributions are of clear interest to the approximation-theory community in deep learning.","major_comments":[],"minor_comments":[{"comment":"The chain establishing Proposition 8 (D^{2} ≺ INN ≺ MLP_LeakyReLU ≺ residual compositions) is long and spans several external lemmas. A short roadmap paragraph at the start of §4.2, listing the three external ingredients and the two internal theorems used, would make the uniform upper bound easier to audit.","section":null},{"comment":"§1.1, last paragraph: “Then purpose of this paper” → “The purpose of this paper”.","section":null},{"comment":"Table 1 footnotes are dense; moving the activation-list footnote (3) into the caption or a short remark would improve readability.","section":null},{"comment":"Definition 10 (ReLU-like) and Proposition 4 are clear, but a one-line remark that the definition is uniform on every compact interval (not merely pointwise) would prevent a possible misreading.","section":null},{"comment":"In the proof of Theorem 6 the appeal to Palais’ extension theorem is correct, yet the text does not explicitly record that the ambient dimension Ω(n,m) always satisfies n ≤ Ω, so the hypothesis of that theorem is met. Adding this observation would close a small expository gap.","section":null},{"comment":"Figure 3 is informative; stating the precise training loss / max-norm error in the caption (already given in the text as < 3·10^{-5}) would make the figure self-contained.","section":null},{"comment":"Several matrix displays in Appendix B.1 use “*” for multiplication; standard juxtaposition or “·” would be more conventional.","section":null},{"comment":"The open gap between the uniform lower and upper bounds is correctly left as future work; a one-sentence explicit statement in §5 that “whether w_sup_min = max{d_x,d_y} remains open” would help readers.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is mathematically solid and the central L^p result is ready for publication. The only reason I prefer minor_revision over outright accept is the length of the Proposition 8 citation chain and a handful of purely expository fixes; none of these affect correctness. Fit for a theory-oriented ML or approximation-theory venue is good."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This paper settles a clean architectural question: for residual nets whose residual branches have inner width one, the minimum block width for L^p approximation on compact sets is exactly max{dx, dy} (LeakyReLU / ReLU / ReLU-like). The same lower bound holds for uniform approximation and, importantly, for any activation family and any inner width. The uniform upper bound they get is min{dx+dy, max{2dx+1, dy}}, which improves Tabuada et al. and removes the dx ≥ dy restriction that appeared in earlier ResNet/ODENet work.\n\nWhat is new is the exact equality for the L^p case under the most restrictive residual branch, the extension of the activation class to a long list of ReLU-like maps, and the activation-independent lower bound that does not depend on inner width. The lower-bound argument (Theorem 4) is elementary and solid: when the ambient dimension is too small you either lose injectivity along a line or land in a proper affine subspace, so you cannot hit simple targets such as the squared norm or the moment curve. The L^p upper bound reduces to known MLP constructions once they show residual blocks of width one can approximate affine maps and coordinate-wise piecewise-linear maps; those reductions (Theorems 1–2) look carefully done.\n\nThe uniform upper bound is longer: it routes through embedding width, Palais extension, and approximation of C^{2}-diffeomorphisms by residual compositions. That chain leans on published ingredients (Teshima, Hwang, their own earlier lemmas) and each link is standard, so I do not see a break, but it is the softest part of the paper. The numerical self-intersection example for the 2-to-2 map is only illustrative; it does not carry the theorems.\n\nMath and citations look careful; no free parameters, no circularity. This is for people who care about width theory of residual and neural-ODE architectures. I would send it to a serious referee and I would cite the exact L^p width and the lower bound myself.","headline":"Exact min block width max{dx,dy} for L^p under inner-width-one ResNets, with a tighter uniform upper bound and an activation-independent lower bound that holds for any inner width.","tokens_in":25046,"tokens_out":535,"would_cite":true,"duration_ms":6296,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["41A46","41A63","41A65","68T07"],"pacs":[],"model":"grok-4.5","headline":"Residual nets with one-neuron residual branches need block width exactly max of input and output dimension for L^p universal approximation, and a slightly larger but explicit width for uniform approximation.","keywords":["minimum width","residual neural networks","compact uniform approximation","universal approximation property","block width","inner width one","ReLU-like activations"],"falsifier":"Either exhibit a continuous map that cannot be uniformly approximated by any residual network of block width min{dx+dy, max{2dx+1, dy}} and inner width one (with the listed activations), or prove that some C^{2}-diffeomorphism of R^w cannot be compactly approximated by such residual compositions.","tokens_in":25118,"feed_emoji":"📐","tokens_out":761,"duration_ms":6483,"temperature":0.7,"pith_summary":"Residual networks are compositions of residual blocks that add a small nonlinear branch to the identity. This paper asks how wide those blocks must be when every residual branch is restricted to a single neuron. For LeakyReLU, ReLU and a large family of ReLU-like activations, the answer for L^p approximation on compact domains is exact: the block width must be at least the larger of the input and output dimensions, and that same width is already enough. For uniform (sup-norm) approximation the same lower bound holds, while an explicit upper bound min{dx+dy, max{2dx+1, dy}} is proved by lifting the input into a higher-dimensional space, approximating diffeomorphisms by residual compositions, and projecting back. The lower bound is universal: no activation family and no larger inner width can succeed with block width smaller than max{dx, dy}. The results remove earlier dimensional restrictions and improve several known sufficient widths, showing that residual architecture can keep universal approximation even under the most severe restriction on the residual branches.","feed_headline":"One-neuron residual branches need width max(input, output)","feed_subtitle":"Exact L^p minimum width proved; uniform approximation needs only a slightly larger explicit width","key_machinery":"Compositions of residual blocks of block width w and inner width 1 (denoted CRB). The paper shows that these compositions can compactly approximate all affine maps and all coordinate-wise piecewise-linear maps of the form fa, and, via single-coordinate diffeomorphisms and embedding width, all C^{2}-diffeomorphisms of R^w; the ResNet is then obtained by sandwiching such a composition between two affine maps.","core_discovery":"When each residual branch has inner width one and the activations are LeakyReLU, ReLU or ReLU-like, the minimal block width for L^p universal approximation of maps from R^{dx} to R^{dy} on compact sets is exactly max{dx, dy}. For uniform approximation the same lower bound holds and the upper bound is min{dx+dy, max{2dx+1, dy}}. Independently of activation family and of inner width, block width strictly less than max{dx, dy} is never sufficient.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Exact min block width max(dx,dy) for residual nets with inner width one","Residual NNs need block width max(input,output) for Lp universal approx","One-neuron residual branches set min block width to max(dx,dy)","Block width min(dx+dy,max(2dx+1,dy)) enables uniform residual approx","No residual net with block width < max(dx,dy) can approximate all maps"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The uniform upper bound rests on residual compositions of inner width one being able to approximate every C^{2}-diffeomorphism on compact sets; if that approximation step fails, the uniform width bound is unproved.","fun_headline_variants_meta":{"raw":{"variants":["Exact min block width max(dx,dy) for residual nets with inner width one","Residual NNs need block width max(input,output) for Lp universal approx","One-neuron residual branches set min block width to max(dx,dy)","Block width min(dx+dy,max(2dx+1,dy)) enables uniform residual approx","No residual net with block width < max(dx,dy) can approximate all maps"]},"model":"grok-4.5","effort":"low","cost_usd":0.010608,"raw_usage":{"total_tokens":2326,"prompt_tokens":789,"num_sources_used":0,"completion_tokens":114,"cost_in_usd_ticks":106080000,"prompt_tokens_details":{"text_tokens":789,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1423,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":789,"tokens_out":114,"duration_ms":10427,"temperature":1.0,"reasoning_tokens":1423,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T16:19:08.129979+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Either exhibit a continuous map that cannot be uniformly approximated by any residual network of block width min{dx+dy, max{2dx+1, dy}} and inner width one (with the listed activations), or prove that some C^{2}-diffeomorphism of R^w cannot be compactly approximated by such residual compositions.","supporting_citations":[],"review_version":2}