{"id":"969320ef-e88a-4840-b9c7-e7046a5e73d8","arxiv_id":"2412.19235","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A collection of deterministic initialization, loss weighting, data-driven initialization, and gradient-free training methods for shallow physics-informed neural networks, tested on ODEs and PDEs.","lead":"This paper proposes deterministic initialization, loss weighting, data-driven initialization, and gradient-free training methods for physics-informed neural networks with one hidden layer, plus a separable two-layer variant for PDEs. A generalist should read it because it tests whether very shallow PINNs can be made competitive, which would make physics-informed models cheaper and easier to interpret.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The only explicit SOTA claim (NbN on Lorentz) is contradicted by the paper's own Table 7 and lacks external PINN baselines, so the 'state-of-the-art' part of the central claim is not supported.","rationale":"I read the paper as claiming practical viability and SOTA rather than a theorem. The deterministic initialization's localization approximation (Eqs. 14–16) is indeed a weak spot, but the numerical experiments give some evidence that later training can compensate, and it is not directly contradicted. By contrast, the SOTA claim is contradicted by the paper's own Table 7 and is essential to the abstract. The reader's rationale already flags this, though the reader's explicit weakest_assumption is the localization issue, so my agreement is partial. The CONDITIONAL verdict remains appropriate: the methods are re-implementable and some results are plausible, but the SOTA language should be removed or substantiated with external baselines and error bars.","tokens_in":30,"tokens_out":8413,"duration_ms":151852,"concrete_test":"Reproduce the Lorenz benchmark (Sec. 4.5.2) on a fixed test grid with Eq. (64), and run NbN against at least four published PINN baselines: causal training [4], modified-MLP conservation laws [5], PirateNet [22], and one recent adaptive or Fourier-feature PINN, using comparable parameter counts and GPU time across 5 seeds. If NbN does not achieve the lowest mean relative L2 error, the SOTA claim is false; if it does, the paper still needs this evidence because Table 7 omits it.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The conclusion states that NbN training demonstrates 'state-of-the-art results on the Lorentz system' (Sec. 6), and the abstract's 'in some cases, state-of-the-art results' is the only concrete instance offered. This is the load-bearing empirical pillar of the central claim. Table 7 contradicts it: for the same Lorenz problem, PIDD initialization (Example 16) gives relative L2 errors (2.0e-4, 6.7e-4, 1.3e-4), while NbN training (Example 20) gives (9.8e-4, 1.4e-3, 6.0e-4); PIDD is 3–5x more accurate. The table's only baseline is 'Causal training [4]'; no external state-of-the-art PINN methods, no seeds, no error bars, and no compute-matched comparison are given. PIDD itself is initialized from odeint/Chebfun reference data (Algorithm 3, Sec. 4.1), so its high accuracy is partly interpolation, but that does not rescue the NbN SOTA claim; if PIDD is excluded as not solver-free, NbN still needs comparison against the wider PINN literature. Thus the strongest claim rests on an internally contradicted table and an absent external benchmark.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a set of initialization, weighting, and training techniques for physics-informed neural networks with one hidden layer, and for separable PINNs (SPINN) with two such layers, for solving ODEs and PDEs. The contributions include a 'strictly deterministic' initialization (Algorithms 1–2), modifications of causal training and gradient normalization, new loss-weighting schemes (detaching, second-derivative weighting, PSBW, RR), a physics-informed data-driven (PIDD) initialization that fits the network to precomputed reference data (Algorithm 3), a generalization metric for parameterized problems (Appendix B), a gradient-free neuron-by-neuron (NbN) fitting algorithm (Algorithm 5), and an extension to 2D problems using SPINN. The central claim, stated in the abstract and conclusions, is that these shallow networks achieve competitive accuracy and, in some cases, state-of-the-art results.","tokens_in":35914,"tokens_out":7596,"duration_ms":72240,"significance":"If the claims are substantiated, the paper would make a useful contribution by demonstrating that very shallow PINNs with carefully chosen initialization and training can be a cheaper alternative to deep architectures. The paper's strengths include a large experimental campaign, concrete algorithms that are easy to reimplement, and a deterministic initialization that removes random-seed variability. The NbN method is genuinely gradient-free and the generalization metric is a step toward quantifying interpolation behavior. However, the significance is diminished by the unsupported state-of-the-art claim, the circular evaluation of PIDD, and the lack of error analysis for the heuristic approximations at the core of the initialization; these issues prevent the current manuscript from supporting its strongest conclusions.","major_comments":[{"comment":"The claim that NbN training gives 'state-of-the-art results on the Lorentz system' is not supported by the evidence presented. Table 7 reports relative L2 errors for NbN training of (9.8e-4, 1.4e-3, 6.0e-4), which are 3-5 times larger than the PIDD initialization errors (2.0e-4, 6.7e-4, 1.3e-4) listed in the same table, and the only external baseline is 'Causal training [4]'. No comparison with other recent PINN methods for the Lorenz system, no seed or error-bar statistics, and no compute-matched comparison are provided. Since this is the only concrete instance of 'state-of-the-art' offered in the paper, the abstract's central claim rests on an internally contradicted and externally unbenchmarked result.","section":"Abstract, Section 6, Table 7"},{"comment":"The PIDD initialization is data-driven: the output weights are set using solutions generated by odeint or Chebfun (e.g., Section 4.2.1 states the data were 'generated with odeint'), and the reported relative L2 errors are then computed against the same reference solutions. This makes the PIDD results an interpolation test on the training data rather than an independent solver-free PINN result. The paper should explicitly acknowledge this circularity and distinguish it from the solver-free NbN results. In particular, Table 7 should not be read as a fair comparison between PIDD and NbN, since PIDD has access to the reference data.","section":"Section 4.1, Algorithms 3-4, Examples 15-16 and 22-23"},{"comment":"The deterministic initialization rests on two heuristic assumptions: that the derivative of the network at x_m is dominated by a single neuron's sigma'(0) term, and that neighboring output weights vary slowly. These assumptions are not quantified, no error bounds are derived, and the key hyperparameter delta-zeta is selected empirically (Section 2.1.3 reports 'the best results were achieved at delta-zeta = 0.7'). Additionally, the derivation leading to Eq. (16) is internally inconsistent: the text derives an approximation involving W_{m-1} at x_m but then states 'we have the following equation for the weights W_m' with N evaluated at x_{m+1}. A numerical check of how well Eq. (15) holds as a function of N and delta-zeta, and a sensitivity analysis, would substantially strengthen the paper.","section":"Section 2, Eqs. (14)-(16); Section 4.1, Eqs. (47)-(49)"},{"comment":"The 'generalization' properties of the proposed network are evaluated only at the midpoints q_{i+1/2} of the training grid, i.e., as interpolation within the domain [q_min, q_max]. This is not extrapolation to unseen parameter ranges, and the paper even shows in Example 18 that the method fails to generalize in chaotic regimes (mu = 0.023 for rho in [25,30]). The claim of 'pronounced generalizing properties' should be qualified as interpolation between precomputed solution snapshots, and the metric mu should be described as an interpolation-error ratio rather than a measure of genuine generalization.","section":"Section 4.3, Appendix B, Examples 17-18"}],"minor_comments":[{"comment":"The spelling 'Lorentz system' appears in Examples 16 and 20, Table 7, and the Conclusions; the correct name is 'Lorenz system' (as used in the equations and in the reference to Lorenz 1963).","section":"Throughout"},{"comment":"The paragraph 'We noticed that changing the weights lambda_ic and lambda_r can lead to better accuracy...' indicates that key hyperparameters were tuned after seeing the test results. The paper would benefit from a description of a validation or cross-validation protocol that separates hyperparameter selection from final performance evaluation.","section":"Section 3.5.1"},{"comment":"The phrase 'under which the calculations in [4] were carried out' would be clearer if the exact reference values of sigma, rho, and beta were stated with the units or conventions used; currently the reader must infer them from the equation.","section":"Section 4.2.2, Example 16"},{"comment":"The definition of the generalization measure mu is correct but the textual description 'the ratio of the relative total L2 error of prediction of set {q_{i+1/2}} to the ratio of the relative total L2 error of prediction of set {q_i}' is garbled; it should say 'the ratio of the error on the training grid to the error on the midpoints'.","section":"Appendix B, Eq. (65)"},{"comment":"The caption 'Contribution to the solution u_{theta,1} of the first 1 (a), 11 (b), 21 (c)...' is missing commas and the first item should read 'first 1 neuron (a), first 11 neurons (b), first 21 neurons (c), ...'.","section":"Figure 7 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is a long preprint with many experimental results but the central SOTA claim is fragile. The authors should either provide a proper external benchmark for NbN on the Lorenz system or soften the claim in the abstract and conclusions. The PIDD evaluation should be reframed as data-driven interpolation. The heuristic initialization derivations need at least a numerical validation of the dominant-neuron assumption. The manuscript would also benefit from thorough proofreading and a more structured presentation; as it stands it reads like a collection of technical reports rather than a single coherent paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful bag of tricks for very shallow PINNs, and the Allen-Cahn number is genuinely competitive. But the \"state-of-the-art\" claim on Lorentz doesn't survive contact with their own Table 7, so read the abstract skeptically.\n\nWhat's actually new: Algorithm 2's strictly deterministic initialization is a clean idea — fix hidden weights by spacing sigmoids and set output weights from local derivative matching. The derivation of the optimal Delta-zeta = ln(2+sqrt(3))/2 is a nice touch, and it matches the value they found empirically. The PIDD initialization, the neuron-by-neuron (NbN) training loop, and the PSBW/RR weighting schemes are also new combinations, described in enough detail to reimplement. The Allen-Cahn result (4.4e-5) is close to the current best (PirateNet at 2.24e-5) and better than causal training. That's real evidence the approach can work on a nontrivial PDE.\n\nThe soft spots are mostly about the claims, not the methods. The conclusion's statement that NbN gives \"state-of-the-art results on the Lorentz system\" is contradicted by Table 7: PIDD initialization gives errors 3-5x smaller (2e-4 vs 9.8e-4 for x), and the only baseline is causal training from [4]. There are no external PINN baselines, no seeds, no error bars, and no code. Many hyperparameters are tuned after seeing the results; Section 3.5.1 admits the best lambdas were noticed after experiments. And PIDD starts from odeint/Chebfun reference data and then reports error against those same references, so its accuracy is partly interpolation, not independent prediction. That's fine if the paper says so, but the abstract's \"state-of-the-art\" framing overreaches.\n\nWho's this for? Practitioners who want to try shallow networks to avoid vanishing gradients. The recipes are concrete and the paper is honest about limitations (e.g., chaotic Lorenz generalization). It deserves a serious referee — a good referee will ask for code, error bars, and external baselines, and will ask the authors to tone down the SOTA claims. I'd send it to review, but I would not cite the SOTA claim until it survives a real comparison.","headline":"A bag of useful tricks for shallow PINNs, but the headline SOTA claim doesn't survive the paper's own Table 7.","tokens_in":36469,"tokens_out":2767,"would_cite":false,"duration_ms":26337,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65L05","65M70"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that one-hidden-layer physics-informed networks, started from a deterministic Euler-like initialization and trained with special loss weighting, match or beat deeper baselines on a range of ODE and PDE benchmarks.","keywords":["physics-informed neural networks","shallow neural networks","deterministic initialization","separable PINN","neuron-by-neuron training","loss weighting","Lorenz system","Allen-Cahn equation"],"falsifier":"Freeze the hidden layer after Algorithm 2 on an ODE with a known analytic solution, compute the full derivative sum in Eq. (14) at every grid point, and compare it to the one-neuron approximation in Eq. (15). If the relative mismatch is large, say above 10%, on any part of the domain—especially with $\\Delta\\zeta=\\ln(2+\\sqrt{3})/2$—then the localization assumption that underlies the output-weight formula is broken, and the claimed source of the method's accuracy is not present.","tokens_in":35352,"feed_emoji":"🧮","tokens_out":10921,"duration_ms":96091,"temperature":0.7,"pith_summary":"This paper tries to establish that very shallow physics-informed neural networks—one hidden layer for ordinary differential equations and a separable product of two single-hidden-layer networks for partial differential equations—are enough to solve a range of physical benchmark problems accurately, provided the weights are initialized deterministically from the differential operator and the loss is weighted with care. The central move is a strictly deterministic initialization that mimics the Euler integration step: hidden weights and biases are fixed so that each sigmoid neuron is centered at one grid point, and output weights are set from local derivative matching, so the initial network already behaves like a first-order numerical integrator. Around that starting point the paper develops training modifications—detaching nonlinear right-hand sides, second-derivative weighting, predicted-solution weighting, relative residuals, causal weighting, and gradient normalization—and a gradient-free neuron-by-neuron fitting procedure. On tests including the harmonic oscillator, charged-sphere potential, relativistic slingshot, exponential growth, the Lorenz system, Allen–Cahn, and a fast advection equation, these methods reach relative L2 errors as low as 1e-5 to 1e-6 and, in the Lorenz case, beat the causal-training baseline cited in the paper. If true, this matters because shallow networks avoid vanishing gradients, train quickly, and are far easier to interpret than the deep PINNs usually deployed.","feed_headline":"A deterministic start makes two-layer PINNs competitive","feed_subtitle":"Fix hidden weights from the ODE itself, train only the rest; Allen-Cahn error drops to 4.4e-5.","key_machinery":"The load-bearing object is the strictly deterministic initialization (Algorithm 2). It fixes the hidden-layer weights to $W_k^{(1)}=2\\Delta\\zeta/\\Delta x$ and biases to $b_k^{(1)}=-2(k+1)\\Delta\\zeta$, and sets the output-layer weights from the local derivative-matching identity $W_k^{(2)}=-2\\Delta x\\,N_l[u(0),x_{k+1}]/\\Delta\\zeta$, with the output bias equal to the initial value; the parameter $\\Delta\\zeta=\\ln(2+\\sqrt{3})/2$ is chosen so that the third derivative of neighboring sigmoids cancels at each neuron center. This construction makes the initial network an Euler-like piecewise-sigmoid integrator, so the subsequent training stages only refine a good starting representation. The separable extension for PDEs, $u_{\\theta}(t,x)=\\sum_j v_j(t)w_j(x)$, is the second central object: it lets the same one-hidden-layer construction be applied along each coordinate and combined multiplicatively, which is what the paper calls a two-hidden-layer SPINN.","core_discovery":"The paper's central claim is that the expressive power of deep PINNs is not needed for the problems considered: a single-hidden-layer network, whose hidden weights and biases are fixed rather than learned and whose output weights are set from the ODE residual at grid points, can be trained to competitive accuracy, and its product with a second single-hidden-layer network (the separable SPINN architecture) extends the same construction to two-dimensional PDEs. The deterministic initialization is derived from the Euler method: with $W_k^{(1)}=2\\Delta\\zeta/\\Delta x$ and $b_k^{(1)}=-2(k+1)\\Delta\\zeta$, the $k$th sigmoid's steepest response sits at $x_{k+1}$, and assuming that one neuron dominates the derivative at each grid point gives $W_k^{(2)}=-2\\Delta x\\,N_l[u(0),x_{k+1}]/\\Delta\\zeta$. The paper chooses $\\Delta\\zeta=\\ln(2+\\sqrt{3})/2\\approx 0.66$ so that third-derivative interference between neighboring sigmoids cancels, and confirms experimentally that $\\Delta\\zeta\\approx 0.7$ works best. On top of this initialization the paper adds loss-weighting schemes—detaching nonlinear operators, second-derivative weighting, predicted-solution-based weighting, relative residuals, δ-causal training, and gradient normalization—plus a data-driven variant called PIDD initialization and a gradient-free neuron-by-neuron fitting rule. The empirical payoff is reported as competitive accuracy across ODE and PDE benchmarks and, for the Lorenz system, errors below the causal-training baseline quoted in the paper.","pith_inferences":["Editorial inference: the Euler-matching initialization effectively replaces random weight guessing with a discretization prior, so the trained network should inherit the convergence order and error profile of a first-order integrator; one testable consequence is that halving the grid step should roughly halve the error floor before optimization, which the paper's scaling plots hint at but do not a","Editorial inference: because only output weights are set analytically and hidden weights stay frozen during the first training stages, the method invites an extreme-learning-machine-style analysis; the paper notes this connection itself, but a rigorous approximation-rate bound for the sigmoid Euler construction is left open.","Editorial inference: the generalization measure introduced in the paper, which compares errors on training parameters versus half-shifted parameters, could be applied to any parameterized PINN, not just the separable construction; the low value in the chaotic Lorenz regime suggests the generalization claim should be read as confined to regular parameter regions.","Editorial inference: the same deterministic initialization could be extended beyond the tested equations to systems with stiffness or shocks by replacing the uniform grid with adaptive collocation points; nothing in the derivation requires uniformity, so adaptive grid spacing is a natural next experiment."],"forward_implications":["For ODEs, the deterministic initialization alone improves the harmonic-oscillator relative L2 error by an order of magnitude over random output-layer initialization, from about 4.7e-3 to 7.7e-4, and the full training pipeline brings it to roughly 2.2e-4.","The combination of detaching, δ-causal weighting, second-derivative weighting, and gradient normalization turns the relativistic-slingshot problem from unusable errors into errors around 1e-3 to 1e-2, with relative L2 values such as 1.2e-3 for h and 1.3e-3 for x.","Predicted-solution-based weighting or relative-residual weighting extends accurate solutions to multi-scale regimes where plain training fails, e.g., exponential growth on [0,10] improves from 0.93 to 1.72e-3, and the Lorenz system on [0,20] reaches errors around 5e-4 to 8e-4.","The gradient-free neuron-by-neuron fitting method reaches Lorenz-system errors below the causal-training baseline reported in the paper (x: 9.8e-4, y: 1.4e-3, z: 6.0e-4) after three sweeps over neurons and without using an optimizer.","On the Allen–Cahn PDE, the two-hidden-layer separable SPINN with deterministic initialization and the full weighting stack reaches relative L2 error 4.40e-5, below the δ-causal training baseline of 6.29e-5 listed in the paper."],"supporting_citations":[{"why":"It defines the PINN loss formulation and the original Allen-Cahn baseline that the paper compares against.","marker":"[3]"},{"why":"It supplies the causal-training baseline for the Lorenz and Allen-Cahn comparisons.","marker":"[4]"},{"why":"It supplies the δ-causal loss weighting and the beta relation that the paper modifies and compares against.","marker":"[10]"},{"why":"It provides the fixed-hidden-weight approximation result that justifies freezing the hidden layer in the deterministic initialization.","marker":"[31]"},{"why":"It supplies a local radial-basis PINN variant whose neuron-center placement is similar to the deterministic bias initialization.","marker":"[37]"},{"why":"It introduces the separable PINN architecture used for the two-dimensional problems.","marker":"[43]"},{"why":"It supplies the gradient-normalization analysis that the paper adapts as the GN1 and GN2 weighting schemes.","marker":"[35]"},{"why":"It supplies the residual-based attention weighting that inspired the predicted-solution and relative-residual weighting schemes.","marker":"[55]"},{"why":"It provides the reference solution used to evaluate Allen-Cahn accuracy.","marker":"[45]"}],"fun_headline_variants":["Single-layer PINNs match deep nets via Euler-derived init","Deterministic start makes one-hidden-layer PINNs competitive","Two hidden layers? ODE-fixed weights make one enough","Physics-informed init for PINNs: depth unnecessary for accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each sigmoid neuron responds mainly at its own grid point, so the network derivative at a point is essentially one neuron's contribution; if neighboring sigmoids overlap heavily, the deterministic start is not a faithful Euler-like approximation and training must repair a bad representation.","fun_headline_variants_meta":{"raw":{"variants":["Single-layer PINNs match deep nets via Euler-derived init","Deterministic start makes one-hidden-layer PINNs competitive","Two hidden layers? ODE-fixed weights make one enough","Physics-informed init for PINNs: depth unnecessary for accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000436,"raw_usage":{"total_tokens":2337,"prompt_tokens":1185,"completion_tokens":1152,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":801,"completion_tokens_details":{"reasoning_tokens":1084}},"tokens_in":801,"tokens_out":1152,"duration_ms":10742,"temperature":1.0,"reasoning_tokens":1084,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:48:30.668353+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Freeze the hidden layer after Algorithm 2 on an ODE with a known analytic solution, compute the full derivative sum in Eq. (14) at every grid point, and compare it to the one-neuron approximation in Eq. (15). If the relative mismatch is large, say above 10%, on any part of the domain—especially with $\\Delta\\zeta=\\ln(2+\\sqrt{3})/2$—then the localization assumption that underlies the output-weight formula is broken, and the claimed source of the method's accuracy is not present.","supporting_citations":[{"cited_title":"Approximation capability of two hidden layer feedforward neural networks with fixed weights,","cited_arxiv_id":null,"evidence_quote":"It provides the fixed-hidden-weight approximation result that justifies freezing the hidden layer in the deterministic initialization."},{"cited_title":"Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks,","cited_arxiv_id":null,"evidence_quote":"It supplies the gradient-normalization analysis that the paper adapts as the GN1 and GN2 weighting schemes."},{"cited_title":"Extreme learning machine: Theory and applications,","cited_arxiv_id":null,"evidence_quote":"It supplies the residual-based attention weighting that inspired the predicted-solution and relative-residual weighting schemes."}],"review_version":1}