{"id":"a781f791-8c35-4f6b-8415-3ff290b553ee","arxiv_id":"2608.11020","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"With a carefully chosen step size, finite-difference derivatives match automatic differentiation in accuracy and beat it in speed and memory for MLP-based physics-informed neural networks.","lead":"The paper tests using finite differences (simple slope estimates) instead of automatic differentiation (the usual method) to compute derivatives inside physics-informed neural networks. On three benchmark equations it finds that finite differences match the accuracy of automatic differentiation, run faster, and use less memory.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Step-size calibration is validated only at initialization; the broad FD/AD interchangeability claim requires the FP32 optimal epsilon to stay inside [1e-6, 1e-1] with a single V-shaped error curve throughout training, which is not verified during shock development.","rationale":"The paper's central claim is empirical: with a properly calibrated step size, FD matches AD on the tested MLP-PINN benchmarks. The strongest condition behind that claim is that the calibration procedure actually finds a good epsilon throughout training. The theoretical analysis of Section 3.2 gives epsilon_opt proportional to epsilon_m^(1/(k+2)) with undetermined function-dependent constants, and Section 3.3 restricts the search to [1e-6, 1e-1] in FP32. The only direct validation of the V-shape and the constant range is in Section 4.2 on a fixed, randomly initialized MLP. The Burgers benchmark is explicitly described as developing a sharp front, which can change the smoothness ratios |f|/|f'''| and |f|/|f''''| by orders of magnitude. If epsilon_opt moves outside the heuristic range, the calibration silently returns a boundary value and FD's derivative error can become large. Figure 5 shows robustness to recalibration frequency, not to the range being correctly bracketed. The reader's weakest assumption identifies exactly this point, and the proposed test would settle it by tracking the empirical argmin across training. I do not see a demonstrated internal inconsistency or a failed benchmark, so the conditional verdict remains appropriate; no adjustment is needed.","tokens_in":14982,"tokens_out":14373,"duration_ms":137026,"concrete_test":"Run one Burgers 1D training run with the same architecture, batch size 8192, and 20k epochs. At checkpoints 0, 1000, 2000, 5000, 10000, and 20000, compute first- and second-derivative FD RMSE versus the AD reference on an extended grid epsilon in [1e-8, 1] (100 log-spaced values). Record the global argmin and the number of local minima for each derivative order. If the argmin is interior to [1e-6, 1e-1] and the curve is unimodal at every checkpoint, the range assumption is confirmed and the concern is resolved. If the argmin lies at a boundary or the curve is multimodal, rerun the main accuracy comparison with an extended calibration range; if FD still matches AD, the conclusion is robust, otherwise the interchangeability claim must be restricted to problems where the calibrated range is verified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline conclusion—that AD, FD, and eFD produce statistically indistinguishable solutions on MLP-based PINNs—depends on the empirical step-size calibration of Section 3.3 returning a genuinely optimal epsilon at every phase of training. Algorithm 2 only searches the FP32 heuristic range [1e-6, 1e-1] and assumes a single V-shaped FD-error curve with constants C1, C2 in epsilon_opt = C * epsilon_m^(1/(k+2)) of order 1. Section 4.2 validates the V-shape and the constant range on a fixed, randomly initialized MLP at a single snapshot; it does not track epsilon_opt as the network develops a sharp front in Burgers 1D or as training proceeds. If the true minimizer falls outside the searched interval at any epoch, Algorithm 2 returns a boundary value, the FD input-derivative error is no longer negligible relative to optimization noise, and the equivalence with AD could break. The reported L2 agreement on the three benchmarks is consistent with the assumption holding, so this is not a demonstrated failure on those problems, but the broad 'practical settings' claim is only as secure as this unmonitored assumption. Figure 5 tests the recalibration interval, not whether the optimum remains inside the candidate range or whether the error landscape remains unimodal.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates replacing automatic differentiation (AD) with finite differences (FD) for computing the derivative terms in physics-informed neural network (PINN) losses. It develops an empirical step-size calibration procedure, analyzes the truncation/roundoff trade-off theoretically, proposes three FD strategies (FD, eFD, sFD), and compares them against AD on three PINNacle benchmark PDEs (Poisson 2D, Burgers 1D, Heat2D-CG). The main empirical claim is that, with calibrated step sizes, AD, FD, and eFD produce statistically indistinguishable solution accuracies on MLP-based PINNs, while FD is faster and more memory-efficient. The paper further identifies a silent failure mode of the batched PyTorch autograd idiom for architectures with inter-sample dependencies (BatchNorm, self-attention) and shows that FD gives a partial mitigation. The experiments use 25-26 seeds per configuration, matched architectures, and an AD baseline that is competitive with PINNacle's best reported results.","tokens_in":15332,"tokens_out":10172,"duration_ms":95273,"significance":"If the results hold, this is a useful and timely contribution. The equivalence of calibrated FD and AD on several standard PINN benchmarks is a nontrivial observation that challenges the common assumption that AD is indispensable for PINN training, and the measured speed/memory advantages are practically relevant. The paper also gives a clear cautionary demonstration that the grad_outputs=ones idiom is silently incorrect for per-sample derivatives in inter-sample architectures, which is an important pitfall for the community. Strengths of the experimental methodology include the large number of seeds, the recalibration ablation spanning four orders of magnitude, the strong AD baseline, and the explicit reporting of means and standard deviations. The claims are falsifiable and the experimental protocol is described in enough detail to be reproduced; however, code and data are only promised for publication. The two main caveats are that the calibration procedure itself relies on an AD reference, so FD is not fully AD-free, and that the step-size optimality assumptions are validated only on a fixed network at initialization rather than throughout training.","major_comments":[{"comment":"The proposed step-size calibration chooses ε by minimizing the RMSE between the FD approximation and an AD-computed reference. This means the central practical claim that FD is a drop-in replacement for AD is only established for the residual-evaluation phase after an AD-based calibration has been performed. Section 5.4 recommends 'Use FD as the default' without mentioning that the default step size is obtained from an AD oracle. Since the final solution accuracy is measured against reference PDE solutions, this is not full circularity, but it does weaken the 'alternative to AD' framing. The authors should either qualify the recommendation (e.g., state that one AD calibration at initialization is required, or that ε can be chosen from the theoretical scaling with fixed constants), or provide an AD-free calibration procedure. This is load-bearing for the paper's practical conclusion.","section":"§3.3, Algorithm 2; §5.4"},{"comment":"The error-versus-ε analysis is validated only on a fixed, randomly initialized MLP at a single snapshot (Section 4.2). The recalibration ablation in Figure 5 varies the refresh interval, but it never checks whether the minimizing ε remains inside the searched FP32 range [1e-6, 1e-1] or whether the error curve remains V-shaped and unimodal later in training. On Burgers 1D, where a sharp front develops around t≈0.3, the function-dependent prefactor (|f|/|f'''|) can change substantially; if the true optimum leaves the candidate range, Algorithm 2 returns a boundary value and the FD derivative error is no longer negligible relative to optimization noise. The reported agreement with AD on the three benchmarks is consistent with the assumption holding, but the broad 'practical settings' claim requires additional evidence. A concrete fix is to report the calibrated ε as a function of epoch for each benchmark (and ideally the error landscape at several epochs), showing that the optimum is tracked rather than clipped at a range boundary.","section":"§4.2; §3.3; Figure 5"},{"comment":"The central equivalence claim is expressed as 'statistically indistinguishable,' but no formal statistical test or confidence interval is reported. With 26 seeds, pairwise differences of the observed magnitude are very likely insignificant, so the claim is plausible; nonetheless, the word 'statistically' invites a hypothesis test. Reporting 95% confidence intervals for the pairwise L2 differences, or a simple nonparametric test, would make the claim precise. Alternatively, the wording should be softened to 'differences are small relative to seed-to-seed variance' if no test is performed.","section":"Tables 3-5; §5.1"}],"minor_comments":[{"comment":"The statement that eFD has '40% higher batch size' appears inconsistent with the stencil counts: FD uses 3 forward evaluations per collocation point (x, x±ε_GM), while eFD uses 5 (x, x±ε1, x±ε2), which is 67% more evaluations. If center evaluations are shared differently or if the comparison is against a different baseline, the accounting should be clarified.","section":"§3.4"},{"comment":"Figure 1's caption reports an L2 relative error of 1.15% for sFD, while Table 2 reports 1.11% ± 0.39% as the mean over 26 seeds. Please clarify whether Figure 1 shows a representative seed, the mean solution, or a different metric.","section":"Figure 1; Table 2"},{"comment":"The manuscript contains numerous typographical errors and awkward phrasings (e.g., 'vinicty,' 'unknwon,' 'durign,' 'substaintally,' 'the on that grid,' 'Similary'), and the text in several places appears to have had spacing stripped, making it difficult to read. A careful proofreading and formatting pass is needed.","section":"Throughout"},{"comment":"The data and code availability statements say the artifacts 'will be made available upon publication.' For a computational paper whose central claims depend on exact training configurations, I encourage the authors to provide the code and data with the revision, rather than only at publication, to allow reviewers to verify the results.","section":"§7-8"},{"comment":"In Appendix B, the FD approximation still disagrees with the per-sample AD reference by 0.79 for BatchNorm and 2.1 for attention. The 'order of magnitude closer' wording is accurate for BatchNorm and roughly a factor of three for attention, but the reader should be reminded that FD is a partial mitigation, not a correct per-sample derivative, as the authors themselves note.","section":"§3.5; Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The core numerical result appears credible and well-supported by the reported experiments. My main concerns are fixable: the AD-based calibration should be acknowledged and the general interchangeability claim should be backed by diagnostics on ε during training, rather than only at initialization. I do not see evidence of misconduct; the requests above are for additional evidence and qualification rather than correction of a demonstrated error."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a genuinely useful paper on a practical question—whether FD can replace AD in PINNs—and it mostly delivers. The genuinely new pieces are the runtime epsilon calibration, the periodic recalibration, and the identification of the grad_outputs=ones bug for BatchNorm/attention architectures. The experiments are carefully done: 25-26 seeds, matched architectures, a strong AD baseline, and a recalibration ablation spanning four orders of magnitude. The claim that calibrated FD matches AD on MLP-based PINNs holds on the three benchmarks tested, and I believe it. The failure-mode identification for inter-sample dependencies is the most original contribution and is well-supported by the appendix experiments.\n\nThe soft spots are real but not fatal. First, no code or data are provided; the paper says they will be made available upon publication. That is a reproducibility problem and should be fixed before publication. Second, the epsilon calibration uses AD as the reference, so the FD/AD equivalence is partly built in. But the final solution accuracy is measured against reference PDE solutions, so the circularity is only partial; I do not think it invalidates the conclusion. Third, the generalization from three benchmarks to \"interchangeable in practical settings\" is a bit broad, especially since the paper itself limits the analysis to first/second derivatives and FP32. I would like to see a fourth problem or a more carefully scoped claim.\n\nThe stress-test concern about epsilon_opt moving outside the candidate range during training is worth checking, but I think the paper already partly addresses it: they recalibrate periodically, and the ablation shows robustness to the recalibration interval. The unverified part is whether the true optimum stays inside [1e-6,1e-1] during shock development in Burgers. The results suggest it does—FD matches AD on that problem—but they never measure epsilon_opt across training. That is a minor gap, not a load-bearing flaw.\n\nI also credit the authors for reporting sFD's failures on Burgers and Heat2D-CG rather than overselling it. The Poisson result is interesting but clearly problem-dependent.\n\nVerdict: this deserves serious peer review. The practical finding is important, the methodology is careful, and the bug identification is original. I would recommend acceptance with minor to moderate revisions, mostly around code/data availability and tempering the generalization claim. Send it to a referee who knows PINNs and one who knows numerical differentiation.","headline":"Solid paper on a practical question; the FD/AD equivalence holds on the tested benchmarks, and the main fixes are code release and a narrower generalization claim.","tokens_in":99,"tokens_out":2058,"would_cite":true,"duration_ms":53339,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Finite-difference derivatives, computed with a calibrated step size, match automatic-differentiation accuracy in physics-informed neural networks.","keywords":["physics-informed neural networks","finite differences","automatic differentiation","step-size calibration","stochastic finite differences","per-sample derivatives","batch normalization","self-attention"],"falsifier":"Track the finite-difference error against step size at several epochs of a single FP32 training run on a problem whose solution develops a sharp front; if the empirical optimal step ever leaves the $[10^{-6},10^{-1}]$ range, or the error curve develops a second minimum, the calibrated FD method will not remain statistically indistinguishable from AD on that problem.","tokens_in":14729,"feed_emoji":"🧮","tokens_out":10912,"duration_ms":91878,"temperature":0.7,"pith_summary":"This paper investigates whether finite differences can substitute for automatic differentiation inside physics-informed neural networks. It argues that they can, provided the finite-difference step size is chosen carefully and recalibrated as training changes the network's smoothness. On three benchmark PDEs, finite differences, exact-order finite differences, and automatic differentiation produce statistically indistinguishable solution accuracy, while finite differences run faster and use less memory. A stochastic variant helps on a stationary problem with a solution-collapse failure mode, but hurts on shock-forming and time-dependent problems. If the paper is right, practitioners can replace automatic differentiation with finite differences for standard MLP-based physics-informed networks without losing accuracy, and gain speed and memory headroom.","feed_headline":"Calibrated finite differences match autodiff accuracy in PINNs","feed_subtitle":"With a periodically recalibrated step size, finite differences are statistically indistinguishable from autodiff on three PDE benchmarks.","key_machinery":"The central object is the three-point central-difference stencil applied to the network output, evaluated on an enlarged batch of perturbed points in a single forward pass. The load-bearing identity is the error balance between truncation and floating-point roundoff: for a $k$-th derivative the error scales roughly as $C_{\\mathrm{trunc}}\\varepsilon^{2}+C_{\\mathrm{round}}\\varepsilon_m/\\varepsilon^{k}$, whose minimum sits at $\\varepsilon_{\\mathrm{opt}}\\propto \\varepsilon_m^{1/(k+2)}$. Because the constant is function-dependent and changes as training changes the network, the paper calibrates $\\varepsilon$ empirically by comparing FD against an AD reference on logarithmically spaced candidates and recalibrating periodically. For mixed-order PDEs it offers three combination rules: a geometric-mean step size (FD), separate order-specific step sizes (eFD), and log-uniform stochastic sampling between the two optima (sFD); the last acts as scale-local regularization.","core_discovery":"On MLP-based PINNs, properly calibrated finite-difference derivative computation is interchangeable with automatic differentiation: on all tested benchmarks the solution differences among AD, FD, and eFD are smaller than seed-to-seed variance, so the two schemes cannot be told apart by their solutions, only by their cost. The optimal step size follows $\\varepsilon_{\\mathrm{opt}}\\propto \\varepsilon_m^{1/(k+2)}$ for a $k$-th derivative, and the paper closes the undetermined constant with an empirical search over logarithmically spaced candidates that is repeated periodically during training. For architectures with inter-sample dependencies, the common batched autograd idiom with a vector of ones as the output gradient gives silently wrong per-sample derivatives; the correct per-sample treatment costs $O(B)$ separate backward passes, while finite differences provide a forward-only approximation that is empirically about an order of magnitude closer to the true derivative on BatchNorm networks and about three times closer on attention networks. The paper reads this as evidence that AD is not essential for training PINNs, against a recently argued position.","pith_inferences":["Editorial inference: the interchangeability result suggests the same calibrated-FD recipe could apply to any derivative-based scientific ML loss where the architecture is sample-independent, such as gradient penalties or score matching, as long as the step size is recalibrated to that loss's scale.","Editorial inference: the silent per-sample-gradient failure is likely broader than the two architectures tested; any layer that mixes information across a batch or sequence, including other normalization schemes, deserves the same FD-versus-reference diagnostic.","Editorial inference: if the error curve becomes multimodal as solutions develop sharp features, a natural extension is to make the step size a learned or scheduled multi-scale distribution rather than a single recalibrated value, turning sFD's noise into a deliberately adaptive prior.","Editorial inference: a direct test no one reported would compare FD and AD on fourth-order PDEs in FP32, where roundoff amplification scales as $\\varepsilon^{-4}$ and the paper's own limitations section leaves the $\\varepsilon_m^{1/6}$ scaling open."],"forward_implications":["On standard MLP PINNs, FD and eFD can be dropped in for AD with periodic step-size recalibration, matching AD's accuracy while cutting per-step time and peak memory on the tested range.","eFD, which assigns each derivative order its own optimal step size, remains within seed-to-seed noise of AD on mixed-order PDEs, so it is a safe choice when a PDE contains both first and second derivatives.","sFD is not a general replacement: it improves the stationary Poisson benchmark but degrades accuracy on shock-forming and time-dependent problems, so it should be reserved for collapse-prone stationary problems.","For networks with BatchNorm or attention, the batched vector-of-ones autograd idiom is wrong; the only exact AD route is a per-sample loop that becomes impractical at PINN batch sizes, while FD offers a forward-only approximation with much smaller error.","Because FD keeps memory nearly constant in derivative order and batch size, it enables batch sizes an order of magnitude or more beyond what correct AD can reach on inter-sample architectures."],"supporting_citations":[{"why":"Provides the canonical V-shaped finite-difference error curve that the paper's step-size calibration reproduces for neural networks.","marker":"Baydin et al. [2018]"},{"why":"Earlier work replacing AD with central differences in PINNs, which this paper extends by adding step-size calibration.","marker":"Lim et al. [2022]"},{"why":"Reports FD-PINN accuracy and convergence gains on steady incompressible flow, supporting the viability of FD in PINNs.","marker":"Jiang et al. [2023]"},{"why":"Shows a forward-pass-only finite-difference estimator speeds up derivative computation in large-scale generative models, supporting the cost claims.","marker":"Wang et al. [2025]"},{"why":"The counterargument that AD is essential for training PDE-solving networks, which the paper directly contests.","marker":"Chen et al. [2025]"},{"why":"Provides the theoretical result that FD-PINNs correspond to true solutions of discrete PDEs on their grids, framing FD as a legitimate numerical scheme.","marker":"Langer [2026]"},{"why":"Supplies the three benchmark PDEs and the reference solutions used in all accuracy comparisons.","marker":"[Hao et al., 2024]"},{"why":"Defines BatchNorm, whose inter-sample statistics create the per-sample derivative failure examined in Section 3.5.","marker":"[Ioffe and Szegedy, 2015]"}],"fun_headline_variants":["Finite differences match autodiff in PINNs, run faster","FD matches AD accuracy in PINNs, uses less memory","AD is silently wrong for dependent samples; FD fixes it","Finite differences beat autodiff for PINNs with BatchNorm","FD equals AD on PINNs; AD fails on dependent data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The calibration procedure assumes that the finite-difference error as a function of step size stays V-shaped with a single minimum inside the candidate range $[10^{-6},10^{-1}]$ throughout training, and that the unknown constant in the optimal-step-size formula remains of order one, so the recalibrated step size stays near the true optimum.","fun_headline_variants_meta":{"raw":{"variants":["Finite differences match autodiff in PINNs, run faster","FD matches AD accuracy in PINNs, uses less memory","AD is silently wrong for dependent samples; FD fixes it","Finite differences beat autodiff for PINNs with BatchNorm","FD equals AD on PINNs; AD fails on dependent data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000748,"raw_usage":{"total_tokens":3306,"prompt_tokens":892,"completion_tokens":2414,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":2328}},"tokens_in":508,"tokens_out":2414,"duration_ms":14723,"temperature":1.0,"reasoning_tokens":2328,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:56:48.973920+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Track the finite-difference error against step size at several epochs of a single FP32 training run on a problem whose solution develops a sharp front; if the empirical optimal step ever leaves the $[10^{-6},10^{-1}]$ range, or the error curve develops a second minimum, the calibrated FD method will not remain statistically indistinguishable from AD on that problem.","supporting_citations":[],"review_version":1}