{"id":"71afcb80-8fad-4e19-8baa-064ba212cda3","arxiv_id":"2507.12659","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Retraining only the final layer of a PINN on a few high-loss collocation points from the adjacent validation interval, plus a learnable activation function, reduces extrapolation error on Allen-Cahn, KdV, and Burgers benchmarks, but the claimed 40-50% average reductions are only achieved when…","lead":"This paper proposes a transfer learning step, retraining only the last layer of a physics-informed neural network on a few high-error points just outside the training time range, together with an adaptive activation function. It reports large reductions in extrapolation error on three benchmark PDEs, though most of the gain comes from the activation function choice rather than the transfer step alone.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 40%/50% gains compare the per-PDE best AF+TL against vanilla tanh, with AF/hyperparameters selected on the extrapolation interval, so the headline overstates the method's practical improvement.","rationale":"Reader's conditional verdict is appropriate. The paper's strongest support is the reproducible code, hard-constraint formulation, and consistent moderate gains from TL in the same-AF comparisons; I do not see an internal inconsistency that would warrant rejection. The main soft spot is the effect-size claim: the abstract's 40%/50% averages are computed as vanilla-tanh-without-TL vs. the per-PDE best-AF-with-TL, and the per-PDE best AF is identified using extrapolation-domain error. This is not a fraud accusation; it is a standard model-selection leakage concern that makes the headline number an upper bound rather than an expected result for a new equation. The reader's weakest_assumption focused on cross-domain transferability; I partially agree, but the more testable and more load-bearing issue is the test-domain selection and baseline mismatch, because even if the transfer heuristic is sound, the paper's headline magnitude is not established. A validation-only selection protocol would settle this. Thus verdict remains CONDITIONAL.","tokens_in":12704,"tokens_out":7345,"duration_ms":86535,"concrete_test":"Using the released code, rerun each PDE with all protocol choices frozen before seeing (0.8,1]: select the final-layer AF and TL hyperparameters by lowest error on the validation interval (0.5,0.8] only, then report L2/MAE on (0.8,1] for that pre-registered choice. Also tabulate the paired same-AF with/without TL averages. If the validation-selected AF differs from the reported best AF, or the average test improvement drops below ~20% for L2, the headline 40%/50% figures are not a reliable estimate of the method's practical gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Tables 1, 3, and 5 report the 'best' AF for each PDE after evaluating extrapolation error on (0.8,1]; the abstract's 40%/50% average is the gap from vanilla tanh without TL to that test-selected best AF with TL. This conflates two effects and makes the headline an oracle estimate. Recomputing same-AF with/without TL gives roughly 15-25% L2 reductions (e.g., AC lctanh 0.16→0.12, KdV lc(x+sin²) 0.10→0.08, Burgers lctanh 0.07→0.06), and Burgers x+sin² shows no TL gain (0.08→0.08). A practitioner facing a new PDE has no reference solution in the extrapolation domain to choose lctanh for AC/Burgers or lc(x+sin²) for KdV; the method as stated provides no selection rule that preserves the reported gains. The paper's own conclusion concedes only 1D regular domains, but the more immediate load-bearing issue is that the central quantitative claim depends on test-domain model selection and a mismatched baseline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a transfer learning (TL) method to improve PINN extrapolation. After training on the temporal domain [0,0.5] with hard-enforced initial/boundary conditions, the method samples 80 collocation points with the highest PDE residual in the validation interval (0.5,0.8], then fine-tunes only the final layer and the parameters of an adaptive activation function on those points, using L2 regularization to limit catastrophic forgetting. It also studies adaptive linear combinations of standard activation functions in the final layer. Experiments on the Allen-Cahn, Korteweg-de Vries, and viscous Burgers equations report relative L2 and MAE errors on the extrapolation interval (0.8,1], and the abstract claims an average 40% reduction in relative L2 error and 50% reduction in MAE. The code is released.","tokens_in":12973,"tokens_out":5090,"duration_ms":55329,"significance":"If the claimed gains are robust, the method would be a low-cost and practically useful way to improve PINN extrapolation, and the release of code plus the hard-constraint encoding are useful contributions. However, the central quantitative claim is currently not supported as stated because the headline numbers mix the effect of the transfer-learning step with a test-selected activation-function choice, and the same-activation-function TL gains are considerably smaller and sometimes negligible. The paper is transparent about its 1D, regular-domain scope, which is a strength, but the evaluation protocol needs to be reworked before the headline claims can be accepted.","major_comments":[{"comment":"The headline 40%/50% reductions are computed by comparing the best per-equation AF with TL against vanilla tanh without TL, so the improvement conflates changing the activation function with the TL contribution. For the same AF, the with-/without-TL differences are smaller and in one case zero: AC lctanh L2 0.16 to 0.12, KdV lc(x + sin2(x)) L2 0.10 to 0.08, Burgers lctanh 0.07 to 0.06, and Burgers x + sin2(x) 0.08 to 0.08. Please report paired same-AF comparisons as the primary evidence for the TL benefit and revise the abstract and introduction claims accordingly.","section":"§5.1–5.3, Tables 1, 3, 5; Abstract"},{"comment":"The choice of AF and of linear-combination size is made after evaluating extrapolation error on the test interval (0.8,1]; for example, the text reports that 'the best ABU-PINN was...' and that 'a linear combination of tanh provided the best extrapolation accuracy.' Since a practitioner solving a new PDE has no reference solution on the extrapolation interval, the reported best-case numbers are oracle estimates. Provide a selection rule based only on training/validation data, or report the average over all candidate AFs, and show the resulting average gain.","section":"§5.1, §5.2, §5.3"},{"comment":"The method's central premise is that the 80 highest-loss collocation points in the validation interval are the most informative points for fine-tuning, but the paper does not compare this selection against random or uniformly spaced points with the same budget. Without such an ablation, the contribution of the residual-based selection, as opposed to the TL fine-tuning itself, is not established.","section":"§4.2"},{"comment":"Results are reported as averages over ten runs with no standard deviations or per-run values. Some same-AF TL differences (e.g., Burgers x + sin2(x) L2 0.08 to 0.08) are within plausible run-to-run variability, so the reader cannot assess whether the TL gains are statistically meaningful. Add uncertainty estimates and, preferably, paired comparisons across seeds.","section":"§5, Tables 1, 3, 5"}],"minor_comments":[{"comment":"The terms 'catastrophic forgetting rate' and 'extrapolation error reduction' are used without definitions; please define them and state exactly how they are computed.","section":"Appendix A, Table S1"},{"comment":"The claim that altering the AF throughout the network can negatively affect convergence is supported by 'our findings (omitted)'; either include the supporting data or remove the unsupported assertion.","section":"Appendix D"},{"comment":"The layer-wise gradient analysis for the AC equation shows that the final three layers have the largest gradient norms, but the method freezes all but the final layer; the text does not explain why fine-tuning only the final layer rather than the last three follows from this figure.","section":"Appendix D, Figure S16"},{"comment":"The claim that the method adds no significant computational cost is supported only for the AC equation; please qualify the claim or provide timings for KdV and Burgers.","section":"Abstract, Table S2"},{"comment":"The abstract describes TL as being applied 'within an extended training domain,' while Section 4.2 states that points are sampled exclusively from the validation interval; please align this terminology.","section":"Section 3 and Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The underlying idea is potentially useful, but the core empirical claim needs to be re-benchmarked before publication. The reported gains are not attributable to the TL component as written, and the AF selection protocol is test-informed. I would be willing to review a revision that reports same-AF paired results, a validation-only selection rule, and a random-sampling ablation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the headline numbers are oracle numbers. They compare the best activation (selected on the unseen test interval) plus transfer learning against vanilla tanh without transfer learning. Same-activation with/without TL gives 15-25% L2 improvements, and Burgers has a zero-gain case (x+sin²). The paper has a real nugget—final-layer retraining on high-loss validation points costs almost nothing and helps—but the abstract oversells it.\n\nWhat's new: the specific combination of final-layer TL on high-loss collocation points from an adjacent validation interval with an adaptive linear-combination activation function is not in the cited prior work. The empirical claim that a single preferred AF (lctanh for AC/Burgers, lc(x+sin²) for KdV) beats diverse ABU-PINN combinations is a useful negative result. They also compare against three SOTA methods, run ten repetitions, and share code. That is real evidence and I give them credit for it.\n\nThe load-bearing issue is the selection rule. The paper picks the best AF per PDE after evaluating extrapolation error on (0.8,1]. A practitioner facing a new PDE has no reference solution in the extrapolation domain, so the method as stated provides no way to reproduce the 40% gain. The same problem affects the SOTA comparisons in Tables 2, 4, and 6, which use the test-selected AF; those numbers are favorable to the authors. The transfer premise itself—that final-layer fine-tuning on (0.5,0.8] transfers to (0.8,1]—is plausible but only supported on three 1D problems; the layer-wise gradient analysis is AC-only. The authors honestly acknowledge the 1D limitation, but the more immediate issue is the missing selection rule.\n\nI'd send it to review, but I'd ask the authors to reframe the headline as same-AF with/without TL (which gives 15-25%), and either add a validation-based selection rule (e.g., pick AF by validation loss on (0.5,0.8]) or pre-register the AF choice. That would make the contribution clearer and the practical gains reproducible. As is, it is a solid empirical study with an overstated central claim.","headline":"The headline 40/50% numbers are oracle estimates—best AF selected on the test interval plus TL versus vanilla tanh—so the practical gains are real but smaller than advertised.","tokens_in":13505,"tokens_out":2492,"would_cite":false,"duration_ms":26500,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65M99"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that retraining only the final layer of a physics-informed neural network on a few dozen high-residual collocation points from a validation interval, together with an adaptive linear combination of activation functions…","keywords":["physics-informed neural networks","extrapolation","transfer learning","adaptive activation functions","collocation point sampling","partial differential equations","Allen-Cahn equation","Burgers equation"],"falsifier":"Run the identical protocol on a 1D PDE outside the three tested, such as Fisher's equation or a Kuramoto-Sivashinsky variant, with the same time split and the same 80-point budget; if the final-layer fine-tune does not reduce the relative L2 error in (0.8,1.0] compared with no transfer learning, or if the training-domain error rises by more than the few percent reported here, the transfer premise fails.","tokens_in":12472,"feed_emoji":"📉","tokens_out":11560,"duration_ms":108752,"temperature":0.7,"pith_summary":"Physics-informed neural networks solve PDEs by training on the equation's residual, but they extrapolate poorly beyond the training time window. This paper argues that the extrapolation failure can be largely repaired at almost no extra cost: freeze all network layers except the last, pick the 80 collocation points in a neighboring validation interval with the largest PDE residual, and retrain only the final layer on those points while letting its activation function be a learnable linear combination of standard activations. On the Allen-Cahn, Korteweg-de Vries, and viscous Burgers equations, the authors report an average 40% reduction in relative L2 error and a 50% reduction in mean absolute error in the held-out extrapolation domain, with only a few extra minutes of training. If the transfer step works as claimed, it would make PINN extrapolation a cheap calibration problem rather than a full retraining problem.","feed_headline":"Retraining one layer cuts PINN extrapolation error by 40%","feed_subtitle":"A physics-informed neural net, frozen except for its last layer, retrains on 80 hard points to see past its window.","key_machinery":"The machinery is two-part: a transfer-learning protocol and an adaptive activation function. The protocol samples collocation points uniformly in [0,T_val), ranks them by PDE residual, keeps the 80 highest-loss points, freezes every layer except the last, and fine-tunes that layer with Adam and L2 regularization for 120-150 epochs. The activation function in the final layer is f(x)=sum_i G(alpha_i) sigma_i(beta_i x), where G is the softmax gate and alpha_i, beta_i are learnable weights and slopes; the best performers in the benchmarks are linear combinations of copies of one PDE-suitable activation, such as tanh for Allen-Cahn and Burgers and x+$sin^{2}$(x) for KdV. Hard-constraint changes of variables such as u(t,x)=$x^{2}$ cos(pi x)+t(1-$x^{2}$)v(t,x) remove the data-loss versus residual-loss trade-off, so the transfer phase is driven by the PDE residual alone.","core_discovery":"The central claim is that targeted transfer learning plus an adaptive activation function substantially improves PINN extrapolation. The time domain is split into training [0,0.5], validation (0.5,0.8], and test (0.8,1.0]. After training with hard-encoded initial and boundary conditions, the authors freeze all weights except the final layer, sample 4,000 collocation points uniformly in [0,0.8), keep the 80 with the highest PDE loss, and fine-tune the final layer with Adam and L2 regularization for 120 to 150 epochs. In that phase the final-layer activation is a softmax-weighted linear combination of standard activations, and the best performer is a linear combination of copies of the single activation type best suited to each PDE, such as tanh for Allen-Cahn and Burgers and x + $sin^{2}$(x) for KdV. Across the three benchmarks this reduces the relative L2 extrapolation error by an average of 40% and the mean absolute error by 50% compared with the same network without transfer learning, and it outperforms the DPM, SA-PINN, and causal sweeping baselines reported in the paper.","pith_inferences":["If the premise generalizes, any time-dependent PINN could be patched for longer prediction by periodically selecting high-residual points at the edge of the solved window and re-fitting only the readout layer, a cheap continual-learning schedule.","The reported averages hide per-benchmark heterogeneity: with tanh alone, transfer learning moves L2 from 0.20 to 0.17 on Allen-Cahn, 0.15 to 0.13 on KdV, and 0.09 to 0.08 on Burgers, so the headline gain depends on pairing transfer learning with the adaptive activation.","Because the paper itself flags that hard-constraint encoding and point selection become harder in higher dimensions, a natural test is whether the same 80-point budget and final-layer-only update survive a two-dimensional or irregular-domain PDE.","One could also check whether validation-interval residual magnitude predicts test-interval improvement: if the 80 highest-loss points do not lie where extrapolation later fails, the selection criterion would need to be revised."],"forward_implications":["If the method transfers, any PINN trained on an initial time window can be patched for extended-time prediction by a short final-layer fine-tune on a few dozen residual-maximizing points, without re-optimizing the whole network.","The best extrapolation accuracy came from adaptive linear combinations of the activation function already suited to the PDE, not from mixing many diverse activations, which suggests activation selection should be shaped by the PDE's qualitative behavior.","The method's computational overhead is small: on the Allen-Cahn benchmark, reported training time increases from 51.6 to 60.7 minutes with the improved activation and transfer learning.","Across the three test equations the method reports lower extrapolation error than the DPM, SA-PINN, and causal sweeping baselines on the same benchmarks.","Because initial and boundary conditions are encoded exactly, the method avoids balancing data versus residual weights in the transfer phase, simplifying the loss."],"supporting_citations":[{"why":"Defines the PINN loss and architecture that the method modifies and benchmarks against.","marker":"[20]"},{"why":"Supplies the adaptive blending unit, the softmax-weighted linear combination of activations that the paper specializes to PDE-preferred single activations.","marker":"[22]"},{"why":"Provides the DPM extrapolation training baseline and the convention of reserving the final portion of the time domain for testing.","marker":"[12]"},{"why":"Provides the self-adaptive PINN baseline compared on Allen-Cahn and Burgers.","marker":"[16]"},{"why":"Provides the causal sweeping baseline compared on Allen-Cahn and KdV.","marker":"[18]"},{"why":"Documents PINN extrapolation failure and growing error with time, motivating the transfer-learning setup.","marker":"[8]"},{"why":"Supplies elastic weight consolidation, the catastrophic-forgetting baseline that L2 regularization outperforms in the supplementary comparison.","marker":"[13]"},{"why":"Justifies freezing early layers and fine-tuning the final layer as a transfer-learning practice.","marker":"[6]"},{"why":"Introduces the x+sin^2(x) activation that the paper uses and adapts for the KdV equation.","marker":"[15]"}],"fun_headline_variants":["Transfer learning slashes PINN extrapolation error by 40%","Retrain one layer, get 40% better PINN extrapolation","Adaptive activation + transfer learning: PINNs extrapolate 40% better","Fine-tune last layer of PINN with 80 points for extrapolation boost","PINN extrapolation error down 40% using transfer learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fine-tuning only the final layer on 80 high-residual collocation points drawn from the validation interval (0.5,0.8] transfers knowledge to the truly unseen extrapolation interval (0.8,1.0] without disrupting the dynamics learned on [0,0.5].","fun_headline_variants_meta":{"raw":{"variants":["Transfer learning slashes PINN extrapolation error by 40%","Retrain one layer, get 40% better PINN extrapolation","Adaptive activation + transfer learning: PINNs extrapolate 40% better","Fine-tune last layer of PINN with 80 points for extrapolation boost","PINN extrapolation error down 40% using transfer learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000755,"raw_usage":{"total_tokens":3391,"prompt_tokens":1013,"completion_tokens":2378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":2282}},"tokens_in":629,"tokens_out":2378,"duration_ms":17666,"temperature":1.0,"reasoning_tokens":2282,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:42:35.941937+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical protocol on a 1D PDE outside the three tested, such as Fisher's equation or a Kuramoto-Sivashinsky variant, with the same time split and the same 80-point budget; if the final-layer fine-tune does not reduce the relative L2 error in (0.8,1.0] compared with no transfer learning, or if the training-domain error rises by more than the few percent reported here, the transfer premise fails.","supporting_citations":[{"cited_title":"Raissi, P","cited_arxiv_id":null,"evidence_quote":"Defines the PINN loss and architecture that the method modifies and benchmarks against."},{"cited_title":"Kim et al","cited_arxiv_id":null,"evidence_quote":"Provides the DPM extrapolation training baseline and the convention of reserving the final portion of the time domain for testing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the self-adaptive PINN baseline compared on Allen-Cahn and Burgers."},{"cited_title":"Penwarden et al","cited_arxiv_id":null,"evidence_quote":"Provides the causal sweeping baseline compared on Allen-Cahn and KdV."},{"cited_title":"Kirkpatrick et al","cited_arxiv_id":null,"evidence_quote":"Supplies elastic weight consolidation, the catastrophic-forgetting baseline that L2 regularization outperforms in the supplementary comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the x+sin^2(x) activation that the paper uses and adapts for the KdV equation."}],"review_version":1}