{"id":"18092ee3-21eb-44e8-9e00-e31e25d83988","arxiv_id":"2501.09395","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ELM-DeepONet trains DeepONets by fixing branch and trunk weights and solving a pseudoinverse least-squares problem for a linking matrix, reporting faster and often more accurate results than backprop-trained DeepONets.","lead":"A new training scheme, ELM-DeepONet, fixes most of a DeepONet's weights at random and learns only a small linking matrix by solving a least-squares problem, cutting training time from minutes to fractions of a second on the paper's benchmarks. The method reports lower test errors than standard DeepONets on nonlinear ODE and PDE examples, though several implementation details remain unclear.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Best experiments use p1 > N, outside the Sec. 3 derivation; the min-norm least-squares formula still holds, but the paper neither proves nor specifies the generalized inverse used and reports inconsistent parameter counts.","rationale":"The reader identified the p1 > N regime as the weakest assumption. I agree that this is the most important gap, but I do not think it invalidates the core least-squares construction: the Moore-Penrose solution W = T^+ G B^+ minimizes ||TWB - G||_F in Frobenius norm for full-rank T and B even when p1 > N, because it is the pseudoinverse of the linear map W -> TWB. The real problem is that the paper's derivation and its stated formula for B^+ are limited to p1 <= N, while the experiments and the conclusion explicitly rely on p1 > N without giving the correct generalized inverse or code. A second, independent inconsistency is the parameter count: the learned W has p1*p2 parameters, which is 100K for the antiderivative setup, not 20K. Either the hyperparameters differ from those stated or the table counts only a subset of parameters; the paper does not say. These issues do not refute the central claim, but they make the 'drastically reduced cost' and 'superior accuracy' claims impossible to verify from the manuscript alone. One additional subtlety worth noting: when p1 >= N, the training-stage predictions equal T T^+ G, independent of B, so the branch network only affects test-time generalization through the projection coefficient B^+ b_test. This is a legitimate kernel-style mechanism, but it differs from the paper's narrative that the branch features and trunk features jointly parameterize the operator. It reinforces the need for a reproducible implementation. Therefore, the CONDITIONAL verdict is appropriate; the issues are addressable with an explicit algorithm, code, and corrected parameter reporting.","tokens_in":9457,"tokens_out":18276,"duration_ms":170613,"concrete_test":"Re-implement the antiderivative experiment with N = 1000, p1 = 1000, p2 = 100 using W = pinv(T) @ G @ pinv(B), where pinv is the Moore-Penrose pseudoinverse. Then (1) verify that pinv(B) equals (B^T B)^{-1} B^T and that the Sec. 3 formula B^T(BB^T)^{-1} is undefined because BB^T is singular; (2) confirm that the training predictions equal T @ pinv(T) @ G, independent of B; (3) rerun the p1 = 10000, p2 = 50 configuration and report the exact training time and the number of entries in W. If these numbers match the paper's Tables 1 and 6, the issue is only a missing proof; if not, the reported speed and accuracy are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 derives W = T^+ G B^+ only under the assumptions p1 <= N and p2 <= M, where B has a right inverse B^+ = B^T(BB^T)^{-1} and T has a left inverse. The sensitivity analysis (Tables 3 and 4) and the inverse-source experiment use p1 = 5000-10000 with N = 1000, explicitly violating the stated condition, and the conclusion admits that this 'violates' the theory. This is load-bearing because, for p1 > N, the Sec. 3 expression B^T(BB^T)^{-1} is not computable: BB^T is singular. The correct generalized inverse is the left pseudoinverse B^+ = (B^T B)^{-1} B^T, and W = T^+ G B^+ is still the minimum-Frobenius-norm least-squares solution when B has full column rank. So the central formula survives, but the paper as written does not specify which solve is actually performed, making the reported 0.14 s training times and best errors hard to reproduce independently. The parameter counts compound the problem: with p1 = 1000 and p2 = 100, the learned matrix W alone has p1*p2 = 100,000 entries, not the 20K listed in Table 1; the reported count appears to refer to something else without saying what. These are reproducibility gaps rather than fatal mathematical errors, but they directly affect the support for the central claim of drastic cost reduction and superior accuracy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ELM-DeepONet, a DeepONet variant in which the branch and trunk networks have fixed (random or predefined) weights and a new linear layer W ∈ R^{p2×p1} is the only trainable parameter. Training is posed as minimizing the Frobenius norm objective ||T W B − G||_F^2, and the authors derive the closed-form solution W = T† G B† under assumptions p1 ≤ N and p2 ≤ M. Experiments on an antiderivative problem, a nonlinear ODE, Darcy flow, and a parabolic inverse source problem report lower test errors and substantially shorter training times than backprop-trained DeepONets, together with a sensitivity analysis of p1 and p2 and sinusoidal trunk-basis variants.","tokens_in":9807,"tokens_out":6183,"duration_ms":60925,"significance":"The core idea is simple and appealing: reformulating DeepONet training as a single least-squares solve over fixed random features is a natural extension of ELM to operator learning. The paper deserves credit for giving an explicit derivation in the p1 ≤ N, p2 ≤ M regime, for comparing several trunk choices, and for reporting large training-time reductions. However, the headline best results are obtained in the p1 > N regime, where the paper's own pseudoinverse formula is not computable as written, and the exact numerical procedure used there is not specified. The parameter-count tables also contain inconsistencies. If these gaps are fixed, the method would be a useful fast baseline for operator learning, but the current evidence is limited to four synthetic benchmarks and the central cost/accuracy claims are not yet fully reproducible.","major_comments":[{"comment":"Under p1 ≤ N and p2 ≤ M, the derivation of W = T† G B† is mathematically correct. However, the sensitivity analysis uses p1 = 5000 and 10000 with N = 1000, and the inverse source experiment uses p1 = 10000, so BB^T is singular and the stated formula B† = B^T (BB^T)^{-1} is not computable. The paper acknowledges in Section 5 that this violates the theoretical condition, but it does not specify which generalized inverse or regularization is actually used in those experiments. Because the best reported errors and the 0.1–0.15 s training times come from this p1 > N regime, please state the exact solve (e.g., left pseudoinverse (B^T B)^{-1} B^T with ridge regularization), the rank assumptions made, and whether the reported timings include that solve. Without this, the central claim is not reproducible.","section":"Sec. 3, Eq. (1)–(2); Sec. 4.2, Tables 3–4; Sec. 4.4"},{"comment":"For the antiderivative example, the text sets p1 = N = 1000 and p2 = M = 100, which gives W exactly p1*p2 = 100,000 entries. Table 1 nevertheless reports 20K parameters for ELM-DeepONet. Please clarify what the reported count represents (trainable W only, fixed networks included, or a different p1/p2 setting). As written, the parameter comparison and the Section 3 claim that p1p2 is 'significantly smaller' than vanilla DeepONet cannot be evaluated.","section":"Sec. 4.1.1, Table 1"},{"comment":"The sensitivity analysis states that relative errors are averaged over 10 trials, but it does not describe a validation protocol for selecting p1 and p2. If the grid search is evaluated on the same test set used for the reported accuracies, the headline errors are optimistically biased. Please specify the train/validation/test split and state explicitly whether hyperparameter selection was performed on a validation set.","section":"Sec. 4.2, Tables 3–4"},{"comment":"The inverse source experiment reports p1 = 10000 and p2 = 50 selected by grid search, but it does not state the number of training functions N or the number of collocation points M. Since the derivation's validity depends on p1 ≤ N and p2 ≤ M, these dataset sizes must be given, along with the exact computation used for W in the p1 > N regime.","section":"Sec. 4.4"}],"minor_comments":[{"comment":"The branch network architectures are written as '1-64-64-64-1', but the branch network in a DeepONet takes an m-dimensional sensor vector as input; please state the value of m and correct the input dimension notation if it is not 1.","section":"Sec. 4.1.1"},{"comment":"The statement that p1p2 is 'significantly smaller' than vanilla DeepONet parameters is not universally true: Table 5 reports 10M parameters for ELM-DeepONet versus 181K for DeepONet-MLP. Please qualify the parameter-count claim.","section":"Sec. 3; Sec. 4.3, Table 5"},{"comment":"The sinusoidal trunk basis includes expressions such as cos(nπx + π/2) and cos(nπx + π/2)sin(nπy); please verify these formulas and state whether they are exactly the functions implemented, since small typographical changes in trigonometric phases would change the basis.","section":"Sec. 4.3"},{"comment":"There are formatting and typographical errors, including the title spacing ('ELM-D EEP ON ETS') and reference list errors such as 'Xin Taia'; a careful proofreading pass is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"I am sympathetic to the paper's core idea, and I believe the main mathematical gap is fixable: the p1 > N case can be handled by specifying the appropriate generalized inverse and regularization. My recommendation of major revision is driven by reproducibility concerns (unspecified solve, inconsistent parameter counts, absent validation protocol) rather than by any suspicion of deliberate misreporting. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on arXiv:2501.09395. The idea is simple and legitimate: fix random features for both trunk and branch, put a learnable matrix W between them, and fit W with one least-squares solve. The math is right in the regime p1 <= N and p2 <= M, and the reported wall-clock times are eye-catching (0.1 s vs 400-1000 s). The author is also candid in the conclusion, noting that the best results violate the stated theoretical condition.\n\nWhat is actually new: I do not know of a prior ELM-DeepONet with this exact two-random-feature-plus-W formulation. It is a close cousin of random feature methods and kernel ridge regression, and the paper does not engage that literature, so the novelty is real but modest.\n\nThe problems are mostly in presentation and reproducibility. First, the derivation uses B^+ = B^T(BB^T)^{-1}, which requires p1 <= N. The sensitivity analysis and the inverse-source experiment use p1 = 5000-10000 with N = 1000 or 2000. In that regime BB^T is singular and the formula is undefined. The correct Moore-Penrose solve (B^T B)^{-1}B^T is still fine, and W = T^+ G B^+ remains the min-norm least-squares solution, so the core method survives, but the paper must say which solve is actually implemented. Second, the parameter counts do not add up: with p1 = 1000 and p2 = 100, W has 100,000 entries, not the 20K listed in Table 1. That looks like an error or a different convention; either way it blocks reconstruction of the cost. Third, comparisons are only against backprop-trained vanilla DeepONets. The paper should position itself against random-feature and kernel operator baselines and cite that literature. Finally, no code or data, and no variance bars in the headline tables, so the accuracy claims are hard to verify as written. The sensitivity tables report averages over 10 trials but no standard deviations, which is minor but easy to fix.\n\nNone of this is fatal. The central argument holds. Whoever referees it should ask for the specifics of the generalized inverse for p1 > N, corrected parameter counts, code, and a comparison to random-feature baselines. I would send it out. It is exactly the kind of paper where a referee can help sharpen a useful but under-specified idea.","headline":"A clean, simple ELM-DeepONet with a correct least-squares core, but the write-up leaves the generalized inverse and parameter counts underspecified in the very regime that produces the best numbers.","tokens_in":10332,"tokens_out":3472,"would_cite":false,"duration_ms":33915,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65F20"],"pacs":[],"model":"deepseek-v4-flash","headline":"DeepONet training reduces to one least-squares solve over fixed random features, and the paper reports this beats backprop-trained DeepONets in accuracy and speed.","keywords":["DeepONets","Extreme Learning Machine","least-squares training","operator learning","Moore-Penrose pseudoinverse","backpropagation-free","forward-inverse problems","fixed basis functions"],"falsifier":"Take an operator whose true outputs lie partly outside the span of the fixed trunk features, e.g. all trunk features even while the target operator has a nonzero odd output component with $p_2=M$; then no $W$ can make $\\|TWB - G\\|_F$ small, and the ELM-DeepONet error would stay far above a backprop-trained DeepONet, contradicting the claim that random fixed features plus a linear map suffice. A second check is to reproduce the antiderivative experiment with $p_1=5000$, where the paper reports low error: the stated formula $B^\\dagger = B^T(BB^T)^{-1}$ is undefined because $BB^T$ is singular, so any finite computed $W$ would show the implementation uses a different solve than the derivation describes.","tokens_in":9221,"feed_emoji":"🧮","tokens_out":14951,"duration_ms":128146,"temperature":0.7,"pith_summary":"The paper tries to establish that a DeepONet can be trained without any gradient-based optimization. By freezing the branch and trunk networks at random initial weights and inserting one extra linear layer between them, operator learning becomes a single least-squares solve for a matrix $W$, with solution $W = T^\\dagger G B^\\dagger$. On several benchmark operators, including nonlinear ODEs, Darcy flow, and an inverse source problem, the method is reported to reach lower or comparable relative $L^2$ error than backprop-trained DeepONets while cutting training time from hundreds of seconds to well under a second. If correct, this would make operator learning much cheaper and shift the question of DeepONet capacity onto the quality of the fixed random features rather than the trained weights.","feed_headline":"One least-squares solve replaces backprop for DeepONets","feed_subtitle":"Fixing random branch and trunk features turns operator learning into a fast matrix solve with lower error.","key_machinery":"The central object is the coupling matrix $W \\in \\mathbb{R}^{p_2 \\times p_1}$. Branch features from the frozen branch network are assembled as $B \\in \\mathbb{R}^{p_1 \\times N}$ for $N$ input functions, and trunk features from the frozen trunk network (or from predefined bases such as sinusoids) are assembled as $T \\in \\mathbb{R}^{M \\times p_2}$ at $M$ collocation points; the whole training objective collapses to $\\|TWB - G\\|_F^2$. Minimizing this gives $W = T^\\dagger G B^\\dagger$, where $T^\\dagger$ and $B^\\dagger$ are Moore-Penrose pseudoinverses, computed as left and right inverses $(T^T T)^{-1}T^T$ and $B^T(BB^T)^{-1}$ under the assumptions $p_2 \\le M$ and $p_1 \\le N$. This machinery turns a neural-network training problem into a linear algebra solve with only $p_1 p_2$ learnable parameters.","core_discovery":"The paper's central claim is that the only genuinely learned part of a DeepONet can be a single matrix $W$ coupling two fixed random feature maps: trunk features $T \\in \\mathbb{R}^{M \\times p_2}$ and branch features $B \\in \\mathbb{R}^{p_1 \\times N}$, so the prediction over the whole training set is $T W B$. Minimizing the Frobenius error $\\|TWB - G\\|_F^2$ over $W$ yields $W = T^\\dagger G B^\\dagger$, computed with Moore-Penrose pseudoinverses under the full-rank conditions $p_2 \\le M$ and $p_1 \\le N$. In the reported experiments the ELM-DeepONet reaches relative $L^2$ errors of 2.12% on an antiderivative operator, 2.91% on a nonlinear ODE, 5.65% on Darcy flow with an MLP branch, and 2.77% on a parabolic inverse source problem, versus 4.12%, 4.87%, 11.77%, and 5.04% for the corresponding vanilla DeepONet baselines; the CNN-branch Darcy variant trails its baseline (6.80% versus 6.32%). Training takes roughly 0.1-3 seconds. The paper also reports that choosing $p_1$ much larger than $N$, beyond the range its derivation covers, improves accuracy, a behavior it flags for future analysis.","pith_inferences":["This suggests, though the paper does not state it outright, that for at least these operator-learning tasks the basis functions do not need to be learned; random fixed features plus a linear readout may carry most of the capacity.","A natural extension, not tested in the paper, is to sample the random features from a distribution matched to the operator's spectrum, since the sinusoidal trunk experiments hint that basis choice matters; this could be verified by comparing singular values of $W$ with the target operator's spectrum.","Because $W$ is fit in one shot, the method should compose with online or incremental learning via pseudoinverse updates, an avenue the paper leaves unexplored.","The reported success of $p_1 > N$ implies the training problem is underdetermined in that regime; checking whether the implementation uses the minimum-norm solution or a regularized solve would clarify how much of the gain depends on that choice, a step the paper does not take."],"forward_implications":["Training time on the reported benchmarks drops to about 0.1-3 seconds on a single GPU, compared with hundreds of seconds for Adam-trained DeepONets.","Relative $L^2$ test errors on the antiderivative and nonlinear ODE problems are roughly half those of the smaller baseline DeepONet (2.12% versus 4.12%, and 2.91% versus 4.87%).","The trunk network can be replaced by fixed basis functions such as sinusoids, and the same least-squares solve still produces competitive errors (2.45% and 3.39% on the two ODE problems).","Using more branch features than training samples ($p_1 > N$) improves accuracy empirically, even though the paper's pseudoinverse derivation assumes $p_1 \\le N$; the best-performing regime is consequently not covered by the derivation.","The method extends to forward-inverse problems: on the parabolic inverse source benchmark the ELM-DeepONet reports 2.52-2.77% relative error compared with 5.04% for the baseline DeepONet."],"supporting_citations":[{"why":"defines the DeepONet branch-trunk architecture and the antiderivative benchmark that the paper replaces with an ELM training scheme","marker":"[10]"},{"why":"supplies the universal approximation result for ELMs that justifies freezing randomly initialized hidden weights","marker":"[18]"},{"why":"precedent for using predefined fixed basis functions in operator networks, which motivates the sinusoidal trunk variants","marker":"[16]"},{"why":"demonstrates ELM-type training inside the PINN framework, the direct precursor the paper extends to DeepONets","marker":"[22]"},{"why":"the Adam optimizer used to train the vanilla DeepONet baselines that the proposed method is compared against","marker":"[24]"},{"why":"establishes well-posedness of the parabolic inverse source problem used as a benchmark in the numerical experiments","marker":"[25]"}],"fun_headline_variants":["ELM-DeepONets: backprop-free training via one least-squares solve","Fix random features, solve for one matrix: DeepONet training speeds up","DeepONet training without backprop: ELM turns it into a matrix solve","Random features + pseudoinverse: ELM-DeepONets cut training cost","No backprop, lower error: ELM-DeepONets solve a linear system"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the fixed random branch and trunk features, together with the chosen $p_1$, $p_2$ and basis functions, are expressive enough for a single linear map $W$ to approximate the target operator; the cleanest derivation also requires $p_1 \\le N$ and $p_2 \\le M$, while the best reported results use $p_1 > N$, leaving the actual computation of $W$ in that regime as an unspecified modeling choice.","fun_headline_variants_meta":{"raw":{"variants":["ELM-DeepONets: backprop-free training via one least-squares solve","Fix random features, solve for one matrix: DeepONet training speeds up","DeepONet training without backprop: ELM turns it into a matrix solve","Random features + pseudoinverse: ELM-DeepONets cut training cost","No backprop, lower error: ELM-DeepONets solve a linear system"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00062,"raw_usage":{"total_tokens":2896,"prompt_tokens":986,"completion_tokens":1910,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":1817}},"tokens_in":602,"tokens_out":1910,"duration_ms":13704,"temperature":1.0,"reasoning_tokens":1817,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:05:00.301735+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an operator whose true outputs lie partly outside the span of the fixed trunk features, e.g. all trunk features even while the target operator has a nonzero odd output component with $p_2=M$; then no $W$ can make $\\|TWB - G\\|_F$ small, and the ELM-DeepONet error would stay far above a backprop-trained DeepONet, contradicting the claim that random fixed features plus a linear map suffice. A second check is to reproduce the antiderivative experiment with $p_1=5000$, where the paper reports low error: the stated formula $B^\\dagger = B^T(BB^T)^{-1}$ is undefined because $BB^T$ is singular, so any finite computed $W$ would show the implementation uses a different solve than the derivation describes.","supporting_citations":[{"cited_title":"Universal approximation using incremental constructive feedforward networks with random hidden nodes","cited_arxiv_id":null,"evidence_quote":"supplies the universal approximation result for ELMs that justifies freezing randomly initialized hidden weights"},{"cited_title":"On stability and regularization for data-driven solution of parabolic inverse source problems","cited_arxiv_id":null,"evidence_quote":"establishes well-posedness of the parabolic inverse source problem used as a benchmark in the numerical experiments"}],"review_version":1}