{"id":"c30d126e-caaf-4a85-ae99-9e548a83525f","arxiv_id":"2607.10735","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A Gaussian process network with GP activation functions can be trained and deployed with O(n q^2) and log(n) costs using a jointly inverse Kalman filter, and it beats standard baselines on several benchmarks.","lead":"GNet is a neural-network-style model whose neuron activations are learned as Gaussian processes, and it is accelerated by a new filter so it never forms large covariance tables. On simulations, UCI data, and a physics emulation problem, it reports better accuracy at far lower cost than exact GPs, approximate GPs, feedforward networks, and KANs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Deep GNet's fixed 10-knot approximation and plug-in last-layer inputs leave the multi-layer model's error unquantified, undermining the central empirical claim.","rationale":"After deriving the JIKF recursions from Lemma 1, the reader's claim about commutativity with \\tilde W_1 is unnecessary: the recursions only reorder products of G_i's, and \\tilde W_1 remains on the right. The reader's identified weakest assumption therefore does not land. However, the deep GNet approximation is a real limitation: Eq. (25) uses 10 fixed knots per hidden neuron, and Eq. (5) conditions on the last-layer inputs, which for L=2 are derived from those approximations. No error bound or convergence result is given, and the empirical superiority of GNet-2L in the UCI and cDFT benchmarks (Figures 4, 6) depends on this approximation being accurate. A knot-sensitivity analysis would reveal whether this is a serious issue. The reader's verdict of CONDITIONAL remains appropriate, but the justification should be the deep-approximation uncertainty, not the commutativity condition.","tokens_in":17346,"tokens_out":11834,"duration_ms":105488,"concrete_test":"For the Branin, DetPep, Park, and cDFT experiments, rerun GNet-2L with m_{l,K} = 10, 20, 50, 100 (keeping other settings fixed) and record held-out NRMSE. If NRMSE changes by >10% when increasing from 10 to 50 knots, the fixed 10-knot choice is not stable and the reported results are specific to that heuristic; if NRMSE is stable, the approximation is adequate. Additionally, for a small-n case (e.g., n=500), compare the GNet-2L predictive mean/variance against a version that integrates out the hidden GP (e.g., by a dense inducing-point approximation with 200 knots or MCMC) to quantify the approximation error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The JIKF recursions in Theorem 1 (Eqs. 15-16) are algebraically correct under pairwise commutativity of the transition matrices; the additional commutativity with \\tilde W_1 mentioned in the reader's weakest_assumption is not required, since \\tilde W_1 always appears on the right of the matrix products in Lemma 1. The genuine load-bearing weakness is in the multi-layer GNet: Section 3 approximates each hidden-layer GP by a fixed set of 10 inducing points (Eq. 25) and the predictive equations (5)-(7) condition on the last-layer inputs \\tilde x_L as known. For L=2, \\tilde x_L is a deterministic function of the approximate hidden-layer draws, so the model is a heuristic: it neither propagates hidden-layer uncertainty nor bounds the approximation error. Since GNet-2L is the best-performing variant in most experiments (Figures 4 and 6), the empirical claims for the deep model rest on an unquantified approximation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces GNet, a Gaussian process regression network in which each neuron's activation function is a GP acting on a one-dimensional projected input. To make the model scalable, the author proposes the jointly inverse Kalman filter (JIKF), an exact O(n q^2) algorithm for DLM-induced covariance-matrix--vector products when transition matrices commute, and derives closed-form gradients for training without forming covariance matrices. Training is further accelerated by combining JIKF with a Nyström-preconditioned conjugate gradient method. The method is tested on simulated nonlinear functions, five UCI regression data sets, and a 201-dimensional classical density functional theory prediction problem, with comparisons to exact and Vecchia-approximated GPs, feedforward neural networks, and Kolmogorov--Arnold networks. The paper claims that GNet achieves lower held-out NRMSE than the non-GNet baselines in most settings and that a pretrained GNet can predict at O(log n) cost per test point.","tokens_in":17615,"tokens_out":9744,"duration_ms":103704,"significance":"If the claims hold, the paper makes a useful contribution. The JIKF recursions in Theorem 1 are elegant and appear correct: I checked the concern that Theorem 1 also requires each G_i to commute with \\tilde W_1; this is not needed because \\tilde W_1 always appears on the right-hand side of the transition products in Lemma 1, and the backward recursion (16) only requires G_{i+1} to commute with later transition matrices, which is implied by (10). The closed-form gradient derivations in Lemmas 2--3 are nontrivial and, if correct, avoid the prohibitive automatic-differentiation graphs of iterative solvers. The empirical study is broad and includes a practically motivated high-dimensional cDFT example. On the negative side, no code or data are provided, and the deep version of the model rests on an unquantified low-rank approximation that is load-bearing for several main empirical claims. The JIKF contribution itself is sufficiently solid that the concerns are addressable in a revision.","major_comments":[{"comment":"For L>1, the model is not a true deep GP but a heuristic finite-rank approximation. Hidden-layer GPs are replaced by 10-knot interpolants with directly optimized whitened values, and the predictive equations condition on the last-layer inputs \\tilde x_L as if they were known. This ignores posterior uncertainty in the hidden layers, and no error bound or sensitivity analysis is given for m_{l,K}=10. Since GNet-2L is the best-performing variant in most of the UCI experiments (Figure 4) and in the cDFT experiment (Figure 6), the central empirical claims for the deep model depend on this unquantified approximation. Please add a knot-convergence study (e.g., m=20,50,100), state clearly that the predictive intervals cover only last-layer noise and not hidden-layer uncertainty, or reframe the deep model as an explicit approximate/parametric model with appropriate qualifications.","section":"Section 3, Eq. (25), and Eqs. (5)--(7)"},{"comment":"The claim that a pretrained GNet predicts each test input in O(log n) time is not substantiated by the presented recursions. Equations (15)--(16) compute b=Σv for all n data locations in O(n q^2); they do not by themselves show how to evaluate a single cross-covariance sum Σ_{i} c(x*, x_i) α_i at an arbitrary test point x* in O(log n) time. The cited Section S3.1 must specify the data structure/algorithm (e.g., prefix sums with sorted inputs and binary search) and any additional assumptions (sorted inputs, commutativity, low-rank preconditioner). This is a central scalability claim for the 'many concurrent users' scenario and should be made precise.","section":"Section 4, Table 1, and Eqs. (15)--(16)"},{"comment":"The definition of NRMSE is nonstandard and likely a typo: the denominator is written as Σ_i (ŷ(x_i) − ȳ)^2, using the model predictions ŷ, rather than the observed values y_i or the latent y_signal(x_i). As written, a model that predicts the constant mean has a zero denominator, and the reported normalized errors are not directly comparable to standard NRMSE in the literature. If this is a typographical error, please correct the denominator to use observed values; if intentional, justify the choice and explain why it does not distort the cross-method comparisons.","section":"Eq. (32)"}],"minor_comments":[{"comment":"The phrase 'For compariosn' should be 'For comparison'.","section":"Section 5, paragraph before Figure 2"},{"comment":"The text says 'we report the the average performance metrics'; remove the duplicated 'the'.","section":"Section 5.1, end of first paragraph"},{"comment":"The symbols k_pcg and m_pcg appear in the table but are not defined in the main text; define them before the table and state how they are set in the experiments.","section":"Section 4, Table 1"},{"comment":"The GNet models use a fixed optimization setting, but some baselines are hand-tuned (e.g., FNN-R learning rates, Vecchia conditioning sizes). This is acceptable, but the text should explicitly say that the comparison is not fully automated/unified across methods and that baseline tuning was chosen to improve baseline performance, not to disadvantage them.","section":"Section 5, benchmark setting"},{"comment":"No code or data availability statement is provided. Given the algorithmic nature of the contribution and the many empirical comparisons, releasing code would substantially strengthen reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The JIKF contribution is sound and the one-layer model is a coherent scalable GP method. The main risk is the deep model's unquantified 10-knot approximation and the O(log n) per-prediction claim, both of which are load-bearing for the paper's broader claims. With a knot-sensitivity analysis and a precise derivation of the per-point prediction algorithm, the paper could be publishable. The self-citation to the inverse Kalman filter [9] is legitimate prior work, but the comparison would be clearer if the relation between JIKF and the earlier IK filter were stated more explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The JIKF algorithm is genuine, the one-layer GNet is on solid ground, and the deep model is the soft spot. The pairwise commutation condition in Theorem 1 is enough; the \\tilde W_1 concern in the stress-test doesn't hold up, since \\tilde W_1 always sits on the right in Lemma 1. The recursions are simple and correct for DLM covariances with commuting transition matrices, and the O(nq^2) cost plus closed-form gradients is a real contribution. The Nyström-PCG integration is sensible, and the log(n) prediction cost for pretrained models is useful.\n\nThe numeric work is broad: simulated functions, UCI regressions, and a cDFT problem with n≈95k and p=201. GNet-1L looks consistently good, and the cDFT results are impressive for a 2,031-parameter model. However, the GNet-2L results need a caveat. Section 3 replaces each hidden GP with a fixed 10-knot predictive process (Eq 25), and the predictive equations then treat the induced last-layer inputs as known. So the two-layer model is an approximation, with no error bound and no propagation of hidden-layer uncertainty. Since GNet-2L is the best method in several experiments, those numbers are evidence about the approximate model, not about an exact deep GNet. That doesn't hurt the JIKF contribution, but it should be stated explicitly and studied.\n\nTwo smaller issues. First, Eq (32)'s NRMSE denominator uses \\hat y - \\bar y rather than y - \\bar y. That makes the scaling depend on the predictions and is non-standard; if that's what was computed, it could distort comparisons. Worth a correction. Second, no code or data are included, and some baselines use non-default hyperparameters. That's not disqualifying, but it makes the empirical claims harder to audit.\n\nOverall, this is a technique-level contribution that deserves a serious referee. The JIKF and closed-form gradients are likely correct and practically useful. I'd engage with the paper, primarily for the one-layer model and the algorithmic machinery. The deep model needs more work before its claims can be taken at face value.","headline":"JIKF is a real O(n q^2) contribution and the one-layer model holds up, but the deep model's fixed 10-knot approximation and a suspicious NRMSE formula mean the multi-layer claims should be read with caution.","tokens_in":18071,"tokens_out":7183,"would_cite":true,"duration_ms":71590,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","62M20","68T07","60G15"],"pacs":[],"model":"deepseek-v4-flash","headline":"GNet replaces neural-network activations with Gaussian processes and uses a new jointly inverse Kalman filter to compute covariance-vector products in O(nq^2), making GP-based networks trainable and predictable without ever forming a covari","keywords":["Gaussian process network","nonparametric neurons","jointly inverse Kalman filter","dynamic linear models","Matérn kernel","closed-form gradients","scalable prediction","density functional theory"],"falsifier":"For a dynamic linear model whose transition matrices commute with each other but not with the stationary covariance W_1, compute Σv directly and via the JIKF recursions; any mismatch would falsify Theorem 1 as stated. Alternatively, in a deep GNet, increase the knot count m_l,K from 10 to 50 and check whether held-out NRMSE changes materially.","tokens_in":17193,"feed_emoji":"⚡","tokens_out":7019,"duration_ms":72032,"temperature":0.7,"pith_summary":"The paper proposes GNet, a network in which each neuron's activation function is a Gaussian process with a one-dimensional input, and shows that such networks can be trained and used at scale. The key enabler is the jointly inverse Kalman filter (JIKF), which computes the product of a DLM-induced covariance matrix with any vector in O(nq^2) operations instead of O(n^2), given commuting transition matrices. Closed-form gradients are also derived, so optimization and prediction avoid forming the n-by-n covariance matrix altogether. In experiments with up to about 95,000 training points and 201-dimensional inputs, GNet models achieve lower held-out normalized RMSE than neural networks, Kolmogorov-Arnold networks, exact Gaussian processes (at small n), and Vecchia-type approximations in most tested scenarios.","feed_headline":"Fast filter makes Gaussian-process networks scale to 95k points","feed_subtitle":"Its jointly inverse Kalman filter computes covariance products in O(nq^2), so no n-by-n matrix is ever formed.","key_machinery":"The jointly inverse Kalman filter (JIKF): two recurrence relations (Eqs. 15–16) that compute the DLM covariance-vector product b = Σv in O(nq^2) time, where q is the latent-state dimension, assuming the transition matrices commute (Eq. 10). It replaces direct O(n^2) matrix-vector multiplication and, combined with closed-form gradients, lets the entire GNet training loop avoid forming the n-by-n covariance matrix. The filter's correctness rests on Lemma 1's factorization of cross-covariances into products of transition matrices times the stationary latent covariance.","core_discovery":"The paper's central claim is that a Gaussian process network with nonparametric neurons is computationally tractable at large scale when the activation kernels come from dynamic linear models. The proof mechanism is the jointly inverse Kalman filter: because the DLM transition matrices commute, the covariance-vector product Σv can be computed exactly by two O(nq^2) recursions, one forward and one backward, without assembling Σ. Together with closed-form loss gradients (Lemmas 2 and 3), training and prediction become matrix-free. Empirically, under a unified optimization setting, the resulting GNet models produce lower held-out normalized RMSE than several mainstream baselines across a range","pith_inferences":["If the JIKF recursions were extended to non-commuting DLM transition families, the same O(nq^2) acceleration could apply to a broader class of state-space and kernel methods, but the paper does not investigate that case.","The strong empirical results for a one-layer, 10-neuron GNet on high-dimensional physical inputs suggest that one-dimensional nonparametric feature maps may be a sufficient inductive bias for certain functionals; testing GNet on other physical quantities would clarify this.","The deep GNet's 10-knot approximation is a pragmatic heuristic; varying the knot count (e.g., from 5 to 50) would provide a sensitivity analysis and reveal bias-variance tradeoffs in new applications.","The paper explicitly leaves reliable uncertainty quantification for deep layers as an open problem; future work could compare GNet's predictive interval coverage with exact Bayesian approaches on a range of datasets."],"forward_implications":["A pretrained GNet can predict each new input in time that grows only logarithmically with the training sample size, making it feasible to serve many concurrent users.","Closed-form gradients let practitioners train GNet without automatic differentiation through iterative filters, avoiding excessive memory and computational graphs.","GNet brings GP-based predictive modeling to sample sizes around 10^5 with high-dimensional inputs, a regime where exact Gaussian processes are computationally prohibitive.","Because the final layer is an exact Gaussian process, GNet provides predictive intervals at modest extra cost, unlike standard neural-network baselines in the study.","The 10-knot representation of hidden layers (Eq. 25) is a concrete approximation that can be refined or adaptively placed to improve deep GNet accuracy."],"fun_headline_variants":["Matrix-free Gaussian process networks scale to 95k points","Inverse Kalman filter speeds Gaussian process networks","GNet: nonparametric neurons with O(nq^2) covariance products","Covariance products without matrices: GNet scales","GNet's jointly inverse Kalman filter cuts storage and compute"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The recursions are exact only if the DLM transition matrices commute with each other and with the stationary latent covariance; the paper explicitly assumes the former but not the latter, and for deep GNet the hidden-layer activations are approximated by 10 knots, so the deep model's accuracy is heuristic.","fun_headline_variants_meta":{"raw":{"variants":["Matrix-free Gaussian process networks scale to 95k points","Inverse Kalman filter speeds Gaussian process networks","GNet: nonparametric neurons with O(nq^2) covariance products","Covariance products without matrices: GNet scales","GNet's jointly inverse Kalman filter cuts storage and compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000148,"raw_usage":{"total_tokens":974,"prompt_tokens":641,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":385,"completion_tokens_details":{"reasoning_tokens":250}},"tokens_in":385,"tokens_out":333,"duration_ms":4280,"temperature":1.0,"reasoning_tokens":250,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T07:08:21.371986+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a dynamic linear model whose transition matrices commute with each other but not with the stationary covariance W_1, compute Σv directly and via the JIKF recursions; any mismatch would falsify Theorem 1 as stated. Alternatively, in a deep GNet, increase the knot count m_l,K from 10 to 50 and check whether held-out NRMSE changes materially.","supporting_citations":[],"review_version":2}