{"id":"81c8c58f-1318-4cbd-8f1e-4f1e13b76026","arxiv_id":"2509.10695","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"By replacing a transformer's output layer with a Bayesian network updated through Kalman smoothing, sequential fine-tuning can be done with one sample of memory, lower compute, and explicit uncertainty estimates, outperforming memory-limited retraining on an inverted-pendulum decision transformer.","lead":"The authors present a sequential fine-tuning method that replaces a transformer's output layer with a Bayesian neural network updated by Kalman filtering, needing only one sample of memory. On a small decision-transformer task, it outperforms memory-constrained retraining while also producing uncertainty estimates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (43) may be a zero gain: under the paper's independence assumption Cov(z_{i-1},w_i)=0, so the backward pass cannot update hidden layers; the claimed sequential Bayesian fine-tuning is underspecified.","rationale":"The reader's weakest_assumption already identifies the independence assumption between weights and activations, and the paper's own Section VI-A concedes that off-diagonal covariance terms are not propagated in the backward pass. My concern sharpens this into a concrete algebraic consequence: Eq. (43) requires Cov(z_{i-1},w_i), which is zero under the stated independence. If so, the RTS smoother has zero gain into z_{i-1}, so the update stops propagating after the last layer. This does not by itself disprove the empirical success-rate claim, which might survive as a last-layer-only adaptation, but it would invalidate the paper's characterization as a Kalman Bayesian Transformer with full-head sequential fine-tuning and would make the reported uncertainty estimates not the claimed Bayesian posterior. Because the reader's CONDITIONAL verdict already requires the authors to address the independence and approximation gaps, this concern reinforces that requirement rather than moving the verdict; I keep the reader's verdict unchanged. The two-layer test would settle whether the gain is in fact zero and whether hidden weights move.","tokens_in":11327,"tokens_out":12765,"duration_ms":111588,"concrete_test":"Implement a two-layer scalar version of Algorithm 2 with the stated independence assumption, compute K_{z_1} from Eq. (43), and record whether w_1 changes after one update. If K_{z_1}=0 and w_1 is unchanged, the backward pass degenerates to last-layer-only adaptation, which would confirm the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-B states \"Assuming w_i and z_{i-1} are independent\" for the forward pass. In the backward pass, Eq. (43) sets K_{z_{i-1}} = Σ_{z_{i-1},w_i} Σ^{-1}_{w_i,w_i}. Under that same independence assumption, Σ_{z_{i-1},w_i}=0, so K_{z_{i-1}}=0, which makes µ^+_{z_{i-1}}=µ_{z_{i-1}} and Σ^+_{z_{i-1},z_{i-1}}=Σ_{z_{i-1},z_{i-1}}. Then at layer i-1 the innovation µ^+_{z_{i-1}}-µ_{z_{i-1}} in Eq. (38) is zero, so no update propagates further; only the last layer's weights change. The paper never gives a separate formula for Σ_{z_{i-1},w_i}; the statement that the cross-covariance \"can be easily calculated similar to (21)\" yields zero under independence. Section VI-A concedes there is no closed-form propagation for off-diagonal covariance terms in the backward pass through nonlinear activations. Thus either the algorithm has a silent zero-gain bug, or it relies on an unstated nonzero correlation model; in neither case is the computed posterior the claimed Bayesian posterior, and the characterization of the method as a Kalman Bayesian Transformer with full-head sequential fine-tuning is not supported by the equations as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Kalman Bayesian Transformer, a method for sequential fine-tuning of transformers under distribution shift and memory constraints. The method freezes the transformer body, replaces the output linear layer with a Bayesian neural network (BNN) initialized to reproduce the pre-trained model, and then updates the BNN weights one data sample at a time. The forward pass propagates means and covariances of activations through linear, ReLU, and softmax layers using closed-form moment propagation and a first-order Taylor approximation; the backward pass applies Rauch-Tung-Striebel smoothing to update weight distributions. The experimental evaluation on a decision transformer for inverted-pendulum stabilization reports higher stabilization success rate than warm-started retraining with memory buffers of 10-50 samples, lower per-sample computation time, and uncertainty estimates that grow with data noise.","tokens_in":11675,"tokens_out":9392,"duration_ms":81300,"significance":"The paper connects Kalman filtering/smoothing with transformer fine-tuning and provides a closed-form, single-pass alternative to sampling-based Bayesian methods. If the algorithm were correct as written, the contribution would be significant: memory-efficient sequential adaptation with explicit uncertainty quantification at low latency. The paper also gives a clear algorithmic description and a concrete experimental setup. However, the central derivation contains a load-bearing inconsistency in the backward pass that, unless resolved, means the method does not implement the full-head Bayesian update it claims. No machine-checked proofs are provided; the theoretical claims rest on moment-propagation formulas from prior work.","major_comments":[{"comment":"The backward pass defines K_{z_{i-1}} = Σ_{z_{i-1},w_i} Σ^{-1}_{w_i,w_i}. Section IV-B explicitly assumes w_i and z_{i-1} are independent, which gives Σ_{z_{i-1},w_i}=0 and hence K_{z_{i-1}}=0. Then (44)-(45) leave µ_{z_{i-1}} and Σ_{z_{i-1},z_{i-1}} unchanged, so in the next lower layer the innovation µ^+_{z_{i-1}} - µ_{z_{i-1}} in (38) is zero and no update reaches layers L-1 and below. The remark that the cross-covariance \"can be easily calculated similar to (21)\" is not sufficient, because (21) computes Σ_{u_i,u_i}, not Σ_{z_{i-1},w_i}, and under the stated assumption this cross-covariance is exactly zero. Section VI-A's admission that off-diagonal covariance terms have no closed-form backward propagation does not fix the inconsistency. Either the algorithm silently updates only the final layer's weights, contradicting the claim of full-head sequential Bayesian fine-tuning, or it relies on an unstated non-independence model that is never derived. This issue must be resolved before the method can be accepted as described.","section":"Section IV-C and Algorithm 2, Eqs. (43)-(45)"},{"comment":"The recursive update (13) is only a valid Bayesian recursion if p(X_k,Y_k|W_B) is a genuine likelihood. The paper never defines this likelihood; instead, Algorithm 2 sets µ^+_{z_L} to the one-hot target and Σ^+_{z_L,z_L} to Σ_data. This is equivalent to a Gaussian observation model y = p + noise for a softmax output p, which is not the categorical likelihood that generated the token data. The Gaussian treatment of a one-hot vector is an approximation whose effect on the posterior is not analyzed. The paper should either specify the likelihood and justify the Gaussian approximation, or present the method as an assumed-density filter rather than exact Bayesian inference.","section":"Section IV-C and Algorithm 2 lines 22-24"},{"comment":"Algorithm 1 preprocesses each autoregressive sequence into a batched representation and the paper states that the augmented data can be treated as i.i.d. However, the individual token targets within a sequence are conditionally dependent, and the correct sequence likelihood is a product of per-token conditional probabilities. A single batched Gaussian measurement update with a common Σ_data does not equal that product likelihood. The preprocessing relabels batches but does not make the tokens within a batch independent. This is a modeling approximation that should be stated explicitly and its consequences for the posterior update discussed.","section":"Section IV-A and Algorithm 1"}],"minor_comments":[{"comment":"The block-matrix notation for the weight initialization is difficult to parse; please specify the exact dimensions of each block and the shape of W_i.","section":"Section IV-C, Eqs. (34)-(36)"},{"comment":"The results are averages over 10 trials, but no error bars, standard deviations, or significance tests are reported; the claim that the proposed method \"does not experience drops\" is based on visual inspection.","section":"Section V-B, Figures 3 and 4"},{"comment":"The experimental section does not state the value of the covariance scale ε or the values of Σ_data used in Figures 3 and 4; please provide these in the setup.","section":"Section V-A"},{"comment":"The comparison with warm-started retraining uses 100 training epochs per memory buffer, while the proposed method uses a single pass; the per-sample computation time comparison in Figure 4 may conflate epoch counts, so the timing comparison should be clarified.","section":"Section V-B, Figure 4"},{"comment":"The first-order Taylor expansion for the softmax mean is correct, but the paper should state explicitly that higher-order terms are neglected and that the approximation error depends on the variance of u_L, which is not reported.","section":"Section IV-B, Eqs. (24)-(27)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is in scope for a machine learning journal and the empirical setup is relevant. The main concern is the backward-pass inconsistency; if the authors can show that the method is intentionally a last-layer-only update and adjust the claims, or provide a legitimate approximation for the cross-covariances, the paper could be acceptable. The comparison to warm-started retraining is somewhat apples-to-oranges because of the different amounts of computation per sample, but this is secondary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2509.10695. First, it is genuinely new in its combination: Kalman Bayesian neural networks, closed-form moment propagation, and a first-order Taylor approximation of softmax moments for sequential transformer fine-tuning. The recursive Bayesian update in Eq. (13) is standard and correct, and the forward pass is carefully derived. Second, the backward pass has a problem that may gut the method. Section IV-B assumes w_i and z_{i-1} are independent to compute forward moments. Under that same assumption, the cross-covariance Σ_{z_{i-1}, w_i} in Eq. (43) is zero, so the gain K_{z_{i-1}} is zero, and Eq. (44) leaves z_{i-1} unchanged. Then no innovation propagates below the last layer; only w_L gets updated. The paper says the cross-covariance can be computed like Eq. (21), but Eq. (21) is for the covariance of u_i, not for the cross-covariance between z_{i-1} and w_i. No formula for that cross-covariance is provided, and under the stated independence it is identically zero. The limitation section (VI-A) concedes there is no closed-form backward propagation for off-diagonal terms, but that is a different issue. So either the equations hide an unstated correlation model, or the algorithm silently reduces to a single-layer update. Both readings contradict the claim of a full-head Kalman Bayesian transformer.\n\nWhat the paper does well: the forward pass with moment propagation through ReLU and the softmax approximation is clean, and the experimental comparison to memory-constrained retraining is relevant. The results on the inverted pendulum look plausible, but without code or full hyperparameters I can't fully verify them. If the zero-gain issue is real, the empirical success might come from adapting only the output layer, which is a much weaker claim than the paper makes.\n\nThis deserves peer review because the core idea is worth debating and the fix might be tractable. But it needs a major revision: derive the cross-covariance in (43) explicitly, state whether the independence assumption is dropped in the backward pass, and provide code. If the gain is really zero, the paper should be repositioned as single-layer adaptation or the backward pass needs an alternative approximation.","headline":"A fresh combination of Bayesian tools for transformer fine-tuning, but a likely zero-gain issue in the backward pass means the advertised full-head update may not be happening.","tokens_in":12164,"tokens_out":7232,"would_cite":false,"duration_ms":58430,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sequential fine-tuning of a transformer can be run as a Kalman-style Bayesian update that stores one sample at a time and, in the paper's experiment, beats replay buffers of up to 50 samples.","keywords":["Kalman Bayesian Transformer","sequential fine-tuning","moment propagation","uncertainty quantification","catastrophic forgetting","decision transformer","Rauch-Tung-Striebel smoother","online Bayesian inference"],"falsifier":"On a small transformer where the true Bayesian posterior can be obtained by Monte Carlo sampling, run the same sequential fine-tuning task and compare the method's updated weight covariance after many samples with the covariance of samples from the true posterior; a large divergence in a regime with small hidden widths and strong weight-activation correlations would show that the independence and Gaussian assumptions are not benign.","tokens_in":11132,"feed_emoji":"🤖","tokens_out":9140,"duration_ms":62552,"temperature":0.7,"pith_summary":"This paper proposes a way to fine-tune a pre-trained transformer sequentially by treating each new sample as a Bayesian measurement that updates a distribution over the weights of the output head. The method, called the Kalman Bayesian Transformer, freezes the transformer body, replaces the linear head with a Bayesian neural network, and propagates means and covariances in closed form through linear, ReLU, and softmax layers using Kalman smoothing. In an inverted-pendulum task, the method needs to store only the latest training sample and achieves higher stabilization success rates than warm-started retraining with replay buffers of 10, 20, 25, and 50 samples, while also returning explicit predictive uncertainty. If this result holds, online fine-tuning in memory-limited, latency-critical settings becomes substantially cheaper and more stable than replay-based retraining.","feed_headline":"Kalman filter fine-tunes transformers with one stored sample","feed_subtitle":"Closed-form Bayesian updates replace replay buffers and raise stabilization success over warm-started retraining.","key_machinery":"The load-bearing identity is the recursive Bayes update $$p(W_B\\mid D_k)\\propto p(W_B\\mid D_{k-1})\\,p(X_k,Y_k\\mid W_B),$$ which turns sequential fine-tuning into a one-sample-memory update. The machinery that makes this tractable is closed-form propagation of first and second moments through linear and ReLU layers, a first-order Taylor approximation of the softmax Jacobian to push moments through the final layer, and an RTS smoother in the backward pass that corrects the means and covariances of weights and intermediate activations layer by layer. The method also initializes the weight prior so that the pre-trained transformer's behavior is exactly reproduced before any data arrive.","core_discovery":"The central discovery is that the recursive Bayesian update $p(W_B|D_k) \\propto p(W_B|D_{k-1}) p(X_k,Y_k|W_B)$ can be implemented in the moment domain for transformer output-head fine-tuning. Means and covariances of activations propagate in closed form through the linear and ReLU layers, softmax moments are approximated by a first-order Taylor expansion, and a Kalman smoothing backward pass updates the weight means and covariances one layer at a time. The paper reports that, on an inverted-pendulum decision-transformer task, this single-sample-memory sequential update outperforms warm-started retraining with replay buffers of 10, 20, 25, and 50 samples in stabilization success rate, does not show the success-rate drops that replay-based retraining exhibits, and produces uncertainty estimates that increase with data noise.","pith_inferences":["The demonstrated setting adapts only the linear output head; the moment-propagation cost of full-covariance updates would grow quickly if applied to attention or embedding parameters, so scaling to deeper adaptation needs further work.","In regimes with strong correlations between weights and activations, the independence and Gaussian assumptions make the updated distribution an assumed-density filter rather than an exact posterior, so the uncertainty numbers should be read as calibrated heuristics until tested against sampling.","A natural testable extension is to apply the same update to a language-model fine-tuning task with high-dimensional softmax outputs, where the first-order Taylor approximation may be the limiting component."],"forward_implications":["Fine-tuning can proceed online with a one-sample memory, eliminating replay buffers and their storage and management costs.","Predictive uncertainty is available in closed form at every step, so a downstream controller or operator can know when the model's confidence is low.","The per-sample update is a single closed-form pass rather than an iterative retraining loop, which suits latency-critical hardware.","Because the prior at each step is the previous posterior, the method is designed to avoid catastrophic forgetting, and the experiment shows no forgetting-induced success-rate drops.","The same recursive update naturally extends as new data arrive indefinitely, without needing to revisit old data."],"supporting_citations":[{"why":"Supplies the Kalman Bayesian neural network machinery, including the RTS smoother weight update equations and the diagonal cross-covariance approximation through the ReLU layer.","marker":"[11]"},{"why":"Supplies the closed-form formulas for propagating mean and covariance through ReLU layers, which make the forward pass analytic.","marker":"[18]"},{"why":"Provides the Bayesian neural network framing and the posterior p(W|D) that the fine-tuning objective is built on.","marker":"[22]"},{"why":"Defines the transformer architecture that is frozen and fine-tuned by replacing the linear head.","marker":"[29]"},{"why":"Supports the claim that Bayesian neural networks avoid catastrophic forgetting in continual learning.","marker":"[31]"}],"fun_headline_variants":["One-sample Kalman beats 50-sample replay for transformer tuning","Kalman Bayesian Transformer: one memory, better stabilization","Moment-domain Kalman fine-tuning outdoes replay with a single sample","Closed-form Bayesian updates: no replay buffer needed for transformers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The formulas assume each layer's weights and the previous layer's activations are independent and that pre-activations are Gaussian, so the closed-form ReLU moment propagation is exact only under those conditions; the paper itself notes that off-diagonal covariance terms have no closed-form backward propagation through nonlinear activations.","fun_headline_variants_meta":{"raw":{"variants":["One-sample Kalman beats 50-sample replay for transformer tuning","Kalman Bayesian Transformer: one memory, better stabilization","Moment-domain Kalman fine-tuning outdoes replay with a single sample","Closed-form Bayesian updates: no replay buffer needed for transformers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1272,"prompt_tokens":870,"completion_tokens":402,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":332}},"tokens_in":486,"tokens_out":402,"duration_ms":343251,"temperature":1.0,"reasoning_tokens":332,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:54:26.672074+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small transformer where the true Bayesian posterior can be obtained by Monte Carlo sampling, run the same sequential fine-tuning task and compare the method's updated weight covariance after many samples with the covariance of samples from the true posterior; a large divergence in a regime with small hidden widths and strong weight-activation correlations would show that the independence and Gaussian assumptions are not benign.","supporting_citations":[{"cited_title":"Kalman Bayesian neural networks for closed-form online learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the Kalman Bayesian neural network machinery, including the RTS smoother weight update equations and the diagonal cross-covariance approximation through the ReLU layer."},{"cited_title":"An analytic solution to covariance propagation in neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the closed-form formulas for propagating mean and covariance through ReLU layers, which make the forward pass analytic."},{"cited_title":"A practical Bayesian framework for backpropa- gation networks,","cited_arxiv_id":null,"evidence_quote":"Provides the Bayesian neural network framing and the posterior p(W|D) that the fine-tuning objective is built on."},{"cited_title":"Continual learning us- ing bayesian neural networks,","cited_arxiv_id":null,"evidence_quote":"Supports the claim that Bayesian neural networks avoid catastrophic forgetting in continual learning."}],"review_version":1}