{"id":"fe5d72cd-d9a7-43c6-8c9b-b6ed570c2299","arxiv_id":"2505.16257","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper claims an optimal weighting parameter and higher-order asymptotic corrections for batch-normalization test-time adaptation, but the central derivations contain sign errors, ad hoc terms, and an invalid consistency assumption.","lead":"This paper proposes a higher-order asymptotic theory for how batch normalization statistics should adapt when a test distribution differs from training, including a formula for the optimal mixing weight and saddlepoint tail estimates. A careful check of the derivations finds multiple algebraic errors and an assumption that fails in the very setting being studied.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 3.2 is unsupported: the 'skewness term' added in Eq. (10) is not part of the mean squared error, so the claimed optimal λ* does not minimize the stated objective.","rationale":"The reader's REJECT verdict is correct. I focus on Eq. (10) rather than the also-real consistency problem in Proposition 3.10 because the false MSE equality directly invalidates the abstract's headline 'optimal weighting parameter that minimizes the mean-squared error' and the subsequent generalization bound. The mistake is not hidden behind an assumption; it is an algebraic non-sequitur. The reader's weakest_assumption (δ = o_P(1) in §3.3) is also valid: under shift δ → −Δ_μ, so Proposition 3.10's expansion and Theorem 3.11 do not apply. However, the Eq. (10) issue is more load-bearing because it affects the first main result of the paper and cannot be fixed by restricting to a no-shift regime. I also note the Lemma 3.1 third-cumulant mismatch between the statement and the appendix proof, which is another independent error. A corrected paper would need to remove Γ from the MSE, which destroys the claimed λ* formula and Theorem 3.3; hence the reader's rejection stands unchanged.","tokens_in":18584,"tokens_out":11518,"duration_ms":94693,"concrete_test":"Analytically recompute Eq. (10) from Eq. (9) without introducing Γ: the squared error is (λ(μ̂_P,n − μ_P − Δ_μ) + (1−λ)(μ̂_Q,m − μ_Q))², whose expectation is λ²(Δ_μ² + σ_P²/n) + (1−λ)²σ_Q²/m because E[μ̂_Q,m − μ_Q] = 0 and the training and test averages are independent. If this calculation is correct, then Proposition 3.2's λ* cannot be the MSE minimizer; a Monte Carlo check with skewed P and Q (e.g., shifted lognormals, n = m = 100) would show the empirical argmin follows the quadratic formula without the κ terms.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central practical claim is Proposition 3.2: λ* in Eq. (13) minimizes the mean-squared error of the adapted BN mean. It does not. Starting from Eq. (8), the error is λ(μ̂_P,n − μ_P − Δ_μ) + (1−λ)(μ̂_Q,m − μ_Q). Squaring and taking the expectation, using independence of training and test samples, gives E(λ) = λ²(Δ_μ² + σ_P²/n) + (1−λ)²σ_Q²/m. The cross term is 2λ(1−λ) E[μ̂_P,n − μ_P − Δ_μ] E[μ̂_Q,m − μ_Q] = 0, and third central moments cannot appear in the expectation of a squared linear statistic. The quantity Γ_{P,Q,n,m} inserted in Eq. (10) is therefore not part of the MSE; Eq. (10) is not a consequence of Eq. (9). Moreover, even if Eq. (12) were adopted as an ad hoc objective, the absolute value is differentiated under an unverified sign condition, so Eq. (13) is at best a stationary point of one branch, not the minimizer of the displayed expression. Proposition 3.2 and the bound in Theorem 3.3, which substitutes this λ*, are unsupported. This is a load-bearing failure of the paper's main adaptation result, not a typographical slip.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a higher-order asymptotic framework for test-time adaptation (TTA) of Batch Normalization statistics under distribution shift. It derives an Edgeworth expansion for the normalized difference between test and training BN means (Lemma 3.1), proposes an optimal weighting parameter lambda* that supposedly minimizes the mean-squared error of the adapted statistic (Proposition 3.2), derives a risk bound (Theorem 3.3), gives saddlepoint density and tail approximations (Propositions 3.5 and 3.7, Theorem 3.8), and reformulates BN TTA as a one-step M-estimator with a higher-order LAN representation (Proposition 3.10, Theorem 3.11). The central practical claim is that lambda* provides a principled, skewness-corrected trade-off between training and test BN statistics.","tokens_in":18897,"tokens_out":10667,"duration_ms":80420,"significance":"If the results were correct, the paper would connect classical Edgeworth and saddlepoint theory to a practically relevant ML problem and would provide falsifiable guidance on how much to trust test-time BN statistics under shift. It would also offer a formal justification for the common heuristic of updating BN statistics at test time. However, the main technical claims contain load-bearing inconsistencies that are not local typos: the Edgeworth coefficient is dimensionally wrong, the proposed skewness term is not part of the MSE being minimized, and the M-estimation section assumes away the distribution shift that motivates the paper. The paper ships no machine-checked proofs, reproducible code, or simulation evidence, so the unsupported claims are not compensated elsewhere. For these reasons, the contribution cannot be accepted in its current form.","major_comments":[{"comment":"The definition of Delta_{3,n,m} in Lemma 3.1 is inconsistent with the characteristic-function calculation in its own proof. In the proof, the characteristic function of T_{n,m} is exp(-V_{n,m} t^2 / 2 + (i/6)(kappa_{3,Q} alpha^3 / sqrt(m) - kappa_{3,P} beta^3 / sqrt(n)) t^3 + ...), so the third cumulant of T_{n,m} is kappa_{3,Q} alpha^3 / sqrt(m) - kappa_{3,P} beta^3 / sqrt(n). The lemma instead defines Delta_{3,n,m} = kappa_{3,Q} alpha^3 sqrt(m) - kappa_{3,P} beta^3 sqrt(n), which differs by factors of m and n. Consequently, the Edgeworth correction in Eq. (7) does not correspond to the actual third cumulant of the normalized statistic. This error propagates to Lemma 3.4 and to the saddlepoint results in Proposition 3.5 and Proposition 3.7, which use the same Delta_{3,n,m}.","section":"Section 3.1, Lemma 3.1"},{"comment":"The claimed optimal lambda* does not minimize the stated objective. Starting from Eq. (8), the exact expectation of (mu_TTA(lambda) - mu_Q)^2 is lambda^2 (Delta_mu^2 + sigma_P^2 / n) + (1-lambda)^2 sigma_Q^2 / m, because the cross term vanishes by independence and third central moments cannot appear in the expectation of a squared linear statistic. The term Gamma_{P,Q,n,m} inserted in Eq. (10) is therefore not part of the MSE; Eq. (10) is not a consequence of Eq. (9). Moreover, even if Eq. (12) were adopted as an ad hoc objective, the displayed expression omits the factor lambda^2 multiplying sigma_hat_{P,n}^2 / n, so differentiating Eq. (12) does not produce Eq. (13). The subsequent discussion of lambda*, Remark 1's monotonicity claim, and the risk bound in Theorem 3.3, which substitutes this lambda*, are all unsupported. This failure is load-bearing for the paper's main adaptation result.","section":"Section 3.1, Eqs. (9)-(13), Proposition 3.2"},{"comment":"The M-estimation reformulation assumes that the initial estimator hat_mu_{P,n} is consistent for the test mean mu_0, i.e., delta = hat_mu_{P,n} - mu_0 = o_P(1). But in the TTA setting with a distribution shift Delta_mu = mu_Q - mu_P != 0, hat_mu_{P,n} converges to mu_P, so delta = -Delta_mu + o_P(1), which is not o_P(1). Proposition 3.10 and Theorem 3.11 therefore do not apply to the shifted-data setting that motivates the paper, and the text does not flag this as a no-shift restriction. In addition, the higher-order expansion in Theorem 3.11 contains the term (psi''_0 / (2 psi'_0^2)) ((1/sqrt{m}) sum psi)^2, which is O_P(1) after multiplying by sqrt{m}. This term does not arise from the preceding algebra under the stated o_P(1) consistency assumption; the standard expansion yields only the leading term plus o_P(1). Thus the LAN representation is not established.","section":"Section 3.3, Proposition 3.10 and Theorem 3.11"}],"minor_comments":[{"comment":"The statistic T_{n,m} is sometimes written T_{m,n} (e.g., Eq. (6) and the surrounding text).","section":"Notation and typos"},{"comment":"The variance term for the training mean is printed as sigma_hat_{P,n}^2 / n without the factor lambda^2, which is inconsistent with Eq. (9) and with the derivation of Eq. (13).","section":"Eqs. (10) and (12)"},{"comment":"The saddlepoint equation is written as K'(\\hat{t}) - x = 0 evaluated at 't=i' instead of at the saddlepoint, and the Gaussian integral expression is missing a closing parenthesis; the proof would benefit from a careful rewrite.","section":"Proof of Proposition 3.5"},{"comment":"The claim that the derivative of lambda* with respect to m is positive is asserted without calculation; from Eq. (13) the sign depends on the relative sizes of the terms and is not obvious.","section":"Remark 1"},{"comment":"The conditions in Lemma 3.6 and Theorem 3.8 are stated informally, and the proof of the uniform error bound is only sketched with phrases like 'one shows that'; a rigorous derivation of the stated rates would be needed even if the other results were corrected.","section":"Section 3.2"}],"recommendation":"reject","confidential_remarks":"The manuscript's central claims fail for reasons that cannot be fixed by local editing: the Edgeworth coefficient is wrong, the proposed skewness-corrected MSE is not the MSE, and the M-estimation section assumes no distribution shift. A complete re-derivation of the main results would be required. In addition, the paper provides no simulations or real-data validation, and Section 4.1's 'verification' is only informal qualitative agreement with prior work. I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a new application of classical Edgeworth, saddlepoint, and one-step M-estimation machinery to the problem of choosing the BN blending weight in test-time adaptation. That is a reasonable thing to want, and the author does engage the right literature. But the main results are not supported by the mathematics, and several errors are of a kind a careful referee would catch quickly.\n\nThe most load-bearing problem is Proposition 3.2. Starting from Eq. (8), the error is lambda*(mu_hat_P,n - mu_P - Delta_mu) + (1-lambda)*(mu_hat_Q,m - mu_Q). Squaring and taking expectations gives lambda^2*(Delta_mu^2 + sigma_P^2/n) + (1-lambda)^2*sigma_Q^2/m, with no cross term. Third central moments cannot enter the expectation of a squared linear statistic. The Gamma term added in Eq. (10) is therefore not part of the MSE, and Eq. (13) is not a minimizer of the stated objective. The sign condition on the absolute value is also unverified. That kills the main adaptation result and the generalization bound built on it.\n\nLemma 3.1 has a separate internal inconsistency: Delta_{3,n,m} is defined with factors sqrt(m) and sqrt(n), but the characteristic function in the proof produces coefficients 1/sqrt(m) and 1/sqrt(n). So the Edgeworth expansion as stated is not the one being proved.\n\nThe M-estimation section has a different fundamental problem. Proposition 3.10 assumes the initial estimator mu_hat_P,n is consistent for mu_0, the true test mean. Under the distribution shift the paper studies, mu_hat_P,n is consistent for mu_P, so delta is a nonzero constant shift, not o_P(1). The one-step estimator and the LAN result in Theorem 3.11 therefore do not apply to the setting. The quadratic term kept in Proposition 3.10 also does not follow from the preceding algebra; after sqrt(m) scaling it would be O_P(1), so the expansion would not be the standard one-step representation.\n\nWeaker but real: Lemma 3.6 and Theorem 3.8 are stated without proof, Remark 1's monotonicity claim is not checked, and there are no simulations or experiments to show the higher-order corrections help in finite samples.\n\nWho is this for? Maybe someone working on formalizing BN/TTA theory, but they would need to rewrite the core derivations. As is, the paper is not in a state where a serious referee should spend time on it. Desk reject.","headline":"A good topic and a sensible toolkit, but the central derivations do not hold up: the claimed optimal lambda is not a minimizer of the stated MSE, and the M-estimation section assumes away the shift it is meant to address.","tokens_in":19400,"tokens_out":5140,"would_cite":false,"duration_ms":39709,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62E20","62F12"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper derives a higher-order asymptotic framework for test-time batch-normalization adaptation, with an optimal blending weight and saddlepoint tail approximations.","keywords":["test-time adaptation","batch normalization","Edgeworth expansion","saddlepoint approximation","one-step M-estimator","local asymptotic normality","distribution shift","higher-order asymptotics"],"falsifier":"Simulate training data from $P=\\mathcal N(0,1)$ and test data from $Q=\\mathcal N(\\delta,1)$ with known $\\delta$, compute the empirical mean-squared error of $\\mu_{\\mathrm{TTA}}(\\lambda)$ over many replications for a grid of $\\lambda$, and compare the empirical minimizer with the closed-form $\\lambda^*$ in Eq. (13) over the region where its sign condition holds; a systematic discrepancy would refute the optimal-weight claim.","tokens_in":18271,"feed_emoji":"📊","tokens_out":10483,"duration_ms":79856,"temperature":0.7,"pith_summary":"The paper claims that the standard test-time practice of mixing training and test batch-normalization statistics is not just a heuristic: it can be analyzed and optimized with higher-order asymptotics. The central result is a closed-form optimal mixing weight $\\lambda^*$ that minimizes the mean-squared error of the blended BN mean, with the optimum depending on the mean shift, the two variances, and the third cumulants of the training and test distributions. Around this, the paper builds an Edgeworth expansion for the normalized training-versus-test mean difference, saddlepoint density and tail approximations that stay accurate in small batches, and a one-step M-estimator representation with a higher-order local asymptotic normality expansion. If these results are correct, batch-normalization test-time adaptation gains a principled, moment-aware rule for how quickly to trust incoming test statistics, and a way to tell when frozen training statistics are safer.","feed_headline":"Formula finds optimal train-test blend for batch-norm stats","feed_subtitle":"Edgeworth and saddlepoint theory turn a momentum heuristic into a moment-aware choice.","key_machinery":"The argument runs on four linked objects. The first is the normalized difference statistic $T_{n,m}$ and its cumulant expansion $K(t)=\\frac12 V_{n,m}t^2+\\frac16\\Delta_{3,n,m}t^3+O(t^4)$, which carries the higher-moment information. The second is the blended estimator $\\mu_{\\mathrm{TTA}}(\\lambda)$ and its mean-squared error, whose minimization over $\\lambda\\in[0,1]$ produces $\\lambda^*$. The third is the saddlepoint equation $K'(\\hat t)=x$, whose solution feeds both the density approximation and the Lugannani--Rice tail formula. The fourth is the one-step M-estimator update $\\hat\\mu_{\\mathrm{TTA}}=\\hat\\mu_{P,n}-\\sum_j\\psi(Y_j,\\hat\\mu_{P,n})/\\sum_j\\psi'(Y_j,\\hat\\mu_{P,n})$, which recovers the test mean for a linear score and yields the higher-order LAN expansion for a skewness-corrected score.","core_discovery":"The central claim is that the normalized discrepancy $T_{n,m}=\\sqrt{nm/(n+m)}(\\hat\\mu_{Q,m}-\\hat\\mu_{P,n}-\\Delta_\\mu)$ between the test and training batch-normalization means admits an Edgeworth expansion whose first correction is governed by the skewness combination $\\Delta_{3,n,m}=\\kappa_{3,Q}\\alpha^3\\sqrt{m}-\\kappa_{3,P}\\beta^3\\sqrt{n}$. From the mean-squared error of the blended statistic $\\mu_{\\mathrm{TTA}}(\\lambda)=\\lambda\\hat\\mu_{P,n}+(1-\\lambda)\\hat\\mu_{Q,m}$, the paper derives the optimal weight $\\lambda^*$ in Eq. (13) that balances squared shift, variance, and skewness. It then replaces truncated Edgeworth series with saddlepoint approximations, giving a Lugannani--Rice tail formula with relative error of order $\\min(1/\\sqrt{n},1/\\sqrt{m})$, and reinterprets the update as a one-step M-estimator to obtain the higher-order expansion $\\sqrt{m}(\\hat\\mu_{\\mathrm{TTA}}-\\mu_0)=Z_m^*/\\psi_0' + \\psi_0''/(2(\\psi_0')^3)(Z_m^*)^2+o_P(1)$, where $\\mu_0$ is the true test mean. The claimed payoff is a bias-variance-skewness accounting of BN test-time adaptation that turns the usual momentum heuristic into an explicit, refinable formula.","pith_inferences":["Beyond the paper: because $\\lambda^*$ is a closed form in estimable moments, a per-layer or per-channel scheduler that recomputes $\\hat\\lambda^*$ from running statistics is a direct implementation the paper leaves implicit.","Beyond the paper: the saddlepoint tail probabilities could double as a test-time anomaly score, flagging batches whose BN statistic falls in a tail the shift model does not predict.","Beyond the paper: the one-step expansion extends naturally to robust scores such as Huber or quantile-based $\\psi$; the paper mentions robust scores but does not develop their higher-order asymptotics."],"forward_implications":["The optimal weight $\\lambda^*$ gives an explicit recipe: small test batches, a large mean shift, or high test variance push the blend toward the training mean, while large test batches and strong test skewness push it toward the test mean.","The saddlepoint results supply uniformly accurate density and tail estimates for the BN TTA statistic, which is the regime where Edgeworth tails are known to be unreliable.","The one-step M-estimator representation connects BN TTA to classical asymptotic statistics and provides a higher-order correction term that can be used to debias or robustify the adapted mean.","The risk bound in Theorem 3.3 quantifies the separate contributions of bias, variance, and skewness to the adapted model's risk, identifying when a fully updated test statistic beats a frozen training statistic."],"supporting_citations":[{"why":"Defines batch normalization and the statistics the paper adapts.","marker":"[10]"},{"why":"Provides the classical validity conditions that justify the Edgeworth expansion used in Lemma 3.1.","marker":"[3]"},{"why":"Introduces the saddlepoint method that Proposition 3.5 adapts to the BN TTA statistic.","marker":"[7]"},{"why":"Supplies the Lugannani-Rice tail formula used in Proposition 3.7.","marker":"[20]"},{"why":"Is the standard source for one-step M-estimators and local asymptotic normality used in Section 3.3.","marker":"[28]"},{"why":"Motivates updating BN statistics with test data and supplies the empirical large-batch behavior that $\\lambda^* \\to 0$ is compared with.","marker":"[31]"},{"why":"Provides empirical evidence that reliance on test statistics should grow with test batch size, the trend the optimal $\\lambda^*$ reproduces.","marker":"[19]"}],"fun_headline_variants":["Higher-order stats sharpen test-time batch-norm adaptation","Optimal batch-norm blend derived from higher-order theory","Saddlepoint approximations refine batch-norm TTA statistics","Moment-aware formula replaces heuristic for BN adaptation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the initial training mean is already a consistent estimate of the true test mean; under an actual distribution shift of the means, that premise is false, so the one-step and local-asymptotic-normality results only apply once a consistent starting estimator is available.","fun_headline_variants_meta":{"raw":{"variants":["Higher-order stats sharpen test-time batch-norm adaptation","Optimal batch-norm blend derived from higher-order theory","Saddlepoint approximations refine batch-norm TTA statistics","Moment-aware formula replaces heuristic for BN adaptation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000418,"raw_usage":{"total_tokens":2193,"prompt_tokens":1026,"completion_tokens":1167,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":642,"completion_tokens_details":{"reasoning_tokens":1112}},"tokens_in":642,"tokens_out":1167,"duration_ms":8461,"temperature":1.0,"reasoning_tokens":1112,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:04:58.985382+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate training data from $P=\\mathcal N(0,1)$ and test data from $Q=\\mathcal N(\\delta,1)$ with known $\\delta$, compute the empirical mean-squared error of $\\mu_{\\mathrm{TTA}}(\\lambda)$ over many replications for a grid of $\\lambda$, and compare the empirical minimizer with the closed-form $\\lambda^*$ in Eq. (13) over the region where its sign condition holds; a systematic discrepancy would refute the optimal-weight claim.","supporting_citations":[{"cited_title":"and Szegedy, C","cited_arxiv_id":null,"evidence_quote":"Defines batch normalization and the statistics the paper adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the classical validity conditions that justify the Edgeworth expansion used in Lemma 3.1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the saddlepoint method that Proposition 3.5 adapts to the BN TTA statistic."},{"cited_title":"and Rice, S","cited_arxiv_id":null,"evidence_quote":"Supplies the Lugannani-Rice tail formula used in Proposition 3.7."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the standard source for one-step M-estimators and local asymptotic normality used in Section 3.3."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides empirical evidence that reliance on test statistics should grow with test batch size, the trend the optimal $\\lambda^*$ reproduces."}],"review_version":1}