{"id":"a6eaf048-2320-4a9a-90e3-c034c3e33b9f","arxiv_id":"2601.13744","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An optimal query-level gate for mixing a language model with a k-NN retriever is derived; the hallucination-discordance reduction from gating converges to a deterministic limit governed by structural Bayes-vs-LM agreement.","lead":"This paper builds a simplified statistical model of retrieval-augmented generation in which a frozen language model is blended with a k-nearest-neighbor retriever by a query-dependent gate, and derives the gate that minimizes cross-entropy plus a geometric trust penalty. A generalist should care because it yields a formal account of when retrieved evidence should be trusted, tying hallucination risk to disagreement between the model and the retrieval neighborhood.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's clean limit depends essentially on the aligned setting; Appendix B concedes this rarely holds and its mismatch limit is different, so the 'solely structural' claim does not carry over.","rationale":"I checked the aligned derivation: Prop. 1's pointwise minimization is valid; Prop. 2's Hoeffding/Chernoff argument gives uniform concentration at a fixed query; Cor. 1's margin argument yields modal-label consistency; continuous mapping then gives Theorem 1 under the stated assumptions. So the internal mathematics is not where the paper fails. The vulnerability is external validity: the one condition that makes the theorem's clean limit true—alignment—is explicitly conceded to be unrealistic, and the appendix's mismatch analysis changes the limit. This is not a disagreement with consensus; it is a statement about the theorem's assumptions. The missing experiments and the uncalibrated discordance interpretation are real but secondary. Therefore the reader's CONDITIONAL verdict is unchanged: the aligned-setting limitation is load-bearing, but the theorem as stated is not internally inconsistent.","tokens_in":23527,"tokens_out":7035,"duration_ms":74445,"concrete_test":"Re-derive Theorem 1 under the hybrid mismatch model (11) for a two-label example: let x∈supp(P_M), P_{Y|X}(1|x)=0.9, P_0(1|x)=0.5, Q=(0.1,0.9), ε=0.2, and no geometric shift. Proposition 4 gives F∞(1|x)=0.74, F∞(2|x)=0.26; hence ℓ_F≈0.406<ℓ_0≈0.693, so the gate switches and ΔH→F∞(1|x)−P_0(1|x)=0.24. The aligned Theorem 1 formula instead gives 1{ℓ_Bayes<ℓ_0}(0.9−0.5)=0.40. If this mismatch analogue is correct, Theorem 1's limit is not robust to the paper's own mismatch model, and the central claim must be re-stated as aligned-only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the identification of the kNN retriever with the true Bayes conditional, used by Theorem 1. Section 2 defines the aligned setting P_{Q,X}=P_{M,X}, P_{Q,Y|X}=P_{M,Y|X}; Theorem 1 then concludes ΔH(x; λ*) → 1{ℓ_Bayes(x)<ℓ_0(x)}(P_{Y|X}(c*(x)|x)−P_0(c*(x)|x)). This requires exactly that identification. Appendix B concedes 'this alignment rarely holds' and Eq. (11) models mismatch. Propositions 3–4 show that under mismatch the retriever does not converge to P_{Y|X}(·|x): for a query x it converges to the memory label law at the projection x* ∈ supp(P_M), F∞(·|x) = (1−ε(x*))P_{Y|X}(·|x*) + ε(x*)Q(·|x*), not P_{Y|X}(·|x). Consequently the limiting gate threshold is not ℓ_Bayes vs ℓ_0 and the limiting ΔH is not of Theorem 1's form whenever ε>0 or the projection is nontrivial. The abstract's 'governed solely by structural agreement between the Bayes rule and the LM' is therefore an aligned-setting artifact; Appendix B does not supply a replacement theorem. The central claim for realistic RAG rests on an assumption the authors themselves call unrealistic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This note proposes a statistical surrogate model for retrieval-augmented generation. A frozen language model P0 and a k-nearest-neighbor retriever Phat are mixed by a query-dependent gate λ, with a trust weight a_fact(x) measuring geometric reliability of the retrieved neighborhood. The population loss is cross-entropy plus a trust penalty ν λ(x)(1-a_fact(x)). Proposition 1 gives the per-query optimal hard gate: retrieve when ℓ_r(x)+ν(1-a_fact(x)) < ℓ0(x). Section 4 defines a discordance-based hallucination score H_disc(x)=a_fact(x)(1-P0(c_hat(x)|x)), and analyzes the change ΔH under the optimal gate. Under the aligned setting (P_Q,X=P_M,X and P_Q,Y|X=P_M,Y|X), with k→∞, k/n→0, Lipschitz conditional distributions and a unique Bayes label, Theorem 1 shows ΔH → 1{ℓ_Bayes<ℓ0}(P_Y|X(c*|x)-P0(c*|x)). Appendix B introduces a mismatch model with geometric deformation and label corruption and gives limits for a_fact and the retriever. The abstract also claims numerical validation on synthetic and real data.","tokens_in":23867,"tokens_out":11060,"duration_ms":105347,"significance":"The theoretical core is sound and clean. Proposition 1 is exact pointwise minimization; Proposition 2's Hoeffding-plus-Chernoff bound and Corollary 1's margin argument are valid; Theorem 1 follows from the continuous mapping theorem under the stated non-degeneracy. The appendix's Propositions 3-4 are useful and establish limits that appear correct. If the paper's scope is restricted to the aligned setting, it provides a useful conceptual reduction: the sign of the gate's effect on discordance is asymptotically governed by Bayes-vs-LM disagreement, not finite-sample kNN noise. The paper does not provide code or experiments, as a theory note; however, the abstract promises numerical validation that is absent.","major_comments":[{"comment":"The claim that the discordance limit is 'governed solely by the structural agreement' between the Bayes rule and the LM is only established in the aligned setting. In the mismatch model of Eq. (11), Propositions 3–4 give F∞(·|x)=(1-ε(x*))P_Y|X(·|x*)+ε(x*)Q(·|x*) for the unique projection x*, and a_fact(x)→exp(-d(x,S)^2). The limiting gate threshold and ΔH then depend on ε, Q, and the projection, not solely on ℓ_Bayes(x) vs ℓ0(x). Appendix B itself states that alignment 'rarely holds' but supplies no mismatch analog of Theorem 1. The abstract should either be qualified to the aligned setting or the theorem should be extended.","section":"Abstract; §4.2; Appendix B"},{"comment":"The abstract says the gating rule 'controls this discordance by activating retrieval only where local evidence appears reliable.' But on the trade-off region ℬ of Eq. (6), ℓ_r+ν(1-a_fact)<ℓ0 while P_hat(c_hat)<P0(c_hat), so ΔH<0: the optimal gate increases discordance. The penalty ν only shrinks ℬ; it does not remove it when a_fact≈1. The mathematical content of §4.1 is a three-regime characterization, not a control guarantee. Please revise the abstract and concluding statements accordingly.","section":"Abstract; §4.1, Eq. (6)"},{"comment":"The abstract states 'and validate the framework numerically on synthetic and real data.' The manuscript contains no experiments, no data description, and no numerical section; the text proceeds from §4.2 to the references and appendices. This is a missing support for a stated claim. Either add the validation or remove/qualify the sentence.","section":"Abstract"},{"comment":"The discordance measure H_disc is defined with a_fact(x) as a multiplicative factor, while a_fact(x) also enters the gate objective as the penalty ν(1-a_fact(x)). In low-trust regions H_disc is small by construction, and the gate is penalized toward λ=0; in high-trust regions a_fact→1 and the asymptotic theorem isolates the Bayes-LM comparison. The finite-sample 'hallucination control' interpretation is therefore partly baked into the proxy. The paper does not validate H_disc against an external factuality metric. Please state this limitation explicitly, or provide evidence that H_disc tracks actual hallucination.","section":"§4, Eq. (4) and Eq. (3)"}],"minor_comments":[{"comment":"The displayed first-order condition writes Σ p_true (Phat-P0)/p_λ + ν(1-a_fact)=0. The derivative of -log((1-λ)P0+λPhat) with respect to λ is (P0-Phat)/p_λ, so the sign of the numerator appears reversed.","section":"§3, soft-gating FOC"},{"comment":"The assumption ℓ_Bayes(x)≠ℓ0(x) is necessary for the indicator to stabilize; if equality holds, the limit may depend on tie-breaking or may not exist. This is stated as an assumption but its role could be clarified, especially because the equality case is not measure-zero in general.","section":"Theorem 1"},{"comment":"The conditional distribution P_Y|X(·|·) is described as Lipschitz 'in its second argument,' but the displayed expressions treat the input x as the second argument. Consider rewording to avoid confusion.","section":"Proposition 2"}],"recommendation":"major_revision","confidential_remarks":"The technical derivations are correct, and the reader's worry about Theorem 1's reliance on the aligned setting is confirmed. The main problems are scope and claims: the abstract oversells the realistic applicability, and the promised numerical validation is absent. With the abstract revised, the mismatch limitation stated in the main text, and the numerical claim either fulfilled or removed, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the core mathematics is solid, and the main text is honestly scoped, but the abstract overclaims in two ways — it advertises numerical validation that is nowhere in the manuscript, and it states a “solely structural” asymptotic limit that, as the stress-test note correctly points out, depends essentially on the aligned setting.\n\nWhat is genuinely new: the trust-weighted gating objective (Eq. 3), the retrieval-trust weight a_fact, the pointwise optimal hard gate (Prop. 1), and the discordance decomposition with its three regimes. Theorem 1 is a clean result: in the aligned setting, the limiting effect of optimal gating on local discordance is governed by whether the Bayes predictor beats the frozen LM in cross-entropy, not by k-NN sampling noise. The proof chain (Prop. 2, Cor. 1, Thm. 1) checks out — Hoeffding plus Chernoff plus a margin argument is standard but correctly executed, and the non-degeneracy condition ℓ_Bayes ≠ ℓ_0 is genuinely needed for the indicator to converge. The mismatch appendix (Props. 3–4) is also correct: the k-NN retriever converges to the memory's label law at the projection onto the support, and the bias bound separating geometric and semantic corruption is a nice touch.\n\nThe soft spots are proportionate but real. The stress-test concern lands: Theorem 1's limit requires the retriever's target to coincide with the true Bayes conditional, and Appendix B's mismatch model replaces it with the memory's local label law at the projection point. So the abstract's phrase “governed solely by the structural agreement between the Bayes rule and the LM” is too strong — it is “solely” only when memory and query have the same conditional law. The authors concede this in Appendix B (“this alignment rarely holds”), which I credit, but the abstract still overreaches.\n\nSecond, the abstract says the framework is “validated numerically on synthetic and real data,” and there are no experiments in the manuscript. That claim should be removed or the experiments supplied. Third, the discordance criterion is explicitly a proxy for hallucination, and the paper deserves credit for flagging it; but there is no calibration showing H_disc tracks actual hallucination rates. That is a minor caveat, not a flaw.\n\nThis paper is for statisticians and theoretical ML people who work on RAG, kNN-LM, or gating/mixture models. It gives future work a precise language and a clear target. It deserves a serious referee — my recommendation is to send it to peer review, and to require that the abstract be aligned with the actual content: drop the numerical-validation claim and qualify the aligned-setting dependence.","headline":"Mathematically sound, honestly scoped theory note whose abstract oversells its reach on two fronts: no experiments behind a numerical-validation claim, and a clean asymptotic limit that only holds in the aligned setting the authors themselves call unrealistic.","tokens_in":24425,"tokens_out":1950,"would_cite":true,"duration_ms":20852,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G20","62H30"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the optimal retrieval gate's hallucination effect is asymptotically determined only by structural Bayes-vs-LM agreement.","keywords":["adaptive gating","hallucination control","nearest neighbor","retrieval-augmented generation","statistical learning","discordance","Bayes-optimal gate","query-memory mismatch"],"falsifier":"Simulate the aligned setting with known P_{Y|X} and a fixed frozen LM, choose query points where ℓ_Bayes(x) < ℓ0(x) and where the Bayes label margin is positive, and verify that the realized ΔH(x; λ*) under the optimal gate converges to the limit in Theorem 1 as n grows; any systematic deviation would falsify the theorem's claim.","tokens_in":23343,"feed_emoji":"🔎","tokens_out":5687,"duration_ms":54587,"temperature":0.7,"pith_summary":"This paper tries to put retrieval-augmented generation (RAG) on a statistical footing by modelling the system as a frozen language model, a k-nearest-neighbor retriever, and a query-dependent gate that mixes them. It derives the Bayes-optimal gate: retrieve exactly when the retriever's cross-entropy advantage over the LM exceeds a geometric trust penalty. The central asymptotic result is that, as the memory grows, the gate's effect on a discordance-based hallucination measure converges to a deterministic quantity decided solely by whether the Bayes predictor agrees with the frozen LM at the query. In other words, finite-sample k-NN noise washes out, and the structural Bayes-vs-LM relationship governs whether retrieval helps or hurts hallucination. This matters because it gives a principled, testable criterion for when grounding should improve factuality.","feed_headline":"Retrieval gating wins only when Bayes beats the LM","feed_subtitle":"The optimal per-query gate reduces hallucination exactly when the Bayes rule outperforms the frozen language model.","key_machinery":"The central object is the retrieval-trust weight a_fact(x), the average of exp(−||x − neighbor||²) over the k retrieved neighbors, which quantifies geometric reliability and acts as a penalty in the gating objective. The gate itself is the measurable hard decision that compares two local cross-entropies, ℓ_r(x) + ν(1 − a_fact(x)) versus ℓ0(x). The asymptotic analysis leans on uniform k-NN consistency (Proposition 2) and mode stability (Corollary 1), which together replace the empirical retriever and trust weight by their Bayes limits, isolating the structural term P_{Y|X}(c*(x)|x) − P0(c*(x)|x) as the sole determinant of the discordance limit.","core_discovery":"Under the aligned setting, where query and memory share the same distribution and label mechanism, the optimal hard gate chooses retrieval precisely when the retriever's local cross-entropy advantage over the frozen LM exceeds a trust penalty scaled by ν. Theorem 1 states that, when k → ∞ and k/n → 0, the local hallucination variation ΔH(x; λ*) converges in probability to λ∞(x)(P_{Y|X}(c*(x)|x) − P0(c*(x)|x)), with λ∞(x) = 1{ℓ_Bayes(x) < ℓ0(x)}. Thus the asymptotic sign and magnitude of the hallucination change is governed solely by the structural difference between the Bayes rule and the LM at the query, not by the finite-sample behavior of the k-NN estimator or the trust weight.","pith_inferences":["A testable practical consequence is that one could build a gating rule without explicitly training a retriever, by comparing a calibrated LM confidence against an entropy estimate from a local memory; if the alignment assumption is violated, the gate should be corrected for projection distance and corruption level.","Because the trade-off region shows likelihood and factual alignment can conflict, a natural extension is to design retrieval-aware training objectives for the LM that reduce structural Bayes-LM disagreement at the modal label, rather than merely improving overall likelihood.","The analytical proxy suggests that RAG failure modes like off-topic retrieval are really failures of the memory's local label law to coincide with the query's true conditional; incorporating a trust penalty as a learned function of distance, rather than fixed exponential, might yield a more robust gate."],"forward_implications":["In the large-memory limit, the optimal gate activates exactly when the Bayes cross-entropy is below the LM cross-entropy, making the decision independent of k-NN estimation noise.","When the retriever improves cross-entropy but assigns less mass to its modal label than the LM, the optimal gate can increase discordance, so likelihood gain does not guarantee hallucination reduction.","The retrieval-trust weight acts as a safeguard only in finite samples; its penalty vanishes asymptotically and its multiplicative factor tends to one, so geometric unreliability matters only through finite-sample effects.","Under the mismatch model, the limiting retriever converges to the memory's local label law at the nearest support point, so the penalty term becomes nonzero and the gate naturally shifts toward the LM as the query moves off the memory support."],"fun_headline_variants":["Optimal RAG gate: retrieve when Bayes beats LM","RAG gating rule: Bayes advantage triggers retrieval","Better RAG via Bayes-LM gate: only when it wins","Query-level RAG gate: Bayes edge decides retrieval","RAG hallucination cut: gate on Bayes vs LM gap"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper's main theorem assumes the aligned setting: queries and memory come from the same distribution with the same label mechanism, which the paper itself notes rarely holds in practice; under mismatch, the retriever converges to the memory's local label law at the projection point, and the clean Bayes-vs-LM limit does not carry over.","fun_headline_variants_meta":{"raw":{"variants":["Optimal RAG gate: retrieve when Bayes beats LM","RAG gating rule: Bayes advantage triggers retrieval","Better RAG via Bayes-LM gate: only when it wins","Query-level RAG gate: Bayes edge decides retrieval","RAG hallucination cut: gate on Bayes vs LM gap"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":977,"prompt_tokens":590,"completion_tokens":387,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":334,"completion_tokens_details":{"reasoning_tokens":305}},"tokens_in":334,"tokens_out":387,"duration_ms":4349,"temperature":1.0,"reasoning_tokens":305,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T09:27:51.035423+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate the aligned setting with known P_{Y|X} and a fixed frozen LM, choose query points where ℓ_Bayes(x) < ℓ0(x) and where the Bayes label margin is positive, and verify that the realized ΔH(x; λ*) under the optimal gate converges to the limit in Theorem 1 as n grows; any systematic deviation would falsify the theorem's claim.","supporting_citations":[],"review_version":1}