A Note on k-NN Gating in RAG

CELESTE); Claire Boyer (IUF; G\'erard Biau (SU; IUF; MEGAVOLT)

REVIEW 4 major objections 3 minor 21 references

A Note on k-NN Gating in RAG

T0 review · 4 major / 3 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that the optimal retrieval gate's hallucination effect is asymptotically determined only by structural Bayes-vs-LM agreement.

desk verdict Mathematically sound, honestly scoped theory note whose abstract oversells its reach on two fronts: no experiments behind a numerical-validation claim, and a clean asymptotic limit that only holds in the aligned setting the authors themselves call unrealistic. read the letter →

arxiv 2601.13744 v2 pith:2C3EWB2Z submitted 2026-01-20 math.ST stat.TH

Gérard Biau (SU , IUF , MEGAVOLT) , Claire Boyer (IUF , CELESTE) This is my paper · ORCID

classification math.STstat.TH MSC 62G2062H30

keywords adaptivegatinghallucinationcontrolnearestneighborretrieval-augmentedgenerationstatisticallearningdiscordanceBayes-optimalgatequery-memorymismatch

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to put retrieval-augmented generation (RAG) on a statistical footing by modelling the system as a frozen language model, a k-nearest-neighbor retriever, and a query-dependent gate that mixes them. It derives the Bayes-optimal gate: retrieve exactly when the retriever's cross-entropy advantage over the LM exceeds a geometric trust penalty. The central asymptotic result is that, as the memory grows, the gate's effect on a discordance-based hallucination measure converges to a deterministic quantity decided solely by whether the Bayes predictor agrees with the frozen LM at the query. In other words, finite-sample k-NN noise washes out, and the structural Bayes-vs-LM relationship governs whether retrieval helps or hurts hallucination. This matters because it gives a principled, testable criterion for when grounding should improve factuality.

What carries the argument

The central object is the retrieval-trust weight a_fact(x), the average of exp(−||x − neighbor||²) over the k retrieved neighbors, which quantifies geometric reliability and acts as a penalty in the gating objective. The gate itself is the measurable hard decision that compares two local cross-entropies, ℓ_r(x) + ν(1 − a_fact(x)) versus ℓ0(x). The asymptotic analysis leans on uniform k-NN consistency (Proposition 2) and mode stability (Corollary 1), which together replace the empirical retriever and trust weight by their Bayes limits, isolating the structural term P_{Y|X}(c*(x)|x) − P0(c*(x)|x) as the sole determinant of the discordance limit.

What would settle it

Simulate the aligned setting with known P_{Y|X} and a fixed frozen LM, choose query points where ℓ_Bayes(x) < ℓ0(x) and where the Bayes label margin is positive, and verify that the realized ΔH(x; λ*) under the optimal gate converges to the limit in Theorem 1 as n grows; any systematic deviation would falsify the theorem's claim.

Watch

Extended reading notes

Core claim

Under the aligned setting, where query and memory share the same distribution and label mechanism, the optimal hard gate chooses retrieval precisely when the retriever's local cross-entropy advantage over the frozen LM exceeds a trust penalty scaled by ν. Theorem 1 states that, when k → ∞ and k/n → 0, the local hallucination variation ΔH(x; λ*) converges in probability to λ∞(x)(P_{Y|X}(c*(x)|x) − P0(c*(x)|x)), with λ∞(x) = 1{ℓ_Bayes(x) < ℓ0(x)}. Thus the asymptotic sign and magnitude of the hallucination change is governed solely by the structural difference between the Bayes rule and the LM at the query, not by the finite-sample behavior of the k-NN estimator or the trust weight.

Load-bearing premise

The paper's main theorem assumes the aligned setting: queries and memory come from the same distribution with the same label mechanism, which the paper itself notes rarely holds in practice; under mismatch, the retriever converges to the memory's local label law at the projection point, and the clean Bayes-vs-LM limit does not carry over.

Editorial extensions

If this is right

In the large-memory limit, the optimal gate activates exactly when the Bayes cross-entropy is below the LM cross-entropy, making the decision independent of k-NN estimation noise.
When the retriever improves cross-entropy but assigns less mass to its modal label than the LM, the optimal gate can increase discordance, so likelihood gain does not guarantee hallucination reduction.
The retrieval-trust weight acts as a safeguard only in finite samples; its penalty vanishes asymptotically and its multiplicative factor tends to one, so geometric unreliability matters only through finite-sample effects.
Under the mismatch model, the limiting retriever converges to the memory's local label law at the nearest support point, so the penalty term becomes nonzero and the gate naturally shifts toward the LM as the query moves off the memory support.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable practical consequence is that one could build a gating rule without explicitly training a retriever, by comparing a calibrated LM confidence against an entropy estimate from a local memory; if the alignment assumption is violated, the gate should be corrected for projection distance and corruption level.
Because the trade-off region shows likelihood and factual alignment can conflict, a natural extension is to design retrieval-aware training objectives for the LM that reduce structural Bayes-LM disagreement at the modal label, rather than merely improving overall likelihood.
The analytical proxy suggests that RAG failure modes like off-topic retrieval are really failures of the memory's local label law to coincide with the query's true conditional; incorporating a trust penalty as a learned function of distance, rather than fixed exponential, might yield a more robust gate.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the core mathematics is solid, and the main text is honestly scoped, but the abstract overclaims in two ways — it advertises numerical validation that is nowhere in the manuscript, and it states a “solely structural” asymptotic limit that, as the stress-test note correctly points out, depends essentially on the aligned setting.

What is genuinely new: the trust-weighted gating objective (Eq. 3), the retrieval-trust weight a_fact, the pointwise optimal hard gate (Prop. 1), and the discordance decomposition with its three regimes. Theorem 1 is a clean result: in the aligned setting, the limiting effect of optimal gating on local discordance is governed by whether the Bayes predictor beats the frozen LM in cross-entropy, not by k-NN sampling noise. The proof chain (Prop. 2, Cor. 1, Thm. 1) checks out — Hoeffding plus Chernoff plus a margin argument is standard but correctly executed, and the non-degeneracy condition ℓ_Bayes ≠ ℓ_0 is genuinely needed for the indicator to converge. The mismatch appendix (Props. 3–4) is also correct: the k-NN retriever converges to the memory's label law at the projection onto the support, and the bias bound separating geometric and semantic corruption is a nice touch.

The soft spots are proportionate but real. The stress-test concern lands: Theorem 1's limit requires the retriever's target to coincide with the true Bayes conditional, and Appendix B's mismatch model replaces it with the memory's local label law at the projection point. So the abstract's phrase “governed solely by the structural agreement between the Bayes rule and the LM” is too strong — it is “solely” only when memory and query have the same conditional law. The authors concede this in Appendix B (“this alignment rarely holds”), which I credit, but the abstract still overreaches.

Second, the abstract says the framework is “validated numerically on synthetic and real data,” and there are no experiments in the manuscript. That claim should be removed or the experiments supplied. Third, the discordance criterion is explicitly a proxy for hallucination, and the paper deserves credit for flagging it; but there is no calibration showing H_disc tracks actual hallucination rates. That is a minor caveat, not a flaw.

This paper is for statisticians and theoretical ML people who work on RAG, kNN-LM, or gating/mixture models. It gives future work a precise language and a clear target. It deserves a serious referee — my recommendation is to send it to peer review, and to require that the abstract be aligned with the actual content: drop the numerical-validation claim and qualify the aligned-setting dependence.

Referee Report

4 major / 3 minor

Summary. This note proposes a statistical surrogate model for retrieval-augmented generation. A frozen language model P0 and a k-nearest-neighbor retriever Phat are mixed by a query-dependent gate λ, with a trust weight a_fact(x) measuring geometric reliability of the retrieved neighborhood. The population loss is cross-entropy plus a trust penalty ν λ(x)(1-a_fact(x)). Proposition 1 gives the per-query optimal hard gate: retrieve when ℓ_r(x)+ν(1-a_fact(x)) < ℓ0(x). Section 4 defines a discordance-based hallucination score H_disc(x)=a_fact(x)(1-P0(c_hat(x)|x)), and analyzes the change ΔH under the optimal gate. Under the aligned setting (P_Q,X=P_M,X and P_Q,Y|X=P_M,Y|X), with k→∞, k/n→0, Lipschitz conditional distributions and a unique Bayes label, Theorem 1 shows ΔH → 1{ℓ_Bayes<ℓ0}(P_Y|X(c*|x)-P0(c*|x)). Appendix B introduces a mismatch model with geometric deformation and label corruption and gives limits for a_fact and the retriever. The abstract also claims numerical validation on synthetic and real data.

Significance. The theoretical core is sound and clean. Proposition 1 is exact pointwise minimization; Proposition 2's Hoeffding-plus-Chernoff bound and Corollary 1's margin argument are valid; Theorem 1 follows from the continuous mapping theorem under the stated non-degeneracy. The appendix's Propositions 3-4 are useful and establish limits that appear correct. If the paper's scope is restricted to the aligned setting, it provides a useful conceptual reduction: the sign of the gate's effect on discordance is asymptotically governed by Bayes-vs-LM disagreement, not finite-sample kNN noise. The paper does not provide code or experiments, as a theory note; however, the abstract promises numerical validation that is absent.

major comments (4)

[Abstract; §4.2; Appendix B] The claim that the discordance limit is 'governed solely by the structural agreement' between the Bayes rule and the LM is only established in the aligned setting. In the mismatch model of Eq. (11), Propositions 3–4 give F∞(·|x)=(1-ε(x*))P_Y|X(·|x*)+ε(x*)Q(·|x*) for the unique projection x*, and a_fact(x)→exp(-d(x,S)^2). The limiting gate threshold and ΔH then depend on ε, Q, and the projection, not solely on ℓ_Bayes(x) vs ℓ0(x). Appendix B itself states that alignment 'rarely holds' but supplies no mismatch analog of Theorem 1. The abstract should either be qualified to the aligned setting or the theorem should be extended.
[Abstract; §4.1, Eq. (6)] The abstract says the gating rule 'controls this discordance by activating retrieval only where local evidence appears reliable.' But on the trade-off region ℬ of Eq. (6), ℓ_r+ν(1-a_fact)<ℓ0 while P_hat(c_hat)<P0(c_hat), so ΔH<0: the optimal gate increases discordance. The penalty ν only shrinks ℬ; it does not remove it when a_fact≈1. The mathematical content of §4.1 is a three-regime characterization, not a control guarantee. Please revise the abstract and concluding statements accordingly.
[Abstract] The abstract states 'and validate the framework numerically on synthetic and real data.' The manuscript contains no experiments, no data description, and no numerical section; the text proceeds from §4.2 to the references and appendices. This is a missing support for a stated claim. Either add the validation or remove/qualify the sentence.
[§4, Eq. (4) and Eq. (3)] The discordance measure H_disc is defined with a_fact(x) as a multiplicative factor, while a_fact(x) also enters the gate objective as the penalty ν(1-a_fact(x)). In low-trust regions H_disc is small by construction, and the gate is penalized toward λ=0; in high-trust regions a_fact→1 and the asymptotic theorem isolates the Bayes-LM comparison. The finite-sample 'hallucination control' interpretation is therefore partly baked into the proxy. The paper does not validate H_disc against an external factuality metric. Please state this limitation explicitly, or provide evidence that H_disc tracks actual hallucination.

minor comments (3)

[§3, soft-gating FOC] The displayed first-order condition writes Σ p_true (Phat-P0)/p_λ + ν(1-a_fact)=0. The derivative of -log((1-λ)P0+λPhat) with respect to λ is (P0-Phat)/p_λ, so the sign of the numerator appears reversed.
[Theorem 1] The assumption ℓ_Bayes(x)≠ℓ0(x) is necessary for the indicator to stabilize; if equality holds, the limit may depend on tie-breaking or may not exist. This is stated as an assumption but its role could be clarified, especially because the equality case is not measure-zero in general.
[Proposition 2] The conditional distribution P_Y|X(·|·) is described as Lipschitz 'in its second argument,' but the displayed expressions treat the input x as the second argument. Consider rewording to avoid confusion.

Circularity Check

1 steps flagged · score 3.0 of 10

Central Theorem 1 is an independent k-NN consistency derivation; the only definitional circularity is the gain-region discordance statement, and the aligned-setting caveat narrows the abstract claim.

self definitional [Section 4.1, Eq. (6) and 'Interpretation via three regimes', regime (i)]
"Δℋ(x; λ*) = 1{ℓ_r(x)+ν(1−a_fact(x))<ℓ_0(x)} a_fact(x) (p̂^{(k)}_{y_r(x)}(x) − p_0(y_r(x)|x)). ... On G = {ℓ_r+ν(1−a_fact)<ℓ_0, p̂^{(k)}_{y_r} ≥ p_0(y_r)}, ... retrieval improves predictive fit while reinforcing factual evidence, implying Δℋ(x; λ*) ≥ 0."

The set G is defined as the conjunction of the gate's switching condition (the indicator in Eq. 5) and the nonnegativity of the last factor in Eq. (4). Since a_fact(x) ≥ 0, conditional on G the product in Eq. (6) is nonnegative by construction. Thus 'optimal gating reduces discordance in the gain region' is an algebraic restatement of the definitions of H_disc, a_fact, and λ*, rather than an independent model-derived prediction. The behavior is true because the 'gain region' was carved out to make it true.

full rationale

The main asymptotic theorem is not circular: Proposition 2 proves uniform k-NN consistency at a fixed query from Hoeffding/Chernoff bounds, Corollary 1 transfers mode consistency, and Theorem 1 combines these with the explicit indicator form of the optimal gate to obtain a limit governed by ℓ_Bayes(x) vs ℓ_0(x). No fitted parameter is relabeled as a prediction, and the only external references are standard k-NN results (Biau & Devroye 2015), not the authors' own prior work. The one definitional near-circularity is the three-regime discussion in §4.1: the gain region G explicitly includes the switch condition and the nonnegativity of the disagreement factor, so ΔH ≥ 0 on G is true by construction. This weakens the rhetorical claim that the gate was found to control hallucination, but it does not invalidate the asymptotic limit. Appendix B concedes 'this alignment rarely holds' in realistic systems and shows that under mismatch the retriever converges to the memory label at the projection point rather than the query's Bayes rule; this is a substantial scope limit on the abstract's 'solely structural' language, but it is a limitation, not a circular derivation. Overall: central theorem independent; one definitional restatement; score 3.

Assumptions & free parameters 2 free parameters · 8 assumptions · 2 invented entities

The paper introduces two new statistics (a_fact, H_disc) and a trust-penalized objective, then analyzes them with classical k-NN tools. No constants are fitted to data in the text; ν and the kernel scale are hand choices that shape all finite-sample behavior. The dominant fragility is the aligned setting, which the authors concede rarely holds and which Theorem 1's structural-limit interpretation requires. The discordance-to-hallucination mapping is asserted without calibration. Axioms are dominated by domain assumptions (alignment, Lipschitz, margin, non-degeneracy) rather than invented entities; the two invented statistics carry no external falsifiable evidence in the text.

free parameters (2)

ν (regularization weight) = not fitted; user-chosen hyperparameter, ν ≥ 0
Appears in the population objective (3) and controls the trust penalty in the optimal gate (Prop. 1). The theory holds for any fixed ν, but the finite-sample gate and all downstream behavior depend on its value; no selection rule or sensitivity analysis is given (the promised numerics are absent).
Trust kernel scale (implicit bandwidth) = fixed at 1: a_fact(x) = (1/k)Σ exp(−‖x−x_j‖²)
The kernel and its scale are chosen by hand in Eq. (1). The behavior of the gate and the discordance score depends on the embedding scale; no bandwidth or scale-invariance argument is provided.

assumptions (8)

domain assumption Aligned setting: memory and query share identical distributions (P_X = P_x, P_{Y|X} = P_{y|x})
Assumed in Section 2 ('Throughout the main analysis we assume...'); required for Theorem 1's identification of the retriever's limit with the true Bayes conditional. Appendix B concedes this 'rarely holds' in practice.
domain assumption Base LM P_0(·|x) is a frozen probability distribution over Y, treated as an estimate of P_{Y|X}
Section 2; the LM is an input to the model whose internal structure is not modeled; all gating conclusions are relative to this fixed P_0.
domain assumption P_{Y|X}(·|·) is L-Lipschitz in its second argument
Required by Proposition 2 and Theorem 1 for the k-NN consistency bound; excludes abrupt label transitions in the embedding space.
domain assumption Unique Bayes label at x with margin δ(x) > 0, and ℓ_Bayes(x) ≠ ℓ_0(x)
Corollary 1 and Theorem 1; the margin ensures the modal label is identifiable, and the cross-entropy inequality ensures the gate's asymptotic indicator is non-degenerate.
domain assumption Query x lies in supp(P_x) for pointwise convergence
Propositions 2 and 3, Theorem 1; requires positive local mass so the k-NN radius shrinks.
ad hoc to paper Discordance H_disc is a valid proxy for hallucination risk
Section 4 ('which we interpret as a risk of hallucination'); an interpretive leap with no calibration against real hallucination metrics in the text.
ad hoc to paper Mismatch model: queries are deformations of memory inputs (X = W + η(W)) with label corruption rate τ(w)
Appendix B, Eq. (11); the deformation and corruption functions are unestimated model primitives; results are stated for arbitrary such functions.
standard math Standard probability tools: Hoeffding, Chernoff, and Lemma 2.2 of Biau & Devroye (2015) on k-NN radii
Used throughout Appendix A and B.3; not proven in the paper.

invented entities (2)

Retrieval-trust weight a_fact(x)
purpose: Quantifies geometric reliability of the k-NN neighborhood (Eq. 1); used as penalty weight in the objective (3) and as multiplier in the discordance score.
A new scalar statistic defined by the paper. No external benchmark or empirical calibration is provided; the promised numerical validation is absent.
Discordance hallucination score H_disc(P_0; x)
purpose: Proxy measure of hallucination risk combining trust weight and LM probability of the retriever's modal label (Section 4).
New criterion; its connection to actual LLM hallucination rates is asserted, not demonstrated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Note on k-NN Gating in RAG." pith.science (2026). https://pith.science/paper/2C3EWB2Z

@misc{pith2026260113744,
  author       = {Pith},
  title        = {Pith review of: A Note on k-NN Gating in RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2C3EWB2Z}},
  note         = {Machine review of arXiv:2601.13744}
}

read the original abstract

We propose a statistical proxy framework for retrieval-augmented generation (RAG) that formalizes how language models balance internal predictions with retrieved evidence. We derive an optimal query-level gate, analyze hallucination via retrieval discordance, model query-memory mismatch, and validate the framework numerically on synthetic and real data.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 1 linked inside Pith

[1]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 '...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Lectures on the Nearest Neighbor Method

G\'erard Biau and Luc Devroye. Lectures on the Nearest Neighbor Method. Springer, Cham, 2015

2015
[4]

Improving language models by retrieving from trillions of tokens

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, et al. Improving language models by retrieving from trillions of tokens. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesv \'a ri, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Lear...

2022
[5]

Switch transformers: S caling to trillion parameter models with simple and efficient sparsity

William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: S caling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23 0 (120): 0 1--39, 2022

2022
[6]

Unbounded cache model for online language modeling with open vocabulary

Edouard Grave, Moustapha M Cisse, and Armand Joulin. Unbounded cache model for online language modeling with open vocabulary. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna Wallach, Rob Fergus, S. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems, volume 30, pages 6044--6054. Curran Associates, Inc., 2017

2017
[7]

A Distribution-Free Theory of Nonparametric Regression

L \'a szl \'o Gy \"o rfi, Michael Kohler, Adam Krzy \.z ak, and Harro Walk. A Distribution-Free Theory of Nonparametric Regression. Springer, New York, 2006

2006
[8]

Atlas : F ew-shot learning with retrieval augmented language models

Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, et al. Atlas : F ew-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24 0 (251): 0 1--43, 2023

2023

Show all 21 references

[9]

Jacobs, Michael I

Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey I. Hinton. Adaptive mixtures of local experts. In Neural Computation, volume 3, pages 79--87, 1991

1991
[10]

Survey of hallucination in natural language generation

Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, et al. Survey of hallucination in natural language generation. ACM Computing Surveys, 55: 0 248,1--38, 2023

2023
[11]

Adam Tauman Kalai and Santosh S. Vempala. Calibrated language models must hallucinate. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing, STOC 2024, pages 160--171, New York, 2024. Association for Computing Machinery

2024
[12]

Generalization through memorization: N earest neighbor language models

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: N earest neighbor language models. In International Conference on Learning Representations, 2020

2020
[13]

Retrieval-augmented generation for knowledge-intensive NLP tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Hugo Larochelle, Marc ' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Haibin Lin, editors, Advances in Neural Information Proces...

2020
[14]

Self C heck GPT : Z ero-resource black-box hallucination detection for generative large language models

Potsawee Manakul, Adian Liusie, and Mark Gales. Self C heck GPT : Z ero-resource black-box hallucination detection for generative large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural La...

2023
[15]

FA ct S core: Fine-grained atomic evaluation of factual precision in long form text generation

Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, et al. FA ct S core: Fine-grained atomic evaluation of factual precision in long form text generation. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural L...

2023
[16]

Outrageously large neural networks: T he sparsely-gated mixture-of-experts layer

Noam Shazeer, Azalia Mirhoseini, Piotr Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: T he sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations, 2017

2017
[17]

REPLUG : Retrieval-augmented black-box language models

Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, et al. REPLUG : Retrieval-augmented black-box language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational...

2024
[18]

Handling out-of-distribution data: A survey

Lakpa Tamang, Mohamed Reda Bouadjenek, Richard Dazeley, and Sunil Aryal. Handling out-of-distribution data: A survey. arXiv:2507.21160, 2025

2025 arXiv
[19]

Calibrating large language models using their generations only

Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Oh. Calibrating large language models using their generations only. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics...

2024
[20]

Xu, Uri Alon, and Graham Neubig

Frank F. Xu, Uri Alon, and Graham Neubig. Why do nearest neighbor language models work? In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, vol...

2023
[21]

Domain generalization: A survey

Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Loy Chen Change. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45: 0 4396--4415, 2023

2023

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 '...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Lectures on the Nearest Neighbor Method

G\'erard Biau and Luc Devroye. Lectures on the Nearest Neighbor Method. Springer, Cham, 2015

2015

[4] [4]

Improving language models by retrieving from trillions of tokens

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, et al. Improving language models by retrieving from trillions of tokens. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesv \'a ri, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Lear...

2022

[5] [5]

Switch transformers: S caling to trillion parameter models with simple and efficient sparsity

William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: S caling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23 0 (120): 0 1--39, 2022

2022

[6] [6]

Unbounded cache model for online language modeling with open vocabulary

Edouard Grave, Moustapha M Cisse, and Armand Joulin. Unbounded cache model for online language modeling with open vocabulary. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna Wallach, Rob Fergus, S. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems, volume 30, pages 6044--6054. Curran Associates, Inc., 2017

2017

[7] [7]

A Distribution-Free Theory of Nonparametric Regression

L \'a szl \'o Gy \"o rfi, Michael Kohler, Adam Krzy \.z ak, and Harro Walk. A Distribution-Free Theory of Nonparametric Regression. Springer, New York, 2006

2006

[8] [8]

Atlas : F ew-shot learning with retrieval augmented language models

Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, et al. Atlas : F ew-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24 0 (251): 0 1--43, 2023

2023