Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Inference Scaling for Bridging Retrieval and Augmented Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Reordering retrieved passages by a fitted, position-debiased utility — recovered from parallel forward passes over permuted prompts — improves RAG answer quality by roughly seven points, with no retraining.

desk verdict Novel, train-free reranking that shows real gains, but the linear decomposition has an unaddressed identifiability problem that keeps the main claim from being fully supported. read the letter →

arxiv 2412.10684 v1 pith:5HM5FSPY submitted 2024-12-14 cs.CL

Youngwon Lee , Seung-won Hwang , Daniel Campos , Filip Graliński , Zhewei Yao , Yuxiong He This is my paper · ORCID
classification cs.CL
keywords retrieval-augmentedgenerationpositionbiasinferencescalingrerankingpermutationinterventionlostinthemiddleself-consistencymixture-of-intervention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Retrieval-augmented generation can get worse when retrieval gets better: a strong relevance reranker such as RankGPT lifts retrieval scores but lowers answer quality, because the generator over-weights passages near the top of the prompt. This paper claims the gap can be closed at inference time, with no retraining, by treating permuted orderings of the same retrieved passages as interventions and estimating two things from the generator's own scores: a per-position bias coefficient and a debiased per-passage utility. Reordering passages by that utility, not by relevance, is reported to improve ROUGE-L on MS MARCO and exact match on HotpotQA by about seven points, and to reduce citation and fact-verification errors as well. If correct, this gives RAG a train-free lever: the same retriever and generator produce better answers purely from a smarter ordering of the same evidence.

What carries the argument

The load-bearing object is the additive decomposition of Equation 4: each permutation's saliency score $s_i$ is approximated as a weighted sum of position-bias coefficients $a_j$ and passage utilities $u_p$, with the $a_j$ constrained to be non-negative and to sum to 1. Given observed scores from $M$ permuted forward passes, a least-squares fit over the $2N$ unknowns recovers both a position-bias curve and a utility per passage, and ranking by $u_p$ is what removes the generator's positional confounding. Two efficiency mechanisms extend the same decomposition: cyclic permutations of the retriever's ranking are chosen as a representative sample of the ordering space (because the first position dominates generation), and a distilled smaller model, trained by minimizing KL divergence between softmax-normalized saliency distributions, can substitute for the generator in scoring while retaining most of the gain at roughly 40% of the cost.

What would settle it

Score a small set of retrieved passages under all $N!$ orderings with the generator, then check whether the utilities fit by Equation 4 reproduce the held-out orderings: large residual error would show the additive decomposition misses something. A sharper version is the duplicate test: if a list containing the same passage twice is scored, the model predicts roughly double the utility of the single copy, whereas a genuine per-passage utility should be unchanged; observing the former refutes the assumption that utility is independent of context.

Watch

Extended reading notes

Core claim

The paper's central claim is that a permutation-level saliency score — how strongly the generator favors a given ordering of passages — can be decomposed into per-position bias weights and per-passage utilities, and that the decomposition is recoverable from a small set of parallel forward passes. Concretely, MOI scores $M$ permuted prompts and solves a constrained least-squares problem (position weights non-negative and summing to 1) for the bias curve $a_j$ and the utilities $u_p$ in the model $s_i \approx \sum_j a_j \cdot u_{\pi_i[j]}$. Ranking passages by descending fitted utility, rather than by retriever or reranker relevance, is what improves generation: on MS MARCO, ROUGE-L rises from 37.75 with the retriever order to 44.30 with MOI, and on HotpotQA, exact match rises from 48.54 with a random order to 55.67, with similar gains on citation and fact-verification benchmarks. The paper further claims the recovered bias curve is monotonically decreasing, quantifying the 'lost in the middle' effect, and that the MOI ordering is so informative that reversing it drops exact match by 18 points, far more than reversing baseline orderings does.

Load-bearing premise

The load-bearing premise is that a passage's contribution to the answer is its own usefulness multiplied by a factor that depends only on its position, with no interaction between passages — so that one number per passage explains every ordering's score.

Editorial extensions

If this is right

  • RAG pipelines can gain substantially from reordering alone: the same passages fed to the same generator in utility order, not relevance order, raise MS MARCO ROUGE-L from 37.75 to 44.30 and HotpotQA exact match from 48.54 to 55.67.
  • Retrieval quality and generation quality decouple: a reranker with higher MRR (RankGPT at 0.634 versus MOI at 0.464 on MS MARCO) can still yield worse answers, so relevance metrics are not a reliable proxy for RAG output quality.
  • Position bias is measurable and monotone: the fitted coefficients decrease from the first position to the last, giving a quantitative, per-model account of the 'lost in the middle' effect.
  • The method's cost can be cut sharply: cyclic permutations plus prefix pruning achieve over 90% cost savings while keeping about half the performance gain, and a distilled small model retains most of the gain at roughly 40% of the inference cost.
  • The intervention transfers across RAG tasks: it reduces false-positive and false-negative citation errors on TREC-RAGgy and improves fact-verification accuracy on FEVER.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the fitted utilities are genuinely position-independent, the bias curve $a_j$ could be estimated once per generator and reused across queries, leaving only the utility fit to be redone — a per-query cost reduction beyond the paper's cyclic-permutation scheme.
  • The decomposition is a general measurement tool: any ordered list an LLM consumes (few-shot examples, candidate tools, reasoning premises) carries the same positional confound, so permutation-intervention could quantify and correct bias there as well.
  • The paper's additive model is most likely to fail on non-additive passage interactions such as near-duplicates or contradictions; an extension with pairwise interaction terms would both test the model and flag when MOI's ranking should be trusted.
  • The 18-point exact-match drop when MOI's order is reversed suggests the fitted ranking sits near a generator-specific optimum; if so, the estimated utilities could double as a cheap proxy for generation probability, useful for answer selection without extra decoding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Mixture-of-Intervention (MOI), an inference-time method for reranking retrieved passages in RAG. Given N retrieved passages, MOI samples M permutations, computes a scalar saliency score s_i for each permutation from the generator LLM, and solves a constrained least-squares problem to decompose s_i into per-passage utilities u_p and position-bias coefficients a_j. The passages are then reordered by the estimated u_p and fed to the generator. The paper reports large improvements over retrieval and reranking baselines on MS MARCO, HotpotQA, CRAG, TREC-RAGgy, and FEVER, and presents cost-reduction variants based on cyclic permutations, input pruning, and distillation to a smaller model.

Significance. If the method is sound, MOI offers a train-free, inference-only mechanism to bridge the retriever–generator gap, directly addressing the known 'lost-in-the-middle' position bias. The reported gains are large and consistent across tasks and backbone models, and the cost-reduction analyses (cyclic permutations, pruning, distillation) are practically valuable. However, the core decomposition in Eq. (4) is not shown to be identifiable, and no statistical uncertainty is reported for any of the main quantitative claims. These issues must be resolved before the specific numerical improvements can be taken as reliable evidence for the method's effectiveness.

major comments (3)
  1. [§3.1, Eq. (4) and §3.2.2] This is the load-bearing issue: without identifiability, the central claim that MOI 'explicitly models the debiased utility' is unsupported.
  2. [§4.1, Tables 1 and 4–6] This is load-bearing for the quantitative claims in the abstract and Table 1.
  3. [§3.1, Eq. (3)] This is a reproducibility issue that affects the validity of all downstream experiments.
minor comments (5)
  1. [§3.1 and Figure 2 caption] There are typos: 'disentagles' should be 'disentangles' and 'obserevations' should be 'observations'.
  2. [§3.2.2] The sentence 'We leave a formal definition of ϕ and an explanation on why ϕ(2) can represent experiments on shaded permutations starting with 2' is misleading because the formal definition is deferred to Appendix B rather than left out; please rephrase, e.g., 'We defer the formal definition to Appendix B.'
  3. [Figure 6 caption] The dashed line is described as representing 'zero position bias,' but since the coefficients sum to 1, the ideal no-bias case is a uniform distribution a_j = 1/N; please clarify.
  4. [Table 5 discussion] The phrase '70/100% of the gain' is ambiguous; state separately that 70% of the EM gain and 100% of the GPT-4 gain are retained.
  5. [Appendix D] 'Kohen's κ' is a typo for 'Cohen's κ'.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: MOI's fitted utility ranking is validated against external gold answers, and the only self-citation is a non-load-bearing analogy.

full rationale

The central chain in MOI is: observe permutation-level scores si from the generator (Eq. 3), fit position-bias coefficients a and passage utilities u by least squares (Eq. 4), rerank by descending u, feed the reranked contexts to the generator, and measure the final answer against external references (ROUGE, EM, GPT-4/human judgment). The reranked order is a function of the fitted u, but the claimed result is not a prediction of the fitted si values; it is the downstream generation quality relative to gold answers. That evaluation is external to the fit, so the improvement is not forced by construction. The identifiability and underdetermination concern for the cyclic variant (N equations for 2N-1 unknowns) is a correctness or stability risk, not a circularity: no quoted equation reduces to its own input, and the paper does not claim the least-squares fit itself is a prediction. The only self-citation is Hwang and Chang (2007), invoked as an analogy for the notion of a 'comprehensive' subset; the paper explicitly leaves the formal definition open and supplies its own empirical support in Appendix B. That citation is not load-bearing, and all other cited results are external works or are additionally supported by the paper's own analyses. Hence there is no significant circularity; the score of 2 reflects the single incidental, non-load-bearing self-citation rather than any circular step.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method's central output (debiased utility u_p) and bias weights a_j are latent variables fit to the same model's outputs via a constrained least-squares objective; they are not derived from first principles or from independent labels. Several additional domain assumptions (linear additivity, first-position dominance, cross-model preference transfer) are made without direct validation. This constitutes substantial fitting content, though downstream evaluation on golden answers provides some independent check.

free parameters (5)
  • position bias coefficients a_j = estimated per model/dataset
    N coefficients fit by minimizing L2 error between predicted and observed permutation scores (Eq. 4); constrained to sum to 1.
  • passage utilities u_p = estimated per query
    N utilities fit along with a_j from M permutation observations; used for final reranking.
  • number of permutations M = M=3N for random, M=N for cyclic
    Hyperparameter controlling number of LLM calls; chosen by the authors.
  • pruning length L = not specified
    Length of prefix used to approximate full permutations; variable pruning explored; exact values omitted.
  • distillation hyperparameters = K=30, LoRA r=8, alpha=32, dropout 0.1, lr 1e-4, batch 4, epochs 5
    Used for training the student model; not the central method but affects cost and performance.
assumptions (4)
  • domain assumption Permutation score s_i can be approximated as a weighted sum of per-passage utilities and position-dependent biases (Eq. 4).
    Central to MOI; no theoretical justification given beyond empirical fit.
  • domain assumption The first passage in the prompt has the greatest influence on generation, so cyclic permutations starting with each passage are a comprehensive sample.
    Basis for 'comprehensive sampling' in Sec. 3.2.2, supported by prior work (Hsieh et al.; Liu et al.) but only heuristic.
  • domain assumption Models from the same family share sufficiently similar permutation preferences to allow direct substitution.
    Used in model substitution experiments (Table 4); no direct analysis of preference alignment across models.
  • domain assumption Distilling softmax-normalized permutation scores from a teacher to a student transfers the ranking behavior.
    Basis of preference distillation (Sec. 3.3.2); no direct evaluation of student-teacher agreement beyond downstream task scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Inference Scaling for Bridging Retrieval and Augmented Generation." pith.science (2026). https://pith.science/paper/5HM5FSPY

@misc{pith2026241210684,
  author       = {Pith},
  title        = {Pith review of: Inference Scaling for Bridging Retrieval and Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5HM5FSPY}},
  note         = {Machine review of arXiv:2412.10684}
}
read the original abstract

Retrieval-augmented generation (RAG) has emerged as a popular approach to steering the output of a large language model (LLM) by incorporating retrieved contexts as inputs. However, existing work observed the generator bias, such that improving the retrieval results may negatively affect the outcome. In this work, we show such bias can be mitigated, from inference scaling, aggregating inference calls from the permuted order of retrieved contexts. The proposed Mixture-of-Intervention (MOI) explicitly models the debiased utility of each passage with multiple forward passes to construct a new ranking. We also show that MOI can leverage the retriever's prior knowledge to reduce the computational cost by minimizing the number of permutations considered and lowering the cost per LLM call. We showcase the effectiveness of MOI on diverse RAG tasks, improving ROUGE-L on MS MARCO and EM on HotpotQA benchmarks by ~7 points.

Figures

Figures reproduced from arXiv: 2412.10684 by the authors.

Figure 1
Figure 1. (Left, RAG) Top-10 passages retrieved by [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (A, baseline) Self-consistency (Wang et al., 2023a) and MoA (Wang et al., 2024a) treat random permuta￾tions of passages as black-box and count the consistency vote for outcomes. (B, proposed) In MOI, permutations are treated as white-box intervention of one another, such that, from the obserevations of p in varying positions, MOI estimates the effect of each passage on generation u along the impact of position bias … view at source ↗
Figure 3
Figure 3. Ideally, wherever a passage p is placed, its contribution to generation, or utility, should be constant (blue line). However, due to position bias of LLMs, the observed orange curve varies by the position and surrounding context. MOI disentangles the effect of position bias (left figure) from observation, to determine the debiased utility up through multiple parallel inter￾ventions. πi , si “ P ` q | pπir1s ; pπir2s… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The distribution of si from an LLM is dis￾tilled to a smaller model by minimizing KL between the normalized probability distributions after softmax. Values colored orange can be pre-computed. This pruning strategy replaces the full permutation ϕ pkq while still preserv…
Figure 6
Figure 6. Figure 6: The values of computed positional coefficients [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The set of all permutations U can be partitioned into disjoint subsets based on the first item. Distance between two permutations can be measured by the L1 distance between the generator’s predicted probability distribution on the first token of the response. Permutati…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 7 canonical work pages

  1. [1]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. https://arxiv.org/abs/1611.09268 Ms marco: A human generated machine reading comprehension dataset . Preprint, arXiv:1611.09268

    arXiv 2018

  2. [2]

    a is b" fail to learn

    Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2024. https://openreview.net/forum?id=GPKTIktA0k The reversal curse: Llms trained on "a is b" fail to learn "b is a" . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    work page 2024

  3. [3]

    Chi, Xuezhi Wang, and Denny Zhou

    Xinyun Chen, Ryan A. Chi, Xuezhi Wang, and Denny Zhou. 2024. https://openreview.net/forum?id=4zAHgkiCQg Premise order matters in reasoning with large language models . In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net

    work page 2024

  4. [4]

    Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. https://doi.org/10.48550/ARXIV.2401.14887 The power of noise: Redefining retrieval for RAG systems . CoRR, abs/2401.14887

    arXiv doi:10.48550/arxiv.2401.14887 2024

  5. [5]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2023. https://arxiv.org/abs/2312.10997 Retrieval-augmented generation for large language models: A survey . Preprint, arXiv:2312.10997

    arXiv 2023

  6. [6]

    Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, and Tomas Pfister. 2024. https://aclanthology.org/2024.findings-acl.890 Found in the middle: Calibrating positional attention bias improves long context utilization . In Findings of the Association for Computat...

    work page 2024

  7. [7]

    Seung - won Hwang and Kevin Chen - Chuan Chang. 2007. https://doi.org/10.1145/1206049.1206054 Optimizing top-k queries for middleware access: A unified cost-based approach . ACM Trans. Database Syst. , 32(1):5

    arXiv 2007

  8. [8]

    Gautier Izacard and Edouard Grave. 2021. https://doi.org/10.18653/v1/2021.eacl-main.74 Leveraging passage retrieval with generative models for open domain question answering . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 874--880, Online. Association for Computational Li...

    doi:10.18653/v1/2021.eacl-main.74 2021

Show all 33 references
  1. [9]

    Gautier Izacard, Patrick S. H. Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi - Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. http://jmlr.org/papers/v24/23-0037.html Atlas: Few-shot learning with retrieval augmented language model...

    2023

  2. [10]

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin - Yew Lin, Yuqing Yang, and Lili Qiu. 2023. https://doi.org/10.48550/ARXIV.2310.06839 Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression . CoRR, abs/2310.06839

    2023 doi arXiv

  3. [11]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  4. [12]

    Zixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. https://aclanthology.org/2024.acl-long.562 Bridging the preference gap between retrievers and LLM s . In Proceedings of the 62nd Annual Meeting of the Association for Computational Lingu...

    2024

  5. [13]

    Shankar Kumar and William Byrne. 2004. https://aclanthology.org/N04-1022 Minimum B ayes-risk decoding for statistical machine translation . In Proceedings of the Human Language Technology Conference of the North A merican Chapter of the Association for Computational Linguistic...

    2004

  6. [14]

    Chin-Yew Lin and Franz Josef Och. 2004. https://doi.org/10.3115/1218955.1219032 Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics . In Proceedings of the 42nd Annual Meeting of the Association for Computational Ling...

    2004

  7. [15]

    Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Rich James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, Luke Zettlemoyer, and Scott Yih. 2023. https://doi.org/10.48550/ARXIV.2310.01352 RA-DIT: retrieval-augmented dual instruction tuning . CoR...

    2023 doi arXiv

  8. [16]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. https://doi.org/10.1162/tacl_a_00638 Lost in the middle: How language models use long contexts . Transactions of the Association for Computational Linguistics, 12...

    2024 doi

  9. [17]

    Thomas Merth, Qichen Fu, Mohammad Rastegari, and Mahyar Najibi. 2024. https://arxiv.org/abs/2404.06910 Superposition prompting: Improving and accelerating retrieval-augmented generation . Preprint, arXiv:2404.06910

    2024 arXiv

  10. [18]

    Niklas Muennighoff. 2022. https://arxiv.org/abs/2202.08904 SGPT: GPT sentence embeddings for semantic search . CoRR, abs/2202.08904

    2022 arXiv

  11. [19]

    Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. https://doi.org/10.18653/v1/2023.acl-long.352 Parallel context windows for large language models . In Proceedings of the 61st Ann...

    2023 doi

  12. [20]

    Revanth Gangi Reddy, JaeHyeok Doo, Yifei Xu, Md Arafat Sultan, Deevya Swain, Avirup Sil, and Heng Ji. 2024. https://arxiv.org/abs/2406.15657 First: Faster improved listwise reranking with single token decoding . Preprint, arXiv:2406.15657

    2024 arXiv

  13. [21]

    Devendra Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.249 Improving passage retrieval with zero-shot question generation . In Proceedings of the 2022 Conference on Empiri...

    2022 doi

  14. [22]

    Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau, and Manzil Zaheer. 2023. https://doi.org/10.1162/tacl_a_00564 Questions are all you need to train a dense passage retriever . Transactions of the Association for Computational Linguistics, 11:600--616

    2023 doi

  15. [23]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.923 Is C hat GPT good at search? investigating large language models as re-ranking agents . In Proceedings of the 2023 ...

    2023 doi

  16. [24]

    Raphael Tang, Crystina Zhang, Xueguang Ma, Jimmy Lin, and Ferhan Ture. 2024. https://doi.org/10.18653/v1/2024.naacl-long.129 Found in the middle: Permutation self-consistency improves listwise ranking in large language models . In Proceedings of the 2024 Conference of the Nort...

    2024 doi

  17. [25]

    Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. 2024 a . https://arxiv.org/abs/2406.04692 Mixture-of-agents enhances large language model capabilities . Preprint, arXiv:2406.04692

    2024 arXiv

  18. [26]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 a . https://openreview.net/pdf?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Confere...

    2023

  19. [27]

    Rizwan Parvez, and Graham Neubig

    Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md. Rizwan Parvez, and Graham Neubig. 2023 b . https://doi.org/10.48550/ARXIV.2311.08377 Learning to filter context for retrieval-augmented generation . CoRR, abs/2311.08377

    2023 doi arXiv

  20. [28]

    Kakade, Hao Peng, and Heng Ji

    Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hao Huang, Chi Han, Shuiwang Ji, Sham M. Kakade, Hao Peng, and Heng Ji. 2024 b . https://arxiv.org/abs/2407.01100 Eliminating position bias of language models: A mechanistic approach . Preprint, arXiv:2407.01100

    2024 arXiv

  21. [29]

    Kejuan Yang, Xiao Liu, Kaiwen Men, Aohan Zeng, Yuxiao Dong, and Jie Tang. 2023. https://doi.org/10.48550/ARXIV.2305.15262 Revisiting parallel context windows: A frustratingly simple alternative and chain-of-thought deterioration . CoRR, abs/2305.15262

    2023 doi arXiv

  22. [30]

    Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, Lingkun Kong, Brian Moran, Jiaqi Wang, Yifan Ethan Xu, An Yan, Chenyu Yang, Eting Yuan, Hanwen Zha, Nan Tang, Lei Chen, Nicolas Scheffer, Yue...

    2024 arXiv

  23. [31]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  24. [32]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  25. [33]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.