Pith. sign in

REVIEW 3 major objections 5 minor 50 references

FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces FlexRAG, an open-source framework that claims to cover text-based, multimodal, and web-based retrieval in one modular pipeline while using roughly one-tenth the CPU and memory of comparable frameworks on large-scale…

desk verdict A genuinely useful RAG framework paper whose headline efficiency claim needs a recall-controlled comparison before I'd trust the order-of-magnitude number. read the letter →

arxiv 2506.12494 v2 pith:OJQX3JRZ submitted 2025-06-14 cs.CL cs.IR

Zhuocheng Zhang , Yang Feng , Min Zhang This is my paper
classification cs.CLcs.IR
keywords retrieval-augmentedgenerationRAGframeworkdenseretrievalmultimodalwebmemory-mappedindexapproximatenearestneighboropen-sourcetoolkit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that the main obstacles to RAG research—reproducing prior systems, sharing new algorithms, and paying high compute overhead—can be removed by one open-source framework. FlexRAG covers text, multimodal, and web retrieval, provides tools for the whole pipeline from parsing to evaluation, and claims that its memory-mapped dense index uses about one-tenth the CPU and memory of comparable frameworks on large-scale retrieval. If that holds, researchers with limited hardware can prototype and share RAG systems that previously required substantial engineering and memory.

What carries the argument

The load-bearing mechanism is FlexRetriever's default index: a sparse/dense hybrid index built on a memory-mapped IVFPQ structure, an inverted-file index with product quantization, with parameters chosen by an empirical formula from ANN-Benchmarks. Memory mapping keeps the index on disk instead of loading it fully into RAM, and persistent caching plus asynchronous execution further cut repeated retrieval overhead; this is what produces the claimed order-of-magnitude CPU and memory reduction.

What would settle it

Measure FlexRAG and FlashRAG on a second large corpus with a different document-length distribution, using the same default index settings on both and recording peak memory, CPU time, and wall-clock time; if the gap is not near an order of magnitude, the resource-efficiency claim is specific to the original test conditions rather than general.

Watch

Extended reading notes

Core claim

On the paper's own terms, FlexRAG is a claim that one framework can span text, multimodal, and web-based RAG across the full lifecycle—parsing, chunking, indexing, retrieval, reranking, generation, and evaluation—and that its design choices, especially memory mapping and default index tuning, give it a decisive resource advantage: roughly one-tenth the CPU time and memory of comparable frameworks on large-scale dense retrieval, while preserving retrieval quality across standard QA benchmarks when components such as retrievers, indexes, rerankers, and generators are swapped.

Load-bearing premise

The framework's headline resource saving depends on the default memory-mapped compressed index being a fair and representative configuration for large-scale retrieval, which the paper tests on one corpus, one rival framework, and one hardware setup.

Editorial extensions

If this is right

  • Replacing the retriever, reranker, or generator in a RAG pipeline becomes a configuration change rather than a reimplementation, so component ablations should become much cheaper to run.
  • A retriever published to the Hugging Face Hub can be re-run by others without rebuilding the environment, directly attacking the reproducibility problem the paper identifies.
  • Because multimodal encoders and web retrievers live in the same pipeline, a single assistant can be built to answer queries that need both live web content and images.
  • Memory-mapped indexing means a large corpus can be searched on a machine whose RAM is much smaller than the index, lowering the hardware floor for RAG research.
  • The included evaluation tasks and metrics give a standard yardstick, so results across RAG systems built with FlexRAG are easier to compare fairly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The one-tenth resource figure comes from one corpus and one comparison framework under default settings; until it is replicated on other corpora and hardware, it is best read as an upper bound on the advantage of disk-mapped compressed indexes rather than a universal ratio.
  • The paper demonstrates each component swap independently but does not test the cost of operating web and multimodal retrieval together; that combined workload is the next natural stress test for the framework's flexibility claim.
  • A boundary case the paper leaves open is very large corpora with long documents or many small fields, where disk-mapped product quantization can behave differently than on short passage-style data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents FlexRAG, an open-source framework for retrieval-augmented generation. The authors argue that existing RAG frameworks suffer from reproducibility issues, a lack of new techniques, and high system overhead, and they position FlexRAG as a comprehensive solution supporting text-based, multimodal, and web-based RAG. The architecture is organized into models, retrievers, system development, and evaluation modules, with the FlexRetriever using memory-mapped IVFPQ indexing as a key efficiency feature. The reported experiments include a modular-assistant evaluation on NQ, TriviaQA, and PopQA, showing that retriever, index, reranker, and generator choices affect QA performance, and a resource-overhead comparison with FlashRAG on MS MARCO, where the authors claim up to an order-of-magnitude lower wall-clock and CPU time and several-times lower memory usage. The paper concludes with a feature comparison against existing RAG frameworks.

Significance. FlexRAG addresses a real need for a flexible, research-oriented RAG toolkit. Its strengths include a public open-source implementation, integration with the Hugging Face Hub for sharing retrievers, support for multimodal and web retrieval, persistent caching, asynchronous processing, and a memory-mapped IVFPQ index. The empirical QA results are plausible and the framework is broad in scope. However, the paper's headline quantitative claim about resource consumption is not yet established: the resource experiment in §4 lacks any retrieval-quality control, and the reported numbers are internally inconsistent with the Introduction's 'one-tenth of the CPU and memory resources' claim. If the resource claims are confirmed with a matched-quality comparison, this would be a valuable systems contribution; in its current state the central claim is supported only conditionally.

major comments (3)
  1. [Introduction and §4] The headline performance claim is internally inconsistent. The Introduction states that FlexRAG consumes 'only one-tenth of the CPU and memory resources required by comparable frameworks,' but §4 reports memory consumption as 'several times' lower, and Figure 6 shows peak and average memory around 10,000 MB for FlexRAG versus roughly 25,000–30,000 MB for FlashRAG, which is approximately a factor of 2.5–3, not an order of magnitude. The authors must reconcile this discrepancy and present a single, data-consistent version of the claim.
  2. [§4] The resource comparison lacks a retrieval-quality control. The paper does not report recall@k, MRR, or any accuracy metric for the MS MARCO retrieval under either framework. FlexRAG's default is an approximate IVFPQ index whose parameters are set by the ANN-Benchmarks empirical formula (§2.2.2), whereas FlashRAG's default dense retriever uses a different, not necessarily equivalent, approximate-search configuration. The observed 10x CPU-time and several-times memory differences may therefore be explained by an accuracy/resource trade-off rather than by an architectural advantage. To establish the claim, the authors should add a recall-matched comparison (e.g., report recall at the same operating point or show that both frameworks achieve identical retrieval quality) or justify that the default configurations are equivalent in retrieval effectiveness.
  3. [§4] The experimental setup is underspecified, which prevents the reader from assessing the generality of the resource claims. The paper does not state the MS MARCO collection size, the number of queries, the embedding dimension, the IVF list count, the number of probes, the product-quantization settings, or whether the measured time includes query encoding or only index search. Only one dataset, one server, and one baseline (FlashRAG) are used. Since 'Superior Performance' is a key differentiator, the authors should provide the full configuration details and at least one additional corpus or indexing configuration to demonstrate that the result is not an artifact of a single default setup.
minor comments (5)
  1. [§4] There is a typo: 'his evaluation' should be 'this evaluation' in the introductory paragraph of Section 4.
  2. [§3] The definition of the Succ metric is incomplete: the paper says it is the 'Success Rate' used to evaluate retrieval quality, but it does not specify what counts as a success (e.g., whether the answer must appear in the top-k retrieved contexts, and what k is).
  3. [Table 1] Table 1 mixes rows for retrievers, indexers, rerankers, and generators, but the text never clearly maps each row to the four experimental groups described in §3. Adding group labels (e.g., 'retriever', 'index', 'reranker', 'generator') would improve interpretability.
  4. [Figure 5] Figure 5 lists a large set of supported tasks, but only three QA datasets are actually evaluated in this paper. The caption or text should clarify that the figure represents framework capability, not experimental coverage in this manuscript.
  5. [§2.2.2] The 'empirical formula' from ANN-Benchmarks (Aumüller et al., 2018) that sets the default IVFPQ parameters is never stated. Providing the formula or a reference with the exact parameter selection rule would make the default configuration reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: FlexRAG is an engineering/system paper whose claims rest on external benchmarks, baselines, and measured resource comparisons, not on self-referential derivations or fitted parameters.

full rationale

The paper makes no theoretical derivation or predictive claim that reduces to its own inputs. FlexRAG is a framework description with an empirical evaluation: retrieval quality is assessed on external benchmarks (NQ, TriviaQA, PopQA, MS MARCO) against external methods (BM25s, Contriever, E5, BGE, Faiss, ScaNN, etc.), and resource overhead is measured against FlashRAG under stated hardware and dataset conditions. The resource-efficiency claim ('one-tenth of the CPU and memory resources') is an empirical measurement reported in Section 4, not a quantity defined by construction. The choice of IVFPQ defaults via the ANN-Benchmarks formula is an engineering configuration choice, not a fitted parameter that is later renamed as a prediction. No cited result is load-bearing in a circular way: the citations cited for encoders, retrievers, and rerankers are external works, and the paper's own framework components are evaluated rather than asserted through self-citation. The noted discrepancy between 'one-tenth' memory in the Introduction and 'several times' lower memory in Section 4, and the lack of a matched retrieval-quality control in the resource experiment, are correctness/rigor concerns, not circularity. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claims rest on a few domain assumptions rather than mathematical axioms or fitted parameters. No free parameters are introduced. The framework assumes that a single default index configuration and a single baseline comparison can support general efficiency claims, which is evidence-limited.

assumptions (3)
  • domain assumption The default IVFPQ index configuration, chosen using the empirical formula from ANN-Benchmarks, is assumed to be suitable for a wide range of large-scale retrieval workloads.
    Invoked in Section 2.2.2 and Section 4; only tested on MS MARCO.
  • domain assumption The component combinations tested in the empirical study (retriever, index, reranker, generator) are representative of FlexRAG's performance across RAG tasks.
    Section 3 tests three QA datasets; generalization to dialogue, multimodal, and web settings is assumed.
  • domain assumption The reported resource comparisons against FlashRAG use fair, comparable default configurations.
    Section 4 states all experiments under default parameter settings, but no detailed configuration is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation." pith.science (2026). https://pith.science/paper/OJQX3JRZ

@misc{pith2026250612494,
  author       = {Pith},
  title        = {Pith review of: FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OJQX3JRZ}},
  note         = {Machine review of arXiv:2506.12494}
}
read the original abstract

Retrieval-Augmented Generation (RAG) plays a pivotal role in modern large language model applications, with numerous existing frameworks offering a wide range of functionalities to facilitate the development of RAG systems. However, we have identified several persistent challenges in these frameworks, including difficulties in algorithm reproduction and sharing, lack of new techniques, and high system overhead. To address these limitations, we introduce \textbf{FlexRAG}, an open-source framework specifically designed for research and prototyping. FlexRAG supports text-based, multimodal, and network-based RAG, providing comprehensive lifecycle support alongside efficient asynchronous processing and persistent caching capabilities. By offering a robust and flexible solution, FlexRAG enables researchers to rapidly develop, deploy, and share advanced RAG systems. Our toolkit and resources are available at \href{https://github.com/ictnlp/FlexRAG}{https://github.com/ictnlp/FlexRAG}.

Figures

Figures reproduced from arXiv: 2506.12494 by the authors.

Figure 1
Figure 1. The architecture of FlexRAG. The light blue boxes represent modules, while the dashed boxes indicate [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. The core components of the WebRetriever module in FlexRAG and its typical workflow. commercial encoders via API calls1,2,3, and de￾ploying encoders using famous frameworks4,5 . Rerankers Rerankers optimize the initially re￾trieved document list through reordering mecha￾nisms, effectively filtering out irrelevant content to reduce noise and enhance input quality for genera￾tors. FlexRAG supports various rerankers, in… view at source ↗
Figure 2
Figure 2. The GUI demonstration provided by FlexRAG. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Architecture of the Preprocessors module in FlexRAG and its typical workflow. with the Hugging Face ecosystem, enabling seam￾less publication, sharing, and reuse of retrievers via the Hugging Face Hub 10. This integration promotes community collaboration and lowers the…
Figure 5
Figure 5. Figure 5: RAG Evaluation Tasks. Tasks without an asterisk (*) correspond to individual datasets, while those [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: The resource overhead of FlexRAG and FlashRAG under different batch sizes. In terms of memory consumption, FlexRAG also demonstrates substantially lower average and peak memory usage, outperforming the baseline by sev￾eral times. These results highlight the tangible pe…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 5 canonical work pages

  1. [1]

    Martin Aumüller, Erik Bernhardsson, and Alexander Faithfull. 2018. https://doi.org/10.48550/arXiv.1807.05614 ANN - Benchmarks : A Benchmarking Tool for Approximate Nearest Neighbor Algorithms . arXiv preprint. ArXiv:1807.05614 [cs]

    arXiv doi:10.48550/arxiv.1807.05614 2018

  2. [2]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, and 1 others. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268

    arXiv 2016

  3. [3]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://doi.org/10.48550/arXiv.2402.03216 BGE M3 - Embedding : Multi - Lingual , Multi - Functionality , Multi - Granularity Text Embeddings Through Self - Knowledge Distillation . arXiv preprint. ArXiv:2402.03216 [cs]

    arXiv doi:10.48550/arxiv.2402.03216 2024

  4. [4]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.48550/arXiv.1810.04805 BERT : Pre -training of Deep Bidirectional Transformers for Language Understanding . arXiv preprint. ArXiv:1810.04805 [cs]

    arXiv doi:10.48550/arxiv.1810.04805 2019

  5. [5]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2025. https://doi.org/10.48550/arXiv.2401.08281 The Faiss library . arXiv preprint. ArXiv:2401.08281 [cs]

    arXiv doi:10.48550/arxiv.2401.08281 2025

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 514 others. 2024. http://arxiv.org/abs/2407.21783 The Llama 3 Herd of Mo...

    arXiv 2024

  7. [7]

    Zhangchi Feng, Dongdong Kuang, Zhongyuan Wang, Zhijie Nie, Yaowei Zheng, and Richong Zhang. 2024. https://doi.org/10.48550/arXiv.2410.10315 EasyRAG : Efficient Retrieval - Augmented Generation Framework for Automated Network Operations . arXiv preprint. ArXiv:2410.10315

    work page Pith review arXiv doi:10.48550/arxiv.2410.10315 2024

  8. [8]

    Jia Fu, Xiaoting Qin, Fangkai Yang, Lu Wang, Jue Zhang, Qingwei Lin, Yubo Chen, Dongmei Zhang, Saravan Rajmohan, and Qi Zhang. 2024. https://doi.org/10.48550/arXiv.2406.19251 AutoRAG - HP : Automatic Online Hyper - Parameter Tuning for Retrieval - Augmented Generation . arXiv preprint. ArXiv:2406.19251

    arXiv doi:10.48550/arxiv.2406.19251 2024

Show all 50 references
  1. [9]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. 2024. https://doi.org/10.48550/arXiv.2312.10997 Retrieval- Augmented Generation for Large Language Models : A Survey . arXiv preprint. ArXiv:2312.1...

    2024 doi arXiv

  2. [10]

    Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. 2020. https://doi.org/10.48550/arXiv.1908.10396 Accelerating Large - Scale Inference with Anisotropic Vector Quantization . arXiv preprint. ArXiv:1908.10396 [cs]

    2020 doi arXiv

  3. [11]

    Yasuto Hoshi, Daisuke Miyashita, Youyang Ng, Kento Tatsuno, Yasuhiro Morioka, Osamu Torii, and Jun Deguchi. 2023. https://doi.org/10.48550/arXiv.2308.10633 RaLLe : A Framework for Developing and Evaluating Retrieval - Augmented Large Language Models . arXiv preprint. ArXiv:2308.10633

    2023 doi arXiv

  4. [12]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. https://doi.org/10.48550/arXiv.2112.09118 Unsupervised Dense Information Retrieval with Contrastive Learning . arXiv preprint. ArXiv:2112.09118 [cs]

    2022 doi arXiv

  5. [13]

    Rohan Jha, Bo Wang, Michael Günther, Georgios Mastrapas, Saba Sturua, Isabelle Mohr, Andreas Koukounas, Mohammad Kalim Akram, Nan Wang, and Han Xiao. 2024. https://doi.org/10.48550/arXiv.2408.16672 Jina- ColBERT -v2: A General - Purpose Multilingual Late Interaction Retriever ...

    2024 doi arXiv

  6. [14]

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.825 LLMLingua : Compressing Prompts for Accelerated Inference of Large Language Models . In Proceedings of the 2023 Conference on Empirical Methods in Natural...

    2023 doi

  7. [15]

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. https://doi.org/10.18653/v1/2024.acl-long.91 LongLLMLingua : Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression . In Proceedings of the 62nd Annua...

    2024 doi

  8. [16]

    Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, and Zhicheng Dou. 2024. https://doi.org/10.48550/arXiv.2405.13576 FlashRAG : A Modular Toolkit for Efficient Retrieval - Augmented Generation Research . arXiv preprint. ArXiv:2405.13576 [cs] version: 1

    2024 doi arXiv

  9. [17]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 TriviaQA : A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational L...

    2017 doi

  10. [18]

    Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. http://arxiv.org/abs/2004.04906 Dense Passage Retrieval for Open - Domain Question Answering . arXiv preprint. ArXiv:2004.04906 [cs]

    2020 arXiv

  11. [19]

    Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. https://doi.org/10.48550/arXiv.2501.03468 MTRAG : A Multi - Turn Conversational Benchmark for Evaluating Retrie...

    2025 doi arXiv

  12. [20]

    Omar Khattab and Matei Zaharia. 2020. https://doi.org/10.48550/arXiv.2004.12832 ColBERT : Efficient and Effective Passage Search via Contextualized Late Interaction over BERT . arXiv preprint. ArXiv:2004.12832 [cs]

    2020 doi arXiv

  13. [21]

    Dongkyu Kim, Byoungwook Kim, Donggeon Han, and Matouš Eibich. 2024 a . https://doi.org/10.48550/arXiv.2410.20878 AutoRAG : Automated Framework for optimization of Retrieval Augmented Generation Pipeline . arXiv preprint. ArXiv:2410.20878

    2024 doi arXiv

  14. [22]

    Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024 b . https://doi.org/10.48550/arXiv.2404.13081 SuRe : Summarizing Retrievals using Answer Candidates for Open -domain QA of LLMs . arXiv preprint. ArXiv:2404.13081 [cs]

    2024 doi arXiv

  15. [23]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  16. [24]

    Thiago Laitz, Konstantinos Papakostas, Roberto Lotufo, and Rodrigo Nogueira. 2024. http://arxiv.org/abs/2401.06910 InRanker : Distilled Rankers for Zero -shot Information Retrieval . arXiv preprint. ArXiv:2401.06910 [cs]

    2024 arXiv

  17. [25]

    Yucheng Li, Bo Dong, Chenghua Lin, and Frank Guerin. 2023. https://doi.org/10.48550/arXiv.2310.06201 Compressing Context to Enhance Inference Efficiency of Large Language Models . arXiv preprint. ArXiv:2310.06201 [cs]

    2023 doi arXiv

  18. [26]

    Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023. http://arxiv.org/abs/2302.07452 How to Train Your DRAGON : Diverse Augmentation Towards Generalizable Dense Retrieval . arXiv preprint. ArXiv:2302.07452 [cs]

    2023 arXiv

  19. [27]

    Xing Han Lù. 2024. https://doi.org/10.48550/arXiv.2407.03618 BM25S : Orders of magnitude faster lexical search via eager sparse scoring . arXiv preprint. ArXiv:2407.03618 [cs]

    2024 doi arXiv

  20. [28]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.48550/arXiv.2212.10511 When Not to Trust Language Models : Investigating Effectiveness of Parametric and Non - Parametric Memories . arXiv preprint. ArXiv:22...

    2023 doi arXiv

  21. [29]

    Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. https://doi.org/10.48550/arXiv.2210.07316 MTEB : Massive Text Embedding Benchmark . arXiv preprint. ArXiv:2210.07316 [cs]

    2023 doi arXiv

  22. [30]

    Rodrigo Nogueira, Zhiying Jiang, and Jimmy Lin. 2020. http://arxiv.org/abs/2003.06713 Document Ranking with a Pretrained Sequence -to- Sequence Model . arXiv preprint. ArXiv:2003.06713 [cs]

    2020 arXiv

  23. [31]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 doi arXiv

  24. [32]

    Vicky Zhao, Lili Qiu, and Dongmei Zhang

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. https://doi.org/10.18653/v1/2024.findings-acl.57 LLMLingua -2: Data Distillation for Efficient a...

    2024 doi

  25. [33]

    Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. https://doi.org/10.48550/arXiv.2009.02252 KILT : a Benc...

    2021 doi arXiv

  26. [34]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. https://doi.org/10.48550/arXiv.2103.00020 Learning Transferable Visual Models From Na...

    2021 doi arXiv

  27. [35]

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. https://doi.org/10.48550/arXiv.2112.01488 ColBERTv2 : Effective and Efficient Retrieval via Lightweight Late Interaction . arXiv preprint. ArXiv:2112.01488 [cs]

    2022 doi arXiv

  28. [36]

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023. https://doi.org/10.48550/arXiv.2302.00093 Large Language Models Can Be Easily Distracted by Irrelevant Context . arXiv preprint. ArXiv:2302.00093 [cs]

    2023 doi arXiv

  29. [37]

    Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, Siyang Qin, Reeve Ingle, Emanuele Bugliarello, Sahar Kazemzadeh, Thomas Mesnard, Ibrahim Alabdulmohsin, Lu...

    2024 doi arXiv

  30. [38]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. http://arxiv.org/abs/2304.09542 Is ChatGPT Good at Search ? Investigating Large Language Models as Re - Ranking Agents . arXiv preprint. ArXiv:2304.09542 [cs]

    2023 arXiv

  31. [39]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024 a . https://doi.org/10.48550/arXiv.2212.03533 Text Embeddings by Weakly - Supervised Contrastive Pre -training . arXiv preprint. ArXiv:2212.03533 [cs]

    2024 doi arXiv

  32. [40]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024 b . https://doi.org/10.48550/arXiv.2409.1...

    2024 doi arXiv

  33. [41]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. https://doi.org/10.48550/arXiv.2310.04408 RECOMP : Improving Retrieval - Augmented LMs with Compression and Selective Augmentation . arXiv preprint. ArXiv:2310.04408 [cs]

    2023 doi arXiv

  34. [42]

    Peng Xu, Wei Ping, Xianchao Wu, Chejian Xu, Zihan Liu, Mohammad Shoeybi, and Bryan Catanzaro. 2024. https://doi.org/10.48550/arXiv.2407.14482 ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities . arXiv preprint. ArXiv:2407.14482 [cs]

    2024 doi arXiv

  35. [43]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 43 others. 2024. https://doi.org/10.4...

    2024 doi arXiv

  36. [44]

    Hao Yu, Aoran Gan, Kai Zhang, Shiwei Tong, Qi Liu, and Zhaofeng Liu. 2024 a . Evaluation of retrieval-augmented generation: A survey. In CCF Conference on Big Data, pages 102--120. Springer

    2024

  37. [45]

    Xiao Yu, Yunan Lu, and Zhou Yu. 2024 b . https://doi.org/10.48550/arXiv.2403.00982 LocalRQA : From Generating Data to Locally Training , Testing , and Deploying Retrieval - Augmented QA Systems . arXiv preprint. ArXiv:2403.00982

    2024 doi arXiv

  38. [46]

    Taolin Zhang, Dongyang Li, Qizhou Chen, Chengyu Wang, Longtao Huang, Hui Xue, Xiaofeng He, and Jun Huang. 2024 a . http://arxiv.org/abs/2405.02659 R4: Reinforced Retriever - Reorder - Responder for Retrieval - Augmented Large Language Models . arXiv preprint. ArXiv:2405.02659 [cs]

    2024 arXiv

  39. [47]

    Xuanwang Zhang, Yunze Song, Yidong Wang, Shuyun Tang, Xinfeng Li, Zhengran Zeng, Zhen Wu, Wei Ye, Wenyuan Xu, Yue Zhang, Xinyu Dai, Shikun Zhang, and Qingsong Wen. 2024 b . https://doi.org/10.48550/arXiv.2408.11381 RAGLAB : A Modular and Research - Oriented Unified Framework f...

    2024 doi arXiv

  40. [48]

    Qiu, and Lili Qiu

    Siyun Zhao, Yuqing Yang, Zilong Wang, Zhiyuan He, Luna K. Qiu, and Lili Qiu. 2024. https://doi.org/10.48550/arXiv.2409.14924 Retrieval Augmented Generation ( RAG ) and Beyond : A Comprehensive Survey on How to Make your LLMs use External Data More Wisely . arXiv preprint. ArXi...

    2024 doi arXiv

  41. [49]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  42. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.