Pith. sign in

REVIEW 3 major objections 6 minor 3 cited by

O1 Embedder: Let Retrievers Think Before Action

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes O1 Embedder, a dense retriever that first generates retrieval-oriented thoughts about a query and then embeds the thought-augmented query, reporting consistent gains over strong baselines on 12 datasets.

desk verdict Useful empirical paper on trainable query expansion for dense retrieval; the thinking claim is overstated but the gains and ablations hold up. read the letter →

arxiv 2502.07555 v2 pith:3YLD3GLM submitted 2025-02-11 cs.CL

Ruiran Yan , Zheng Liu , Defu Lian This is my paper · ORCID
classification cs.CL
keywords denseretrievalthought-augmentedembeddingreasoningzero-shotmulti-tasktrainingbehaviorcloningcontrastivelearningLLMmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to give dense retrieval the slow-thinking ability that reasoning LLMs have: before a query is embedded, the model first generates several thoughts about what information the query is really asking for. To train this, the authors synthesize thought data by having a strong LLM propose candidate thoughts and a committee of retrievers vote on which thought is closest to the known relevant document, then fine-tune one model with two losses: next-token prediction on the chosen thought and contrastive learning on query and thought-augmented query embeddings. The central claim is that this single model, called O1 Embedder, outperforms existing dense retrievers on 12 datasets in both in-domain and zero-shot settings, with the largest gains on multi-hop and code-search tasks. If true, it shows that reasoning before embedding is a general capability a retriever can learn, not a task-specific trick.

What carries the argument

The load-bearing piece is the thought-augmented query embedding: the model writes k thoughts for a query, encodes query plus thought for each, and mean-pools the vectors before matching against document embeddings. Training rests on two mechanisms: an exploration-refinement data pipeline, in which an LLM generates candidate thoughts and a retrieval committee scores each candidate against the ground-truth document and selects the majority-vote winner as the training target; and multi-task fine-tuning, which combines a generation loss (behavior cloning of the chosen thought) with a composite contrastive loss over both the plain query and the thought-augmented query. A dedicated <emb> token is appended after the generation-stopping token so the embedding objective does not collapse the generation task.

What would settle it

Train the same O1 Embedder pipeline but select training thoughts by retrieval-committee score while masking out any candidate thought whose tokens substantially overlap the ground-truth document; if the zero-shot BEIR gains collapse to near the no-thinking baseline while in-domain gains survive, the committee score is rewarding paraphrase rather than transferable reasoning.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a pre-trained LLM can be fine-tuned to do both jobs at once: generate retrieval-utility thoughts for a query and embed the query together with those thoughts into one dense vector. The authors report that this thought-augmented embedding beats strong LLM-based baselines on MS MARCO, TREC DL19/DL20, and all nine out-of-domain sets they test, including HotPotQA and CosQA, and that the thinking component is responsible for most of the gain. The paper also demonstrates that jointly training the two capabilities matters: using an external thought generator with an already-trained retriever yields inconsistent results, whereas the jointly trained model improved on every dataset.

Load-bearing premise

The load-bearing premise is that a retriever's similarity score between a candidate thought and the ground-truth document measures how useful that thought is for retrieval; if that score mainly rewards thoughts that restate the answer, the training signal may teach answer regurgitation instead of transferable reasoning.

Editorial extensions

If this is right

  • A single fine-tuned model can both generate retrieval thoughts and embed thought-augmented queries, so retrieval systems can add a slow-thinking stage without a separate generator module.
  • The thinking operation contributes most on tasks that need multi-hop or code reasoning, which is where direct semantic matching falls short.
  • Joint training of generation and contrastive learning is what makes thoughts usable; bolting an external thought generator onto a fixed retriever does not reliably help.
  • The training recipe transfers across backbone LLMs of different families and sizes, with performance roughly tracking model scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of whether the committee teaches transferable reasoning rather than answer paraphrasing would be to edit selected thoughts to remove n-gram overlap with the target document and re-measure zero-shot gains.
  • The same thought-augmented retrieval could be combined with instruction-tuned retrievers or iterative multi-round thinking, since the aggregation step currently uses simple mean pooling.
  • Generated thoughts could double as human-readable explanations for why a query retrieved what it did, which is a debugging affordance the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes O1 Embedder, a dense retrieval model built on a pre-trained LLM that generates one or more 'thoughts' for an input query before producing a thought-augmented embedding for retrieval. The method has two main components: (1) a data synthesis pipeline in which an LLM generates candidate thoughts and a retrieval committee selects the most useful one by similarity to the gold document under majority voting, and (2) a multi-task training procedure that combines behavior-cloning generation loss with contrastive retrieval loss, using a separate <emb> token for embeddings. The model is trained on MS MARCO and evaluated on MS MARCO dev, TREC DL19/DL20, and nine BEIR datasets, reporting consistent improvements over RepLLaMA and Promptriever in both in-domain and zero-shot settings.

Significance. If the reported results are reliable, O1 Embedder is a practical extension of LLM-based dense retrieval: it shows consistent gains across diverse benchmarks without task-specific fine-tuning, and the joint training design (shared encoding, separate embedding token, composite contrastive loss) is a useful technical contribution. The paper also provides a clear, reproducible-looking data production workflow based on an LLM teacher and a retrieval committee. However, the stronger claim that the model acquires a general 'slow-thinking' reasoning capability is not yet supported: the thought-selection criterion is based on similarity to the gold document, and the provided case study shows the selected thought closely paraphrases the answer, leaving open that the mechanism is answer regurgitation rather than transferable reasoning. The absence of code, data, key hyperparameters, and significance tests further limits the strength of the conclusions.

major comments (3)
  1. [Section 3.2 and Table 5] The retrieval committee selects the 'golden' thought by taking each retriever's argmax similarity sigma_r(t_i,d) to the ground-truth document d, and the Table 5 case study shows that the selected thought ('admitted to Morehouse College at age 15') is nearly a restatement of the answer in the positive document. This operationalizes retrieval utility as proximity to the target passage, so the model may be learning to generate answer-like text rather than transferable reasoning. The claim that O1 Embedder provides a 'slow-thinking' reasoning capability is therefore not established by the current evidence; the method could be viewed as a well-engineered HyDE variant with a distilled answer generator. Please add an ablation comparing against HyDE with the same teacher LLM (without committee filtering) and an analysis of whether the selected thoughts contain reasoning steps beyond answer extraction, particularly for multi-hop queries. If the gains are largely attributable to answer paraphrase, the writing should be revised to avoid overclaiming the reasoning contribution.
  2. [Section 4.1.3 and Eq. (10)] The loss weight lambda in Eq. (10), the number of candidate thoughts k used in data production (Section 3.2) and inference (Section 3.4), and the number of in-context examples m in Eq. (4) are not reported. Because the paper states that code and data will be made publicly available but does not currently supply them, these hyperparameters are essential for reproducibility and for assessing whether the main results are sensitive to their values. Please report these values, together with the decoding temperature used for thought generation at inference.
  3. [Table 2] On several BEIR datasets the improvements over the strongest baseline are small (e.g., +0.9 nDCG@10 on TREC-COVID, +1.7 on FiQA, and +1.8 on SciFact relative to Promptriever), and no significance tests or variance estimates are provided. Given the paper's claim of 'substantial improvements' across 12 datasets, please add significance testing (e.g., paired bootstrap over queries) or run-level statistics to confirm that these differences are not within run-to-run noise.
minor comments (6)
  1. [Throughout] The method name is inconsistently spelled 'Promptriver' (Sections 1, 2.1, 4.1.2) whereas the cited work is 'Promptriever'; please correct.
  2. [Table 2 and Table 5] There are typos: 'O1 embeder' in Table 2, 'gnenrated' in the Table 5 caption, and 'tamplate' in Algorithm 1; please correct them.
  3. [Eq. (7)] The summation upper limit in the generation loss appears to be |q_i|+|t_i|, which would include one more token than the thought's |t_i| tokens; please define the indexing explicitly (e.g., j = |q_i|+1, ..., |q_i|+|t_i|−1 or clarify that the special end token is included).
  4. [Section 4.2.1] The statement 'MRR@10 of 43.1 (+1.9% improvement)' should read '+1.9 points' or '+4.6% relative', as the current phrasing conflates absolute and relative improvements.
  5. [Section 4.1.1 and Tables 1-2] The description of the 'O1 embedder w/o T' baseline should clarify whether this variant is trained with the same multi-task objective and merely disables thought at inference, or is trained without the generation loss; this affects the interpretation of the 'Impact of Thought' comparison.
  6. [Appendix D] The equation for attention scores uses variables q_i and k_i that are not defined in the main text; please define them.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the held-out retrieval evaluation is external, and the committee-based thought selection in Section 3.2 is a training-label construction choice rather than a test-time prediction.

full rationale

The derivation chain is self-contained. In Section 3.2, candidate thoughts are scored by a retrieval committee using similarity to the gold document d, and the highest-scoring thought becomes the behavior-cloning target; in Eq. (9), the same d is the positive for contrastive learning. This shared use of d is ordinary supervised learning: the training label is derived from the gold document, and the model is then evaluated on unseen queries with no test-set labels used for training or threshold selection. The committee members (BM25, BGE, GTE, Stella) are pre-trained retrievers, not the model being evaluated, so the selection is not defined by the final model's own output. No fitted parameter is renamed a prediction, no uniqueness theorem is imported from prior work, and the paper's self-citations (e.g., BGE, RetroMAE) are background references rather than load-bearing premises. The principal caveat is that 'optimal retrieval utility' is defined operationally by committee similarity to d, making the data-production claim definitional; whether this yields transferable reasoning rather than answer paraphrase is a construct-validity concern, not a circularity of the derivation. The evaluation on MS MARCO dev, TREC DL, and BEIR remains an external, held-out test of the claimed improvements.

Assumptions & free parameters 6 free parameters · 6 assumptions · 1 invented entities

The central claim depends on the training pipeline, not on a closed-form model, so the ledger is dominated by hand-set hyperparameters and domain assumptions. The most important unverified premise is that retrieval-committee similarity to the target document selects thoughts with genuine retrieval utility; the paper provides no independent validation of this premise.

free parameters (6)
  • candidate/retrieval thought count k = not reported
    The number of independently generated thoughts per query during data synthesis and at retrieval; required to reproduce the pipeline and directly affects the thought-augmented embedding in Eq. 11.
  • in-context example count m = 3
    Appendix A states "randomly select 3 query and positive document pair" for the teacher prompt; chosen by hand and affects thought quality.
  • max thought tokens = 256
    Section 4.1.3 sets 256 max tokens for thought generation; truncation could affect retrieval utility.
  • batch size = 64
    Section 4.1.3 uses batch size 64; contrastive learning in Eq. 9 depends on in-batch negatives, so batch size affects the embedding quality.
  • hard negatives per query = 15
    Section 4.1.3 uses 15 hard negatives from an unspecified pre-trained embedder; the hard-negative source is not named, affecting reproducibility.
  • loss weight lambda = not reported
    Eq. 10 defines the total loss as lambda times generation loss plus (1-lambda) times contrastive loss, but the value of lambda is never given.
assumptions (6)
  • domain assumption Inner-product similarity between query and document embeddings is a sufficient relevance signal for dense retrieval (Eq. 1).
    The whole method optimizes and evaluates this objective; Eq. 1 defines retrieval as argmax inner product.
  • domain assumption Retrieval committee similarity between a candidate thought and the ground-truth document is a valid proxy for the thought's retrieval utility.
    Section 3.2 selects the golden thought by majority voting on sigma_r(t_i,d); this premise is load-bearing for the data synthesis.
  • domain assumption The teacher LLM (Llama-3.1-70B-Instruct) can generate useful candidate thoughts across source and target domains.
    Section 4.1.3 uses this model for all thought generation; if it hallucinates on specialized domains, the paper's own analysis in Section 4.2.2 admits performance gains shrink.
  • domain assumption The BEIR protocol is a valid zero-shot generalization measure without train/test contamination from the teacher model or retrieval committee.
    Section 4.1.1 uses BEIR for out-of-domain evaluation; contamination would inflate the reported generalization.
  • ad hoc to paper Using a separate <emb> token prevents interference between the generation and embedding tasks.
    Section 3.3.2 introduces <emb> with this motivation but provides no analysis isolating its effect.
  • standard math Standard cross-entropy and InfoNCE-style contrastive losses are appropriate training objectives for generation and embedding.
    Eqs. 7 and 9 define the losses; these are background ML objectives, not proved in the paper.
invented entities (1)
  • Retrieval thought (t_i)
    purpose: An intermediate text generated for a query before encoding, intended to expose hidden information needs and improve dense retrieval.
    The thought is a modeling artifact. Its only observed evidence is the benchmark improvements in Tables 1 and 2; no external measurable signature is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of O1 Embedder: Let Retrievers Think Before Action." pith.science (2026). https://pith.science/paper/3YLD3GLM

@misc{pith2026250207555,
  author       = {Pith},
  title        = {Pith review of: O1 Embedder: Let Retrievers Think Before Action},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3YLD3GLM}},
  note         = {Machine review of arXiv:2502.07555}
}
read the original abstract

The growing power of large language models (LLMs) has revolutionized how people access and utilize information. Notably, the LLMs excel at performing fine-grained data representation, which facilitates precise retrieval of information. They also generate high-quality answers based on external references, enabling the production of useful knowledge. The recent introduction of reasoning models, like OpenAI O1 and DeepSeek R1, marks another leap forward, highlighting LLMs' ability to think progressively before delivering final answers. This breakthrough significantly improves the ability to address complex tasks, e.g., coding and math proofs. Inspired by this progress, we aim to develop similar capabilities for retrieval models, which hold great promise for tackling critical challenges in the field, including multi-task retrieval, zero-shot retrieval, and tasks requiring intensive reasoning of complex relationships. With this motivation, we propose a novel approach called O1 Embedder, which generates useful thoughts for the input query before making retrieval for the target documents. To realize this objective, we conquer two technical difficulties. First, we design a data synthesis workflow, creating training signals for O1 Embedder by generating initial thoughts from an LLM-expert and subsequently refining them using a retrieval committee. Second, we optimize the training process, enabling a pre-trained model to be jointly fine-tuned to generate retrieval thoughts via behavior cloning and perform dense retrieval through contrastive learning. Our approach is evaluated by comprehensive experiments, where substantial improvements are achieved across 12 popular datasets, spanning both in-domain and out-of-domain scenarios. These results highlight O1 Embedder's remarkable accuracy and generalizability, paving the way for the development of next-generation IR foundation models.

Figures

Figures reproduced from arXiv: 2502.07555 by the authors.

Figure 1
Figure 1. O1 Embedder. First of all, the model generates the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The production of thought data. In the first step, the LLM is prompted to generate candidates thoughts about the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Training and Retrieval process of O1 Embedder. During the training process, O1 embedder minimizes two losses: the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Top 20 Attention score from <emb> token in the [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering

    cs.CL 2026-01 conditional novelty 7.0 of 10

    On ChemKGMultiHopQA, iterative retrieval-reasoning outperformed oracle gold-context static RAG for all 11 LLMs tested, with gains up to 25.6 percentage points.

  2. MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning

    cs.IR 2026-03 conditional novelty 6.0 of 10

    MemSifter trains a 4B proxy with an outcome-driven, rank-sensitive RL reward to sift LLM memory, and on eight benchmarks it matches or beats embedding, graph, and long-context baselines.

  3. Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?

    cs.IR 2025-05 conditional novelty 6.0 of 10

    Standard pointwise rerankers beat reasoning-based pointwise rerankers under identical training, and removing the reasoning process at inference partially recovers the gap.

Reference graph

Works this paper leans on

74 extracted references · 8 canonical work pages · cited by 3 Pith papers

  1. [1]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. arXiv:1611.09268 [cs.CL] https://arxiv.org/abs/1611.09268

    arXiv 2018

  2. [2]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 17682–17690

    2024

  3. [3]

    Alexander Bondarenko, Maik Fröbe, Meriem Beloucif, Lukas Gienapp, Yamen Ajjour, Alexander Panchenko, Chris Biemann, Benno Stein, Henning Wachsmuth, Martin Potthast, et al . 2020. Overview of Touché 2020: argument retrieval. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conference of the CLEF Association, CLEF...

    2020

  4. [4]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. https://doi.org/10.48550/ arXiv.2402.03216 arXiv:2402.03216 [cs]

    arXiv doi:10.48550/arxiv.2402.03216 2024

  5. [5]

    Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. 2024. Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems. arXiv:2403.02419 [cs.LG] https: //arxiv.org/abs/2403.02419

    arXiv 2024

  6. [6]

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track. arXiv:2102.07662 [cs.IR] https://arxiv.org/ abs/2102.07662

    arXiv 2021

  7. [7]

    Voorhees

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. arXiv:2003.07820 [cs.IR] https://arxiv.org/abs/2003.07820

    arXiv 2020

  8. [8]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    arXiv 2018

Show all 74 references
  1. [9]

    Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang

  2. [10]

    Luyu Gao and Jamie Callan. 2022. Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Smaranda Muresan, Preslav Nakov, and Aline ...

    2022 doi

  3. [11]

    Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2022. Precise zero-shot dense retrieval without relevance labels. arXiv preprint arXiv:2212.10496 (2022)

    2022 arXiv

  4. [12]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 (2023)

    2023 arXiv

  5. [13]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, A...

    2018 arXiv

  6. [14]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  7. [15]

    Faegheh Hasibi, Fedor Nikolaev, Chenyan Xiong, Krisztian Balog, Svein Erik Bratsberg, Alexander Kotov, and Jamie Callan. 2017. DBpedia-Entity v2: A Test Collection for Entity Search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in I...

    2017

  8. [16]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 (2022)

    2022 arXiv

  9. [17]

    Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Al- lan Hanbury. 2021. Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling. https://doi.org/10.48550/arXiv.2104.06967 arXiv:2104.06967 [cs]

    2021 doi arXiv

  10. [18]

    Junjie Huang, Duyu Tang, Linjun Shou, Ming Gong, Ke Xu, Daxin Jiang, Ming Zhou, and Nan Duan. 2021. CoSQA: 20,000+ Web Queries for Code Search and Question Answering. In Proceedings of the 59th Annual Meeting of the Associa- tion for Computational Linguistics and the 11th Inte...

    2021

  11. [19]

    Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. CodeSearchNet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436 (2019)

    2019 arXiv

  12. [20]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Information Retrieval with Contrastive Learning. arXiv:2112.09118 [cs.IR] https://arxiv.org/abs/2112.09118

    2022 arXiv

  13. [21]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thoma...

    2023 arXiv

  14. [22]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)

    2020 arXiv

  15. [23]

    Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. arXiv preprint arXiv:2004.04906 (2020)

    2020 arXiv

  16. [24]

    Mei Kobayashi and Koichi Takeda. 2000. Information retrieval on the web. ACM computing surveys (CSUR) 32, 2 (2000), 144–173

    2000

  17. [25]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  18. [26]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. arXiv preprint arXiv:2405.17428 (2024)

    2024 arXiv

  19. [27]

    Hyunji Lee, Sohee Yang, Hanseok Oh, and Minjoon Seo. 2022. Generative Multi- hop Retrieval. arXiv:2204.13596 [cs.IR] https://arxiv.org/abs/2204.13596

    2022 arXiv

  20. [28]

    Chaofan Li, Zheng Liu, Shitao Xiao, Yingxia Shao, and Defu Lian. 2024. Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrieval. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Lun-Wei ...

    2024 doi

  21. [29]

    Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. 2024. Making text embedders few-shot learners. arXiv preprint arXiv:2409.15700 (2024)

    2024 arXiv

  22. [30]

    Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. 2024. Making Text Embedders Few-Shot Learners. https://doi.org/10.48550/arXiv.2409.15700 arXiv:2409.15700 [cs]

    2024 doi arXiv

  23. [31]

    Lei Li, Xiangxu Zhang, Xiao Zhou, and Zheng Liu. 2024. AutoMIR: Ef- fective Zero-Shot Medical Information Retrieval without Relevance Labels. arXiv:2410.20050 [cs.IR] https://arxiv.org/abs/2410.20050

    2024 arXiv

  24. [32]

    Xiangyang Li, Kuicai Dong, Yi Quan Lee, Wei Xia, Yichun Yin, Hao Zhang, Yong Liu, Yasheng Wang, and Ruiming Tang. 2024. CoIR: A Comprehensive Benchmark for Code Information Retrieval Models. arXiv:2407.02883 [cs.IR] https://arxiv.org/abs/2407.02883

    2024 arXiv

  25. [33]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281 (2023)

    2023 arXiv

  26. [34]

    Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 364 (2019)

    2019 arXiv

  27. [35]

    Zheng Liu, Shitao Xiao, Yingxia Shao, and Zhao Cao. 2023. RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers...

    2023 doi

  28. [36]

    Zheng Liu, Yujia Zhou, Yutao Zhu, Jianxun Lian, Chaozhuo Li, Zhicheng Dou, Defu Lian, and Jian-Yun Nie. 2024. Information Retrieval Meets Large Language Models. In Companion Proceedings of the ACM Web Conference 2024 (Singapore, Singapore) (WWW ’24). Association for Computing ...

    2024

  29. [37]

    Kun Luo, Minghao Qin, Zheng Liu, Shitao Xiao, Jun Zhao, and Kang Liu. 2024. Large language models as foundations for next-gen dense retrieval: A compre- hensive empirical assessment. arXiv preprint arXiv:2408.12194 (2024)

    2024 arXiv

  30. [38]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine- Tuning LLaMA for Multi-Stage Text Retrieval. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’24) . Association for Computing Mac...

    2024

  31. [39]

    Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018. Www’18 open challenge: financial opinion mining and question answering. In Companion proceedings of the the web conference 2018. 1941–1942

    2018

  32. [40]

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski S...

    2022 doi arXiv

  33. [41]

    Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022. Large Dual Encoders Are Generalizable Retrievers. InProceedings of the 2022 Conference on Empirical Methods in Natural Language P...

    2022 doi

  34. [42]

    Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxi- ang Dong, Hua Wu, and Haifeng Wang. 2020. RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2010.08191 (2020)

    2020 arXiv

  35. [43]

    Nikhil Sardana, Jacob Portes, Sasha Doubov, and Jonathan Frankle. 2023. Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. arXiv preprint arXiv:2401.00448 (2023)

    2023 arXiv

  36. [44]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test- time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314 (2024)

    2024 arXiv

  37. [45]

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, and Tao Yu. 2022. One embedder, any task: Instruction-finetuned text embeddings. arXiv preprint arXiv:2212.09741 (2022)

    2022 arXiv

  38. [46]

    Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han-yu Wang, Haisu Liu, Quan Shi, Zachary S Siegel, Michael Tang, et al. 2024. Bright: A realistic and challenging benchmark for reasoning-intensive retrieval. arXiv preprint arXiv:2407.12883 (2024)

    2024 arXiv

  39. [47]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models. https://doi.org/10.48550/arXiv.2104.08663 arXiv:2104.08663 [cs]

    2021 doi arXiv

  40. [48]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal

  41. [49]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucu- rull, David Esiobu, Jude Fernandes, Jeremy...

    2023 arXiv

  42. [50]

    Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang

    Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R. Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang. 2021. TREC-COVID: constructing a pandemic information retrieval test collection. SI- GIR Forum 54, 1, Article 1 (Feb. 2021), 12 pages. ht...

    2021 doi

  43. [51]

    David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or Fiction: Verifying Scientific Claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , Bonnie Webber...

    2020 doi

  44. [52]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533 (2022)

    2022 arXiv

  45. [53]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2023. SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval. https://doi.org/10.48550/arXiv.2207. 02578 arXiv:2207.02578 [cs]

    2023 doi arXiv

  46. [54]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024. Text Embeddings by Weakly- Supervised Contrastive Pre-training. https://doi.org/10.48550/arXiv.2212.03533 arXiv:2212.03533 [cs]

    2024 doi arXiv

  47. [55]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023. Improving Text Embeddings with Large Language Models.arXiv preprint arXiv:2401.00368 (2023)

    2023 arXiv

  48. [56]

    Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. 2024. Uniir: Training and benchmarking universal multimodal information retrievers. In European Conference on Computer Vision . Springer, 387–404

    2024

  49. [57]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  50. [58]

    Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. 2024. Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models. https://doi.org/10.48550/arXiv.2409.11136 arXiv:2409.11136 [cs]

    2024 doi arXiv

  51. [59]

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024. Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS’24

    2024

  52. [60]

    Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. 2022. RetroMAE: Pre- Training Retrieval-oriented Language Models Via Masked Auto-Encoder. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Yoav Goldberg, Zornitsa Kozareva, and Yue Zh...

    2022 doi

  53. [61]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-Pack: Packed Resources For General Chinese Embeddings. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, Washi...

    2024

  54. [62]

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. https://doi.org/10. 48550/arXiv.2007.00808 arXiv:2007.00808 [cs]

    2020 doi arXiv

  55. [63]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for di- verse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600 (2018)

    2018 arXiv

  56. [64]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems 36 (2024)

    2024

  57. [65]

    Peitian Zhang, Shitao Xiao, Zheng Liu, Zhicheng Dou, and Jian-Yun Nie

  58. [66]

    Siyun Zhao, Yuqing Yang, Zilong Wang, Zhiyuan He, Luna K Qiu, and Lili Qiu

  59. [67]

    Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. 2024. Dense text retrieval based on pretrained language models: A survey. ACM Transactions on Information Systems 42, 4 (2024), 1–60

    2024

  60. [68]

    Junjie Zhou, Zheng Liu, Shitao Xiao, Bo Zhao, and Yongping Xiong. 2024. VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval. arXiv preprint arXiv:2406.04292 (2024)

    2024 arXiv

  61. [69]

    Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. 2023. Large lan- guage models for information retrieval: A survey. arXiv preprint arXiv:2308.07107 (2023)

    2023

  62. [70]

    arXiv preprint arXiv:2409.14924 (2024)

    Retrieval augmented generation (rag) and beyond: A comprehensive sur- vey on how to make your llms use external data more wisely. arXiv preprint arXiv:2409.14924 (2024)

    2024 arXiv

  63. [74]

    Don’t repeat the query, just give the response

    Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. 2024. Large Language Models for Information Retrieval: A Survey. https://doi.org/10.48550/ arXiv.2308.07107 arXiv:2308.07107 [cs]. Conference ...

    2024 doi

  64. [2018]

    FEVER: a Large-scale Dataset for Fact Extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Marilyn Walker, Heng Ji, and Amanda Ste...

    2018

  65. [2023]

    arXiv preprint arXiv:2310.07554 (2023)

    Retrieve anything to augment large language models. arXiv preprint arXiv:2310.07554 (2023)

    2023

  66. [2024]

    Advances in Neural Information Processing Systems 36 (2024)

    Towards revealing the mystery behind chain of thought: a theoretical perspective. Advances in Neural Information Processing Systems 36 (2024)

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.