Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

In deep research, passage-level lexical retrieval with BM25 plus a small reranker beats much larger neural rankers and approaches GPT-5-class agent accuracy, provided the ranking setup matches the web-search-style queries agents actually is

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 21:01 UTC pith:6Q2J6F5D

load-bearing objection A credible, useful reproduction study of IR components in deep research; the main empirical findings look solid, but the headline 'near GPT-5 parity' claim compares across different protocols and should not stand as written. the 3 major comments →

arxiv 2602.21456 v2 pith:6Q2J6F5D submitted 2026-02-25 cs.IR cs.AIcs.CL

Revisiting Text Ranking in Deep Research

Chuan Meng , Litu Ou , Sean MacAvaney , Jeff Dalton This is my paper
classification cs.IR cs.AIcs.CL
keywords text rankingdeep researchBM25neural retrievalre-rankingquery-to-question reformulationpassage retrievalBrowseComp-Plus
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish how well established text-ranking methods work when the consumer is a deep-research LLM agent, rather than a human searcher. Using a fixed-corpus benchmark with human-verified relevance judgments, it finds that passage-level retrieval units outperform truncated full documents for agents with limited context windows; that BM25, tuned for document length, beats far larger neural retrievers because agents issue keyword-heavy, quote-laden web-search queries; and that a cheap reranker on top of BM25 lifts an open 20B agent to 0.689 answer accuracy, close to the 0.701 reported for a GPT-5-based agent. It also shows that a simple query-to-question reformulation, using the agent's reasoning trace, substantially repairs the training-inference query mismatch that otherwise cripples neural rankers. The significance is that transparent, open, and far cheaper ranking pipelines can nearly match commercial agent systems, and that the ranking component deserves deliberate design in deep research rather than being treated as an opaque oracle.

Core claim

The paper establishes that established text-ranking findings survive the move to deep research, but with three corrections: passage-level retrieval units dominate truncated full documents for agents with limited context; BM25, when paired with length-normalisation-appropriate parameters, outperforms much larger neural retrievers because agents issue web-search-style keyword and exact-phrase queries; and a cheap reranker on top of BM25 lifts an open 20B agent to accuracy close to a GPT-5-class agent. The paper also shows that the training-inference query mismatch is a real and fixable problem: translating agent queries into natural-language questions with reasoning context significantly impro

What carries the argument

The paper constructs a 250-word passage corpus with titles from the fixed BrowseComp-Plus document corpus; evaluates passage retrieval with Max-P mapping, assigning each document the maximum score among its retrieved passages; and proposes Q2Q, a reformulator that turns agent-issued web-search queries into natural-language questions using the agent's own reasoning trace. The load-bearing mechanism is the observation that deep-research agents produce keyword-heavy, quote-laden queries: this determines why BM25 and exact-match-friendly retrievers succeed, why single-vector dense models fail, and why re-ranking depth amplifies gains.

Load-bearing premise

The headline results rest on BrowseComp-Plus's human-verified relevance judgments, the Max-P mapping from passage scores to document-level judgments, and LLM-as-judge answer scoring being faithful, and on the tuned BM25 parameters not being overfit to the test queries, as the paper itself cautions.

What would settle it

Run the same agent pipelines on a second fixed-corpus deep research benchmark where agents are prompted to issue full natural-language questions instead of keyword queries; if BM25 no longer matches or beats SPLADE-v3 and ColBERTv2, and Q2Q reformulation stops improving neural retrieval, the query-mismatch explanation for these results would be refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Passage-level retrieval should become a default design choice for deep research agents, especially those with smaller context windows, because it improves accuracy and completion rates without a full-document reader.
  • Open, reproducible pipelines can approach commercial agent quality: BM25 passage retrieval plus a 3B reranker with a 20B agent reaches 0.689 accuracy on BrowseComp-Plus, close to the 0.701 reported for a GPT-5-based agent.
  • BM25 results on long documents are highly sensitive to length-normalisation settings; prior default parameters substantially understate its performance, so fair comparisons need either tuned parameters or passage-level units.
  • Neural rankers deployed in deep research need to handle web-search-style queries; Q2Q reranking with reasoning context is a cheap and effective mitigation, lifting SPLADE-v3 accuracy by about eight percent.
  • Re-ranking depth and initial retriever strength are coupled: deeper re-ranking improves both answer accuracy and search efficiency, reducing the number of search calls the agent needs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural follow-up the paper leaves implicit is training rankers on mixed query formats, both web-search-style and natural questions, so that no additional reformulation step is needed at inference time.
  • The reasoning-reranker failure may be an artifact of pointwise, keyword-driven scoring; listwise rerankers that see the full surrounding reasoning trace might behave differently and deserve direct testing.
  • Because the findings rest on two agents that were trained to issue web-search-style queries, the BM25 advantage is likely to shrink if future agents are trained to ask natural-language questions; the Q2Q results bracket that counterfactual.
  • The passage-corpus advantage should transfer to other agentic search tasks with long documents and limited context windows, but the 250-word passage size is a free parameter worth tuning on other benchmarks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper empirically revisits established IR text-ranking methods in an open deep-research setting, using the BrowseComp-Plus fixed-corpus benchmark. The authors construct a passage-level corpus, compare passage versus truncated-document retrieval units with two open LLM agents, evaluate BM25, SPLADE-v3, RepLLaMA, Qwen3-Embed-8B, and ColBERTv2 as retrievers and monoT5-3B, RankLLaMA-7B, and Rank1-7B as re-rankers, and propose a Q2Q reformulation to mitigate the mismatch between agent-issued web-search-style queries and the natural-language queries used to train neural rankers. Main reported findings: passage units outperform truncated documents for the shorter-context gpt-oss-20b agent; BM25 is strongest on passages; re-ranking consistently helps; and Q2Q (Q+R) improves neural retrieval and re-ranking. The paper also claims that its best open pipeline (BM25+monoT5 with gpt-oss-20b, 0.689 accuracy) approaches a GPT-5-based agent's 0.701 accuracy.

Significance. If the empirical claims hold, the paper provides useful practical guidance and a reproducible testbed for ranking components in deep research, an area where black-box search APIs have made systematic analysis difficult. Strengths include the open release of code, data, and encrypted agent traces, the use of the dataset's official evaluation code, and broad retriever/reranker coverage. The Q2Q finding is actionable and supported by a paired significance test in the key RQ3 comparisons. However, the headline near-parity with a GPT-5-class agent is not supported by the in-paper evidence, and the absence of repeated runs weakens several small-difference comparisons. With those corrected, this would be a solid contribution.

major comments (3)
  1. [§4.3, Table 8; §4.1, Table 2] The headline that BM25–monoT5 (0.689) is 'comparable to a GPT-5–based agent using Qwen3-Embed-8B (0.701; Table 1 in [6])' is not supported by the evidence in the paper. The 0.701 figure is an external number from [6]'s protocol. The paper's own GPT-5.2 run in Table 2 obtains only 0.451 accuracy on the document corpus, with 354/830 queries hitting the iteration limit and scored 0. No GPT-5-class agent is run under the same passage+reranking protocol used for the 0.689 result. Thus the near-parity claim rests on an uncontrolled cross-protocol comparison. Please either run a GPT-5-class agent under the same protocol or remove/qualify the parity claim.
  2. [§4.2, Figure 1, Table 7] The BM25 hyperparameter analysis tunes k1 and b on all 830 BrowseComp-Plus queries, as the manuscript itself acknowledges: 'the parameter selection was performed directly on all BrowseComp-Plus queries (due to the absence of a validation set).' The tuned configurations are then used in Table 7 and inform the broad conclusion that 'BM25 with appropriate setup outperforms neural rankers in most cases.' This is test-set fitting. Although the passage-corpus BM25 result in Table 3 uses default parameters, the document-corpus comparisons in Table 7 are contaminated. Please use a held-out split for tuning, or clearly label tuned results as exploratory and keep the central comparisons on fixed/default parameters.
  3. [Tables 3, 4, 8, 9, 11] All agent-pipeline numbers appear to come from single stochastic runs. Agent search traces are non-deterministic, and several contrasts are small (e.g., passage vs. document accuracy 0.516 vs. 0.476; Q2Q gains 0.557 vs. 0.516; reranker differences in Table 8). Without repeated runs or confidence intervals, these differences cannot be separated from run-to-run noise. Only Table 11 reports statistical significance. Please provide variance estimates or repeated runs for at least the headline comparisons, or soften the corresponding claims.
minor comments (4)
  1. [§4.1, Table 2] The sentence 'newer vLLM versions (v0.15 and v0.13) lead to higher performance compared to the old one' overstates an observational comparison; other factors may vary across the three studies. Please rephrase as a descriptive observation.
  2. [Figure 1] The red × and green + marks are hard to locate in the heatmaps. Consider adding contour lines or numerical labels for the marked parameter settings.
  3. [§3.2.6] The Max-P recall measure maps retrieved passages to documents, but the agent only reads a small number of retrieved passages. Please clarify whether recall is computed over all retrieved passages or only those actually fed to the agent, and discuss the sensitivity of the passage-corpus conclusions to this choice.
  4. [§4.2] The statement that k1 and b are 'corpus-dependent parameters rather than query-specific ones' would benefit from a direct citation or a brief justification, since it is used to motivate the grid-search analysis.

Circularity Check

0 steps flagged

No significant circularity: the paper is an empirical reproduction study whose claims are measured against an external fixed corpus and evaluation protocol.

full rationale

This paper reports empirical comparisons of retrievers, re-rankers, retrieval units, and query reformulation on the external BrowseComp-Plus benchmark, using the dataset's fixed corpus and the dataset authors' released evaluation code. There is no derivation chain in which a claimed prediction reduces by construction to a fitted input or to a self-citation. The BM25 hyperparameter grid search is performed on the full test set, and the paper explicitly cautions that the tuned results should be compared carefully; this is a disclosed evaluation-protocol limitation rather than a fitted value being renamed as a prediction. The Max-P strategy for mapping passage scores to document-level judgments is a standard, externally specified evaluation choice and does not by construction force the passage-level conclusions; passage-level performance is measured, not derived. The comparison to a GPT-5-based agent's 0.701 accuracy from [6] rests on an external result under a different protocol, and the paper's own reproduced GPT-5.2 run gives 0.451; this is a correctness/interpretation risk about baseline comparability, not a circularity. Self-citations (e.g., [29], [31]) are not load-bearing: the relevant findings are re-tested in the paper's own experiments. No claim in the paper reduces to its own input by definition, so the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The central claims rest on the validity of BrowseComp-Plus judgments, the Max-P passage-evaluation mapping, LLM-as-judge accuracy, and the representativeness of two agents' query behavior. No new entities are postulated. One free hyperparameter pair (BM25 k1,b) is tuned on the full test set.

free parameters (1)
  • BM25 k1 and b for document corpus = k1=10, b=1 (also k1=3.8, b=0.87 tested)
    Chosen via grid search over all 830 BrowseComp-Plus queries with no validation split (§4.2, Figure 1, Table 7); used to show BM25 is competitive on the document corpus, so it is a fitted value that affects the comparison.
axioms (5)
  • domain assumption BrowseComp-Plus relevance judgments (gold and evidence) are valid ground truth for deep research retrieval
    All retrieval metrics are computed against these judgments (§3.2.3, §3.2.6). If judgments are noisy, comparisons across retrievers inherit that noise.
  • domain assumption LLM-as-judge accuracy estimates reflect true answer quality
    Accuracy is computed by an LLM judge following the dataset protocol (§3.2.6); the paper does not validate the judge against human ratings.
  • domain assumption The Max-P passage-to-document scoring faithfully evaluates passage retrieval
    The passage corpus has no passage-level judgment labels; each document is assigned the maximum score of its retrieved passages (§3.2.6). This mapping could favor retrievers that surface many passages from the same document.
  • domain assumption Two open-source agents (gpt-oss-20b and GLM-4.7-Flash) represent deep research agent query behavior
    All agent-issued query distribution conclusions derive from these two models (§3.2.1); other agents may issue different query styles.
  • domain assumption Splitting documents into ≤250-word spaCy passages with titles prepended preserves usable retrieval units
    The passage corpus is constructed this way (§3.2.4); fragmentation could alter lexical scores and passage advantages.

pith-pipeline@v1.3.0-alltime-deepseek · 20756 in / 13001 out tokens · 106342 ms · 2026-08-02T21:01:16.979844+00:00 · methodology

0 comments
read the original abstract

Deep research has emerged as an important task that aims to address hard queries that need extensive open-web exploration. To tackle it, most prior work equips large language model (LLM)-based agents with opaque web search APIs, enabling agents to iteratively issue search queries, retrieve external evidence, and reason over it. Despite search's essential role in deep research, black-box web search APIs leave the behaviour of established text ranking methods in deep research largely unclear. To fill this gap, we reproduce key findings and best practices for text ranking methods in deep research. We examine their effectiveness from three perspectives: (i) retrieval units (documents vs. passages), (ii) pipeline configurations (different retrievers, re-rankers, and re-ranking depths), and (iii) query characteristics (the mismatch between agent-issued queries and the training queries of text rankers). We perform experiments on BrowseComp-Plus, a deep research dataset with a fixed corpus, evaluating 2 open-source agents, 5 retrievers, and 3 re-rankers. We find that agent-issued queries typically follow web-search-style syntax (e.g., quoted exact matches), favouring lexical, learned sparse, and multi-vector retrievers; passage-level units are more efficient under limited context windows, and avoid the difficulties of document length normalisation in lexical retrieval; re-ranking is highly effective. We further propose a query-to-question (Q2Q) method that translates agent-issued queries into natural language questions, significantly reducing the query mismatch.

Figures

Figures reproduced from arXiv: 2602.21456 by Chuan Meng, Jeff Dalton, Litu Ou, Sean MacAvaney.

Figure 1
Figure 1. Figure 1: Heatmap from a grid search on BrowseComp-Plus using the original full queries (not end-to-end), showing the [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Towards Retrieving Interaction Spaces for Agentic Search

    cs.IR 2026-06 unverdicted novelty 7.0

    RISE uses BM25 to bound interaction spaces for agentic search and pre-processes documents for shell navigation, matching direct corpus interaction accuracy at roughly one-quarter the cost on BrowseComp-Plus.

  2. UEmbed: Unified Sparse and Dense Multimodal Embeddings

    cs.CV 2026-08 conditional novelty 6.0

    UEmbed uses 16 special tokens over a partitioned vocabulary to make a decoder-only multimodal model emit dense and sparse embeddings in one forward pass; the 9B model scores 71.8 dense / 71.0 sparse on MMEB-v2.

Reference graph

Works this paper leans on

69 extracted references · 22 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al . 2025. gpt-oss-120b & gpt-oss-20b Model Card.arXiv preprint arXiv:2508.10925(2025)

    Pith/arXiv arXiv 2025

  2. [2]

    Mohammad Kalim Akram, Saba Sturua, Nastia Havriushenko, Quentin Herreros, Michael Günther, Maximilian Werk, and Han Xiao. 2026. jina-embeddings-v5- text: Task-Targeted Embedding Distillation.arXiv preprint arXiv:2602.15547 (2026)

    Pith/arXiv arXiv 2026

  3. [3]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. InICLR

    2024

  4. [4]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset.arXiv preprint arXiv:1611.09268(2018)

    Pith/arXiv arXiv 2018

  5. [5]

    Antoine Chaffin and Raphaël Sourty. 2025. PyLate: Flexible Training and Retrieval for Late Interaction Models. InCIKM. 6334–6339

    2025

  6. [6]

    Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, et al. 2025. BrowseComp- Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent.arXiv preprint arXiv:2508.06600(2025)

    Pith/arXiv arXiv 2025

  7. [7]

    Voorhees

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2019. Overview of the TREC 2019 Deep Learning Track. InREC 2019

    2019

  8. [8]

    Zhuyun Dai and Jamie Callan. 2019. Deeper Text Understanding for IR with Contextual Neural Language Modeling. InSIGIR. 985–988

    2019

  9. [9]

    Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. 2021. TREC CAsT 2021: The Conversational Assistance Track Overview. InTREC

    2021

  10. [10]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InNAACL. 4171–4186

    2019

  11. [11]

    Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant

  12. [12]

    Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. InSIGIR. 2288–2292

    2021

  13. [13]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.arXiv preprint arXiv:2501.12948(2025)

    Pith/arXiv arXiv 2025

  14. [14]

    Ben He and Iadh Ounis. 2005. Term Frequency Normalisation Tuning for BM25 and DFR Models. InECIR. 200–214

    2005

  15. [15]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. InCOLING. 6609–6625

    2020

  16. [16]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2021. LoRA: Low-Rank Adaptation of Large Language Models. InICLR

    2021

  17. [17]

    Tiansheng Hu, Yilun Zhao, Canyu Zhang, Arman Cohan, and Chen Zhao. 2026. SAGE: Benchmarking and Improving Retrieval for Deep Research Agents.arXiv preprint arXiv:2602.05975(2026)

    arXiv 2026

  18. [18]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. InCOLM

    2025

  19. [19]

    Marcin Kaszkiel and Justin Zobel. 1997. Passage Retrieval Revisited. InACM SIGIR Forum, Vol. 31. ACM New York, NY, USA, 178–185

    1997

  20. [20]

    Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. InSIGIR. 39–48

    2020

  21. [21]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural Questions: A Benchmark for Question Answering Research. TACL7 (2019), 453–466

    2019

  22. [22]

    Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant. 2024. SPLADE-v3: New baselines for SPLADE.arXiv preprint arXiv:2403.06789(2024)

    Pith/arXiv arXiv 2024

  23. [23]

    Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, et al. 2025. WebSailor: Nav- igating Super-human Reasoning for Web Agent.arXiv preprint arXiv:2507.02592 (2025)

    Pith/arXiv arXiv 2025

  24. [24]

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic Search-Enhanced Large Reasoning Models.arXiv preprint arXiv:2501.05366(2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards General Text Embeddings with Multi-stage Contrastive Learning.ArXiv(2023)

    2023

  26. [26]

    2022.Pretrained transformers for text ranking: Bert and beyond

    Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. 2022.Pretrained transformers for text ranking: Bert and beyond. Springer Nature

    2022

  27. [27]

    Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, Aili Chen, Ke Ji, Weiyu Cheng, Zijia Wu, Chengyu Du, Qidi Xu, et al. 2025. WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents.arXiv preprint arXiv:2509.06501(2025)

    arXiv 2025

  28. [28]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine- Tuning LLaMA for Multi-Stage Text Retrieval. InSIGIR. 2421–2425

    2024

  29. [29]

    Chuan Meng, Negar Arabzadeh, Arian Askari, Mohammad Aliannejadi, and Maarten de Rijke. 2024. Ranked List Truncation for Large Language Model-based Re-Ranking. InSIGIR. 141–151

    2024

  30. [30]

    Chuan Meng, Jiqun Liu, Mohammad Aliannejadi, Fengran Mo, Jeff Dalton, and Maarten de Rijke. 2026. Re-Rankers as Relevance Judges.arXiv preprint arXiv:2601.04455(2026)

    arXiv 2026

  31. [31]

    Chuan Meng, Francesco Tonolini, Fengran Mo, Nikolaos Aletras, Emine Yilmaz, and Gabriella Kazai. 2025. Bridging the Gap: From Ad-hoc to Proactive Search in Conversations. InSIGIR. 64–74

    2025

  32. [32]

    Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, et al. 2025. UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations. InACL. 6936–6949

    2025

  33. [33]

    Ariane Mueller and Craig Macdonald. 2025. Semantically Proportioned nDCG for Explaining ColBERT’s Learning Process. InECIR. 341–356

    2025

  34. [34]

    Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage Re-ranking with BERT. arXiv preprint arXiv:1901.04085(2019)

    Pith/arXiv arXiv 2019

  35. [35]

    Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Docu- ment Ranking with a Pretrained Sequence-to-Sequence Model. InEMNLP. 708– 718

    2020

  36. [36]

    Trippas, and Svitlana Vakulenko

    Paul Owoicho, Jeffrey Dalton, Mohammad Aliannejadi, Leif Azzopardi, Johanne R. Trippas, and Svitlana Vakulenko. 2022. TREC CAsT 2022: Going Beyond User Ask and System Retrieve with Initiative and Response Generation. InTREC

    2022

  37. [37]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.Journal of Machine Learning Research21, 140 (2020), 1–67

    2020

  38. [38]

    Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995. Okapi at TREC-3.Nist Special Publication Sp109 (1995), 109

    1995

  39. [39]

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. InNAACL. 3715–3734

    2022

  40. [40]

    Rulin Shao, Rui Qiao, Varsha Kishore, Niklas Muennighoff, Xi Victoria Lin, Daniela Rus, Bryan Kian Hsiang Low, Sewon Min, Wen-tau Yih, Pang Wei Koh, et al. 2025. ReasonIR: Training Retrievers for Reasoning Tasks.arXiv preprint arXiv:2504.20595(2025)

    Pith/arXiv arXiv 2025

  41. [41]

    Sahel Sharifymoghaddam and Jimmy Lin. 2026. Rerank Before You Reason: Analyzing Reranking Tradeoffs through Effective Token Cost in Deep Search Agents.arXiv preprint arXiv:2601.14224(2026)

    Pith/arXiv arXiv 2026

  42. [42]

    Zhengliang Shi, Yiqun Chen, Haitao Li, Weiwei Sun, Shiyu Ni, Yougang Lyu, Run-Ze Fan, Bowen Jin, Yixuan Weng, Minjun Zhu, et al. 2025. Deep Research: A Systematic Survey.arXiv preprint arXiv:2512.02038(2025)

    arXiv 2025

  43. [43]

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning.arXiv preprint arXiv:2503.05592 (2025)

    Pith/arXiv arXiv 2025

  44. [44]

    Hongjin SU, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu. 2025. BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval. InICLR

    2025

  45. [45]

    Weihang Su, Yichen Tang, Qingyao Ai, Junxi Yan, Changyue Wang, Hongning Wang, Ziyi Ye, Yujia Zhou, and Yiqun Liu. 2025. Parametric Retrieval Augmented Generation. InSIGIR. 1240–1250

    2025

  46. [46]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)

    2021

  47. [47]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models.arXiv preprint arXiv:2307.09288(2023)

    Pith/arXiv arXiv 2023

  48. [48]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal

  49. [49]

    Yibo Wang, Lei Wang, Yue Deng, Keming Wu, Yao Xiao, Huanjin Yao, Liwei Kang, Hai Ye, Yongcheng Jing, and Lidong Bing. 2026. DeepResearchEval: An Auto- mated Framework for Deep Research Task Construction and Agentic Evaluation. arXiv preprint arXiv:2601.09688(2026)

    arXiv 2026

  50. [50]

    Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese

  51. [51]

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al

  52. [52]

    Orion Weller, Michael Boratko, Iftekhar Naim, and Jinhyuk Lee. 2025. On the Theoretical Limitations of Embedding-Based Retrieval.arXiv preprint arXiv:2508.21038(2025)

    arXiv 2025

  53. [53]

    Orion Weller, Kathryn Ricci, Eugene Yang, Andrew Yates, Dawn Lawrie, and Ben- jamin Van Durme. 2025. Rank1: Test-Time Compute for Reranking in Information Retrieval. InCOLM

    2025

  54. [54]

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. InICLR

    2021

  55. [55]

    Emergent Abilities of Large Language Models.Transactions on Machine Learning Research(2022)

    2022

  56. [56]

    Yilong Xu, Zhi Zheng, Xiang Long, Yujun Cai, and Yiwei Wang. 2026. Self- Manager: Parallel Agent Loop for Long-form Deep Research.arXiv preprint arXiv:2601.17879(2026)

    arXiv 2026

  57. [57]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388(2025)

    Pith/arXiv arXiv 2025

  58. [58]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2. 5 Technical Report.arXiv preprint arXiv:2412.15115(2024)

    Pith/arXiv arXiv 2024

  59. [59]

    Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I Hsu, Jun Yan, Vishy Tirumalashetty, Eunsol Choi, Tomas Pfister, Chen-Yu Lee, et al. 2026. SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback. arXiv preprint arXiv:2601.18202(2026)

    arXiv 2026

  60. [60]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. InEMNLP. 2369–2380

    2018

  61. [61]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. ReAct: Synergizing Reasoning and Acting in Language Models. InThe eleventh international conference on learning representations

    2022

  62. [62]

    Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. 2025. GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models.arXiv preprint arXiv:2508.06471 (2025)

    Pith/arXiv arXiv 2025

  63. [63]

    Eugene Yang, Andrew Yates, Kathryn Ricci, Orion Weller, Vivek Chari, Benjamin Van Durme, and Dawn Lawrie. 2025. Rank-K: Test-Time Reasoning for Listwise Reranking.arXiv preprint arXiv:2505.14432(2025)

    Pith/arXiv arXiv 2025

  64. [64]

    Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al . 2025. BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese. arXiv preprint arXiv:2504.19314(2025)

    Pith/arXiv arXiv 2025

  65. [65]

    Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon, and Daxin Jiang. 2022. Bridging the Gap Between Indexing and Re- trieval for Differentiable Search Index with Query Generation.arXiv preprint arXiv:2206.10128(2022)

    Pith/arXiv arXiv 2022

  66. [67]

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al . 2025. Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models.arXiv preprint arXiv:2506.05176(2025)

    Pith/arXiv arXiv 2025

  67. [2021]

    arXiv preprint arXiv:2109.10086(2021)

    SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval. arXiv preprint arXiv:2109.10086(2021)

    Pith/arXiv arXiv 2021

  68. [2022]

    Revisiting Text Ranking in Deep Research

    MuSiQue: Multihop Questions via Single-hop Question Composition.TACL 10 (2022), 539–554. Revisiting Text Ranking in Deep Research

    2022

  69. [2025]

    arXiv preprint arXiv:2504.12516(2025)

    BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. arXiv preprint arXiv:2504.12516(2025)

    Pith/arXiv arXiv 2025