Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

TongSearch-QR: Reinforced Query Reasoning for Retrieval

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that small language models (7B and 1.5B) trained with reinforcement learning on a similarity-improvement reward can rewrite queries for reasoning-intensive retrieval as well as or better than much larger models such as…

desk verdict A practical small-model query reasoner with a plausible but unproven headline result; the reward proxy and possible data overlap need checking before the 1.4-point GPT-4o gap is taken at face value. read the letter →

arxiv 2506.11603 v2 pith:DLVOFUUK submitted 2025-06-13 cs.IR

Xubo Qin , Jun Bai , Jiaqi Li , Zixia Jia , Zilong Zheng This is my paper
classification cs.IR
keywords queryreasoningrewritingreinforcementlearningGRPOreasoning-intensiveretrievalsmalllanguagemodelsBRIGHTbenchmarksemi-rule-basedreward
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that query reasoning for retrieval need not be done by expensive frontier LLMs: a small model can be trained to rewrite hard queries into reasoned ones that a plain BM25 retriever then uses successfully. The claim is that with a semi-rule-based reward measuring how much a rewritten query increases embedding similarity to known relevant documents, group-relative policy optimization turns Qwen2.5-7B-Instruct and Qwen2.5-1.5B-Instruct into query reasoners that rival GPT-4o and other large models on the BRIGHT benchmark. If true, it matters because query rewriting has been a bottleneck for real retrieval-augmented systems, which often cannot afford GPT-4-class models or expose data to external APIs. The paper further claims that the trained models combine with reasoning-intensive retrievers to improve performance beyond the large-model baselines, and that explicit chain-of-thought output is unnecessary.

What carries the argument

The central mechanism is the semi-rule-based reward $R(q,q') = (\mathrm{score}_{q'} - \mathrm{score}_q)/|D^+|$, which measures the average per-positive-document gain in embedding cosine similarity achieved by rewriting the query. It is called semi-rule-based because the relevance scores come from a fixed pretrained embedding model rather than from retrieval metrics, retrieval collections, or human labels. In GRPO training, the reward is group-normalized into advantages, so the policy is pushed toward rewrites that increase similarity to all known relevant documents. This design lets the model train on noisy user-selected StackExchange answers without per-token supervision, and it avoids the need for explicit think chains: the reasoned query itself is the output.

What would settle it

Correlate the reward $R(q,q')$ computed by bge-base-en-v1.5 with the actual change in BM25 nDCG@10 on a sample of BRIGHT queries after rewriting. If the correlation is at or below zero across subtasks, the training signal is not aligned with the retrieval metric and the central comparison loses its foundation. A second check is to verify that none of the StackExchange training questions overlap with BRIGHT's 1,384 evaluation queries.

Watch

Extended reading notes

Core claim

TongSearch-QR trains small language models to rewrite a query $q$ into a reasoned query $q'$ that a retriever then uses. The training signal is the semi-rule-based reward $R(q,q') = (\mathrm{score}_{q'} - \mathrm{score}_q)/|D^+|$, where each score is the sum of cosine similarities between the query and the known positive documents $D^+$, computed by a frozen bge-base-en-v1.5 embedding model. With GRPO on top of Qwen2.5-7B-Instruct and Qwen2.5-1.5B-Instruct, the resulting TongSearch-QR-7B reaches 27.9 nDCG@10 on the BRIGHT benchmark under a BM25 retriever, above GPT-4o's 26.5, while TongSearch-QR-1.5B reaches 24.6. Combined with the reasoning-intensive retriever ReasonIR, TongSearch-QR-7B reaches 31.9 nDCG@10, above the 29.9 achieved by GPT-4 reasoned queries. The paper also claims roughly 100 times lower API cost than GPT-4o for the 7B model.

Load-bearing premise

The paper assumes that increasing a frozen embedding model's cosine similarity between the rewritten query and the known relevant documents reliably improves BM25 nDCG@10, yet it reports no correlation analysis linking the reward to the retrieval metric.

Editorial extensions

If this is right

  • With BM25, TongSearch-QR-7B outperforms prompt-based GPT-4o query reasoning on BRIGHT (27.9 vs 26.5 nDCG@10), so small local models can substitute for large external reasoning models in retrieval pipelines.
  • The 1.5B variant reaches 24.6 nDCG@10, close to several large baselines, which makes query reasoning feasible on resource-constrained or privacy-sensitive deployments.
  • Combining TongSearch-QR with the reasoning-intensive retriever ReasonIR raises nDCG@10 to 31.9 for the 7B model, beating GPT-4 reasoned queries at 29.9 and showing the two research directions are complementary.
  • At the API prices reported in the paper, TongSearch-QR-7B costs about 100 times less per output token than GPT-4o, making its performance-per-dollar efficiency roughly two orders of magnitude higher.
  • The explicit thinking format is not needed: models trained to output only the reasoned query match those trained with think chains, which simplifies deployment and reduces token cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested corollary of the reward design is that the same pipeline could train query reasoners for other retrieval backends, provided the backend's ranking correlates with the embedding model used in the reward; swapping bge-base for a sparse model already changed results in the paper.
  • The paper treats user-selected StackExchange answers as free supervision (V2) and finds they beat distilled answers from frontier models, which suggests a scalable recipe for building query reasoners from existing community Q&A archives that the paper only demonstrates for English Stack Exchange.
  • The paper does not report whether its training data overlaps the BRIGHT evaluation queries, so part of the claimed transfer may in principle be memorization rather than reasoning; checking overlap would settle that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces TongSearch-QR, a family of small language models (7B and 1.5B) trained with GRPO to perform query reasoning and rewriting for reasoning-intensive retrieval. The training signal is a semi-rule-based reward function that measures the average increase in cosine similarity between a rewritten query and gold documents, computed with a frozen bge-base-en-v1.5 embedding model. Training data are curated from the H4 Stack Exchange Preferences dataset, using either answers generated by DeepSeek-R1 or QwQ-32B or user-selected answers. The models are evaluated on the BRIGHT benchmark with BM25 as the retriever and with the ReasonIR retriever, reporting nDCG@10. The headline result is that TongSearch-QR-7B reaches 27.9 nDCG@10 with BM25, outperforming GPT-4o's 26.5, while also claiming much lower inference cost. Ablation studies examine data source, reward model, explicit thinking, and RL versus SFT.

Significance. If the empirical claims hold, the paper offers a practical recipe for training compact query reasoners without human labels, a cheap reward that avoids reward hacking, and an automatic data-curation pipeline that can be extended to other domains. The code and training details are released, and the ablations (Tables 3 and 4) are informative, especially the demonstration that RL tolerates noisy user-selected answers far better than SFT. However, the central 'significantly outperform GPT-4o' claim rests on three unverified assumptions: that the embedding-similarity reward correlates with BM25 nDCG@10, that the training data do not overlap the BRIGHT evaluation corpus, and that single-run score differences are meaningful. These assumptions are checkable and should be addressed before the results can be considered robust.

major comments (4)
  1. [Abstract and Table 1] The abstract and Section 4.2 claim that TongSearch-QR models 'significantly outperform' baselines, but Table 1 reports a single deterministic nDCG@10 value per model with no variance, no repeated runs, and no significance testing. On a 1,384-query benchmark, a 1.4-point gap (27.9 vs 26.5 in Table 1) may well be within run-to-run noise or query-level variance. Please report standard deviations across at least three independent runs, bootstrap confidence intervals, or a paired significance test (e.g., Wilcoxon signed-rank test across queries) to support the word 'significantly'.
  2. [Section 3.2 and Table 1] The reward function R(q,q') = (score_q' - score_q)/|D+| is computed as the average change in cosine similarity to gold documents under a frozen bge-base-en-v1.5 model, while the headline evaluation is BM25 nDCG@10. The paper provides no evidence that this embedding-similarity reward is a reliable proxy for lexical retrieval quality. Please report the correlation (e.g., Spearman's rho) between the reward and BM25 nDCG@10 on a held-out validation set, or include a variant where the reward is computed with BM25 itself, to validate the training-to-evaluation transfer. Without this, the training signal and the reported metric could be optimizing different objectives.
  3. [Section 3.3 and Appendix B] The training data are sampled from the H4 Stack Exchange Preferences dataset with categories listed in Appendix B (including 'biology', 'earthscience', 'economics', 'robotics', 'stackoverflow', 'sustainability', etc.), whereas BRIGHT evaluation subtasks in Table 1 include StackOverflow, Biology, Earth Science, Economics, Psychology, Robotics, and Sustainability. Because both draw from StackExchange communities, there is a clear risk of overlap between training questions and BRIGHT queries or documents. The paper does not report any decontamination or overlap analysis. Please provide n-gram overlap statistics between the training set and BRIGHT queries/corpora, and report results after removing overlapping items or on a version of BRIGHT that excludes StackExchange-derived subtasks. If overlap exists and is not handled, the reported gains could be inflated by lexical memorization.
  4. [Table 2 and Section 4.2] The cost-effectiveness claim relies on cost values that are not well-defined. The paper states that TongSearch-QR-7B is priced as Qwen2.5-7B-Instruct on OpenRouter and that TongSearch-QR-1.5B is assigned a cost of 0.01 because the base model is free. These are arbitrary assumptions, and the efficiency scores (279.0 and 2460.0) would change by orders of magnitude under a self-hosted compute-cost model or under different API pricing. Please report actual deployment costs (e.g., GPU-hours or dollar cost per 1,000 queries) or provide a sensitivity analysis over a realistic range of price assumptions. As written, the efficiency comparison in Table 2 is not a robust basis for the paper's cost advantage claim.
minor comments (6)
  1. [Abstract] In the abstract, 'e,g,' should be 'e.g.,'.
  2. [Section 3.2] The GRPO objective expression is notationally unclear: the expectation is written over (q,a) ∼ π_θ, but the generated variable is q' and the symbol 'a' is never defined. Please clarify the sampling distribution and the role of the weight w_g.
  3. [Table 1] The Seed1.5-Embedding row is taken from an external model card; please mark all externally sourced numbers explicitly so that readers know which numbers were computed by the authors.
  4. [Section 4.2] There is a typo: 'incidate' should be 'indicate'.
  5. [Section 4.3.2] There is a stray hyphen before 'While' in the sentence beginning 'These results indicate that...'.
  6. [Figure 2] The x-axis label 'Cost' should specify the unit and whether it is per 1M output tokens; the y-axis label should be spelled out as 'BRIGHT nDCG@10' in the caption for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RL reward is a frozen-embedding surrogate and the evaluation is an external BM25/nDCG benchmark, so the central claim does not reduce to its training objective.

full rationale

The paper's derivation chain is an empirical training-and-evaluation pipeline rather than a closed definitional loop. The reward function R(q,q') = (score_q' - score_q)/|D+| in Section 3.2 is computed from cosine similarities under a fixed pretrained bge-base-en-v1.5 embedding model against training positives from the H4 StackExchange Preferences dataset. The headline result, however, is nDCG@10 on the external BRIGHT benchmark using BM25 as the retriever. The training objective is therefore not the same object as the evaluation metric: maximizing dense embedding similarity to training answers is not, by construction, equivalent to maximizing lexical BM25 nDCG@10 on BRIGHT relevance judgments. No parameter is fitted to BRIGHT before evaluation, and no BRIGHT score is used as a training signal. The paper also does not rely on load-bearing self-citations; its citations to BRIGHT, DeepSeek-R1, QwQ-32B, and ReasonIR are external prior work. The possible StackExchange domain overlap between training data and the BRIGHT benchmark is a data-contamination and correctness risk, not a circularity of the derivation chain, since the paper's equations do not make the evaluation outcome equivalent to its inputs.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central result rests on a small set of domain assumptions rather than free parameters. The only notable hand-chosen number is the arbitrary cost of 0.01 USD/M for the 1.5B model used in the headline efficiency score. The main axioms are that embedding similarity aligns with BM25 retrieval quality and that training data from StackExchange does not leak into the BRIGHT benchmark.

free parameters (2)
  • TongSearch-QR-1.5B cost for efficiency metric = 0.01 USD/M tokens
    Arbitrarily assigned because Qwen2.5-1.5B API is free; used to compute Eff=2460.0, the headline cost-effectiveness number.
  • Sampling cap per category for training data = 1200 (V1) / 1500 (V2)
    Chosen by hand; affects the data mix and could influence which subtasks improve most.
assumptions (3)
  • domain assumption Cosine similarity measured by bge-base-en-v1.5 is a reliable proxy for retrieval relevance measured by BM25 nDCG@10
    The reward R(q,q') is the average embedding-similarity increment, while evaluation is lexical BM25; no correlation analysis is provided in Section 3.2.
  • domain assumption No meaningful overlap exists between the StackExchange training data and the BRIGHT evaluation queries
    Training samples up to 1500 questions per StackExchange category; BRIGHT includes a StackExchange subtask, and no decontamination step is reported in Section 3.3 or Appendix B.
  • domain assumption GRPO with a frozen-embedding reward generalizes to out-of-distribution BRIGHT subtasks
    The model is trained on Q&A pairs and evaluated on 12 diverse subtasks; the transfer is asserted from Table 1 without additional analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TongSearch-QR: Reinforced Query Reasoning for Retrieval." pith.science (2026). https://pith.science/paper/DLVOFUUK

@misc{pith2026250611603,
  author       = {Pith},
  title        = {Pith review of: TongSearch-QR: Reinforced Query Reasoning for Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DLVOFUUK}},
  note         = {Machine review of arXiv:2506.11603}
}
read the original abstract

Traditional information retrieval (IR) methods excel at textual and semantic matching but struggle in reasoning-intensive retrieval tasks that require multi-hop inference or complex semantic understanding between queries and documents. One promising solution is to explicitly rewrite or augment queries using large language models (LLMs) to elicit reasoning-relevant content prior to retrieval. However, the widespread use of large-scale language models like GPT-4 or LLaMA3-70B remains impractical due to their high inference cost and limited deployability in real-world systems. In this work, we introduce TongSearch QR (Previously Known as "TongSearch Reasoner"), a family of small-scale language models for query reasoning and rewriting in reasoning-intensive retrieval. With a novel semi-rule-based reward function, we employ reinforcement learning approaches enabling smaller language models, e,g, Qwen2.5-7B-Instruct and Qwen2.5-1.5B-Instruct, to achieve query reasoning performance rivaling large-scale language models without their prohibitive inference costs. Experiment results on BRIGHT benchmark show that with BM25 as retrievers, both TongSearch QR-7B and TongSearch QR-1.5B models significantly outperform existing baselines, including prompt-based query reasoners and some latest dense retrievers trained for reasoning-intensive retrieval tasks, offering superior adaptability for real-world deployment.

Figures

Figures reproduced from arXiv: 2506.11603 by the authors.

Figure 1
Figure 1. An example query reasoning with LLM. The query is sampled from the Biology Subtask of BRIGHT (Su et al., 2024) benchmark. This highlights a fundamental challenge in information retrieval for complex queries: the need to bridge a reasoning gap between the user’s implicit intent and the relevant knowledge. An effective retrieval system must go beyond matching surface-level expressions—it needs to infer what the user i… view at source ↗
Figure 2
Figure 2. Cost vs. Performance comparison of different models. Details about the cost and performance can be found in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. shows the prompt template for the instructions of chain-of-thought query reasoning. The reasoner model takes the instructions and original query as input, and return a “pseudo-answer” with thoughts including as much relevant information as possible. The “pseudo-answer” can be used as the reasoned query, and the retriever can benefit from the external information provided by the reasoned query. Instructions: 1. Ident… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Antares: Foundation Models for Agentic Vulnerability Localization

    cs.CR 2026-08 conditional novelty 6.0 of 10

    Antares-3B, a 3B model trained with SFT plus GRPO, matches GPT-5.5 on repository-scale vulnerability localization at roughly 1/100th the inference cost.

  2. RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    RICE-PO trains retrieval agents using retrieval scores of intermediate summaries as local rewards, gated by influence and residual-stability estimates, outperforming group-based RL baselines on BRIGHT and BEIR.

  3. Query-focused and Memory-aware Reranker for Long Context Processing

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Training 16 pre-selected query-focused attention heads with a listwise contrastive loss produces a 4B reranker that beats larger pointwise and listwise baselines and complex memory systems on long-context QA and dialo...

Reference graph

Works this paper leans on

42 extracted references · 7 canonical work pages · cited by 3 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Ms marco: A human generated machine reading comprehension dataset, 2018

    Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., Rosenberg, M., Song, X., Stoica, A., Tiwary, S., and Wang, T. Ms marco: A human generated machine reading comprehension dataset, 2018. URL https://arxiv.org/abs/1611.09268

    arXiv 2018

  3. [3]

    BGE M3 - Embedding : Multi - Lingual , Multi - Functionality , Multi - Granularity Text Embeddings Through Self - Knowledge Distillation , June 2024 a

    Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. BGE M3 - Embedding : Multi - Lingual , Multi - Functionality , Multi - Granularity Text Embeddings Through Self - Knowledge Distillation , June 2024 a . URL http://arxiv.org/abs/2402.03216. arXiv:2402.03216 [cs]

    arXiv 2024

  4. [4]

    Q., Hanin, B., Bailis, P., Stoica, I., Zaharia, M., and Zou, J

    Chen, L., Davis, J. Q., Hanin, B., Bailis, P., Stoica, I., Zaharia, M., and Zou, J. Are more llm calls all you need? towards scaling laws of compound inference systems, 2024 b . URL https://arxiv.org/abs/2403.02419

    arXiv 2024

  5. [5]

    Datta, R., Joshi, D., Li, J., and Wang, J. Z. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv., 40 0 (2), May 2008. ISSN 0360-0300. doi:10.1145/1348246.1348248. URL https://doi.org/10.1145/1348246.1348248

    work page arXiv 2008

  6. [6]

    DeepSeek-AI, Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., Dai, D., Guo, D., Yang, D., Chen, D., Ji, D., Li, E., Lin, F., Dai, F., Luo, F., Hao, G., Chen, G., Li, G., Zhang, H., Bao, H., Xu, H., Wang, H., Zhang, H., Ding, H., Xin, H., Gao, H., Li, H., Qu, H., Cai, J. L., Liang, J., Guo, J., Ni, J., Li, J., ...

    arXiv 2025

  7. [7]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Devlin, J. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

  8. [8]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Face, H. Open r1: A fully open reproduction of deepseek-r1, January 2025. URL https://github.com/huggingface/open-r1

    2025

Show all 42 references
  1. [9]

    Precise zero-shot dense retrieval without relevance labels

    Gao, L., Ma, X., Lin, J., and Callan, J. Precise zero-shot dense retrieval without relevance labels. arXiv preprint arXiv:2212.10496, 2022

    2022 arXiv

  2. [10]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  3. [11]

    Query Expansion by Prompting Large Language Models , May 2023

    Jagerman, R., Zhuang, H., Qin, Z., Wang, X., and Bendersky, M. Query Expansion by Prompting Large Language Models , May 2023. URL http://arxiv.org/abs/2305.03653. arXiv:2305.03653 [cs]

    2023 arXiv

  4. [12]

    Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning, 2025

    Jiang, P., Lin, J., Cao, L., Tian, R., Kang, S., Wang, Z., Sun, J., and Han, J. Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning, 2025. URL https://arxiv.org/abs/2503.00223

    2025 arXiv

  5. [13]

    Dense passage retrieval for open-domain question answering, 2020

    Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and tau Yih, W. Dense passage retrieval for open-domain question answering, 2020. URL https://arxiv.org/abs/2004.04906

    2020 arXiv

  6. [14]

    and Zaharia, M

    Khattab, O. and Zaharia, M. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp.\ 39--48, 2020

    2020

  7. [15]

    H., Gonzalez, J

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention, 2023. URL https://arxiv.org/abs/2309.06180

    2023 arXiv

  8. [16]

    Huggingface h4 stack exchange preference dataset, 2023

    Lambert, N., Tunstall, L., Rajani, N., and Thrush, T. Huggingface h4 stack exchange preference dataset, 2023. URL https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences

    2023

  9. [17]

    Nv-embed: Improved techniques for training llms as generalist embedding models

    Lee, C., Roy, R., Xu, M., Raiman, J., Shoeybi, M., Catanzaro, B., and Ping, W. Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428, 2024

    2024 arXiv

  10. [18]

    Towards general text embeddings with multi-stage contrastive learning

    Li, Z., Zhang, X., Zhang, Y., Long, D., Xie, P., and Zhang, M. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023

    2023 arXiv

  11. [19]

    Roberta: A robustly optimized bert pretraining approach

    Liu, Y. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019

    1907 arXiv

  12. [20]

    Pre-trained language model for web-scale retrieval in baidu search, 2021

    Liu, Y., Huang, G., Liu, J., Lu, W., Cheng, S., Li, Y., Shi, D., Wang, S., Cheng, Z., and Yin, D. Pre-trained language model for web-scale retrieval in baidu search, 2021. URL https://arxiv.org/abs/2106.03373

    2021 arXiv

  13. [21]

    Large language models as foundations for next-gen dense retrieval: A comprehensive empirical assessment

    Luo, K., Qin, M., Liu, Z., Xiao, S., Zhao, J., and Liu, K. Large language models as foundations for next-gen dense retrieval: A comprehensive empirical assessment. arXiv preprint arXiv:2408.12194, 2024

    2024 arXiv

  14. [22]

    Fine- Tuning LLaMA for Multi - Stage Text Retrieval

    Ma, X., Wang, L., Yang, N., Wei, F., and Lin, J. Fine- Tuning LLaMA for Multi - Stage Text Retrieval . In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR '24, pp.\ 2421--2425, New York, NY, USA, July 2024....

    2024

  15. [23]

    Judgerank: Leveraging large language models for reasoning-intensive reranking, 2024

    Niu, T., Joty, S., Liu, Y., Xiong, C., Zhou, Y., and Yavuz, S. Judgerank: Leveraging large language models for reasoning-intensive reranking, 2024. URL https://arxiv.org/abs/2411.00142

    2024 arXiv

  16. [24]

    Gpt-4o system card, 2024

    OpenAI. Gpt-4o system card, 2024. URL https://arxiv.org/abs/2410.21276

    2024 arXiv

  17. [25]

    Qwen2.5 technical report, 2025

    Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin,...

    2025 arXiv

  18. [26]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters

    Rasley, J., Rajbhandari, S., Ruwase, O., and He, Y. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '20, pp.\ 3505–35...

    2020

  19. [27]

    and Zaragoza, H

    Robertson, S. and Zaragoza, H. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3 0 (4): 0 333–389, April 2009. ISSN 1554-0669. doi:10.1561/1500000019. URL https://doi.org/10.1561/1500000019

    2009 doi

  20. [28]

    V., Rus, D., Low, B

    Shao, R., Qiao, R., Kishore, V., Muennighoff, N., Lin, X. V., Rus, D., Low, B. K. H., Min, S., tau Yih, W., Koh, P. W., and Zettlemoyer, L. Reasonir: Training retrievers for reasoning tasks, 2025. URL https://arxiv.org/abs/2504.20595

    2025 arXiv

  21. [29]

    K., Wu, Y., and Guo, D

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300

    2024 arXiv

  22. [30]

    S., Tang, M., et al

    Su, H., Yen, H., Xia, M., Shi, W., Muennighoff, N., Wang, H.-y., Liu, H., Shi, Q., Siegel, Z. S., Tang, M., et al. Bright: A realistic and challenging benchmark for reasoning-intensive retrieval. arXiv preprint arXiv:2407.12883, 2024

    2024 arXiv

  23. [31]

    Lref: A novel llm-based relevance framework for e-commerce search

    Tang, T., Tian, Z., Zhu, Z., Wang, C., Hu, H., Tang, G., Liu, L., and Xu, S. Lref: A novel llm-based relevance framework for e-commerce search. In Companion Proceedings of the ACM on Web Conference 2025, WWW '25, pp.\ 468–475, New York, NY, USA, 2025. Association for Computing...

    2025

  24. [32]

    The llama 3 herd of models, 2024

    Team, L. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  25. [33]

    Text embeddings by weakly-supervised contrastive pre-training

    Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., and Wei, F. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022

    2022 arXiv

  26. [34]

    Improving text embeddings with large language models

    Wang, L., Yang, N., Huang, X., Yang, L., Majumder, R., and Wei, F. Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368, 2023

    2023 arXiv

  27. [35]

    Uniir: Training and benchmarking universal multimodal information retrievers

    Wei, C., Chen, Y., Chen, H., Hu, H., Zhang, G., Fu, J., Ritter, A., and Chen, W. Uniir: Training and benchmarking universal multimodal information retrievers. In European Conference on Computer Vision, pp.\ 387--404. Springer, 2024

    2024

  28. [36]

    V., Zhou, D., et al

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  29. [37]

    Weller, O., Ricci, K., Yang, E., Yates, A., Lawrie, D., and Durme, B. V. Rank1: Test-time compute for reranking in information retrieval, 2025. URL https://arxiv.org/abs/2502.18418

    2025 arXiv

  30. [38]

    Inference scaling laws: An empirical analysis of compute-optimal inference for llm problem-solving

    Wu, Y., Sun, Z., Li, S., Welleck, S., and Yang, Y. Inference scaling laws: An empirical analysis of compute-optimal inference for llm problem-solving. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24, 2024

    2024

  31. [39]

    Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning, 2025

    Xie, T., Gao, Z., Ren, Q., Luo, H., Hong, Y., Dai, B., Zhou, J., Qiu, K., Wu, Z., and Luo, C. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning, 2025. URL https://arxiv.org/abs/2502.14768

    2025 arXiv

  32. [40]

    Enhancing asymmetric web search through question-answer generation and ranking

    Ye, D., Liu, J., Fan, J., Tian, B., Zhou, T., Chen, X., and Ma, J. Enhancing asymmetric web search through question-answer generation and ranking. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, pp.\ 6127–6136, New York, NY, US...

    2024

  33. [41]

    The lessons of developing process reward models in mathematical reasoning

    Zhang, Z., Zheng, C., Wu, Y., Zhang, B., Lin, R., Yu, B., Liu, D., Zhou, J., and Lin, J. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025

    2025 arXiv

  34. [42]

    Large language models for information retrieval: A survey

    Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., and Wen, J.-R. Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.