Pith. sign in

REVIEW 4 major objections 5 minor 64 references

Zero-shot pseudo-document prompting can match or beat RL-trained query rewriting, and a hybrid that applies RL to pseudo-document generation performs best overall.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:05 UTC pith:SZ3NJE3B

load-bearing objection Useful comparison and a sensible hybrid, but the headline 'prompting beats RL' rests on a GPT-4o-mini vs Qwen 3B/7B comparison that is never squared with the promised compute-aware setting. the 4 major comments →

arxiv 2510.17139 v3 pith:SZ3NJE3B submitted 2025-10-20 cs.CL cs.IR

Rethinking On-policy Optimization for Query Augmentation

Zhichao Xu , Shengyao Zhuang , Xueguang Ma , Bingsen Chen , Yijun Tian , Fengran Mo , Tao Li , Jie Cao
show 1 more author
Vivek Srikumar
This is my paper
classification cs.CL cs.IR
keywords query augmentationpseudo-document generationreinforcement learningon-policy optimizationquery rewritinginformation retrievalLLM promptingblack-box retrieval
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to settle whether query augmentation for information retrieval should be done by prompting a large language model to write a pseudo-document or by training a policy with reinforcement learning to rewrite queries. It claims that under a compute-aware comparison, simple zero-shot prompting often performs on par with or better than expensive RL-trained rewriting, especially when the prompted model is powerful. It then proposes OPQE, which keeps the pseudo-document format of prompting but uses on-policy RL to optimize the generated pseudo-document against retrieval rewards. If correct, the main value of RL for query augmentation is not rewriting queries but learning to generate better pseudo-documents, and prompting should be a standard low-cost baseline in future work.

Core claim

The central discovery is that the object being optimized matters more than the optimization machinery: RL policies trained to rewrite a query into another query underperform a zero-shot prompt that generates a pseudo-document and uses the concatenation of query and pseudo-document as the augmented query, yet RL becomes the strongest method when the same on-policy objective is applied to generating the pseudo-document itself. The paper merges the generative structure of pseudo-document prompting with the targeted optimization of RL: OPQE conditions the policy on writing a hypothetical passage, concatenates it with the query, and rewards the result with retrieval metrics. Across evidence-seeki

What carries the argument

The central mechanism is the pseudo-document as the policy's output space. Standard RL query augmentation treats the policy as a rewriter that outputs a finished query and rewards it by retrieval effectiveness. OPQE redefines the action as generating a full hypothetical document, concatenates it with the original query, and computes retrieval rewards over that concatenated string. This accomplishes two things: it converts asymmetric query-document matching into symmetric document-document matching for dense retrievers, and it gives the RL process a warm start because zero-shot pseudo-documents already carry lexical and semantic signal. The reward is a product of a format indicator and a retr

Load-bearing premise

The paper's central empirical claim rests on comparing a zero-shot prompt from a large proprietary model with PPO-trained small open-weight models, with no compute or cost accounting; if the same backbone were used on both sides, the conclusion that prompting matches or surpasses RL could reverse, as the paper's own limitations acknowledge results may change with more performant base models and different on-policy algorithms.

What would settle it

Run SPQE with the exact small open-weight backbones used for RL training on the same benchmarks and compare against the RL-trained policies; if the small prompted models fall clearly below the RL policies, the 'training-free prompting matches RL' claim is an artifact of model scale. Alternatively, run OPQE on tool retrieval with a dense retriever; if it also fails to beat no augmentation, the pseudo-document reformulation does not by itself fix RL's dense-retrieval weakness.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Simple, training-free pseudo-document prompting can serve as a cheap standard baseline in query augmentation studies, competitive with learned rewriters.
  • RL-trained query rewriting is not uniformly beneficial: on dense tool retrieval it can reduce performance below no augmentation, while prompting remains robust.
  • Training an RL policy to produce pseudo-documents rather than rewritten queries yields the best overall dense retrieval accuracy, so task reformulation can matter as much as reward engineering.
  • Smaller open-weight models trained with on-policy optimization can approach the level of much larger prompted models, making RL useful when a powerful API model is unavailable or too costly.
  • Query-side augmentation alone can improve black-box retrievers, which matters when the retriever cannot be fine-tuned.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Extension: the headline comparison 'prompting beats RL' is scale-confounded, because the prompting baseline uses a large proprietary model while the RL policies use small open-weight models with no compute or cost accounting; a fair test would hold the backbone fixed.
  • Extension: OPQE was only tested on evidence-seeking and ad hoc retrieval, not on tool retrieval where dense RL failed; a natural next experiment is whether pseudo-document RL also rescues dense tool retrieval or inherits the same failure.
  • Extension: the warm-start explanation predicts that OPQE's margin grows with the base model's zero-shot pseudo-document quality; this could be tested by ablating the prompt or initializing from a weaker generator.
  • Extension: because the reward is a product of format and retrieval scores, the method depends on format following; relaxing or shaping that reward may change the conclusions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports a systematic comparison between prompt-based query augmentation (SPQE, zero-shot pseudo-document generation) and RL-based query rewriting (DeepRetrieval-style PPO) across evidence-seeking, ad hoc, and tool retrieval. It finds that the training-free SPQE often matches or beats RL-trained policies, and then proposes OPQE, which trains a policy to generate pseudo-documents under retrieval rewards. The authors report that OPQE outperforms both standalone prompting and RL-based rewriting on several benchmarks, especially for dense retrieval, and release code for reproducibility.

Significance. If the headline finding held as stated, this would be an important result for practitioners: it would suggest that expensive RL fine-tuning for query augmentation is often unnecessary when a strong instruction-following LLM is available, and that the hybrid OPQE formulation offers a better use of RL. The paper is also among the first to compare these paradigms under a common retrieval environment, and it extends the comparison to tool retrieval. The release of code and the use of public benchmarks are strengths. However, the headline comparison is currently confounded by base-model choice, and the 'compute-aware' framing is not backed by any cost or compute measurement, so the significance of the main claim cannot be assessed as presented.

major comments (4)
  1. [§4.2, Backbone Models; Abstract] The central comparison between SPQE and RL is confounded with the base model. SPQE uses GPT-4o-mini, while the RL policies are initialized from Qwen2.5-3B/7B-Instruct. Appendix B reports SPQE only with GPT-4o-mini, Qwen3-32B, and GPT-OSS-120B — never with Qwen2.5-3B/7B. Thus 'simple prompting surpasses RL' may simply reflect the parametric knowledge of a much larger proprietary model. The abstract's claim of a 'compute-aware comparison setting' is also unfulfilled: no compute or cost figures are reported, only GPU counts for RL. This is load-bearing for the paper's main conclusion and needs to be fixed, e.g., by running SPQE on the same Qwen2.5-3B/7B backbones or by reporting a proper cost-aware comparison.
  2. [§5.2, Table 3] The claim that OPQE 'outperforms both standalone prompting and RL-based rewriting' is not uniformly supported. On sparse evidence-seeking retrieval, SPQE achieves 78.2 average Hit@20, while OPQE-7B achieves 76.5; on dense evidence-seeking, the differences between SPQE (79.7), OPQE-3B (80.2), and OPQE-7B (79.8) are within normal noise. No error bars, multiple seeds, or significance tests are reported anywhere in the paper, so the 'consistently best' language in Section 5.2 is stronger than the evidence warrants.
  3. [§4.4, Table 2] The tool retrieval experiment has an internal inconsistency: Section 4.4 states 'we adopt BM25 and E5-base-v2 as the retrievers', but Table 2's caption says the dense retriever is Contriever-msmarco. This makes the experimental setup ambiguous and hinders reproduction. In addition, the sentence 'looking at Recall@10, HyDE, another generative approach, consistently outperforms all other methods' refers to HyDE, but Table 2 contains no HyDE column — only SPQE. Either the wrong method name or the wrong table is cited.
  4. [Limitations and Ethical Risks] The manuscript's own limitation statement admits that 'our observations and conclusions may be subject to change with more performant base models and different on-policy algorithms.' This directly undercuts the abstract's broad claim that training-free prompting 'often performs on par with, or even surpasses' RL-based counterparts. The limitation should be reflected in the abstract and conclusion, or the experiments need to be expanded to cover the same base models for both paradigms.
minor comments (5)
  1. [Abstract; §1] Typo: 'paramatric knowledge' should be 'parametric knowledge'. Also, the abstract's sentence about OPQE is grammatically awkward: 'in which the LLM policy learns to generate a pseudo-document that maximizes retrieval performance, rather than rewriting the query' is fine, but the later clause 'merging the flexibility...' is a sentence fragment.
  2. [Table 5] In Table 5, the dataset is listed as 'Squad' while elsewhere it is 'SQuAD'. Use a consistent capitalization.
  3. [References] Reference to Ellen M. V oorhees and 'V oorhees' has a spurious space. This appears in the references and in the text.
  4. [§4.4] The paper refers to 'Qwen-3-32B-Instruct' and 'GPT-OSS-120B' but elsewhere uses 'Qwen3-32B' and 'GPT-OSS-120B'; please standardize the model names.
  5. [§4.2, Implementation Details] The description of the retrieval environment is brief; specifying the exact Pyserini and Faiss configurations (e.g., BM25 parameters, indexing settings) would improve reproducibility.

Circularity Check

0 steps flagged

No mathematical circularity; main empirical comparison has an unquantified confound (model scale) but not a definitional or self-citation circle.

full rationale

The paper contains no derivation chain in which an output is smuggled into an input. The RL objective (Eq. 1) maximizes a retrieval-based reward, the OPQE method is a distinct policy that generates pseudo-documents rather than rewritten queries, and all evaluations use external, pre-existing benchmark corpora (NQ, TriviaQA, BEIR, ToolRet) with standard metrics. The reward functions (Appendix C) are defined directly from retrieval ranks/NDCG/Completeness, and the reported test scores are on held-out evaluation sets, not on the training rewards, so there is no fitted-parameter-renamed-as-prediction pattern. The comparison between SPQE and RL does conflate method with base model: 'Our main experimental result is based on proprietary GPT-4o-mini' while RL uses Qwen2.5-3B/7B, and the paper's own limitation concedes 'Our observations and conclusions may be subject to change with more performant base models.' This is a legitimate validity concern (confounding and unquantified 'compute-aware' claim), but it is not circularity under the specified definitions: the results are externally falsifiable and the central OPQE result (Qwen2.5-3B/7B OPQE vs. Qwen2.5-3B/7B RL) holds base models fixed. The paper also appropriately frames RL as borrowing DeepRetrieval's reward design rather than re-deriving it, and no load-bearing argument rests on a self-citation chain. Therefore the appropriate score is 1: no significant circularity, with a minor caveat about the headline comparison's design rather than any logical circle.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The ledger captures the experimental choices on which the empirical claims rest. OPQE and SPQE have no mathematical derivation; their validity is entirely empirical. Reward shaping, PPO hyperparameters, the single-pseudo-document concatenation, and the transfer assumption are the main ingredients beyond standard retrieval pipelines. No new theoretical entities are introduced.

free parameters (4)
  • PPO training hyperparameters (LR=1e-6, global batch=128, actor/critic minibatch=32, rollout temperature=0.6)
    Reported in Section 4.2, but no seeds or early stopping are given; these choices shape the RL comparison and are not swept.
  • KL regularization coefficient beta in Eq. (1)
    The objective in Eq. (1) includes beta, but its value is never reported; it controls the trade-off between retrieval reward and policy deviation, directly affecting the RL results.
  • Evidence-seeking retrieval reward thresholds (5.0/4.0/2.0/1.0/0.5/0.1/-2.5 by rank bin) = 5.0,4.0,2.0,1.0,0.5,0.1,-2.5
    Table 8 specifies a hand-designed piecewise reward for rank; this reward shaping, inherited from DeepRetrieval-style training, affects the learned policy and was not varied.
  • Tool retrieval reward metric Completeness@10 = Completeness@10
    The authors choose to optimize Completeness@10 for tool retrieval; as they acknowledge, different reward designs could change the conclusions.
axioms (4)
  • domain assumption Retrieval metrics (NDCG@10, Hit@20, Completeness@10) are valid proxies for query augmentation quality.
    The paper optimizes and evaluates on these metrics; a different reward could reverse the RL results, especially in tool retrieval where RL underperforms on dense retrieval.
  • domain assumption A single concatenated (query, pseudo-document) is a stronger augmented query than a rewritten query or multi-document aggregation.
    SPQE deliberately drops HyDE's multi-embedding aggregation step and treats one pseudo-document as sufficient; OPQE inherits this assumption as the generation target.
  • standard math PPO with Eq. (1) and the format-reward product is a valid on-policy optimization framework.
    The objective is a standard PPO/RLHF objective from cited prior work. No proof is needed, but the specific product of format indicator and retrieval reward is an assumed design choice.
  • domain assumption Training on the ToolRet train corpus transfers to the official evaluation splits despite a corpus mismatch.
    Section 4.4 trains and evaluates on different tool sets; the paper assumes the learned query augmentation generalizes across candidate-tool environments.

pith-pipeline@v1.3.0-alltime-deepseek · 18199 in / 11098 out tokens · 96864 ms · 2026-08-04T09:05:20.933748+00:00 · methodology

0 comments
read the original abstract

Recent advances in large language models (LLMs) have led to a surge of interest in query augmentation for information retrieval (IR). Two main approaches have emerged. The first prompts LLMs to generate answers or pseudo-documents that serve as new queries, relying purely on the model's parametric knowledge or contextual information. The second applies reinforcement learning (RL) to fine-tune LLMs for query rewriting, directly optimizing retrieval metrics. While having respective advantages and limitations, the two approaches have not been compared under consistent experimental conditions. In this work, we present the first systematic comparison of prompting-based and RL-based query augmentation across diverse benchmarks, including evidence-seeking, ad hoc, and tool retrieval. Our key finding is that under a compute-aware comparison setting, simple, training-free query augmentation often performs on par with, or even surpasses, more expensive RL-based counterparts, especially when using powerful LLMs. Motivated by this discovery, we introduce a novel hybrid method, On-policy Pseudo-document Query Expansion (OPQE), in which the LLM policy learns to generate a pseudo-document that maximizes retrieval performance, rather than rewriting the query, thus merging the flexibility and generative structure of prompting with the targeted optimization of RL. We show OPQE outperforms both standalone prompting and RL-based rewriting, demonstrating that a synergistic approach yields the best results. We open source our implementation to facilitate reproducibility.

Figures

Figures reproduced from arXiv: 2510.17139 by Bingsen Chen, Fengran Mo, Jie Cao, Shengyao Zhuang, Tao Li, Vivek Srikumar, Xueguang Ma, Yijun Tian, Zhichao Xu.

Figure 1
Figure 1. Figure 1: Reward curves of OPQE vs. RL on FEVER and TriviaQA datasets across different retrievers. [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The RL (DeepRetrieval) prompt template for NQ dataset — sparse retrieval. Notice the boolean operators. [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The RL (DeepRetrieval) prompt template for NQ dataset — dense retrieval. [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: SPQE prompt template for NQ dataset. We use same prompt template for sparse and dense retrieval. [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The OPQE prompt template for NQ dataset. We use same prompt template for sparse and dense retrieval. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

64 extracted references · 1 canonical work pages

  1. [1]

    Kenya Abe, Kunihiro Takeoka, Makoto P Kato, and Masafumi Oyamada. 2025. Llm-based query expansion fails for unfamiliar and ambiguous queries. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3035--3039

    2025

  2. [2]

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925

    Pith/arXiv arXiv 2025

  3. [3]

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, and 1 others. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268

    Pith/arXiv arXiv 2016

  4. [4]

    Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. 2016. A full-text learning to rank dataset for medical information retrieval. In European Conference on Information Retrieval, pages 716--722. Springer

    2016

  5. [5]

    Xin Cao, Gao Cong, Bin Cui, Christian S ndergaard Jensen, and Ce Zhang. 2009. The use of categorization information in language models for question retrieval. In Proceedings of the 18th ACM conference on Information and knowledge management, pages 265--274

    2009

  6. [6]

    Chia-Yuan Chang, Zhimeng Jiang, Vineeth Rakesh, Menghai Pan, Chin-Chia Michael Yeh, Guanchu Wang, Mingzhi Hu, Zhichao Xu, Yan Zheng, Mahashweta Das, and Na Zou. 2025. MAIN - RAG : Multi-Agent Filtering Retrieval-Augmented Generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2...

    doi:10.18653/v1/2025.acl-long.131 2025

  7. [7]

    Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghaddam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, and Jimmy Lin. 2025. https://arxiv.org/abs/2508.06600 Browsecomp-plus: A more fair and transparent evaluation b...

    Pith/arXiv arXiv 2025

  8. [8]

    Jo \ a o Coelho, Bruno Martins, Jo \ a o Magalh \ a es, and Chenyan Xiong. 2025. Aligning web query generation with ranking objectives via direct preference optimization. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2982--2986

    2025

  9. [9]

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M Voorhees, and Ian Soboroff. 2025. Overview of the trec 2022 deep learning track. arXiv preprint arXiv:2507.10865

    Pith/arXiv arXiv 2025

  10. [10]

    Guglielmo Faggioli, Thibault Formal, Stefano Marchesin, St \'e phane Clinchant, Nicola Ferro, and Benjamin Piwowarski. 2023. Query performance prediction for neural ir: Are we there yet? In European Conference on Information Retrieval, pages 232--248. Springer

    2023

  11. [11]

    Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. https://doi.org/10.18653/v1/2023.acl-long.99 Precise zero-shot dense retrieval without relevance labels . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1762--1777, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.acl-long.99 2023

  12. [12]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    Pith/arXiv arXiv 2025

  13. [13]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. https://openreview.net/forum?id=jKN1pXi7b0 Unsupervised dense information retrieval with contrastive learning . Transactions on Machine Learning Research

    2022

  14. [14]

    Pengcheng Jiang, Jiacheng Lin, Lang Cao, Runchu Tian, SeongKu Kang, Zifeng Wang, Jimeng Sun, and Jiawei Han. 2025. https://arxiv.org/abs/2503.00223 Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning . Preprint, arXiv:2503.00223

    Pith/arXiv arXiv 2025

  15. [15]

    Jeff Johnson, Matthijs Douze, and Herv \'e J \'e gou. 2019. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535--547

    2019

  16. [16]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601--1611, Vancouver, Canada. Assoc...

    doi:10.18653/v1/p17-1147 2017

  17. [17]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. https://doi.org/10.1162/tacl_a_00276 Natural questions: A benchma...

    doi:10.1162/tacl_a_00276 2019

  18. [18]

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, and 1 others. 2024. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124

    Pith/arXiv arXiv 2024

  19. [19]

    Yibin Lei, Yu Cao, Tianyi Zhou, Tao Shen, and Andrew Yates. 2024. https://doi.org/10.18653/v1/2024.eacl-short.34 Corpus-steered query expansion with large language models . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 393--401, St. Julian ' s, Malta. Associat...

    doi:10.18653/v1/2024.eacl-short.34 2024

  20. [20]

    Yibin Lei, Tao Shen, and Andrew Yates. 2025. https://arxiv.org/abs/2506.09260 Thinkqe: Query expansion via an evolving thinking process . Preprint, arXiv:2506.09260

    arXiv 2025

  21. [21]

    Minghan Li, Xinxuan Lv, Junjie Zou, Tongna Chen, Chao Zhang, Suchao An, Ercong Nie, and Guodong Zhou. 2025. https://arxiv.org/abs/2509.07794 Query expansion in the age of pre-trained and large language models: A comprehensive survey . Preprint, arXiv:2509.07794

    Pith/arXiv arXiv 2025

  22. [22]

    Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: An easy-to-use python toolkit to support replicable ir research with sparse and dense representations. arXiv preprint arXiv:2102.10073

    Pith/arXiv arXiv 2021

  23. [23]

    Hang Liu, Meng Chen, Youzheng Wu, Xiaodong He, and Bowen Zhou. 2021. Conversational query rewriting with self-supervised learning. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7628--7632. IEEE

    2021

  24. [24]

    Gonzalez, and Ion Stoica

    Shu Liu, Sumanth Hegde, Shiyi Cao, Alan Zhu, Dacheng Li, Tyler Griggs, Eric Tang, Akshay Malik, Kourosh Hakhamaneshi, Richard Liaw, Philipp Moritz, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025. Skyrl-sql: Matching gpt-4o and o4-mini on text2sql with multi-turn rl

    2025

  25. [25]

    Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.322 Query rewriting in retrieval-augmented large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303--5315, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.322 2023

  26. [26]

    Xueguang Ma, Kai Sun, Ronak Pradeep, and Jimmy Lin. 2021. A replication study of dense passage retriever. arXiv preprint arXiv:2104.05740

    Pith/arXiv arXiv 2021

  27. [27]

    Chuan Meng, Negar Arabzadeh, Mohammad Aliannejadi, and Maarten De Rijke. 2023. Query performance prediction: From ad-hoc to conversational search. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2583--2593

    2023

  28. [28]

    Jacob Menick, Kevin Lu, Shengjia Zhao, E Wallace, H Ren, H Hu, N Stathas, and F Petroski Such. 2024. Gpt-4o mini: advancing cost-efficient intelligence. Open AI: San Francisco, CA, USA

    2024

  29. [29]

    Fengran Mo, Kelong Mao, Yutao Zhu, Yihong Wu, Kaiyu Huang, and Jian-Yun Nie. 2023. https://doi.org/10.18653/v1/2023.acl-long.274 C onv GQR : Generative query reformulation for conversational search . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4998--5012, Toronto, Canada. Associ...

    doi:10.18653/v1/2023.acl-long.274 2023

  30. [30]

    Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2024. Gorilla: Large language model connected with massive apis. Advances in Neural Information Processing Systems, 37:126544--126565

    2024

  31. [31]

    Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang, Chaojun Xiao, and 1 others. 2024. Tool learning with foundation models. ACM Computing Surveys, 57(4):1--40

    2024

  32. [32]

    Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2024. Towards completeness-oriented tool retrieval for large language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 1930--1940

    2024

  33. [33]

    Qwen. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2

    Pith/arXiv arXiv 2024

  34. [34]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  35. [35]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. https://doi.org/10.18653/v1/D16-1264 SQ u AD : 100,000+ questions for machine comprehension of text . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383--2392, Austin, Texas. Association for Computational Linguistics

    doi:10.18653/v1/d16-1264 2016

  36. [36]

    Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, and 1 others. 1995. Okapi at TREC-3. British Library Research and Development Department

    1995

  37. [37]

    J. J. Rocchio. 1971. Relevance feedback in information retrieval. In G. Salton, editor, The Smart retrieval system - experiments in automatic document processing, pages 313--323. Englewood Cliffs, NJ: Prentice-Hall

    1971

  38. [38]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    Pith/arXiv arXiv 2017

  39. [39]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    Pith/arXiv arXiv 2024

  40. [40]

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279--1297

    2025

  41. [41]

    Zhengliang Shi, Yuhan Wang, Lingyong Yan, Pengjie Ren, Shuaiqiang Wang, Dawei Yin, and Zhaochun Ren. 2025. https://doi.org/10.18653/v1/2025.findings-acl.1258 Retrieval models aren ' t tool-savvy: Benchmarking tool retrieval for large language models . In Findings of the Association for Computational Linguistics: ACL 2025, pages 24497--24524, Vienna, Austr...

    doi:10.18653/v1/2025.findings-acl.1258 2025

  42. [42]

    Richard S Sutton, Andrew G Barto, and 1 others. 1998. Reinforcement learning: An introduction, volume 1. MIT press Cambridge

    1998

  43. [43]

    Nandan Thakur, Nils Reimers, Andreas R \"u ckl \'e , Abhishek Srivastava, and Iryna Gurevych. 2021. https://openreview.net/forum?id=wCu6T5xFjeJ BEIR : A heterogeneous benchmark for zero-shot evaluation of information retrieval models . In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)

    2021

  44. [44]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. https://doi.org/10.18653/v1/N18-1074 FEVER : a large-scale dataset for fact extraction and VER ification . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pap...

    doi:10.18653/v1/n18-1074 2018

  45. [45]

    Voorhees

    Ellen M. Voorhees. 1994. Query expansion using lexical-semantic relations. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '94, page 61–69, Berlin, Heidelberg. Springer-Verlag

    1994

  46. [46]

    Ellen M Voorhees, Nick Craswell, Bhaskar Mitra, Daniel Campos, and Emine Yilmaz. 2020. Overview of the trec 2019 deep learning track

    2020

  47. [47]

    David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.609 Fact or fiction: Verifying scientific claims . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534--7550, Online. Association for Com...

    doi:10.18653/v1/2020.emnlp-main.609 2020

  48. [48]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533

    Pith/arXiv arXiv 2022

  49. [49]

    Liang Wang, Nan Yang, and Furu Wei. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.585 Query2doc: Query expansion with large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9414--9423, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.585 2023

  50. [50]

    Xiao Wang, Craig Macdonald, and Iadh Ounis. 2020. Deep reinforced query reformulation for information retrieval. arXiv preprint arXiv:2007.07987

    Pith/arXiv arXiv 2020

  51. [51]

    Zeqiu Wu, Yi Luan, Hannah Rashkin, David Reitter, Hannaneh Hajishirzi, Mari Ostendorf, and Gaurav Singh Tomar. 2022. Conqrr: Conversational query rewriting for retrieval with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10000--10014

    2022

  52. [52]

    Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, and Vivek Srikumar. 2024. Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15359--15396, Miami, Florida, USA. Association for Computational Linguistics. 10.18653/v1/2024.findings-emnlp.901 https://doi.o...

    doi:10.18653/v1/2024.findings-emnlp.901 2024

  53. [53]

    Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang, Jimmy Lin, and Vivek Srikumar. 2025. A Survey of Model Architectures in Information Retrieval. arXiv preprint arXiv:2502.14822

    arXiv 2025

  54. [54]

    Zhichao Xu, Zhiqi Huang, Shengyao Zhuang, and Vivek Srikumar. 2025. Distillation versus Contrastive Learning: How to Train Your Rerankers. arXiv preprint arXiv:2507.08336

    arXiv 2025

  55. [55]

    Zhichao Xu, Jinghua Yan, Ashim Gupta, and Vivek Srikumar. 2025. State Space Models are Strong Text Rerankers. In Proceedings of the 10th Workshop on Representation Learning for NLP (RepL4NLP-2025), pages 152--169, Albuquerque, NM. Association for Computational Linguistics. 10.18653/v1/2025.repl4nlp-1.12 https://doi.org/10.18653/v1/2025.repl4nlp-1.12

    work page doi:10.18653/v1/2025.repl4nlp-1.12 2025

  56. [56]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

    Pith/arXiv arXiv 2025

  57. [57]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369--2380, Brussels...

    doi:10.18653/v1/d18-1259 2018

  58. [58]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR)

    2023

  59. [59]

    Fanghua Ye, Meng Fang, Shenghui Li, and Emine Yilmaz. 2023. Enhancing conversational search: Large language model-aided informative query rewriting. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5985--6006

    2023

  60. [60]

    Shi Yu, Jiahua Liu, Jingqin Yang, Chenyan Xiong, Paul Bennett, Jianfeng Gao, and Zhiyuan Liu. 2020. https://doi.org/10.1145/3397271.3401323 Few-shot generative conversational query rewriting . In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 1933–1936, New York, NY, USA. As...

    arXiv 2020

  61. [61]

    Chengxiang Zhai and John Lafferty. 2001. Model-based feedback in the language modeling approach to information retrieval. In Proceedings of the tenth international conference on Information and knowledge management, pages 403--410

    2001

  62. [62]

    Changtai Zhu, Siyin Wang, Ruijun Feng, Kai Song, and Xipeng Qiu. 2025. Convsearch-r1: Enhancing query reformulation for conversational search with reasoning via reinforcement learning. arXiv preprint arXiv:2505.15776

    arXiv 2025

  63. [63]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  64. [64]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...