REVIEW 4 major objections 6 minor 4 cited by
From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper argues that agentic deep research—an LLM-driven loop of planning, searching, reasoning, and synthesizing—will become the dominant paradigm for information seeking and introduces a test-time scaling law to explain why.
desk verdict A timely, well-organized position paper on agentic deep research whose useful taxonomy is weakened by a scaling 'law' that is really an interpolation of two unrelated benchmarks. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanisms are two. First, the Agentic Deep Research loop: an LLM plans queries, retrieves, reasons, and iterates, with reasoning and search co-evolving rather than staged in sequence. Second, the Test-Time Scaling law: a hypothesized linear relation between normalized performance and inference-time compute along two axes, internal reasoning depth and external knowledge exploration, visualized as a diagonal plane interpolating two empirical scaling curves. The law frames a task-dependent token-budget trade-off between search and reasoning that the paper expects future systems to allocate adaptively.
What would settle it
Run one integrated deep-research agent on a fixed set of open-ended web-research questions at several total token budgets, holding the task constant. If normalized accuracy does not rise roughly linearly with total inference tokens, or if doubling the search budget and doubling the reasoning budget together yields less than the sum of the individual gains, the linear test-time scaling law is falsified.
Extended reading notes
Core claim
The paper's core claim is that Agentic Deep Research—defined as an LLM-driven feedback loop in which reasoning decides when, what, and how to search, and retrieved evidence in turn refines reasoning—will inevitably become the dominant paradigm for information seeking. The supporting formal object is the Test-Time Scaling law for Deep Research: when performance is normalized, it improves linearly with scaling of internal reasoning depth (longer chains of thought, self-refinement, self-consistency) and of external knowledge exploration (more iterative retrieval steps and long-context access). The paper is explicit that this law is a hypothesis: the diagonal plane in Figure 3b interpolates between reasoning data from S1 on AIME24 and search data from IterDRAG on MuSiQue, and different task types align with different planes in the search–reasoning trade-off. It further argues that reinforcement learning, rather than prompting or supervised fine-tuning, is the principled mechanism for incentivizing reasoning-driven search in open-ended environments.
Load-bearing premise
The scaling law's load-bearing premise is that performance trends measured separately on a math benchmark and a retrieval benchmark can be merged into one linear law that predicts how an integrated agent behaves on real research tasks, even though no integrated system was measured to produce that law.
Editorial extensions
If this is right
- If the scaling law holds, spending more inference tokens on either deeper reasoning or more search steps should yield predictable, roughly linear accuracy gains on knowledge-intensive tasks.
- Standard web search and single-pass retrieval-augmented generation will be inadequate for complex multi-step queries, pushing users toward agentic systems that plan and iterate.
- Reinforcement learning, rather than prompting or supervised fine-tuning, becomes the key training method for teaching search agents to explore, self-correct, and adapt.
- Benchmark gaps of the kind reported—51.5% versus below 10% on BrowseComp, 26.6% versus under 20% on HLE—suggest that agentic deep research is not a marginal improvement but a categorical jump.
- Open-source momentum around agentic research implementations indicates the paradigm shift is already happening in the developer community, not just in commercial products.
Reading between the lines
- An editorial extension: if the linear test-time scaling law is real, search engines may evolve into compute-budget allocators that decide how many tokens go to retrieval versus reasoning for each query, making token cost a first-class search metric.
- The paper's two-axis trade-off suggests a concrete design target: an agent that predicts whether a query is search-heavy or reasoning-heavy and sets its own budget accordingly, which could be trained by reinforcement learning on reward signals for answer quality minus compute cost.
- A testable consequence the paper does not state: the same linear trend should appear on a single integrated benchmark if total inference budget is varied, and not just as an interpolation of two separate curves.
- If the position is correct, evaluating information-seeking systems shifts from retrieval metrics like precision and recall to end-user outcomes such as report correctness, synthesis quality, and reduced cognitive load.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper is a position paper and survey arguing that a new paradigm, 'Agentic Deep Research' (LLMs with tightly integrated reasoning, iterative search, and synthesis), will become the dominant mode of information seeking, superseding traditional web search, LLM chatbots, and naive RAG. It organizes the evolution of search into stages, surveys prompting/SFT/RL approaches for incentivizing search with reasoning agents, reports benchmark numbers for OpenAI Deep Research on BrowseComp, BrowseComp-ZH, and Humanity's Last Exam, analyzes GitHub star trends of open-source implementations, and proposes a 'test-time scaling law' claiming that normalized performance improves linearly with scaling of internal reasoning depth and external knowledge exploration. It also discusses the alternative view of human-centered search and lists open problems. The main contributions are the taxonomy, the curated resource collection, and the TTS conjecture.
Significance. If the position is correct, the paper identifies a real shift in information access and charts a useful design space around the search-reasoning trade-off. The survey of recent RL-based methods (Search-R1, R1-Searcher, DeepResearcher, and others) and the curated GitHub repository are valuable resources for the community. The 'test-time scaling law' is a thought-provoking conjecture that could stimulate useful empirical work, and the paper explicitly includes an alternative-view section and human-in-the-loop caveats, which is a strength for a position paper. However, the paper provides no new experimental evidence, and its only formal quantitative contribution, the TTS law, is an interpolation between two external single-axis curves rather than a validated empirical law. The significance lies in the framing and resource synthesis rather than in a demonstrated scaling result, and the central dominance claim is a prediction that the manuscript supports with citations and qualitative trends rather than with a decisive experiment.
major comments (4)
- [Section 4, Figure 3b] The proposed 'Test-Time Scaling law' is not supported as stated. The data consist of two independent curves: s1 on AIME24 for the reasoning axis (roughly 2^9-2^13 inference tokens) and IterDRAG on MuSiQue for the search axis (16K-5M tokens), each normalized separately to [0,1]. The 'diagonal plane' is an interpolation between these two series, with no equation, no error bars, no held-out points, and no evaluation on any integrated deep-research system that jointly performs reasoning and search. Because the two normalizations are independent, the slopes are not commensurable, and the visual linearity in Figure 3b does not constitute evidence of a shared scaling law. Since this law is the paper's only formal quantitative contribution, it should be reframed as an illustrative conjecture with explicit caveats, or substantiated with an actual equation, error analysis, and validation on an integrated system.
- [Abstract, Section 1, Section 4, Section 4.3] The paper is internally inconsistent about the status of the TTS claim: the abstract and Section 1 call it a 'test-time scaling law,' Section 4 opens with 'we introduce the Test-Time Scaling (TTS) law' and then calls it 'an hypothesis,' and Section 4.3 says the trade-off law 'is anticipated.' A 'law' implies an empirical regularity, while the text repeatedly concedes that the claim is a hypothesis. This distinction matters because the paper's quantitative contribution rests entirely on this claim; the authors should choose one framing and apply it consistently throughout.
- [Section 3.2, Figure 2] The benchmark comparison in Section 3.2 is based on external reports and product announcements, not on new evaluations by the authors. The text says 'we adopt and evaluate three representative benchmarks' and later concludes 'we demonstrate that agentic systems significantly outperform existing approaches,' but Figure 2 reports numbers from third-party sources (e.g., OpenAI Deep Research at 51.5% on BrowseComp, 42.9% on BrowseComp-ZH, 26.6% on HLE) without standard errors, system versions, evaluation protocols, or dates of access. This is a citation, not a demonstration. The authors should either perform an independent evaluation with transparent methodology or explicitly state that these are reported third-party results, and soften the conclusion accordingly.
- [Section 3.2, Figure 3a, Appendix B] The GitHub-star analysis is presented as 'empirical momentum' that reinforces the dominance claim, but it excludes the two most-starred and two least-starred repositories and then asserts a 'clear upward trajectory' without statistical tests, a well-defined sample period, or a comparison baseline. This is an anecdotal, cherry-picked trend rather than a rigorous empirical result. The authors should either provide a formal analysis (e.g., regression on star counts with confidence intervals) or characterize this evidence qualitatively and not as 'empirical momentum' that supports the central position.
minor comments (6)
- [Page 1, footnote] The footnote 'Work does note related the author's position at Amazon' appears to be a typographical error for 'Work does not reflect the author's position at Amazon'; please fix.
- [Section 3.2] The phrase 'information seekin' in the final paragraph is missing a 'g'; it should read 'information seeking'.
- [Section 2.2] There are several run-on or unclear sentences, such as 'engage users through interactive dialogue with integrate and tailored solutions that not directly available online' and 'LLM chatbots maintain conversation history throughout interactions, Beyond that, they aggregate...'; the prose needs careful proofreading.
- [Figure 2] Figure 2 does not state the data sources or the date each benchmark number was collected; adding a footnote with the source and access date would improve reproducibility and clarity.
- [Appendix B, Table 1] Table 1 contains typos such as '2Wikimultihopqa' and inconsistent capitalization, and many entries under 'Evaluation Data' are 'N/A'; the table should be cleaned up and the 'N/A' cases clarified.
- [Section 4.2] The statement that 'under optimal inference parameters, the performance improves nearly linearly' would benefit from a specific reference to the corresponding figure or table in Yue et al. (2025) and a definition of what constitutes 'optimal inference parameters.'
Circularity Check
No circularity: the TTS law is an explicitly hypothetical interpolation of independent external results, and the paradigm-dominance claim is a forward-looking position rather than a derived consequence.
full rationale
This is a position/survey paper rather than a derivation, and no load-bearing step reduces to its own input. The central quantitative element, the Test-Time Scaling law, is presented as an explicit hypothesis: 'the diagonal plane connecting empirical data points interpolated in the three-dimensional plot represents our hypothetical TTS for Agentic Deep Research' (Section 4/Figure 3b). The two input curves come from independent external work (Muennighoff et al. on s1/AIME24 and Yue et al. on IterDRAG/MuSiQue), and the normalization and linear interpolation are the construction of the figure, not a parameter fitted to the paper's own conclusions. The paper does not fit any model to its own outcomes and then rename the fit as a prediction. The dominance claim is supported by external benchmark numbers (OpenAI Deep Research on BrowseComp, BrowseComp-ZH, and Humanity's Last Exam) and external open-source star trends; even though BrowseComp-ZH is co-authored by members of this paper's author list, the reported scores are for an external product, and the benchmark is used as an evaluation instrument rather than as a premise that entails the paper's conclusion. The many remaining self-citations (e.g., Zou et al. 2025a-c, Gu et al. 2025, Luo et al. 2025a, Zhang et al. 2025b) are contextual, non-load-bearing, and none is invoked as a uniqueness theorem or as the source of an ansatz. The paper's inconsistent labeling of the TTS relation as a 'law' in Section 1 and as 'hypothetical'/'anticipated' in Sections 4 and 4.3 is an overclaim or rigor problem, not a circularity. No equation or conclusion in the paper is equivalent to its input by construction, so the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption Test-time scaling for reasoning is linear and continues at the observed rate for deep research tasks.
- domain assumption IterDRAG on MuSiQue represents search-phase test-time scaling for agentic deep research.
- ad hoc to paper Interpolating between two independent datasets defines a valid scaling plane.
- domain assumption GitHub star growth is a valid measure of paradigm adoption.
Cite this review
Pith. "Pith review of From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents." pith.science (2026). https://pith.science/paper/O6LNXFLX
@misc{pith2026250618959,
author = {Pith},
title = {Pith review of: From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents},
year = {2026},
howpublished = {\url{https://pith.science/paper/O6LNXFLX}},
note = {Machine review of arXiv:2506.18959}
}
read the original abstract
Information retrieval is a cornerstone of modern knowledge acquisition, enabling billions of queries each day across diverse domains. However, traditional keyword-based search engines are increasingly inadequate for handling complex, multi-step information needs. Our position is that Large Language Models (LLMs), endowed with reasoning and agentic capabilities, are ushering in a new paradigm termed Agentic Deep Research. These systems transcend conventional information search techniques by tightly integrating autonomous reasoning, iterative retrieval, and information synthesis into a dynamic feedback loop. We trace the evolution from static web search to interactive, agent-based systems that plan, explore, and learn. We also introduce a test-time scaling law to formalize the impact of computational depth on reasoning and search. Supported by benchmark results and the rise of open-source implementations, we demonstrate that Agentic Deep Research not only significantly outperforms existing approaches, but is also poised to become the dominant paradigm for future information seeking. All the related resources, including industry products, research papers, benchmark datasets, and open-source implementations, are collected for the community in https://github.com/DavidZWZ/Awesome-Deep-Research.
Figures
Forward citations
Cited by 4 Pith papers
-
Contextual Information Policy Optimization for Search Agents
CIPO trains search agents with a dense reward that measures how much a reasoning step depends on newly retrieved evidence, improving F1 across seven QA benchmarks.
-
SciDataSailor: Deep Scientific Data Exploring
SciDataSailor uses Monte Carlo tree search to synthesize tool-use trajectories for training and benchmarking LLM agents that explore raw scientific repositories.
-
Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptions
Most MCP tool descriptions (97.1%) contain quality smells, and augmenting them improves agent success by a median of 5.85 percentage points at a 67.46% increase in execution steps.
-
LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services
LocalSearchBench—1.3M merchant records and 900 multi-hop local-life QA tasks across 9 Chinese cities—shows the best reasoning agent reaches only 35.6% correctness.
Reference graph
Works this paper leans on
-
[1]
M., Zobeiri, A., Dehghani, M., Mohammadkhani, M., Mohammadi, B., Ghahroodi, O., Baghshah, M
Abootorabi, M. M., Zobeiri, A., Dehghani, M., Mohammadkhani, M., Mohammadi, B., Ghahroodi, O., Baghshah, M. S., and Asgari, E. (2025). Ask in any modality: A comprehensive survey on multimodal retrieval-augmented generation. arXiv preprint arXiv:2502.08826
arXiv 2025
-
[2]
L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774
arXiv 2023
-
[3]
Alzubi, S., Brooks, C., Chiniya, P., Contente, E., von Gerlach, C., Irwin, L., Jiang, Y., Kaz, A., Nguyen, W., Oh, S., et al. (2025). Open deep search: Democratizing search with open-source reasoning agents. arXiv preprint arXiv:2503.20201
arXiv 2025
-
[4]
Amendola, M., Passarella, A., and Perego, R. (2023). Social search: Retrieving information in online social platforms--a survey. Online Social Networks and Media , 36:100254
2023
-
[5]
Anthropic (2023). Claude. https://www.anthropic.com. Accessed: 2025-06-03
2023
-
[6]
Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. (2023). Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations
2023
-
[7]
K., Cucerzan, S., and Hwang, S
Baek, J., Jauhar, S. K., Cucerzan, S., and Hwang, S. J. (2024). Researchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738
arXiv 2024
-
[8]
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022). Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862
arXiv 2022
Show all 115 references
-
[9]
A., Cavanaugh, M., Clark, K., Karsch-Mizrachi, I., Lipman, D
Benson, D. A., Cavanaugh, M., Clark, K., Karsch-Mizrachi, I., Lipman, D. J., Ostell, J., and Sayers, E. W. (2012). Genbank. Nucleic acids research , 41(D1):D36--D42
2012
-
[10]
Bi, J., Liang, S., Zhou, X., Liu, P., Guo, J., Tang, Y., Song, L., Huang, C., Sun, G., He, J., et al. (2025). Why reasoning matters? a survey of advancements in multimodal reasoning (v1). arXiv preprint arXiv:2504.03151
2025
-
[11]
and Page, L
Brin, S. and Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems , 30(1-7):107--117
1998
-
[12]
Broder, A. (2002). A taxonomy of web search. In ACM Sigir forum , volume 36, pages 3--10. ACM New York, NY, USA
2002
-
[13]
S., Martin, L
Callison-Burch, C., Tomar, G. S., Martin, L. J., Ippolito, D., Bailis, S., and Reitter, D. (2022). Dungeons and dragons as a dialog challenge for artificial intelligence. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 9379--9393
2022
-
[14]
and Shu, K
Chen, C. and Shu, K. (2023). Can llm-generated misinformation be detected? In NeurIPS 2023 Workshop on Regulatable ML
2023
-
[15]
and Yih, W.-t
Chen, D. and Yih, W.-t. (2020). Open-domain question answering. In Proceedings of the 58th annual meeting of the association for computational linguistics: tutorial abstracts , pages 34--37
2020
-
[17]
Z., et al
Chen, M., Li, T., Sun, H., Zhou, Y., Zhu, C., Yang, F., Zhou, Z., Chen, W., Wang, H., Pan, J. Z., et al. (2025b). Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470
2025 arXiv
-
[18]
Chen, Q., Qin, L., Liu, J., Peng, D., Guan, J., Wang, P., Hu, M., Zhou, Y., Gao, T., and Che, W. (2025c). Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567
2025 arXiv
-
[19]
Cheng, M., Luo, Y., Ouyang, J., Liu, Q., Liu, H., Li, L., Yu, S., Zhang, B., Cao, J., Ma, J., et al. (2025). A survey on knowledge-oriented retrieval-augmented generation. arXiv preprint arXiv:2503.10677
2025 arXiv
-
[20]
C., Austine-Orimoloye, O., Azov, A
Dyer, S. C., Austine-Orimoloye, O., Azov, A. G., Barba, M., Barnes, I., Barrera-Enriquez, V. P., Becker, A., Bennett, R., Beracochea, M., Berry, A., et al. (2025). Ensembl 2025. Nucleic Acids Research , 53(D1):D948--D957
2025
-
[21]
Fuhr, N. (1992). Probabilistic models in information retrieval. The computer journal , 35(3):243--255
1992
-
[22]
Ghosh, S., Evuru, C. K. R., Kumar, S., S, R., Aneja, D., Jin, Z., Duraiswami, R., and Manocha, D. (2024). A closer look at the limitations of instruction tuning. In Proceedings of the 41st International Conference on Machine Learning , pages 15559--15589
2024
-
[23]
P., Chen, Y., Liu, A., Zhang, W., and Yu, P
Gu, Z., Zou, H. P., Chen, Y., Liu, A., Zhang, W., and Yu, P. S. (2025). Scaling laws for many-shot in-context learning with self-generated annotations. arXiv preprint arXiv:2503.03062
2025 arXiv
-
[24]
Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. (2025). Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948
2025 arXiv
-
[25]
L., Ying, R., and Leskovec, J
Hamilton, W. L., Ying, R., and Leskovec, J. (2017). Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems , pages 1025--1035
2017
-
[26]
Hao, S., Gu, Y., Ma, H., Hong, J., Wang, Z., Wang, D., and Hu, Z. (2023). Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics
2023
-
[27]
M., Fathima, N
Hendriksen, G., Dinzinger, M., Farzana, S. M., Fathima, N. A., Fr \"o be, M., Schmidt, S., Zerhoudi, S., Granitzer, M., Hagen, M., Hiemstra, D., et al. (2024). The open web index: Crawling and indexing the web for public use. In European Conference on Information Retrieval , p...
2024
-
[28]
Huang, J., Madala, S., Sidhu, R., Niu, C., Hockenmaier, J., and Zhang, T. (2025a). Rag-rl: Advancing retrieval-augmented generation via rl and curriculum learning. arXiv preprint arXiv:2503.12759
2025 arXiv
-
[29]
Huang, X., Liu, W., Chen, X., Wang, X., Wang, H., Lian, D., Wang, Y., Tang, R., and Chen, E. (2024). Understanding the planning of llm agents: A survey. arXiv preprint arXiv:2402.02716
2024 arXiv
-
[30]
Huang, Z., Yuan, X., Ju, Y., Zhao, J., and Liu, K. (2025b). Reinforced internal-external knowledge synergistic reasoning for efficient adaptive search agent. arXiv preprint arXiv:2505.07596
2025 arXiv
-
[31]
V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., Koura, P
Iyer, S., Lin, X. V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., Koura, P. S., et al. (2022). Opt-iml: Scaling language model instruction meta learning through the lens of generalization. arXiv preprint arXiv:2212.12017
2022 arXiv
-
[32]
Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Carney, A., et al. (2024). Openai o1 system card. arXiv preprint arXiv:2412.16720
2024 arXiv
-
[33]
Jiang, Z., Sun, M., Liang, L., and Zhang, Z. (2024). Retrieve, summarize, plan: Advancing multi-hop question answering with an iterative approach. arXiv preprint arXiv:2407.13101
2024 arXiv
-
[34]
F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., and Neubig, G
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., and Neubig, G. (2023). Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages 7969--7992
2023
-
[35]
Jin, B., Zeng, H., Yue, Z., Wang, D., Zamani, H., and Han, J. (2025). Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516
2025 arXiv
-
[36]
u chemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., G \
Kasneci, E., Se ler, K., K \"u chemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., G \"u nnemann, S., H \"u llermeier, E., et al. (2023). Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual ...
2023
-
[37]
Khder, M. A. (2021). Web scraping or web crawling: State of art, techniques, approaches and application. International Journal of Advances in Soft Computing & Its Applications , 13(3)
2021
-
[38]
M., Franklin, M., Oler, E., Wilson, A., Pon, A., Cox, J., Chin, N
Knox, C., Wilson, M., Klinger, C. M., Franklin, M., Oler, E., Wilson, A., Pon, A., Cox, J., Chin, N. E., Strawbridge, S. A., et al. (2024). Drugbank 6.0: the drugbank knowledgebase for 2024. Nucleic acids research , 52(D1):D1265--D1275
2024
-
[39]
J., Lee, J
Landrum, M. J., Lee, J. M., Benson, M., Brown, G. R., Chao, C., Chitipiralla, S., Gu, B., Hart, J., Hoffman, D., Jang, W., et al. (2018). Clinvar: improving access to variant interpretations and supporting evidence. Nucleic acids research , 46(D1):D1062--D1067
2018
-
[40]
Leake, D. B. and Scherle, R. (2001). Towards context-based search engine selection. In Proceedings of the 6th international conference on Intelligent user interfaces , pages 109--112
2001
-
[41]
Li, M., Su, N., Qu, F., Zhong, Z., Chen, Z., Tu, Z., and Li, X. (2025a). Vista: Enhancing vision-text alignment in mllms via cross-modal mutual information maximization. arXiv preprint arXiv:2505.10917
2025 arXiv
-
[42]
Li, S., He, Y., Guo, H., Bu, X., Bai, G., Liu, J., Liu, J., Qu, X., Li, Y., Ouyang, W., et al. (2024). Graphreader: Building graph-based agent to enhance long-context abilities of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024 ,...
2024
-
[43]
Li, X., Dong, G., Jin, J., Zhang, Y., Zhou, Y., Zhu, Y., Zhang, P., and Dou, Z. (2025b). Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366
2025 arXiv
-
[44]
Li, X., Jin, J., Dong, G., Qian, H., Zhu, Y., Wu, Y., Wen, J.-R., and Dou, Z. (2025c). Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.21776
2025 arXiv
-
[45]
Li, X. L. and Liang, P. (2021). Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pa...
2021
-
[46]
Li, Z., Dong, Q., Ma, J., Zhang, D., and Sui, Z. (2025d). Selfbudgeter: Adaptive token allocation for efficient llm reasoning. arXiv preprint arXiv:2505.11274
2025 arXiv
-
[47]
Linden, G., Meek, C., Chickering, M., and Meek, C. (2009). The pollution effect: Optimizing keyword auctions by favoring relevant advertising. In Fifth workshop on Ad Auctions
2009
-
[48]
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., Wu, Z., Zhao, L., Zhu, D., Li, X., Qiang, N., Shen, D., Liu, T., and Ge, B. (2023). Summary of chatgpt-related research and perspective towards the future of large language models. Meta-R...
2023
-
[49]
T., Foerster, J., Clune, J., and Ha, D
Lu, C., Lu, C., Lange, R. T., Foerster, J., Clune, J., and Ha, D. (2024). The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292
2024 arXiv
-
[50]
Luo, J., Zhang, W., Yuan, Y., Zhao, Y., Yang, J., Gu, Y., Wu, B., Chen, B., Qiao, Z., Long, Q., et al. (2025a). Large language model agent: A survey on methodology, applications and challenges. arXiv preprint arXiv:2503.21460
2025 arXiv
-
[51]
Luo, K., Liu, Z., Zhang, P., Qian, H., Zhao, J., and Liu, K. (2025b). Does rag really perform bad for long-context processing? arXiv preprint arXiv:2502.11444
2025 arXiv
-
[52]
Ma, X., Gong, Y., He, P., Zhao, H., and Duan, N. (2023). Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages 5303--5315
2023
-
[53]
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems , 36:46534--46594
2023
-
[54]
Mao, K., Liu, Z., Qian, H., Mo, F., Deng, C., and Dou, Z. (2024). Rag-studio: Towards in-domain adaptation of retrieval augmented generation through self-alignment. In Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 725--735
2024
-
[55]
M., and Tielman, M
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., and Tielman, M. L. (2024). A systematic review on fostering appropriate trust in human-ai interaction: Trends, opportunities and challenges. ACM Journal on Responsible Computing , 1(4):1--45
2024
-
[56]
Mo, F., Mao, K., Zhao, Z., Qian, H., Chen, H., Cheng, Y., Li, X., Zhu, Y., Dou, Z., and Nie, J.-Y. (2024). A survey of conversational search
2024
-
[57]
L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Cand \`e s, E., and Hashimoto, T
Muennighoff, N., Yang, Z., Shi, W., Li, X. L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Cand \`e s, E., and Hashimoto, T. (2025). s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393
2025 arXiv
-
[58]
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021). Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332
2021 arXiv
-
[59]
Noruzi, A. (2005). Google scholar: The new generation of citation indexes
2005
-
[60]
Page, L., Brin, S., Motwani, R., and Winograd, T. (1999). The pagerank citation ranking: Bringing order to the web. Technical report, Stanford infolab
1999
-
[61]
Peng, B., Galley, M., He, P., Cheng, H., Xie, Y., Hu, Y., Huang, Q., Liden, L., Yu, Z., Chen, W., et al. (2023). Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813
2023 arXiv
-
[62]
Phan, L., Gatti, A., Han, Z., Li, N., Hu, J., Zhang, H., Zhang, C. B. C., Shaaban, M., Ling, J., Shi, S., et al. (2025). Humanity's last exam. arXiv preprint arXiv:2501.14249
2025 arXiv
-
[63]
and Berndt, D
Prabhune, S. and Berndt, D. J. (2024). Deploying large language models with retrieval augmented generation. arXiv preprint arXiv:2411.11895
2024 arXiv
-
[64]
A., and Lewis, M
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M. (2023). Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 5687--5711
2023
-
[65]
Robertson, S., Zaragoza, H., et al. (2009). The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval , 3(4):333--389
2009
-
[66]
M., Rasheed, Z., Kemell, K.-K., Waseem, M., Kilamo, T., Saari, M., Duc, A
Sami, A. M., Rasheed, Z., Kemell, K.-K., Waseem, M., Kilamo, T., Saari, M., Duc, A. N., Systä, K., and Abrahamsson, P. (2024). System for systematic literature review using multiple ai agents: Concept and an empirical evaluation
2024
-
[67]
Schick, T., Dwivedi-Yu, J., Dess \` , R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems , 36:68539--68551
2023
-
[68]
Schmidgall, S., Su, Y., Wang, Z., Sun, X., Wu, J., Yu, X., Liu, J., Liu, Z., and Barsoum, E. (2025). Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227
2025 arXiv
-
[69]
Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., and Chen, W. (2023). Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. In Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 9248--9274
2023
-
[70]
Shneiderman, B. (2022). Human-centered AI . Oxford University Press
2022
-
[71]
Singh, A., Ehtesham, A., Kumar, S., and Khoei, T. T. (2025). Agentic retrieval-augmented generation: A survey on agentic rag. arXiv preprint arXiv:2501.09136
2025 arXiv
-
[72]
Snell, C., Lee, J., Xu, K., and Kumar, A. (2024). Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314
2024 arXiv
-
[73]
X., Fang, L., and Wen, J.-R
Song, H., Jiang, J., Min, Y., Chen, J., Chen, Z., Zhao, W. X., Fang, L., and Wen, J.-R. (2025). R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592
2025 arXiv
-
[74]
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Chen, H., et al. (2025). Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419
2025 arXiv
-
[75]
Sun, H., Qiao, Z., Guo, J., Fan, X., Hou, Y., Jiang, Y., Xie, P., Huang, F., and Zhang, Y. (2025a). Zerosearch: Incentivize the search capability of llms without searching. arXiv preprint arXiv:2505.04588
2025 arXiv
-
[76]
Sun, Z., Wang, Q., Yu, W., Zang, X., Zheng, K., Xu, J., Zhang, X., Yang, S., and Li, H. (2025b). Rearter: Retrieval-augmented reasoning with trustworthy process rewarding. arXiv preprint arXiv:2501.07861
2025 arXiv
-
[77]
Tam, D., Mascarenhas, A., Zhang, S., Kwan, S., Bansal, M., and Raffel, C. (2023). Evaluating the factual consistency of large language models through news summarization. In Findings of the Association for Computational Linguistics: ACL 2023 , pages 5220--5255
2023
-
[78]
J., Ting, D
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., and Ting, D. S. W. (2023). Large language models in medicine. Nature medicine , 29(8):1930--1940
2023
-
[79]
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288
2023 arXiv
-
[80]
Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2023). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...
2023
-
[81]
UniProt Consortium, T. (2018). Uniprot: the universal protein knowledgebase. Nucleic acids research , 46(5):2699--2699
2018
-
[82]
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. (2018). Graph attention networks. In International Conference on Learning Representations
2018
-
[83]
Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L., and Sun, H. (2023a). Towards understanding chain-of-thought prompting: An empirical study of what matters. In The 61st Annual Meeting Of The Association For Computational Linguistics
2023
-
[84]
Wang, B., Ping, W., Mcafee, L., Xu, P., Li, B., Shoeybi, M., and Catanzaro, B. (2024a). Instructretro: Instruction tuning post retrieval-augmented pretraining. In International Conference on Machine Learning , pages 51255--51272. PMLR
2024
-
[85]
Wang, H., Fu, T., Du, Y., Gao, W., Huang, K., Liu, Z., Chandak, P., Liu, S., Van Katwyk, P., Deac, A., et al. (2023b). Scientific discovery in the age of artificial intelligence. Nature , 620(7972):47--60
2023
-
[86]
Wang, J., Lu, T., Li, L., Huang, D., et al. (2024b). Enhancing personalized search with ai: a hybrid approach integrating deep learning and cloud computing. Journal of Advanced Computing Systems , 4(10):1--13
2024
-
[87]
Wang, X., Feng, M., Qiu, J., Gu, J., and Zhao, J. (2024c). From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection. Advances in Neural Information Processing Systems , 37:58118--58153
2024
-
[88]
X., Ouyang, J., Xu, Y., and Shi, W
Wang, Z., Teo, S. X., Ouyang, J., Xu, Y., and Shi, W. (2024d). M-rag: Reinforcing large language model performance through retrieval-augmented generation with multiple partitions. arXiv preprint arXiv:2405.16420
2024 arXiv
-
[89]
W., Passos, A
Wei, J., Sun, Z., Papay, S., McKinney, S., Han, J., Fulford, I., Chung, H. W., Passos, A. T., Fedus, W., and Glaese, A. (2025a). Browsecomp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516
2025 arXiv
-
[90]
V., Zhou, D., et al
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824--24837
2022
-
[91]
Wei, Z., Yao, W., Liu, Y., Zhang, W., Lu, Q., Qiu, L., Yu, C., Xu, P., Zhang, C., Yin, B., et al. (2025b). Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning. arXiv preprint arXiv:2505.16421
2025
-
[92]
Wu, J., Zhu, J., and Liu, Y. (2025). Agentic reasoning: Reasoning llms with tools for the deep research. arXiv preprint arXiv:2502.04644
2025 arXiv
-
[93]
Xiong, G., Jin, Q., Wang, X., Zhang, M., Lu, Z., and Zhang, A. (2024). Improving retrieval-augmented generation in medicine with iterative follow-up questions. In Biocomputing 2025: Proceedings of the Pacific Symposium , pages 199--214. World Scientific
2024
-
[94]
Yang, W., Zhang, W., Liu, Y., Han, Y., Wang, Y., Lee, J., and Yu, P. S. (2025). Cold-start recommendation with knowledge-guided retrieval-augmented generation. arXiv preprint arXiv:2505.20773
2025
-
[95]
Yang, Y., Yih, W.-t., and Meek, C. (2015). Wikiqa: A challenge dataset for open-domain question answering. In Proceedings of the 2015 conference on empirical methods in natural language processing , pages 2013--2018
2015
-
[96]
Yao, J.-Y., Ning, K.-P., Liu, Z.-H., Ning, M.-N., Liu, Y.-Y., and Yuan, L. (2023a). Llm lies: Hallucinations are not bugs, but features as adversarial examples. arXiv preprint arXiv:2310.01469
2023 arXiv
-
[97]
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. (2023b). Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems , 36:11809--11822
2023
-
[98]
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. (2023c). React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR)
2023
-
[99]
Yue, Z., Zhuang, H., Bai, A., Hui, K., Jagerman, R., Zeng, H., Qin, Z., Wang, D., Wang, X., and Bendersky, M. (2025). Inference scaling for long-context retrieval augmented generation. In The Thirteenth International Conference on Learning Representations
2025
-
[100]
Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al. (2022). Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414
2022 arXiv
-
[101]
Zhang, E., Wang, X., Gong, P., Lin, Y., and Mao, J. (2024a). Usimagent: Large language models for simulating search users. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages 2687--2692
2024
-
[102]
Zhang, G., Yue, Y., Sun, X., Wan, G., Yu, M., Fang, J., Wang, K., Chen, T., and Cheng, D. (2024b). G-designer: Architecting multi-agent communication topologies via graph neural networks. arXiv preprint arXiv:2410.11782
2024 arXiv
-
[103]
Zhang, N., Zhang, C., Tan, Z., Yang, X., Deng, W., and Wang, W. (2025a). Credible plan-driven rag method for multi-hop question answering. arXiv preprint arXiv:2504.16787
2025
-
[104]
V., et al
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022). Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068
2022 arXiv
-
[105]
P., Huang, Z., Wang, Z., Gao, Y., et al
Zhang, W., Zhang, X., Zhang, C., Yang, L., Shang, J., Wei, Z., Zou, H. P., Huang, Z., Wang, Z., Gao, Y., et al. (2025b). Personaagent: When large language model agents meet personalization at test time. arXiv preprint arXiv:2506.06254
2025 arXiv
-
[106]
S., and Zhang, M
Zhao, Y., Zhang, Q., Luo, X., Zhang, W., Xiao, Z., Ju, W., Yu, P. S., and Zhang, M. (2025). Dynamic text bundling supervision for zero-shot inference on text-attributed graphs. arXiv preprint arXiv:2505.17599
2025
-
[107]
Zheng, Y., Fu, D., Hu, X., Cai, X., Ye, L., Lu, P., and Liu, P. (2025a). Deepresearcher: Scaling deep research via reinforcement learning in real-world environments. arXiv preprint arXiv:2504.03160
2025 arXiv
-
[108]
Zheng, Z., Ni, X., and Hong, P. (2025b). Multiple abstraction level retrieve augment generation. arXiv preprint arXiv:2501.16952
2025 arXiv
-
[109]
Zhong, W., Guo, L., Gao, Q., Ye, H., and Wang, Y. (2024). Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 19724--19731
2024
-
[110]
Zhou, P., Leon, B., Ying, X., Zhang, C., Shao, Y., Ye, Q., Chong, D., Jin, Z., Xie, C., Cao, M., et al. (2025). Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese. arXiv preprint arXiv:2504.19314
2025 arXiv
-
[111]
I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. (2022). Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations
2022
-
[112]
Zhu, Y., Zhang, P., Zhang, C., Chen, Y., Xie, B., Liu, Z., Wen, J.-R., and Dou, Z. (2024). Inters: Unlocking the power of large language models in search with instruction tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume...
2024
-
[113]
Zhuge, M., Wang, W., Kirsch, L., Faccio, F., Khizbullin, D., and Schmidhuber, J. (2024). Gptswarm: Language agents as optimizable graphs. In International Conference on Machine Learning , pages 62743--62767. PMLR
2024
-
[114]
P., Gu, Z., Zhou, Y., Chen, Y., Zhang, W., Fang, L., Wang, Y., Li, Y., Liu, K., and Yu, P
Zou, H. P., Gu, Z., Zhou, Y., Chen, Y., Zhang, W., Fang, L., Wang, Y., Li, Y., Liu, K., and Yu, P. S. (2025a). Testnuc: Enhancing test-time computing approaches and scaling through neighboring unlabeled data consistency. arXiv preprint arXiv:2502.19163
2025 arXiv
-
[115]
P., Huang, W.-C., Wu, Y., Chen, Y., Miao, C., Nguyen, H., Zhou, Y., Zhang, W., Fang, L., He, L., et al
Zou, H. P., Huang, W.-C., Wu, Y., Chen, Y., Miao, C., Nguyen, H., Zhou, Y., Zhang, W., Fang, L., He, L., et al. (2025b). A survey on large language model based human-agent systems. arXiv preprint arXiv:2505.00753
2025 arXiv
-
[116]
P., Huang, W.-C., Wu, Y., Miao, C., Li, D., Liu, A., Zhou, Y., Chen, Y., Zhang, W., Li, Y., et al
Zou, H. P., Huang, W.-C., Wu, Y., Miao, C., Li, D., Liu, A., Zhou, Y., Chen, Y., Zhang, W., Li, Y., et al. (2025c). A call for collaborative intelligence: Why human-agent systems should precede ai autonomy. arXiv preprint arXiv:2506.09420
2025 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.