Pith. sign in

REVIEW 5 major objections 5 minor 56 references

MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MassTool claims that jointly training tool-usage detection with tool retrieval improves accuracy across three benchmarks, beating the best prior retriever with p<0.001.

desk verdict Useful dataset, plausible multi-task recipe, but the graph module's evaluation leaks or is inert, and the table has a suspicious duplicate — needs revision before the central claims hold. read the letter →

arxiv 2507.00487 v2 pith:4VBCUCLW submitted 2025-07-01 cs.IR cs.CL

Jianghao Lin , Xinyuan Wang , Xinyi Dai , Menghui Zhu , Bo Chen , Ruiming Tang , Yong Yu , Weinan Zhang This is my paper
classification cs.IRcs.CL
keywords toolretrievallargelanguagemodelsusagedetectionmulti-tasklearningquery-centricgraphconvolutionsearch-basedintentmodelingdense
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that tool retrieval for large language models is improved by treating it as two decisions in sequence: first decide whether the query needs any external tool, then decide which tools to return. To carry this out, the authors introduce MassTool, a two-tower framework that trains a tool-usage detector alongside a retriever on the same query, so that knowledge about whether a call is needed leaks into the matching step. On the standard ToolLens, ToolBenchG2 and ToolBenchG3 benchmarks, MassTool reports statistically significant gains over the best prior baseline, with Recall@3 improved by 2.69% to 12.02% depending on the backbone. The paper also releases ToolDet, 31,229 queries that need no tool invocation, which can be added to existing retrieval datasets to enable this dual-step training. If the claim holds, tool-augmented LLM pipelines get both smaller, higher-quality candidate sets and an explicit early exit for queries that need no tools.

What carries the argument

The argument is carried by three modules on top of a frozen dense retriever: the query-centric graph convolution network (QC-GCN), which runs graph message passing over a query-tool graph and a query-scene graph to capture collaborative patterns and scene-level tool co-occurrence; the search-based user intent modeling (SUIM), which retrieves the query's nearest neighbours from a global query pool, filters them by a cosine-similarity threshold, and aggregates them with an intent attention network so out-of-distribution phrasings are enriched by semantically close queries; and the adaptive knowledge transfer module (AdaKT), which multiplies the detection tower's hidden state through a sigmoid gate into the search-enhanced query representation, letting the 'does this need a tool?' decision filter which aspects of the query drive the ranking. The three query representations (graph, search, gated) are normalized and added before computing cosine matching scores against normalized tool representations. The learning objective combines the list-wise retrieval loss, a binary detection loss, and a contrastive regularization loss aligning query-tool and query-scene views.

What would settle it

Take ToolBench's test set, remove a random 10% of golden tools from each query's label set, and re-run MassTool and COLT; if MassTool's Recall@3 advantage shrinks or reverses compared with the original labels, the reported gains would be shown to depend on label completeness rather than on stronger query understanding.

Watch

Extended reading notes

Core claim

MassTool's central claim is that retrieval accuracy is gated by query comprehension, not just by tool representation, and that the missing ingredient is a dual-step decision: whether to call any tool at all, and if so which ones. Concretely, the paper shows that jointly optimizing a tool-usage detection head (binary cross-entropy), a sampled list-wise retrieval loss, and an in-batch contrastive loss over query-tool and query-scene graphs yields higher Recall@K and NDCG@K than existing retrievers that assume every query requires a tool call. The detection tower contributes a hidden 'detective knowledge' vector that is injected into the retrieval tower through an element-wise gating module, and this injection is what lets the detector's confidence shape the final query-tool matching scores. Across four dense-retrieval backbones and three benchmarks, every reported MassTool configuration outperforms the strongest baseline COLT with p<0.001.

Load-bearing premise

The load-bearing premise is that the golden tool sets in ToolLens and ToolBench are complete and correct; if many queries are missing relevant tools in the labels, then higher Recall and NDCG could reflect better mimicry of incomplete labels rather than genuinely better tool selection.

Editorial extensions

If this is right

  • Tool-augmented LLM pipelines can stop early: when the detection tower says no tool is needed, the system can answer directly and skip the retrieval step entirely.
  • Existing retrieval datasets can be upgraded to dual-step training by appending ToolDet, which the paper shows improves retrieval metrics over using tool-only queries.
  • Retrieval quality on complex, multi-tool queries — where the golden set is larger — improves more than on simple queries, suggesting the gain concentrates where intent is hardest to parse.
  • The framework is backbone-agnostic: the same relative gains appear with ANCE, TAS-B, coCondensor, and Contriever as frozen encoders.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the golden labels are noisy, part of the reported margin may be MassTool fitting the label pattern rather than finding better tools; a human-audited subset of queries is the clean test.
  • The same 'should I retrieve at all?' gate before ranking could transfer to other search settings, such as personal knowledge assistants, where many queries should not trigger retrieval.
  • Because SUIM's query pool can be expanded online, MassTool becomes a semi-nonparametric retriever; one testable prediction is that retrieval accuracy keeps rising as the pool grows beyond the training queries.
  • The detection score could also save inference cost by routing no-tool queries to a smaller model, a latency trade-off the paper notes is left for future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. MassTool proposes a two-tower multi-task framework for tool retrieval in LLM pipelines. The paper formulates tool retrieval as a dual-step decision process: first predict whether a query requires tool invocation, then retrieve the relevant tools. The retrieval tower combines a query-centric graph convolution network (QC-GCN) built from golden query-tool and query-scene graphs, a search-based user intent modeling module (SUIM) that aggregates semantically similar training queries, and an adaptive knowledge transfer module (AdaKT) that fuses the detection tower's hidden state into the retrieval query representation. The model is trained with a tool usage detection loss, a sampled list-wise retrieval loss, and a contrastive regularization loss. The authors also construct and release ToolDet, a 31,229-query dataset of non-tool-invocation utterances. Experiments on ToolLens, ToolBenchG2, and ToolBenchG3 across four backbone retrievers report consistent and large improvements over COLT and other baselines, with relative Recall@3 gains of roughly 2.7%–12.0%.

Significance. If the reported results hold, MassTool is a potentially useful contribution: it introduces a multi-task detection-plus-retrieval formulation, releases the first open tool-usage detection dataset, and demonstrates consistent gains over strong baselines across multiple backbones and datasets. The code and dataset are promised to be open-sourced, which would support reproducibility. However, the central empirical claims are currently blocked by an unspecified and potentially problematic graph construction protocol at test time, a likely reporting error in the Raw baseline rows of Table 1, and the complete absence of any evaluation of the detection tower. These issues are load-bearing for the paper's main narrative rather than presentation details, so the manuscript needs substantive revision before the claims can be accepted.

major comments (5)
  1. [§4.2.1, Eq. (13)] The evaluation protocol for QC-GCN is unspecified, and the two natural readings are both problematic. The query-tool and query-scene graphs in §4.2.1 are built from golden tool sets T_q, and LightGCN in Eq. (13) aggregates over neighbor sets N_q and N_t. For a test query that is not inserted into the graph, N_q is empty, so e_graph_q in Eq. (4) reduces to e0_q, and the 'graph-enhanced query representation' used in Eq. (6) contributes nothing on the query side; in that case the reported gains cannot be attributed to QC-GCN's query understanding. If test queries are instead inserted with their golden edges, the query representation directly aggregates the gold tools, leaking the answer into the matching score. The paper must state exactly which graph is used at test time. If the graph is transductive, an inductive query-side graph module, or at minimum a controlled experiment in which test queries have no golden edges, is needed. The ablation in §6.5 does not resolve this issue because all variants share the same graph construction.
  2. [Table 1, Raw rows] The Raw rows for ToolLens and ToolBenchG2 are identical for every backbone: for example, ANCE Raw R@3 = 0.2083 and R@5 = 0.2656 on both datasets, TAS-B Raw R@3 = 0.1910 on both, coCondensor Raw R@3 = 0.1533 on both, and Contriever Raw R@3 = 0.2567 on both. ToolLens and ToolBenchG2 are different datasets with different queries and tool pools, so exact equality to four decimal places across four metrics is not credible. This appears to be a copy-and-paste error, and because the Raw rows anchor the baseline comparison, the error propagates into the relative improvement numbers. Please correct Table 1 and verify every entry against experimental logs.
  3. [§4.3, Eq. (11)] The contrastive loss in Eq. (11) is written as a negative sum of two softmax probability terms, not as the negative logarithm of those terms. The standard InfoNCE/NCE objective cited in [6] contains a logarithm; without it, Eq. (11) is a different objective with different gradient behavior and cannot be described as InfoNCE regularization. Please correct the equation or explicitly state that a non-standard variant is used and justify that choice.
  4. [§5 and §6] The paper's first stated contribution is the dual-step modeling with tool usage detection, and Section 5 releases ToolDet for this task. However, no detection performance is reported anywhere: there is no accuracy, F1, AUC, or precision/recall on the ToolDet test split or on the integrated retrieval datasets. Without these numbers, the claim that the detection tower 'predicts the need for function calls' and that the dual-step formulation improves retrieval through detective knowledge is not directly supported; the retrieval gains could come purely from the auxiliary loss. Please add a detection evaluation and, ideally, an end-to-end evaluation of the two-step pipeline.
  5. [§6.5, Figure 4] The ablation results in Figure 4 are plotted on extremely narrow y-axis ranges (for example, ANCE on ToolLens Recall@5 spans roughly 0.9820–0.9850), and the text describes differences as 'considerable' and 'significant' without error bars, standard deviations, or significance tests. At this scale, the component-wise claims for w/o AdaKT, w/o SUIM, w/o DF, and w/o CL are not established. Please report multiple seeds with variance or statistical significance tests, or soften the corresponding claims.
minor comments (5)
  1. [§4.2.1] The update equations for the query-scene graph are not given explicitly; Eq. (13) only shows the query-tool message passing, even though the text states that the operations are symmetric and the scene-node updates are used in Eq. (4). Please provide the scene-node aggregation equations or a clear reference to them.
  2. [§4.3, Eq. (10)] The objective in Eq. (10) is a weighted binary cross-entropy over positive and negative candidates rather than a standard listwise ranking loss; please clarify the relationship to listwise losses and why this specific surrogate was chosen.
  3. [§6.2] The significance claim is described only as a t-test with p < 0.001; please report whether the test is paired, over how many samples it is computed, and how variance is estimated.
  4. [§5] The ToolDet construction uses a semantic similarity range of 0.4–0.6 from Sentence-BERT against tool-dependent queries, but the paper does not report how sensitive the final dataset is to this threshold or how many queries were removed by the LLM and human verification stages; adding these statistics would strengthen the dataset description.
  5. [§1] The paper states it is the first to formulate tool retrieval as a dual-step decision problem, but the related work already cites MetaTool [11], which explicitly addresses deciding whether to use tools and which tools to use; please soften the novelty claim or clarify the distinction.

Circularity Check

1 steps flagged · score 6.0 of 10

QC-GCN query representations are built by aggregating the golden tool set, so the reported retrieval gains are either label leakage at test time or an inert graph module; the central claim about graph-enhanced query understanding is circular as written.

  1. self definitional [Section 4.2.1 (QC-GCN); Eq. (4), Eq. (7), Eq. (13); Appendix B.1]
    "The query-tool graph links a query q to each tool t in its golden tool set Tq. ... At the l-th layer for the query-tool graph, the message propagation can be written as: e_l_q = Σ_{t∈N_q} e_t^{l-1}/√(|N_q||N_t|), ... where N_q and N_t are the neighbor sets for query q and tool t, respectively. ... s(q, t) = e_q^T e_t."

    By construction, N_q for the query-tool graph is the golden tool set T_q. Eq. (13) aggregates neighbor embeddings into e_l_q, so egraph_q in Eq. (4) is a function of T_q. The matching score s(q,t)=e_q^T e_t then ranks tools using a query representation that already contains the golden tools whenever test queries are placed in the graph. This makes the retrieval prediction depend on the label it is supposed to output; the reported Recall/NDCG gains are therefore not independent evidence for QC-GCN. If test queries are not placed in the graph, the same equations give empty aggregation and the module is inert, so the claim is unsupported either way.

full rationale

MassTool's remaining machinery—SUIM over training neighbors, AdaKT gating, list-wise and contrastive losses—is standard supervised retrieval training with hyperparameters tuned on validation data; those parts are not circular, and the paper's self-citations are not load-bearing. The circularity is localized to QC-GCN: the query-tool graph is defined by each query's golden tool set, and Eq. (13) makes the graph-enhanced query representation an aggregate over those golden tools before the matching score is computed. If the same graph construction is used for test queries, the retrieval target T_q is an input to the scoring function, so Recall/NDCG are inflated by construction; if test queries are instead inserted without golden edges, every aggregation layer is empty and the claimed graph enhancement degenerates to the initial PLM embedding. Either way, Table 1 cannot validate the paper's central claim that QC-GCN improves query comprehension. The ablation study does not resolve this because all variants keep the same ambiguous graph construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the 'scene' concept is borrowed from COLT. The free parameters are standard hyperparameters tuned on validation, and the axioms are the usual dataset and modeling assumptions for retrieval evaluation.

free parameters (6)
  • lambda (tool usage detection loss weight) = selected from {0.08, 0.2, 0.5, 1.0}, best near 0.2 (Figure 5)
    Balances detection and retrieval losses; tuned on validation data.
  • beta (contrastive loss weight) = 0.04
    Fixed hyperparameter for the InfoNCE regularization term.
  • epsilon (dynamic filtering threshold) = selected from {0.6, 0.7, 0.8, 0.9, 1.0}, optimal in [0.7, 0.9]
    Controls how many nearest neighbors are kept in SUIM.
  • K (number of nearest neighbors in SUIM) = selected from {10, 20, 25, 30, 35, 40}
    Number of candidate neighbor queries considered for intent modeling.
  • tau (temperature in InfoNCE) = not reported
    Temperature for the contrastive loss in Eq. 11; not specified in the paper.
  • learning rate = selected from {1e-3, 5e-3, 1e-4, 5e-4, 1e-5}
    Adam optimizer learning rate, tuned per configuration.
assumptions (4)
  • domain assumption Golden tool sets in ToolLens and ToolBench faithfully represent the tools needed for each query.
    Evaluation metrics and graph edges are built on these sets; if they are incomplete, the reported gains may be artifacts of label noise (Section 6.2, Appendix C.1).
  • domain assumption ToolDet queries are genuinely tool-independent.
    ToolDet is constructed via a proprietary LLM and human review; its label quality is not measured (Section 5).
  • domain assumption Semantically nearest neighbor queries carry transferable intent signals for tool selection.
    SUIM assumes similar questions need similar tools; this can fail for genuinely novel OOD queries where no good neighbor exists (Section 4.2.2).
  • domain assumption Pretrained dense retriever embeddings (ANCE, TAS-B, coCondensor, Contriever) provide a sound starting point.
    The backbone is frozen and used as the base encoder; if these embeddings are poorly suited to tool descriptions, the graph and search modules may not compensate (Appendix C.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models." pith.science (2026). https://pith.science/paper/4VBCUCLW

@misc{pith2026250700487,
  author       = {Pith},
  title        = {Pith review of: MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4VBCUCLW}},
  note         = {Machine review of arXiv:2507.00487}
}
read the original abstract

Tool retrieval is a critical component in enabling large language models (LLMs) to interact effectively with external tools. It aims to precisely filter the massive tools into a small set of candidates for the downstream tool-augmented LLMs. However, most existing approaches primarily focus on optimizing tool representations, often neglecting the importance of precise query comprehension. To address this gap, we introduce MassTool, a multi-task search-based framework designed to enhance both query representation and tool retrieval accuracy. MassTool employs a two-tower architecture: a tool usage detection tower that predicts the need for function calls, and a tool retrieval tower that leverages a query-centric graph convolution network (QC-GCN) for effective query-tool matching. It also incorporates search-based user intent modeling (SUIM) to handle diverse and out-of-distribution queries, alongside an adaptive knowledge transfer (AdaKT) module for efficient multi-task learning. By jointly optimizing tool usage detection loss, list-wise retrieval loss, and contrastive regularization loss, MassTool establishes a robust dual-step sequential decision-making pipeline for precise query understanding. Extensive experiments demonstrate its effectiveness in improving retrieval accuracy. Our code is available at https://github.com/wxydada/MassTool.

Figures

Figures reproduced from arXiv: 2507.00487 by the authors.

Figure 1
Figure 1. (a) The pipeline of tool-augmented large language models with tool retriever. (b) The [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall framework of MassTool. 4 Methodology As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The performance of MassTool and COLT on different testing subsets w.r.t. the number of [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The performance of different MassTool variants on ToolLens (top row) and ToolBenchG3 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The performance of MassTool w.r.t. different tool usage detection loss weight [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: The performance of MassTool w.r.t. different thresholds [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: The t-SNE visualization for query representations from QC-GCN (in [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 30 canonical work pages

  1. [6]

    Gutmann Michael, Hyvärinen Aapo. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models // Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010. 297–304

    work page 2010

  2. [1]

    Deep learning using rectified linear units (relu) // arXiv preprint arXiv:1803.08375

    Agarap AF. Deep learning using rectified linear units (relu) // arXiv preprint arXiv:1803.08375. 2018

    arXiv 2018

  3. [2]

    GeckOpt: LLM System Efficiency via Intent-Based Tool Selection // Proceedings of the Great Lakes Symposium on VLSI 2024

    Fore Michael, Singh Simranjit, Stamoulis Dimitrios. GeckOpt: LLM System Efficiency via Intent-Based Tool Selection // Proceedings of the Great Lakes Symposium on VLSI 2024. 2024. 353–354

    work page 2024

  4. [3]

    PTR: Precision-Driven Tool Recommendation for Large Language Models // arXiv preprint arXiv:2411.09613

    Gao Hang, Zhang Yongfeng. PTR: Precision-Driven Tool Recommendation for Large Language Models // arXiv preprint arXiv:2411.09613. 2024

    work page arXiv 2024

  5. [4]

    Unsupervised corpus aware language model pre-training for dense passage retrieval // arXiv preprint arXiv:2108.05540

    Gao Luyu, Callan Jamie. Unsupervised corpus aware language model pre-training for dense passage retrieval // arXiv preprint arXiv:2108.05540. 2021

    arXiv 2021

  6. [5]

    Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum // Proceedings of the AAAI Conference on Artificial Intelligence

    Gao Shen, Shi Zhengliang, Zhu Minghang, Fang Bowen, Xin Xin, Ren Pengjie, Chen Zhumin, Ma Jun, Ren Zhaochun . Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum // Proceedings of the AAAI Conference on Artificial Intelligence. 38, 16. 2024. 18030–18038

    work page 2024

  7. [7]

    He Xiangnan, Deng Kuan, Wang Xiang, Li Yan, Zhang Yongdong, Wang Meng . Lightgcn: Simplifying and powering graph convolution network for recommendation // Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 2020. 639–648

    work page 2020

  8. [8]

    Hofstätter Sebastian, Lin Sheng-Chieh, Yang Jheng-Hong, Lin Jimmy, Hanbury Allan . Effi- ciently teaching an effective dense retriever with balanced topic aware sampling // Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021. 113–122

    work page 2021

Show all 56 references
  1. [9]

    A Comprehensive Survey on Retrieval Methods in Recommender Systems // arXiv preprint arXiv:2407.21022

    Huang Junjie, Chen Jizheng, Lin Jianghao, Qin Jiarui, Feng Ziming, Zhang Weinan, Yu Yong. A Comprehensive Survey on Retrieval Methods in Recommender Systems // arXiv preprint arXiv:2407.21022. 2024

    2024

  2. [10]

    Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios // arXiv preprint arXiv:2401.17167

    Huang Shijue, Zhong Wanjun, Lu Jianqiao, Zhu Qi, Gao Jiahui, Liu Weiwen, Hou Yutai, Zeng Xingshan, Wang Yasheng, Shang Lifeng, others . Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios // arXiv preprint arXiv:2401....

    2024 arXiv

  3. [11]

    Metatool benchmark for large language models: Deciding whether to use tools and which to use // arXiv preprint arXiv:2310.03128

    Huang Yue, Shi Jiawen, Li Yuan, Fan Chenrui, Wu Siyuan, Zhang Qihui, Liu Yixin, Zhou Pan, Wan Yao, Gong Neil Zhenqiang, others . Metatool benchmark for large language models: Deciding whether to use tools and which to use // arXiv preprint arXiv:2310.03128. 2023

    arXiv

  4. [12]

    Unsupervised dense information retrieval with contrastive learning // arXiv preprint arXiv:2112.09118

    Izacard Gautier, Caron Mathilde, Hosseini Lucas, Riedel Sebastian, Bojanowski Piotr, Joulin Armand, Grave Edouard. Unsupervised dense information retrieval with contrastive learning // arXiv preprint arXiv:2112.09118. 2021

    2021 arXiv

  5. [13]

    Joko Hideaki, Chatterjee Shubham, Ramsay Andrew, De Vries Arjen P , Dalton Jeff, Hasibi Faegheh. Doing personal laps: Llm-augmented dialogue construction for personalized multi- session conversational search // Proceedings of the 47th International ACM SIGIR Conference on Rese...

    2024

  6. [14]

    Natural language processing for dialects of a language: A survey // arXiv preprint arXiv:2401.05632

    Joshi Aditya, Dabre Raj, Kanojia Diptesh, Li Zhuang, Zhan Haolan, Haffari Gholamreza, Dippold Doris. Natural language processing for dialects of a language: A survey // arXiv preprint arXiv:2401.05632. 2024

    2024 arXiv

  7. [15]

    Improving Tool Retrieval by Leveraging Large Language Models for Query Generation // arXiv preprint arXiv:2412.03573

    Kachuee Mohammad, Ahuja Sarthak, Kumar Vaibhav, Xu Puyang, Liu Xiaohu . Improving Tool Retrieval by Leveraging Large Language Models for Query Generation // arXiv preprint arXiv:2412.03573. 2024. 10

    2024 arXiv

  8. [16]

    Api-bank: A comprehensive benchmark for tool-augmented llms // arXiv preprint arXiv:2304.08244

    Li Minghao, Zhao Yingxiu, Yu Bowen, Song Feifan, Li Hangyu, Yu Haiyang, Li Zhoujun, Huang Fei, Li Yongbin. Api-bank: A comprehensive benchmark for tool-augmented llms // arXiv preprint arXiv:2304.08244. 2023

    2023 arXiv

  9. [17]

    Agent4ranking: Semantic robust ranking via personalized query rewriting using multi-agent llm // arXiv preprint arXiv:2312.15450

    Li Xiaopeng, Su Lixin, Jia Pengyue, Zhao Xiangyu, Cheng Suqi, Wang Junfeng, Yin Dawei . Agent4ranking: Semantic robust ranking via personalized query rewriting using multi-agent llm // arXiv preprint arXiv:2312.15450. 2023

    2023 arXiv

  10. [18]

    How can recommender systems benefit from large language models: A survey // ACM Transactions on Information Systems

    Lin Jianghao, Dai Xinyi, Xi Yunjia, Liu Weiwen, Chen Bo, Zhang Hao, Liu Yong, Wu Chuhan, Li Xiangyang, Zhu Chenxu, others . How can recommender systems benefit from large language models: A survey // ACM Transactions on Information Systems. 2025. 43, 2. 1–47

    2025

  11. [19]

    A Graph-Enhanced Click Model for Web Search // Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Lin Jianghao, Liu Weiwen, Dai Xinyi, Zhang Weinan, Li Shuai, Tang Ruiming, He Xiuqiang, Hao Jianye, Yu Yong. A Graph-Enhanced Click Model for Web Search // Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021. 1259–1268

    2021

  12. [20]

    Hammer: Robust Function-Calling for On-Device Language Models via Function Masking // arXiv preprint arXiv:2410.04587

    Lin Qiqiang, Wen Muning, Peng Qiuying, Nie Guanyu, Liao Junwei, Wang Jun, Mo Xiaoyun, Zhou Jiamu, Cheng Cheng, Zhao Yin, others. Hammer: Robust Function-Calling for On-Device Language Models via Function Masking // arXiv preprint arXiv:2410.04587. 2024

  13. [21]

    ToolACE: Winning the Points of LLM Function Calling // arXiv preprint arXiv:2409.00920

    Liu Weiwen, Huang Xu, Zeng Xingshan, Hao Xinlong, Yu Shuai, Li Dexun, Wang Shuai, Gan Weinan, Liu Zhengying, Yu Yuanqing, others. ToolACE: Winning the Points of LLM Function Calling // arXiv preprint arXiv:2409.00920. 2024

    2024 arXiv

  14. [22]

    DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task // arXiv preprint arXiv:2406.16332

    Liu Wenhan, Zhu Yutao, Dou Zhicheng. DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task // arXiv preprint arXiv:2406.16332. 2024

    2024 arXiv

  15. [23]

    From summary to action: Enhancing large language models for complex tasks with open world apis // arXiv preprint arXiv:2402.18157

    Liu Yulong, Yuan Yunlong, Wang Chunwei, Han Jianhua, Ma Yongqiang, Zhang Li, Zheng Nanning, Xu Hang. From summary to action: Enhancing large language models for complex tasks with open world apis // arXiv preprint arXiv:2402.18157. 2024

    2024 arXiv

  16. [24]

    Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets // arXiv preprint arXiv:2406.18518

    Liu Zuxin, Hoang Thai, Zhang Jianguo, Zhu Ming, Lan Tian, Kokane Shirley, Tan Juntao, Yao Weiran, Liu Zhiwei, Feng Yihao, others. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets // arXiv preprint arXiv:2406.18518. 2024

    2024 arXiv

  17. [25]

    Improve Mathematical Reasoning in Language Models by Automated Process Supervision // arXiv preprint arXiv:2406.06592

    Luo Liangchen, Liu Yinxiao, Liu Rosanne, Phatale Samrat, Lara Harsh, Li Yunxuan, Shu Lei, Zhu Yun, Meng Lei, Sun Jiao, others . Improve Mathematical Reasoning in Language Models by Automated Process Supervision // arXiv preprint arXiv:2406.06592. 2024

    2024 arXiv

  18. [26]

    Visualizing data using t-SNE

    Maaten Laurens Van der, Hinton Geoffrey. Visualizing data using t-SNE. // Journal of machine learning research. 2008. 9, 11

    2008

  19. [27]

    Efficient and Scalable Estimation of Tool Representations in Vector Space // arXiv preprint arXiv:2409.02141

    Moon Suhong, Jha Siddharth, Erdogan Lutfi Eren, Kim Sehoon, Lim Woosang, Keutzer Kurt, Gholami Amir. Efficient and Scalable Estimation of Tool Representations in Vector Space // arXiv preprint arXiv:2409.02141. 2024

    2024 arXiv

  20. [28]

    Adaptive Selection for Homogeneous Tools: An Instantiation in the RAG Scenario // arXiv preprint arXiv:2406.12429

    Mu Feiteng, Jiang Yong, Zhang Liwen, Liu Chu, Li Wenjie, Xie Pengjun, Huang Fei. Adaptive Selection for Homogeneous Tools: An Instantiation in the RAG Scenario // arXiv preprint arXiv:2406.12429. 2024

    2024 arXiv

  21. [29]

    Talm: Tool augmented language models // arXiv preprint arXiv:2205.12255

    Parisi Aaron, Zhao Yao, Fiedel Noah. Talm: Tool augmented language models // arXiv preprint arXiv:2205.12255. 2022

    2022 arXiv

  22. [30]

    Gorilla: Large language model connected with massive apis // arXiv preprint arXiv:2305.15334

    Patil Shishir G, Zhang Tianjun, Wang Xin, Gonzalez Joseph E. Gorilla: Large language model connected with massive apis // arXiv preprint arXiv:2305.15334. 2023

    2023 arXiv

  23. [31]

    Large language model based long-tail query rewriting in taobao search // Companion Proceedings of the ACM on Web Conference 2024

    Peng Wenjun, Li Guiyang, Jiang Yue, Wang Zilong, Ou Dan, Zeng Xiaoyi, Xu Derong, Xu Tong, Chen Enhong. Large language model based long-tail query rewriting in taobao search // Companion Proceedings of the ACM on Web Conference 2024. 2024. 20–28

    2024

  24. [32]

    ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

    Qin Yujia, Liang Shihao, Ye Yining, Zhu Kunlun, Yan Lan, Lu Yaxi, Lin Yankai, Cong Xin, Tang Xiangru, Qian Bill, Zhao Sihan, Tian Runchu, Xie Ruobing, Zhou Jie, Gerstein Mark, Li Dahai, Liu Zhiyuan, Sun Maosong. ToolLLM: Facilitating Large Language Models to Master 16000+ Real...

    2023

  25. [33]

    Tool Learning with Large Language Models: A Survey // arXiv preprint arXiv:2405.17935

    Qu Changle, Dai Sunhao, Wei Xiaochi, Cai Hengyi, Wang Shuaiqiang, Yin Dawei, Xu Jun, Wen Ji-Rong. Tool Learning with Large Language Models: A Survey // arXiv preprint arXiv:2405.17935. 2024

    2024 arXiv

  26. [34]

    Towards completeness-oriented tool retrieval for large language models // Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

    Qu Changle, Dai Sunhao, Wei Xiaochi, Cai Hengyi, Wang Shuaiqiang, Yin Dawei, Xu Jun, Wen Ji-Rong. Towards completeness-oriented tool retrieval for large language models // Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 2024. 1930–1940

    2024

  27. [35]

    Sentence-bert: Sentence embeddings using siamese bert- networks // arXiv preprint arXiv:1908.10084

    Reimers Nils, Gurevych Iryna . Sentence-bert: Sentence embeddings using siamese bert- networks // arXiv preprint arXiv:1908.10084. 2019

    1908 arXiv

  28. [36]

    The probabilistic relevance framework: BM25 and beyond // Foundations and Trends® in Information Retrieval

    Robertson Stephen, Zaragoza Hugo, others . The probabilistic relevance framework: BM25 and beyond // Foundations and Trends® in Information Retrieval. 2009. 3, 4. 333–389

    2009

  29. [37]

    Satpute Ankit, Gießing Noah, Greiner-Petter André, Schubotz Moritz, Teschke Olaf, Aizawa Akiko, Gipp Bela. Can llms master math? investigating large language models on math stack exchange // Proceedings of the 47th International ACM SIGIR Conference on Research and Development...

    2024

  30. [38]

    Toolformer: Language models can teach themselves to use tools // Advances in Neural Information Processing Systems

    Schick Timo, Dwivedi-Yu Jane, Dessì Roberto, Raileanu Roberta, Lomeli Maria, Hambro Eric, Zettlemoyer Luke, Cancedda Nicola, Scialom Thomas. Toolformer: Language models can teach themselves to use tools // Advances in Neural Information Processing Systems. 2024. 36

    2024

  31. [39]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face // Advances in Neural Information Processing Systems

    Shen Yongliang, Song Kaitao, Tan Xu, Li Dongsheng, Lu Weiming, Zhuang Yueting. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face // Advances in Neural Information Processing Systems. 2024. 36

    2024

  32. [40]

    Siro Clemencia, Aliannejadi Mohammad, Rijke Maarten de . Rethinking the evaluation of dialogue systems: Effects of user feedback on crowdworkers and LLMs // Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2024. 1952–1962

    2024

  33. [41]

    A statistical interpretation of term specificity and its application in retrieval // Journal of documentation

    Sparck Jones Karen. A statistical interpretation of term specificity and its application in retrieval // Journal of documentation. 1972. 28, 1. 11–21

    1972

  34. [42]

    Toolal- paca: Generalized tool learning for language models with 3000 simulated cases // arXiv preprint arXiv:2306.05301

    Tang Qiaoyu, Deng Ziliang, Lin Hongyu, Han Xianpei, Liang Qiao, Cao Boxi, Sun Le. Toolal- paca: Generalized tool learning for language models with 3000 simulated cases // arXiv preprint arXiv:2306.05301. 2023

    2023 arXiv

  35. [43]

    BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models

    Thakur Nandan, Reimers Nils, Rücklé Andreas, Srivastava Abhishek, Gurevych Iryna. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models. 2021

    2021

  36. [44]

    Attention is all you need // Advances in neural information processing systems

    Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N, Kaiser Łukasz, Polosukhin Illia. Attention is all you need // Advances in neural information processing systems. 2017. 5998–6008

    2017

  37. [45]

    Tool-LMM: A Large Multi-Modal Model for Tool Agent Learning // arXiv preprint arXiv:2401.10727

    Wang Chenyu, Luo Weixin, Chen Qianyu, Mai Haonan, Guo Jindi, Dong Sixun, Li Zhengxin, Ma Lin, Gao Shenghua, others . Tool-LMM: A Large Multi-Modal Model for Tool Agent Learning // arXiv preprint arXiv:2401.10727. 2024

    2024 arXiv

  38. [46]

    Towards efficient and effective unlearning of large language models for recommendation // Frontiers of Computer Science

    Wang Hangyu, Lin Jianghao, Chen Bo, Yang Yang, Tang Ruiming, Zhang Weinan, Yu Yong. Towards efficient and effective unlearning of large language models for recommendation // Frontiers of Computer Science. 2025. 19, 3. 193327

    2025

  39. [47]

    Chain-of-thought prompting elicits reasoning in large language models // Advances in neural information processing systems

    Wei Jason, Wang Xuezhi, Schuurmans Dale, Bosma Maarten, Xia Fei, Chi Ed, Le Quoc V , Zhou Denny, others . Chain-of-thought prompting elicits reasoning in large language models // Advances in neural information processing systems. 2022. 35. 24824–24837

    2022

  40. [48]

    Xi Yunjia, Liu Weiwen, Lin Jianghao, Chen Bo, Tang Ruiming, Zhang Weinan, Yu Yong. Memo- CRS: Memory-enhanced Sequential Conversational Recommender Systems with Large Lan- guage Models // Proceedings of the 33rd ACM International Conference on Information and Knowledge Managem...

    2024

  41. [49]

    Approximate nearest neighbor negative contrastive learning for dense text retrieval // arXiv preprint arXiv:2007.00808

    Xiong Lee, Xiong Chenyan, Li Ye, Tang Kwok-Fung, Liu Jialin, Bennett Paul, Ahmed Junaid, Overwijk Arnold. Approximate nearest neighbor negative contrastive learning for dense text retrieval // arXiv preprint arXiv:2007.00808. 2020

    2007 arXiv

  42. [50]

    On the tool ma- nipulation capability of open-source large language models // arXiv preprint arXiv:2305.16504

    Xu Qiantong, Hong Fenglu, Li Bo, Hu Changran, Chen Zhengyu, Zhang Jian. On the tool ma- nipulation capability of open-source large language models // arXiv preprint arXiv:2305.16504. 2023

    2023 arXiv

  43. [51]

    React: Synergizing reasoning and acting in language models // arXiv preprint arXiv:2210.03629

    Yao Shunyu, Zhao Jeffrey, Yu Dian, Du Nan, Shafran Izhak, Narasimhan Karthik, Cao Yuan. React: Synergizing reasoning and acting in language models // arXiv preprint arXiv:2210.03629. 2022

    2022 arXiv

  44. [52]

    Easytool: Enhancing llm-based agents with concise tool instruction // arXiv preprint arXiv:2401.06201

    Yuan Siyu, Song Kaitao, Chen Jiangjie, Tan Xu, Shen Yongliang, Kan Ren, Li Dongsheng, Yang Deqing. Easytool: Enhancing llm-based agents with concise tool instruction // arXiv preprint arXiv:2401.06201. 2024

    2024 arXiv

  45. [53]

    Zhang Yuxiang, Fan Xin, Wang Junjie, Chen Chongxian, Mo Fan, Sakai Tetsuya, Yamana Hayato. Data-Efficient Massive Tool Retrieval: A Reinforcement Learning Approach for Query-Tool Alignment with Language Models // Proceedings of the 2024 Annual International ACM SIGIR Conferenc...

    2024

  46. [54]

    P , Zhang Hao, Gonzalez Joseph E., Stoica Ion

    Zheng Lianmin, Chiang Wei-Lin, Sheng Ying, Zhuang Siyuan, Wu Zhanghao, Zhuang Yonghao, Lin Zi, Li Zhuohan, Li Dacheng, Xing Eric. P , Zhang Hao, Gonzalez Joseph E., Stoica Ion . Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. 2023

    2023

  47. [55]

    ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval // arXiv preprint arXiv:2403.06551

    Zheng Yuanhang, Li Peng, Liu Wei, Liu Yang, Luan Jian, Wang Bin. ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval // arXiv preprint arXiv:2403.06551. 2024

    2024 arXiv

  48. [56]

    Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning // arXiv preprint arXiv:2502.14361

    Zhu Jiachen, Zheng Congmin, Lin Jianghao, Du Kounianhua, Wen Ying, Yu Yong, Wang Jun, Zhang Weinan. Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning // arXiv preprint arXiv:2502.14361. 2025. 13 A Broader Impact Our work contributes to improving...

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.