Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Advancing and Benchmarking Personalized Tool Invocation for LLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning a 7B open model on synthesized personalized tool-invocation data makes it surpass GPT-4-turbo on the new PTBench benchmark without hurting general abilities.

desk verdict Useful new task formulation and a real benchmark, but the train/test circularity makes the headline generalization claim narrower than stated; still deserves peer review. read the letter →

arxiv 2505.04072 v1 pith:A4FWNLIW submitted 2025-05-07 cs.CL cs.AI

Xu Huang , Yuefeng Huang , Weiwen Liu , Xingshan Zeng , Yasheng Wang , Ruiming Tang , Hong Xie , Defu Lian This is my paper · ORCID
classification cs.CLcs.AI
keywords personalizedtoolinvocationpreferenceprofile-dependentqueryPToolPTBenchdatasynthesissupervisedfine-tuningcalling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces the concept of personalized tool invocation, which requires an LLM to choose among functionally similar tools according to a user's preferences and to fill query-omitted tool parameters from the user's profile. To study it, the authors build PTool, a three-stage synthetic-data pipeline, and PTBench, a manually verified benchmark of 1,083 samples across 80 users and five everyday scenarios. The central claim is that supervised fine-tuning a 7B open-source model on PTool data raises PTBench overall accuracy from 0.0738 to 0.2678, above GPT-4-turbo's 0.1847, and that the gain carries to users not seen in training. The authors also report no degradation on standard general and function-calling benchmarks. The reason to care is that, if true, personalized tool use could be trained from synthetic profiles rather than collected from real users.

What carries the argument

PTool, the carried mechanism, is a three-stage data synthesis framework: tool generation uses depth-first expansion over an API tree, with several platforms per scenario to create functionally interchangeable tools; user profile construction clusters platform parameters into a feature tree and assigns values top-down, producing basic features, implicit preferences, and simulated behavioral history; query and solution generation uses a user agent and an assistant agent, followed by rule-based and LLM-based verification, to create profile-grounded queries and tool calls. PTBench is its manually checked test subset, with separate trained and untrained user splits. This machinery lets the authors compare fine-tuned and API models on the same personalized distribution and attribute the gains to the synthetic data.

What would settle it

Run the fine-tuned model on real logged tool-invocation requests with ground-truth platform preferences and profile-sourced parameter values; if its accuracy is no higher than a generic instruction-tuned baseline, the claimed transfer to real users fails.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a small open-source model can be taught personalized tool invocation by synthetic data alone. After parameter-efficient fine-tuning on PTool-generated data, Qwen2.5-7B-Instruct reaches 0.2678 overall accuracy on PTBench, compared with 0.1847 for GPT-4-turbo; on platform selection, the fine-tuned model scores 0.7374 versus 0.5484, and on profile-sourced parameter values 0.7341 versus 0.6832. On six held-out users the overall score is 0.2660, nearly matching the trained-user score of 0.2701, which the authors take as evidence of generalization rather than memorization. They further find that removing behavioral history from the user profile degrades tool-preference accuracy more than removing basic features degrades tool-invocation accuracy, and that the trained model's scores on MMLU, HumanEval, GSM8K, CommonsenseQA, and BFCL non-live do not fall relative to the base model.

Load-bearing premise

The benchmark's synthetic user profiles, platform sets, and missing-parameter patterns are assumed to stand in for real-world personalized tool use.

Editorial extensions

If this is right

  • A 7B open-source model fine-tuned with PTool data reaches 0.2678 overall PTBench accuracy, above GPT-4-turbo's 0.1847, with platform accuracy 0.7374 versus 0.5484.
  • The gains generalize to six unseen users: 0.2660 overall accuracy on untrained users, close to the trained-user performance of 0.2701.
  • Training does not degrade general abilities: MMLU, HumanEval, GSM8K, CommonsenseQA, and BFCL non-live scores are maintained or improved relative to the base Qwen2.5-7B-Instruct model.
  • Profile-dependent parameter filling is harder than query-dependent filling for all models, but the PTool-trained model improves profile-value accuracy to 0.7341, above GPT-4-turbo's 0.6832.
  • Removing behavioral history from the user profile disproportionately hurts platform accuracy, while removing basic features hurts tool-invocation accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If PTBench's distribution transfers, the same recipe could be used in deployed assistants: a 7B-class model fine-tuned on synthetic profiles can deliver personalized tool selection and parameter filling without collecting real user data.
  • The strongest untested risk is distribution shift: the benchmark fixes platform catalogues and profile schemas, so a direct follow-up is to add new platforms or new profile fields after training and measure whether the gains survive.
  • The ablation pattern suggests a testable extension: training on profiles with carefully curated behavioral histories may matter more than adding more basic features, since removing history hurts platform accuracy most.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces the task of Personalized Tool Invocation, decomposing it into Tool Preference (selecting among functionally similar tools based on user preferences) and Profile-dependent Query (inferring missing tool arguments from a user profile). The authors propose PTool, a three-stage LLM-based synthesis framework that generates tools via a hierarchical API tree, constructs user profiles through bottom-up feature clustering and top-down assignment, and produces query-solution pairs via a two-agent role-playing pipeline with rule-based and model-based verification. They then build PTBench, a 1,083-sample, manually verified benchmark drawn from the same PTool distribution, and fine-tune Qwen2.5-7B with LoRA on 7,096 PTool queries. The fine-tuned model is reported to reach 0.2678 overall accuracy on PTBench, outperforming GPT-4-turbo (0.1847) and other API/open-source baselines, including 0.2660 on held-out untrained users, while also improving BFCL non-live scores without degrading MMLU, HumanEval, GSM8K, or CSQA.

Significance. If the results hold, this is a useful first benchmark and data-synthesis framework for personalized tool invocation, a genuinely underexplored aspect of tool use that combines user profiling with function calling. The public release of the benchmark, the manual verification of the test set, and the external BFCL gain are concrete strengths that support the practical value of the contribution. However, the main quantitative claims are measured inside a closed synthetic distribution that the training data and test data share, so the headline numbers should be interpreted as in-distribution improvements rather than demonstrated real-world generalization. The benchmark itself is likely to be a useful resource for future work, and the paper's task formulation is a reasonable starting point, but the evaluation design currently overstates the strength of the generalization evidence.

major comments (4)
  1. [§5.1, Table 1; Findings 3 and 4] The train/test split holds out users but not tools or scenarios: both the training set and PTBench share the same 5 scenarios, 15 platforms, and 360 APIs, and the 'untrained users' split (6 users, 609 queries) only means those user profiles were absent from SFT. The platforms and APIs in those queries are exactly the ones seen during training. The reported platform accuracy of 0.7374 and untrained-user overall accuracy of 0.2660 are therefore consistent with the model memorizing name-level platform/attribute associations from 7,096 training queries and applying them to new profiles, rather than with a generalizable profile-to-tool-attribute rule. This makes the cross-user generalization claim in Finding 3 and the headline win over GPT-4-turbo (0.2678 vs 0.1847) an in-tool-universe result. The authors should add a held-out platform or scenario split, or an API-level hold-out, to support the generalization claim, or explicitly restrict the conclusions to the closed tool universe. The Limitations section does not currently mention this shared-universe issue.
  2. [Appendix A.1; Tables 2-5] Equations (2)-(11) define accuracy rates but leave the notion of a 'correct sample' underspecified. It is not stated whether parameter names and values are compared by exact string matching, by semantic equivalence, or by partial credit; how the multiple parameters of a single invocation are aggregated into a sample-level correctness score; or how platform accuracy is treated when the correct platform is chosen together with the wrong API. In addition, all results are from single runs with no error bars, so differences of a few points in Tables 2 and 3 cannot be distinguished from noise. An operational definition of the evaluation script is needed for the benchmark numbers to be reproducible and meaningful.
  3. [§5.2, Finding 2; Table 2] The headline comparison against API models is asymmetric. The fine-tuned Qwen2.5-7B model has been trained on PTool queries and on the exact platforms and APIs used in PTBench, while GPT-4-turbo, GPT-4o, DeepSeek, Claude, and the open-source baselines are evaluated zero-shot on a synthetic tool distribution they have never seen. The statement in Finding 2 that most models 'fail to select suitable tools' and the overall accuracy comparison in Table 2 therefore compare an in-distribution tuned model with out-of-distribution zero-shot models. To make the cross-model comparison informative, the authors should add a few-shot or in-context baseline that provides API models with a small set of PTool examples, or explicitly caveat that the advantage reflects in-distribution training rather than general superiority in personalized tool invocation.
  4. [Limitations] The limitations paragraph mentions only limited scenario coverage and limited task coverage. It does not acknowledge that the evaluation is carried out on a test set synthesized by the same pipeline that produced the training data, that no real user data or behavioral logs are used, and that no tool/scenario generalization test exists. Since the abstract claims that training on PTool improves 'personalized tool invocation capabilities' in general, the manuscript should either add an external validation signal, such as a semi-realistic or real-user evaluation, or explicitly state that the demonstrated gains are confined to the PTool distribution. The BFCL improvement in §5.5 is a useful external signal but does not measure personalization, so it cannot by itself establish real-world validity for the personalization claim.
minor comments (6)
  1. [§5.3] The 'All w/o History' bullet says 'The behavioral history is given,' but the variant name indicates that behavioral history is omitted; the text should be corrected to say 'omitted.'
  2. [Appendix A.1, Eq. (10)] The definition of untrained overall accuracy says it 'indicate the personalized tool selection ability on trained users'; 'trained' should be replaced with 'untrained.'
  3. [§5.5] 'CommonSenceQA' is a typo and should be 'CommonsenseQA.'
  4. [Tables 2, 4, 5] The 'Trained' and 'Untrained' overall columns are not defined in the captions; a note should indicate that these refer to the user split described in §5.1.
  5. [Figure 3] The vertical axis of the bar charts is not labeled, so the reader cannot determine the scale or the metric being reported.
  6. [§4.3 and Figure 7] The user agent is described as using basic features and implicit preferences, but the evaluation prompt in Figure 7 exposes only basic_features and user_history; please clarify whether implicit preferences are visible to the solver and whether this matches the actual evaluation setup.

Circularity Check

0 steps flagged · score 2.0 of 10

No material circularity: the only self-citation (ToolACE API-tree) is not load-bearing, and the benchmark's self-constructed nature is an external-validity limitation rather than a derivation-level circularity.

full rationale

The paper's central claim is empirical: PTool synthesizes training data, the authors SFT Qwen2.5-7B on it, and the trained model is scored on PTBench. PTBench is generated by the same PTool pipeline, but the test split is human-verified and disjoint from training queries and user IDs; evaluating on a held-out sample of the same synthetic distribution is standard benchmark practice and does not make the improvement true by construction. The untrained-user split keeps the same platforms, APIs, scenarios, and feature vocabulary, so it demonstrates in-distribution user generalization only; the paper never claims unseen-tool or real-user generalization, though the Limitations section could have stated this caveat. The BFCL non-live gain and the MMLU/GSM8K/HumanEval/CSQA checks provide an external signal. The only self-citation that touches methodology is 'Similar to ToolACE, we also developed a structure akin to an API Tree' (Section 4.1); this is a design inspiration, not a load-bearing theorem or uniqueness argument, and the paper's main results do not reduce to it. No equation in the paper turns a fitted parameter into a predicted quantity, and no uniqueness result is imported from the authors' prior work. Thus the residual concern is benchmark external validity, not circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 2 invented entities

The central claim rests on the quality of the synthetic data and the validity of the benchmark. The synthesis design counts are hand-chosen but not fitted to achieve a particular outcome. The main unproven inputs are the realism of GPT-4-turbo role-playing and the correctness of LLM-generated solutions, mitigated partially by manual review of the test set.

free parameters (3)
  • Synthesis design counts (5 scenarios, 3 platforms, 24 APIs) = 5/3/24
    Hand-selected scope of the synthetic universe; determines benchmark difficulty and coverage. Not fitted to data, but central to the benchmark design.
  • Dataset split ratio (6 untrained users, ~6% queries of remaining 74 users) = 80 total users, 1,083 test queries
    Arbitrary split used to create PTBench test set; affects generalization measurements.
  • LoRA training hyperparameters (rank 8, alpha 16, lr 1e-4, 1 epoch) = r=8, alpha=16, lr=1e-4, epoch=1
    Chosen for limited resources; not optimized. The central claim depends on performance with these settings.
assumptions (3)
  • domain assumption LLM role-playing with GPT-4-turbo can generate realistic user profiles, behavior histories, and queries.
    Sections 4.2 and 4.3 rely on GPT-4-turbo to simulate users; no real-world data is used to validate realism.
  • domain assumption Manually verified LLM-generated solutions are correct ground truth for personalized tool invocation.
    Section 4.3 states manual inspection of test set parameters, but no inter-annotator agreement or external validation is provided.
  • domain assumption Tool preferences are inferable from explicit basic features plus behavioral history without hidden psychological attributes.
    Section 4.2 deliberately excludes psychological attributes, which may limit the expressiveness of personalization.
invented entities (2)
  • PTBench benchmark independent evidence
    purpose: Evaluation dataset for personalized tool invocation, released publicly.
    Public GitHub repo allows external use and scrutiny, providing a falsifiable handle outside this paper.
  • PTool synthesis framework
    purpose: Generates personalized tool invocation training and evaluation data.
    Its effectiveness is measured on PTBench, which PTool itself generates, so no independent evidence within the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancing and Benchmarking Personalized Tool Invocation for LLMs." pith.science (2026). https://pith.science/paper/A4FWNLIW

@misc{pith2026250504072,
  author       = {Pith},
  title        = {Pith review of: Advancing and Benchmarking Personalized Tool Invocation for LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A4FWNLIW}},
  note         = {Machine review of arXiv:2505.04072}
}
read the original abstract

Tool invocation is a crucial mechanism for extending the capabilities of Large Language Models (LLMs) and has recently garnered significant attention. It enables LLMs to solve complex problems through tool calls while accessing up-to-date world knowledge. However, existing work primarily focuses on the fundamental ability of LLMs to invoke tools for problem-solving, without considering personalized constraints in tool invocation. In this work, we introduce the concept of Personalized Tool Invocation and define two key tasks: Tool Preference and Profile-dependent Query. Tool Preference addresses user preferences when selecting among functionally similar tools, while Profile-dependent Query considers cases where a user query lacks certain tool parameters, requiring the model to infer them from the user profile. To tackle these challenges, we propose PTool, a data synthesis framework designed for personalized tool invocation. Additionally, we construct \textbf{PTBench}, the first benchmark for evaluating personalized tool invocation. We then fine-tune various open-source models, demonstrating the effectiveness of our framework and providing valuable insights. Our benchmark is public at https://github.com/hyfshadow/PTBench.

Figures

Figures reproduced from arXiv: 2505.04072 by the authors.

Figure 1
Figure 1. Example of Personalized Tool Invocation. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Framework of our personalized tool invocation data synthesis framework: PTool. The pipeline comprises [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Ablation study on user profile in evaluation [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Error Analysis on PTBench. T-wrong, T-missing, and T-excessive represent wrong tools, missing tools and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Study of model scaling. The base models are [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: General Capabilities Analysis. Our model is [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Illustration of personalized tool invocation task. The blue part represents the expected output. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SPIEval: Evaluating Large Language Models as Mobile Assistants over Scattered Personal Information

    cs.CL 2026-08 conditional novelty 6.0 of 10

    A new human-curated benchmark for LLM mobile assistants shows models struggle to locate scattered personal information, with the best model at 57.3% accuracy.

  2. UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs

    cs.LG 2026-08 conditional novelty 6.0 of 10

    UserToolBench evaluates LLMs on personalized tool-use decisions with hidden user profiles and incomplete requests, and finds the best model reaches only 49.36% exact trajectory accuracy.

Reference graph

Works this paper leans on

54 extracted references · 4 canonical work pages · cited by 2 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Mahyar Abbasian, Zhongqi Yang, Elahe Khatibi, Pengfei Zhang, Nitish Nagesh, Iman Azimi, Ramesh Jain, and Amir M Rahmani. 2024. Knowledge-infused llm-powered conversational health agent: A case study for diabetes patients. arXiv preprint arXiv:2402.10153

    arXiv 2024

  4. [4]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  5. [5]

    Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1007--1014

    2023

  6. [6]

    Anastasiya Belyaeva, Justin Cosentino, Farhad Hormozdiari, Krish Eswaran, Shravya Shetty, Greg Corrado, Andrew Carroll, Cory Y McLean, and Nicholas A Furlotte. 2023. Multimodal llms for health grounded in individual-specific data. In Workshop on Machine Learning for Multimodal Healthcare Data, pages 86--102. Springer

    2023

  7. [7]

    Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, et al. 2024. When large language models meet personalization: Perspectives of challenges and opportunities. World Wide Web, 27(4):42

    2024

  8. [8]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, and et al. 2021. https://arxiv.org/abs/2107.03374 Evaluating large language models trained on code

    arXiv 2021

Show all 54 references
  1. [9]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    2021 arXiv

  2. [10]

    Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, et al. 2023. Educhat: A large-scale language model-based chatbot system for intelligent education. arXiv preprint arXiv:2308.02773

    2023 arXiv

  3. [11]

    DeepSeek-AI. 2024. https://arxiv.org/abs/2412.19437 Deepseek-v3 technical report . Preprint, arXiv:2412.19437

    2024 arXiv

  4. [12]

    DeepSeek-AI. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning . Preprint, arXiv:2501.12948

    2025 arXiv

  5. [13]

    Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2024. Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings. Advances in neural information processing systems, 36

    2024

  6. [14]

    Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021 a . Aligning ai with shared human values. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  7. [15]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 b . Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  8. [16]

    Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2024. Large language models are zero-shot rankers for recommender systems. In European Conference on Information Retrieval, pages 364--381. Springer

    2024

  9. [17]

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  10. [18]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  11. [19]

    Mingyu Jin, Qinkai Yu, Dong Shu, Chong Zhang, Lizhou Fan, Wenyue Hua, Suiyuan Zhu, Yanda Meng, Zhenting Wang, Mengnan Du, et al. 2024. Health-llm: Personalized retrieval-augmented disease prediction system. arXiv preprint arXiv:2402.00746

    2024 arXiv

  12. [20]

    u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \

    Enkelejda Kasneci, Kathrin Se ler, Stefan K \"u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \"u nnemann, Eyke H \"u llermeier, et al. 2023. Chatgpt for good? on opportunities and challenges of large language models for education....

    2023

  13. [21]

    Kausik Lakkaraju, Sai Krishna Revanth Vuruma, Vishal Pallagani, Bharath Muppasani, and Biplav Srivastava. 2023. Can llms be good financial advisors?: An initial study in personal decision making for optimized outcomes. arXiv preprint arXiv:2307.07422

    2023 arXiv

  14. [22]

    Yuxuan Lei, Jianxun Lian, Jing Yao, Xu Huang, Defu Lian, and Xing Xie. 2024. Recexplainer: Aligning large language models for explaining recommendation models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1530--1541

    2024

  15. [23]

    Ruyu Li, Wenhao Deng, Yu Cheng, Zheng Yuan, Jiaqi Zhang, and Fajie Yuan. 2023. Exploring the upper limits of text-based collaborative filtering using large language models: Discoveries and insights. arXiv preprint arXiv:2305.11700

    2023

  16. [24]

    Qiqiang Lin, Muning Wen, Qiuying Peng, Guanyu Nie, Junwei Liao, Jun Wang, Xiaoyun Mo, Jiamu Zhou, Cheng Cheng, Yin Zhao, Jun Wang, and Weinan Zhang. 2024. https://arxiv.org/abs/2410.04587 Hammer: Robust function-calling for on-device language models via function masking . Prep...

    2024

  17. [25]

    Weiwen Liu, Xingshan Zeng, Xu Huang, xinlong hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong WANG, Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Wang Xinzhi, Yong Liu, Yasheng Wang, Duyu Tang, Dandan Tu, Lifeng Shang, Xin Jiang, Rui...

    2025

  18. [26]

    Xiao-Yang Liu, Guoxuan Wang, Hongyang Yang, and Daochen Zha. 2023. Fingpt: Democratizing internet-scale data for financial large language models. arXiv preprint arXiv:2307.10485

    2023 arXiv

  19. [27]

    Controlllm: augment language models with tools by searching on graphs (2023)

    Z Liu, Z Lai, Z Gao, E Cui, Z Li, X Zhu, L Lu, Q Chen, Y Qiao, J Dai, et al. Controlllm: augment language models with tools by searching on graphs (2023). arXiv preprint arXiv:2310.17796

    2023 arXiv

  20. [28]

    Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, et al. 2024. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. arXiv preprint arXiv:2406.18518

    2024 arXiv

  21. [29]

    Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, and Jiebo Luo. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.39 LLM -rec: Personalized recommendation via prompting large language models . In Findings of the ...

    2024 doi

  22. [30]

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332

    2021 arXiv

  23. [31]

    Ha-Thanh Nguyen. 2023. A brief report on lawgpt 1.0: A virtual legal assistant based on gpt-3. arXiv preprint arXiv:2302.05729

    2023 arXiv

  24. [32]

    Minju Park, Sojung Kim, Seunghyun Lee, Soonwoo Kwon, and Kyuseok Kim. 2024. Empowering personalized learning through a conversation-based tutoring system with student modeling. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1--10

    2024

  25. [33]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, dahai li, Zhiyuan Liu, and Maosong Sun. 2024. https://openreview.net/forum?id=dHng2O0Jjr...

    2024

  26. [34]

    Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2025. Tool learning with large language models: A survey. Frontiers of Computer Science, 19(8):198343

    2025

  27. [35]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2024. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36

    2024

  28. [36]

    Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, and Zhaochun Ren. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.624 Learning to use tools via cooperative and interactive agents . In Findings of the Associ...

    2024 doi

  29. [37]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. https://doi.org/10.18653/v1/N19-1421 C ommonsense QA : A question answering challenge targeting commonsense knowledge . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associa...

    2019 doi

  30. [38]

    Qwen Team. 2024 a . Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  31. [39]

    Qwen Team. 2024 b . https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  32. [40]

    Lei Wang, Songheng Zhang, Yun Wang, Ee-Peng Lim, and Yong Wang. 2023. https://doi.org/10.18653/v1/2023.emnlp-industry.64 LLM 4 V is: Explainable visualization recommendation using C hat GPT . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proces...

    2023 doi

  33. [41]

    Wei Wei, Xubin Ren, Jiabin Tang, Qinyong Wang, Lixin Su, Suqi Cheng, Junfeng Wang, Dawei Yin, and Chao Huang. 2024. Llmrec: Large language models with graph augmentation for recommendation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, ...

    2024

  34. [42]

    Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, Hui Xiong, and Enhong Chen. 2023. A survey on large language models for recommendation. CoRR, abs/2305.19860

    2023 arXiv

  35. [43]

    Yunjia Xi, Weiwen Liu, Jianghao Lin, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Weinan Zhang, and Yong Yu. 2024. Towards open-world recommendation with knowledge augmentation from large language models. In Proceedings of the 18th ACM Conference on Recommender ...

    2024

  36. [44]

    Yang Xu, Yunlong Feng, Honglin Mu, Yutai Hou, Yitong Li, Xinghao Wang, Wanjun Zhong, Zhongyang Li, Dandan Tu, Qingfu Zhu, Min Zhang, and Wanxiang Che. 2024. https://doi.org/10.18653/v1/2024.findings-acl.974 Concise and precise context compression for tool-using language models...

    2024 doi

  37. [45]

    Patil, Ion Stoica, and Joseph E

    Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2024. Berkeley function calling leaderboard. https://gorilla.cs.berkeley.edu/blogs/8_berkeley_function_calling_leaderboard.html

    2024

  38. [46]

    Fan Yang, Zheng Chen, Ziyan Jiang, Eunah Cho, Xiaojiang Huang, and Yanbin Lu. 2023 a . Palr: Personalization aware llms for recommendation. arXiv preprint arXiv:2305.07622

    2023 arXiv

  39. [47]

    Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. 2023 b . https://openreview.net/forum?id=cwjh8lqmOL GPT 4tools: Teaching large language model to use tools via self-instruction . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  40. [48]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629

    2022 arXiv

  41. [49]

    Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Ren Kan, Dongsheng Li, and Deqing Yang. 2024. Easytool: Enhancing llm-based agents with concise tool instruction. arXiv preprint arXiv:2401.06201

    2024 arXiv

  42. [50]

    Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Juntao Tan, Thai Hoang, Liangwei Yang, Yihao Feng, Zuxin Liu, et al. 2024 a . Agentohana: Design unified data and training pipeline for effective agent learning. arXiv preprint arXiv:2402.15506

    2024 arXiv

  43. [51]

    Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, et al. 2024 b . xlam: A family of large action models to empower ai agent systems. arXiv preprint arXiv:2409.03215

    2024 arXiv

  44. [52]

    Recommendation as instruction following: A large language model empowered recommendation approach

    Junjie Zhang, Ruobing Xie, Yupeng Hou, Xin Zhao, Leyu Lin, and Ji-Rong Wen. Recommendation as instruction following: A large language model empowered recommendation approach. ACM Transactions on Information Systems

  45. [53]

    Zhehao Zhang, Ryan A Rossi, Branislav Kveton, Yijia Shao, Diyi Yang, Hamed Zamani, Franck Dernoncourt, Joe Barrow, Tong Yu, Sungchul Kim, et al. 2024 c . Personalization of large language models: A survey. arXiv preprint arXiv:2411.00027

    2024 arXiv

  46. [54]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.