Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Doc2Agent claims that raw REST API docs can be automatically converted into validated Python tools, and that agents using those tools outperform direct API calling by 55.1% relative at roughly one-tenth the cost per task.

desk verdict A real engineering pipeline for turning unstructured API docs into validated Python tools, but the headline WebArena gain compares across two different evaluators and is not established as stated. read the letter →

arxiv 2506.19998 v1 pith:YAPFLXX7 submitted 2025-06-24 cs.CL

Xinyi Ni , Haonan Jian , Qiuyang Wang , Vedanshi Chetan Shah , Pengyu Hong This is my paper
classification cs.CL
keywords APIdocumentationtool-usingagentsRESTAPIsautomatedtoolgenerationcodeparametervalueinferencevalidationlargelanguagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Doc2Agent aims to remove the manual effort that currently stands between real-world REST API documentation and a deployed tool-using agent. The paper's central claim is that an automated loop, generating Python tools from free-text docs, testing them against live servers, having a code agent fix failures, and re-validating, can produce reliable tools from documentation that is unstructured, incomplete, or outdated. On a five-site web-agent benchmark, agents equipped with the generated tools improved average success rates by 55.1% relative to a direct-API-calling baseline while spending about 10% of the cost per task. Across a 167-page real-world API corpus with 744 endpoints, 59.5% of the endpoints yielded tools that passed automated validation. If these numbers hold, the main obstacle to scaling tool agents becomes having the documentation itself, not hand-curating uniform tool sets.

What carries the argument

The load-bearing mechanism is a generate-validate-refine-deploy loop. Generation converts unstructured HTML or Markdown documentation into structured JSON via LLM extraction, then into Python functions that wrap HTTP calls; a target-oriented variant first writes simplified function 'fingerprints' that define a use case and expected inputs for flexible, search-like endpoints. Validation executes each function against the live server and asks an LLM to classify the response as valid information, a code error, a server error, or a request error, with verification requiring alignment between the actual response and the predicted expectation. Refinement feeds failed tools and their error messages to a code agent, which rewrites the function and is re-validated. The distinctive sub-mechanism is an automatically built parameter database: JSON responses from already-verified tools are stored in a vector database alongside parameter names and descriptions, and for an unknown parameter the system retrieves semantically similar key-value pairs from other APIs, using implicit dependencies between services to infer parameter values without domain expertise. Deployment exports the verified tools through a standardized tool-serving protocol or as an OpenAPI specification, making them usable across agent architectures.

What would settle it

Obtain the direct-API-calling baseline's action logs for the same five web-task sites, run them through the API-specified evaluator used for Doc2Agent, and check whether the average stays near 29.2; if it rises to the level of the hybrid agent, about 42.2, the reported 55.1% relative improvement is largely an artifact of the evaluation change. A second check: apply the pipeline to a fresh random sample of API documentation and measure whether the verified-tool rate still approaches 59.5% without human intervention.

Watch

Extended reading notes

Core claim

The paper's central claim is that wrapping REST APIs as validated Python functions changes the agent's job from low-level request construction to high-level task logic, and that this shift is what drives the gains. The pipeline first transforms each endpoint's documentation into a structured specification and then into a callable function, using direct wrapping for simple docs and task-oriented generation for flexible endpoints. Every generated function is executed against the real API, and an LLM judge compares the actual response with the response predicted from the tool's description, so a tool is verified only when behavior matches expectation. Failed functions are handed to a code agent together with the documentation, error information, and candidate parameter values, and the loop repeats for up to three refinement rounds. The paper reports 443 verified tools from 744 real-world endpoints, a 47.6% boost in tool pass rate from refinement, and 70 refined tools for glycoscience research APIs. On the web-task benchmark, its tool-using agent reached an average 45.3% success, a 55.1% relative gain over the published direct-API baseline, at $0.12 per task versus $1.20.

Load-bearing premise

The central result collapses if the prior direct-API-calling agent's published scores under the original evaluator cannot be compared with Doc2Agent's scores under the stricter API-specific evaluator; the paper could not re-score the baseline because its logs were unavailable.

Editorial extensions

If this is right

  • If the claim holds, developers no longer need to hand-craft uniform tool sets before an agent can use a new service; messy documentation becomes an acceptable starting point.
  • The order-of-magnitude cost reduction makes it economical for agents to call many tools repeatedly in multi-step tasks, widening the range of tasks that are feasible to automate.
  • Because every tool is exercised against a live server during validation, broken or outdated documentation is caught at build time rather than at agent runtime.
  • The cross-API parameter inference means tools can become usable even when no human expert supplies example inputs, which is the situation for most long-tail and research APIs.
  • Verified tools exported through a standard deployment protocol should be reusable across different agent frameworks, so tool-building effort does not have to be repeated per system.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to turn the validation loop into a self-improving production system, where every successful agent call seeds the parameter database, so the toolset improves with use instead of only during the initial build.
  • The paper's reported 95.6% relative gain on the least-organized site suggests the main failure mode of direct API calling is missing parameter knowledge rather than model reasoning; ablating the parameter database would test that attribution directly.
  • The paper's discussion of truncated, verbose JSON responses points to an obvious next design: generate response-filtering or summarization wrappers as part of each tool, so agents receive task-sized payloads instead of raw server output.
  • If docs-to-tools conversion becomes reliable, benchmark construction could be inverted: future benchmarks could sample real API documentation pages and let a pipeline like this one build the tools, matching real-world documentation quality instead of hand-curated schemas.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Doc2Agent, a pipeline that takes unstructured REST API documentation and automatically generates, validates, refines, and deploys Python-based tools for LLM agents. The pipeline combines direct, JSON-schema-based tool generation with target-oriented tool generation, an LLM-based validation loop, iterative refinement by a code agent, and a parameter-value database built from previously validated API responses. The authors evaluate Doc2Agent on 167 real-world API documentation pages (744 endpoints), WebArena, and glycoscience research APIs. They report 443 validated real-world tools, a 55.1% relative improvement over direct API calling on WebArena at 10% of the cost, and a domain-specific glycomaterial research agent built entirely from automatically generated tools.

Significance. If the quantitative claims held, Doc2Agent would address a real bottleneck in tool-agent deployment: unstructured, heterogeneous API documentation and missing parameter values. The pipeline design is coherent, the authors release code, and the paper is candid about several limitations, including stateful-API validation, attempts by the code agent to bypass validation, and the lack of a suitable benchmark for open-domain API usage. However, the central WebArena improvement claim is not supported as stated because it compares results under two different evaluation protocols, and the research-agent result relies on a filtered, LLM-judged evaluation. These issues currently reduce the contribution to a promising but unverified pipeline.

major comments (5)
  1. [§4.2, Table 2] The headline 55.1% relative improvement is computed across two different evaluation protocols. The Δ row compares Doc2Agent's average of 45.3 under the new 'API-Specified Evaluation' with the API-based agent's published average of 29.2 under the vanilla WebArena evaluator. Section 4.2 states that the prior agent could not be re-evaluated because its logs are unavailable, and that the new evaluator was applied only to Doc2Agent and the hybrid agent. Appendix E.1 shows that the two protocols can disagree per task in both directions (Task 22 is a false positive under vanilla, while Task 395 requires functional-equivalence reasoning to be counted as a success), so the aggregate difference cannot be attributed to the tool-generation pipeline. To support the improvement claim, the direct-API baseline must be run under the identical API-specified evaluator; otherwise the comparison should be reported only as a cross-protocol observation, and the 55.1% claim should be removed from the abstract and Section 1.
  2. [§4.2, Table 2] No measure of variability is reported for any WebArena cell. Each score appears to be a single run or an aggregation with unspecified replication, and no error bars, standard deviations, or significance tests are provided. Given the small per-site differences in some rows, the claimed gains (e.g., 95.6% on CMS) cannot be distinguished from run-to-run noise. The authors should report multiple independent runs with confidence intervals or explicitly state the number of runs underlying each entry in Table 2.
  3. [§2.2, Table 1] The tool-validation protocol uses an LLM-generated expected response to decide whether an API response is valid, yet Section 2.2 asserts 'strong agreement with human judgment' without providing the human study, sample size, or agreement measure. Because the same model families are used for tool generation and for validation, the pass rates in Table 1 may partly reflect self-consistency in the LLM's expectations rather than genuine API correctness. The authors should add a human-annotated validation sample with reported agreement statistics or use an independent judge before treating the Table 1 pass rates as verified.
  4. [§4.3, Table 3] The glycoscience agent evaluation is not yet a reliable demonstration of domain adaptability. The 50 tasks are generated by GPT-4o from the tool descriptions, the success judgments are made by an LLM judge, and the reported percentages are computed over a 'Filtered' set defined as the union of tasks successfully completed by at least one agent. Constructing the denominator from the systems under test inflates relative rankings and makes the absolute success rates impossible to interpret. The authors should report unfiltered success rates, release the task set, and use independently verified or pre-specified ground-truth answers.
  5. [§1, Table 1] The claim that Doc2Agent generates verified tools for 59.5% of real-world APIs is a denominator error. Table 1 lists 443 validated tools from 744 real-world endpoints, which is 59.5% of endpoints, not of the 167 collected API documentation pages; the same ambiguity affects the 81.5% research-API figure, which is not derivable from the 70/131 entries in Table 1. The abstract and Section 1 should state that the rate is per endpoint, or provide the correct per-documentation calculation.
minor comments (6)
  1. [§1] Typo: 'programmic' should be 'programmatic' in the first paragraph.
  2. [§2.1] Typo: 'circumstanses' should be 'circumstances' in the sentence about unexpected situations during automatic testing.
  3. [§5] Typo: 'argentic services' should be 'agentic services' in the Related Works section.
  4. [§4.2] The cost comparison ($0.12 vs. $1.20 per task) does not state which costs are included; clarify whether it covers only inference tokens and whether tool generation and validation overhead are excluded.
  5. [Table 3] Table 3's header is ambiguous: it is unclear whether the numeric columns after 'Filtered' are per-cell success rates or per-row totals; add explicit column headers and units.
  6. [Appendix F] The 'Tool Generation Example' text refers to the Pokemon TCG API, but the displayed code is the OSRM general_request function; align the example label with the code shown.

Circularity Check

3 steps flagged · score 4.0 of 10

Tool generation is externally grounded by live API calls and WebArena, but the 'validated tools' counts, the parameter-inference loop, and the glycan-agent success rates are defined through LLM self-agreement and self-seeded data, giving partial circularity.

  1. self definitional [Section 2.2 (Tool Validation)]
    "To validate a tool, we call it using the provided example inputs and compare the actual API response against an expected output generated by a language model, conditioned on the tool’s description. A tool is considered verified if the response aligns with the model-predicted expectation."

    The ground truth for 'verified' is not an independent oracle but an expectation generated by an LLM from the tool's own description, and the tools and descriptions are produced by the same LLM family (GPT-4o and Claude 3.7 Sonnet). The reported validated-tool rates (59.5% real-world, 81.5% research) therefore measure model-to-model agreement as much as API correctness. Live HTTP responses provide some external signal, and human agreement is asserted but not quantified, so the circularity is partial. Section 7's admission that the code agent 'cheated' by generating try-catch blocks to bypass validation shows the loop is gameable by the entity being validated.

  2. fitted input called prediction [Section 2.3 (Tool Refinement, Parameter Value Inference)]
    "We leverage two primary sources of information: (1) parameter examples from other API documentations, particularly from the same domain, and (2) JSON responses from previously validated API tools, which contain rich domain-specific key-value pairs. ... When a new parameter value is used, it is recorded as an example and added to the parameter database."

    The parameter values used to rescue failing tools are mined from tools whose 'validated' status was itself established by the LLM-expectation validator of Section 2.2, and every newly accepted value is written back into the same database. The leave-one-API-out 'Pass@10' comparison (33 vs 17) is then scored by an LLM-as-a-judge on whether the tool call 'executed correctly.' The inferred values are fitted to the pipeline's own self-validated outputs rather than to an independent parameter ground truth; actual JSON responses and HTTP status codes break full circularity, but the pass/fail criterion remains model-generated.

1 more flagged steps
  1. other [Section 4.3 (Task Generation and Results)]
    "We prompted GPT-4o to generate 50 research-oriented tasks based on the tool descriptions ... we used an LLM-as-a-judge approach (see template in Appendix H.3) to estimate task success. Since not all generated tasks are guaranteed to be solvable, we report success rates over the Filtered set using the union of all tasks successfully completed by at least one agent to provide a fair basis for relative performance comparison."

    The research-agent tasks are generated from the very tool descriptions Doc2Agent produced, success is estimated by an LLM judge, and the denominator is restricted to tasks that at least one agent solved. The reported success rates (e.g., Claude 36.0% overall, 58.1% filtered) are therefore internal consistency scores among the pipeline's own descriptions, the agents' trajectories, and the LLM judge, not independent benchmark outcomes. This affects the adaptability claim for glycomaterial science, while WebArena remains an external benchmark.

full rationale

Doc2Agent's core derivation is not circular by construction: tools are generated from raw HTML/Markdown API docs, executed against live REST endpoints, and refined based on HTTP responses, and the WebArena runs use an external task environment. Those external signals keep the pipeline from being equivalent to its inputs. However, three load-bearing evaluation claims are partially self-referential: (1) 'verified' is defined as agreement between an API response and an LLM-generated expectation conditioned on the tool description, while the same LLM family creates the tools and descriptions, making the validated-tool counts partly measures of model self-consistency; (2) parameter values are inferred from previously self-validated tools and written back into the same database, so the parameter-inference success rates are a closed loop around the LLM validator rather than an independent test; (3) the glycan research-agent tasks are generated from the pipeline's own tool descriptions and judged by an LLM with a denominator filtered to tasks some agent solved, so those success rates are internal. The WebArena headline comparison also mixes two evaluators (Doc2Agent scored with the new API-specified evaluator versus the prior agent's published vanilla-evaluator score), but under the hard rules that is an evaluation-protocol confound, not a definitional circularity, so it does not itself raise the circularity score. Overall: partial circularity in the self-validation and research-agent evaluations, with independent external content in the WebArena and live-API components; score 4.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The pipeline relies on hand-chosen limits (three refinement rounds, ten candidate values, ten tools per API) rather than fitted parameters. The key assumption is that LLM judges can reliably tell valid API responses from errors, since both validation pass rates and final task scores use these judges. No new physical or conceptual entities are postulated; the parameter database and MCP server are engineering components built from existing tools.

free parameters (3)
  • refinement rounds = 3
    The pipeline stops after three refinement rounds per tool; reported pass rates depend on this choice (Section 4.1).
  • candidate parameter values sampled = 10
    Up to 10 candidate values are retrieved from the vector database for parameter inference (Section 2.3).
  • target-oriented tools per API = 10
    Up to 10 task-specific tools are generated per API document in target-oriented mode (Section 3.2).
assumptions (4)
  • domain assumption LLM judges (GPT-4o, Claude) can reliably classify API responses as valid information versus errors
    Used throughout validation, refinement, and final evaluation (Section 2.2, Appendix H.1). If the judge is lenient, tool pass rates and task success rates are inflated.
  • domain assumption REST APIs can be invoked without authentication for the selected documentation set
    Selection criteria filter for APIs that do not require API keys (Appendix A.1), limiting generality to unauthenticated APIs.
  • domain assumption Python function wrappers are sufficient encapsulation for real-world APIs
    The pipeline does not handle stateful API workflows; Section 7 admits validation is insufficient for stateful APIs.
  • domain assumption API documentation pages collected from APIList.com are representative of real-world API quality
    The dataset is self-curated, and no external benchmark or inter-annotator agreement is reported (Appendix A).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation." pith.science (2026). https://pith.science/paper/YAPFLXX7

@misc{pith2026250619998,
  author       = {Pith},
  title        = {Pith review of: Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YAPFLXX7}},
  note         = {Machine review of arXiv:2506.19998}
}
read the original abstract

REST APIs play important roles in enriching the action space of web agents, yet most API-based agents rely on curated and uniform toolsets that do not reflect the complexity of real-world APIs. Building tool-using agents for arbitrary domains remains a major challenge, as it requires reading unstructured API documentation, testing APIs and inferring correct parameters. We propose Doc2Agent, a scalable pipeline to build agents that can call Python-based tools generated from API documentation. Doc2Agent generates executable tools from API documentations and iteratively refines them using a code agent. We evaluate our approach on real-world APIs, WebArena APIs, and research APIs, producing validated tools. We achieved a 55\% relative performance improvement with 90\% lower cost compared to direct API calling on WebArena benchmark. A domain-specific agent built for glycomaterial science further demonstrates the pipeline's adaptability to complex, knowledge-rich tasks. Doc2Agent offers a generalizable solution for building tool agents from unstructured API documentation at scale.

Figures

Figures reproduced from arXiv: 2506.19998 by the authors.

Figure 1
Figure 1. Bridging the Gap Between Real-World APIs and AI Agents (Top) Conventional agent development relies on high-quality APIs and manual integration, leading to scalability issues. (Bottom) Our approach automates tool generation from natural language API documentation, enabling self-validation, refinement, and seamless deployment of AI agents. We argue that it is essential to develop an automated pipeline for scalable age… view at source ↗
Figure 2
Figure 2. Comparison of function-based API using and direct API calling Existing API-calling approaches (Song et al., 2025b) allow LLMs to read API documen￾tation and generate URLs to invoke REST APIs directly. However, this method is cumbersome: lengthy documentation con￾sumes context window space, wastes to￾kens, and can cause the model to lose focus in multi-turn interactions (Liu et al., 2023; Laban et al., 2025). To addr… view at source ↗
Figure 3
Figure 3. Overview of Doc2Agent: Automated pipeline for generating AI agents from API [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: A parameter database is automatically con [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Making OpenAPI Documentation Agent-Ready: Detecting Documentation and REST Smells with a Multi-Agent LLM System

    cs.SE 2026-05 unverdicted novelty 5.0 of 10

    Hermes uses multi-agent LLMs to detect 2450 documentation and REST smells across 600 OpenAPI endpoints, demonstrating that structurally valid microservice APIs are often not semantically ready for agent consumption.

  2. OpenClaw and Ollama in Agentic AI: Toward Fully Autonomous and Scalable AI Agent Systems

    cs.AI 2026-04 reject novelty 3.0 of 10

    A review-plus-demo claiming agentic capabilities emerge from system integration, backed by a 15-task benchmark whose C1→C3 performance gap is largely built into the test design.

Reference graph

Works this paper leans on

69 extracted references · 25 canonical work pages · cited by 2 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  4. [4]

    Rahmani, and Ramesh Jain

    Mahyar Abbasian, Iman Azimi, Amir M. Rahmani, and Ramesh Jain. Conversational health agents: A personalized llm-powered agent framework, 2024. URL https://arxiv.org/abs/2310.02374

    arXiv 2024

  5. [5]

    fastapi mcp

    Shahar Abramov. fastapi mcp. https://mem0.ai/openmemory-mcp, 2025

    work page 2025

  6. [6]

    Glyconnect: glycoproteomics goes visual, interactive, and analytical

    Davide Alocci, Julien Mariethoz, Alessandra Gastaldello, Elisabeth Gasteiger, Niclas G Karlsson, Daniel Kolarich, Nicolle H Packer, and Fr \'e d \'e rique Lisacek. Glyconnect: glycoproteomics goes visual, interactive, and analytical. Journal of proteome research, 18 0 (2): 0 664--677, 2018

    work page 2018

  7. [7]

    Automate logins with browser bots

    axiom.ai. Automate logins with browser bots. https://axiom.ai/automate/login, 2024. Accessed: 2024-10-02

    work page 2024

  8. [8]

    Researchagent: Iterative research idea generation over scientific literature with large language models, 2024

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Researchagent: Iterative research idea generation over scientific literature with large language models, 2024. URL https://arxiv.org/abs/2404.07738

    arXiv 2024

Show all 69 references
  1. [9]

    Web services, service-oriented architectures, and cloud computing

    Douglas K Barry. Web services, service-oriented architectures, and cloud computing. Elsevier, 2003

    2003

  2. [10]

    Chemcrow: Augmenting large-language models with chemistry tools, 2023

    Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Chemcrow: Augmenting large-language models with chemistry tools, 2023. URL https://arxiv.org/abs/2304.05376

    2023 arXiv

  3. [11]

    On microservice analysis and architecture evolution: A systematic mapping study

    Vincent Bushong, Amr S Abdelfattah, Abdullah A Maruf, Dipta Das, Austin Lehman, Eric Jaroszewski, Michael Coffey, Tomas Cerny, Karel Frajtak, Pavel Tisnovsky, et al. On microservice analysis and architecture evolution: A systematic mapping study. Applied Sciences, 11 0 (17): 0...

    2021

  4. [12]

    Karlsson, Jie Fu, and Yemin Shi

    Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F. Karlsson, Jie Fu, and Yemin Shi. Autoagents: A framework for automatic agent generation, 2024. URL https://arxiv.org/abs/2309.17288

    2024 arXiv

  5. [13]

    Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, and Huan Sun

    Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, Vishal Dey, Mingyi Xue, Frazier N. Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, and Huan Sun. Scienceagentbench: Toward ...

    2025 arXiv

  6. [14]

    Structured information extraction from scientific text with large language models

    John Dagdelen, Alexander Dunn, Sanghoon Lee, Nicholas Walker, Andrew S Rosen, Gerbrand Ceder, Kristin A Persson, and Anubhav Jain. Structured information extraction from scientific text with large language models. Nature Communications, 15 0 (1): 0 1418, 2024

    2024

  7. [15]

    Nmr chemical shift prediction and structural elucidation of linker-containing oligo-and polysaccharides using the computer program casper

    Kevin M Dorst and G \"o ran Widmalm. Nmr chemical shift prediction and structural elucidation of linker-containing oligo-and polysaccharides using the computer program casper. Carbohydrate research, 533: 0 108937, 2023

    2023

  8. [16]

    Yiming Du, Wenyu Huang, Danna Zheng, Zhaowei Wang, Sebastien Montella, Mirella Lapata, Kam-Fai Wong, and Jeff Z. Pan. Rethinking memory in ai: Taxonomy, operations, topics, and future directions, 2025. URL https://arxiv.org/abs/2505.00675

    2025

  9. [17]

    Anytool: Self-reflective, hierarchical agents for large-scale api calls, 2024

    Yu Du, Fangyun Wei, and Hongyang Zhang. Anytool: Self-reflective, hierarchical agents for large-scale api calls, 2024. URL https://arxiv.org/abs/2402.04253

    2024 arXiv

  10. [18]

    Retool: Reinforcement learning for strategic tool use in llms, 2025

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms, 2025. URL https://arxiv.org/abs/2504.11536

    2025 arXiv

  11. [19]

    From llm reasoning to autonomous ai agents: A comprehensive review, 2025

    Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah. From llm reasoning to autonomous ai agents: A comprehensive review, 2025. URL https://arxiv.org/abs/2504.19678

    2025 arXiv

  12. [20]

    Introducing structured outputs in the api

    OpenAI GPTs. Introducing structured outputs in the api. https://openai.com/index/introducing-structured-outputs-in-the-api/, 2024. Accessed: 2025-5-01

    2024

  13. [21]

    From rag to memory: Non-parametric continual learning for large language models, 2025

    Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. From rag to memory: Non-parametric continual learning for large language models, 2025. URL https://arxiv.org/abs/2502.14802

    2025 arXiv

  14. [22]

    A comprehensive survey on vector database: Storage and retrieval technique, challenge, 2023

    Yikun Han, Chunjiang Liu, and Pengfei Wang. A comprehensive survey on vector database: Storage and retrieval technique, challenge, 2023. URL https://arxiv.org/abs/2310.11703

    2023

  15. [23]

    Kegg as a glycome informatics resource

    Kosuke Hashimoto, Susumu Goto, Shin Kawano, Kiyoko F Aoki-Kinoshita, Nobuhisa Ueda, Masami Hamajima, Toshisuke Kawasaki, and Minoru Kanehisa. Kegg as a glycome informatics resource. Glycobiology, 16 0 (5): 0 63R--70R, 2006

    2006

  16. [24]

    Webvoyager: Building an end-to-end web agent with large multimodal models, 2024

    Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. Webvoyager: Building an end-to-end web agent with large multimodal models, 2024. URL https://arxiv.org/abs/2401.13919

    2024 arXiv

  17. [25]

    Model context protocol (mcp): Landscape, security threats, and future research directions, 2025

    Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. Model context protocol (mcp): Landscape, security threats, and future research directions, 2025. URL https://arxiv.org/abs/2503.23278

    2025 arXiv

  18. [26]

    Unilectin, a one-stop-shop to explore and study carbohydrate-binding proteins

    Anne Imberty, Fran c ois Bonnardel, and Fr \'e d \'e rique Lisacek. Unilectin, a one-stop-shop to explore and study carbohydrate-binding proteins. Current Protocols, 1 0 (11): 0 e305, 2021

    2021

  19. [27]

    Pubchem substance and compound databases

    Sunghwan Kim, Paul A Thiessen, Evan E Bolton, Jie Chen, Gang Fu, Asta Gindulyte, Lianyi Han, Jane He, Siqian He, Benjamin A Shoemaker, et al. Pubchem substance and compound databases. Nucleic acids research, 44 0 (D1): 0 D1202--D1213, 2016

    2016

  20. [28]

    Large language models are zero-shot reasoners, 2023

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2023. URL https://arxiv.org/abs/2205.11916

    2023 arXiv

  21. [29]

    Llms get lost in multi-turn conversation, 2025

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. Llms get lost in multi-turn conversation, 2025. URL https://arxiv.org/abs/2505.06120

    2025 arXiv

  22. [30]

    Microservice api evolution in practice: A study on strategies and challenges

    Alexander Lercher, Johann Glock, Christian Macho, and Martin Pinzger. Microservice api evolution in practice: A study on strategies and challenges. Journal of Systems and Software, 215: 0 112110, September 2024. ISSN 0164-1212. doi:10.1016/j.jss.2024.112110. URL http://dx.doi....

    2024

  23. [31]

    St-webagentbench: A benchmark for evaluating safety and trustworthiness in web agents, 2024

    Ido Levy, Ben Wiesel, Sami Marreed, Alon Oved, Avi Yaeli, and Segev Shlomov. St-webagentbench: A benchmark for evaluating safety and trustworthiness in web agents, 2024. URL https://arxiv.org/abs/2410.06703

    2024 arXiv

  24. [32]

    Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance, 2023

    Yang Li, Yangyang Yu, Haohang Li, Zhi Chen, and Khaldoun Khashanah. Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance, 2023. URL https://arxiv.org/abs/2309.03736

    2023 arXiv

  25. [33]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts, 2023. URL https://arxiv.org/abs/2307.03172

    2023 arXiv

  26. [34]

    The hidden dangers of browsing ai agents, 2025

    Mykyta Mudryi, Markiyan Chaklosh, and Grzegorz Wójcik. The hidden dangers of browsing ai agents, 2025. URL https://arxiv.org/abs/2505.13076

    2025 arXiv

  27. [35]

    The Proteins API: accessing key integrated protein and genome information

    Andrew Nightingale, Ricardo Antunes, Emanuele Alpi, Borisas Bursteinas, Leonardo Gonzales, Wudong Liu, Jie Luo, Guoying Qi, Edd Turner, and Maria Martin. The Proteins API: accessing key integrated protein and genome information . Nucleic Acids Research, 45 0 (W1): 0 W539--W544...

    2017 doi

  28. [36]

    Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to m...

    2023 arXiv

  29. [37]

    Tool learning with large language models: A survey

    Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. Tool learning with large language models: A survey. Frontiers of Computer Science, 19 0 (8): 0 198343, 2025

    2025

  30. [38]

    Qwen agent github page

    QwenLM. Qwen agent github page. https://github.com/QwenLM/Qwen-Agent, 2025

    2025

  31. [39]

    Rapidapi hub

    RapidAPI. Rapidapi hub. https://rapidapi.com/hub, 2024. Accessed: 2024-12-31

    2024

  32. [40]

    Rcsb protein data bank: architectural advances towards integrated searching and efficient access to macromolecular structure data from the pdb archive

    Yana Rose, Jose M Duarte, Robert Lowe, Joan Segura, Chunxiao Bi, Charmi Bhikadiya, Li Chen, Alexander S Rose, Sebastian Bittrich, Stephen K Burley, et al. Rcsb protein data bank: architectural advances towards integrated searching and efficient access to macromolecular structu...

    2021

  33. [41]

    Database resources of the national center for biotechnology information

    Eric W Sayers, Jeffrey Beck, Evan E Bolton, Devon Bourexis, James R Brister, Kathi Canese, Donald C Comeau, Kathryn Funk, Sunghwan Kim, William Klimke, et al. Database resources of the national center for biotechnology information. Nucleic acids research, 49 0 (D1): 0 D10--D17, 2021

    2021

  34. [42]

    Tool learning in the wild: Empowering language models as automatic tool agents, 2025

    Zhengliang Shi, Shen Gao, Lingyong Yan, Yue Feng, Xiuyi Chen, Zhumin Chen, Dawei Yin, Suzan Verberne, and Zhaochun Ren. Tool learning in the wild: Empowering language models as automatic tool agents, 2025. URL https://arxiv.org/abs/2405.16533

    2025 arXiv

  35. [43]

    Paloma Sodhi, S. R. K. Branavan, Yoav Artzi, and Ryan McDonald. Step: Stacked llm policies for web actions, 2024. URL https://arxiv.org/abs/2310.03720

    2024 arXiv

  36. [44]

    Bissyandé, Andrey Boytsov, Ulrick Ble, and Anne Goujon

    Yewei Song, Xunzhu Tang, Cedric Lothritz, Saad Ezzini, Jacques Klein, Tegawendé F. Bissyandé, Andrey Boytsov, Ulrick Ble, and Anne Goujon. Callnavi, a challenge and empirical study on llm function calling and routing, 2025 a . URL https://arxiv.org/abs/2501.05255

    2025 arXiv

  37. [45]

    Restgpt: Connecting large language models with real-world restful apis, 2023

    Yifan Song, Weimin Xiong, Dawei Zhu, Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, Cheng Li, Ke Wang, Rong Yao, Ye Tian, and Sujian Li. Restgpt: Connecting large language models with real-world restful apis, 2023. URL https://arxiv.org/abs/2306.06624

    2023 arXiv

  38. [46]

    Beyond browsing: Api-based web agents, 2025 b

    Yueqi Song, Frank Xu, Shuyan Zhou, and Graham Neubig. Beyond browsing: Api-based web agents, 2025 b . URL https://arxiv.org/abs/2410.16464

    2025 arXiv

  39. [47]

    Automating the generation of prompts for llm-based action choice in pddl planning, 2025

    Katharina Stein, Daniel Fišer, Jörg Hoffmann, and Alexander Koller. Automating the generation of prompts for llm-based action choice in pddl planning, 2025. URL https://arxiv.org/abs/2311.09830

    2025 arXiv

  40. [48]

    Openapi specification

    Swagger. Openapi specification. https://swagger.io/specification/, 2024. Accessed: 2024-10-02

    2024

  41. [49]

    Seeberger, and Chi Huey Wong

    Naoyuki Taniguchi, Tamao Endo, Gerald Warren Hart, Peter H. Seeberger, and Chi Huey Wong. Glycoscience: Biology and medicine. Springer Japan, January 2015. ISBN 9784431548416. doi:10.1007/978-4-431-54841-6

    2015 doi

  42. [50]

    Glytoucan: an accessible glycan structure repository

    Michael Tiemeyer, Kazuhiro Aoki, James Paulson, Richard D Cummings, William S York, Niclas G Karlsson, Frederique Lisacek, Nicolle H Packer, Matthew P Campbell, Nobuyuki P Aoki, et al. Glytoucan: an accessible glycan structure repository. Glycobiology, 27 0 (10): 0 915--919, 2017

    2017

  43. [51]

    Appworld: A controllable world of apps and people for benchmarking interactive coding agents, 2024

    Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. Appworld: A controllable world of apps and people for benchmarking interactive coding agents, 2024. URL https://arxiv.org/abs/2407.18901

    2024 arXiv

  44. [52]

    Glycodata

    Fangxi Wang, Swarnadeep Seth, Saikiran Reddy Ramacharla, and Sanket A Deshmukh. Glycodata. glycodata.org/, 2025 a . Accessed: 2025-1-15

    2025

  45. [53]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18 0 (6), March 2024 a . I...

    2024 doi

  46. [54]

    Toolgen: Unified tool retrieval and calling via generation, 2025 b

    Renxi Wang, Xudong Han, Lei Ji, Shu Wang, Timothy Baldwin, and Haonan Li. Toolgen: Unified tool retrieval and calling via generation, 2025 b . URL https://arxiv.org/abs/2410.03439

    2025 arXiv

  47. [55]

    Executable code actions elicit better llm agents, 2024 b

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents, 2024 b . URL https://arxiv.org/abs/2402.01030

    2024 arXiv

  48. [56]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. URL https://arxiv.org/abs/2201.11903

    2023 arXiv

  49. [57]

    Glycam web: Website builders, 2025

    Woods Group . Glycam web: Website builders, 2025. URL http://glycam.org. Accessed 2025

    2025

  50. [58]

    The human o-glcnacome database and meta-analysis

    Eugenia Wulff-Fuentes, Rex R Berendt, Logan Massman, Laura Danner, Florian Malard, Jeet Vora, Robel Kahsay, and Stephanie Olivier-Van Stichelen. The human o-glcnacome database and meta-analysis. Scientific data, 8 0 (1): 0 25, 2021

    2021

  51. [59]

    The glycosmos portal: a unified and comprehensive web resource for the glycosciences

    Issaku Yamada, Masaaki Shiota, Daisuke Shinmachi, Tamiko Ono, Shinichiro Tsuchiya, Masae Hosoda, Akihiro Fujita, Nobuyuki P Aoki, Yu Watanabe, Noriaki Fujita, et al. The glycosmos portal: a unified and comprehensive web resource for the glycosciences. Nature Methods, 17 0 (7):...

    2020

  52. [60]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  53. [61]

    Survey on evaluation of llm-based agents, 2025

    Asaf Yehudai, Lilach Eden, Alan Li, Guy Uziel, Yilun Zhao, Roy Bar-Haim, Arman Cohan, and Michal Shmueli-Scheuer. Survey on evaluation of llm-based agents, 2025. URL https://arxiv.org/abs/2503.16416

    2025 arXiv

  54. [62]

    Glygen: computational and informatics resources for glycoscience

    William S York, Raja Mazumder, Rene Ranzinger, Nathan Edwards, Robel Kahsay, Kiyoko F Aoki-Kinoshita, Matthew P Campbell, Richard D Cummings, Ten Feizi, Maria Martin, et al. Glygen: computational and informatics resources for glycoscience. Glycobiology, 30 0 (2): 0 72--73, 2020

    2020

  55. [63]

    Api agents vs

    Chaoyun Zhang, Shilin He, Liqun Li, Si Qin, Yu Kang, Qingwei Lin, and Dongmei Zhang. Api agents vs. gui agents: Divergence and convergence, 2025. URL https://arxiv.org/abs/2503.11069

    2025 arXiv

  56. [64]

    Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, and Yu Su

    Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, and Yu Su. Skillweaver: Web agents can self-improve by discovering and honing skills, 2025. URL https://arxiv.org/abs/2504.07079

    2025 arXiv

  57. [65]

    Complexfuncbench: Exploring multi-step and constrained function calling under long-context scenario, 2025

    Lucen Zhong, Zhengxiao Du, Xiaohan Zhang, Haiyi Hu, and Jie Tang. Complexfuncbench: Exploring multi-step and constrained function calling under long-context scenario, 2025. URL https://arxiv.org/abs/2501.10132

    2025 arXiv

  58. [66]

    Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig

    Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents, 2024. URL https://arxiv.org/abs/2307.13854

    2024 arXiv

  59. [67]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  60. [68]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  61. [69]

    [ !1 "AQ 2a #BRVqҡ 3CTbrt 467 .e ȥ] =h=n I3&m T bM^ Ŀsąz( / Ox& OE J>ܸkŹ3_]#ntܼr ߛ9̱^Oo|A^yW 7E3'z= (qm\ ^ Q9 w bd >_5-SBr 1i? ?6 SeT] O[I1h

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.