Pith. sign in

REVIEW 3 major objections 5 minor 97 references

A Survey on Agent Workflow -- Status and Future

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Agent workflows have become the AI ecosystem's backbone, and they lack a shared standard, a 24-system survey finds.

desk verdict A useful broad map of agent workflows, but the comparison tables aren't yet auditable; fixable with methodology and corrections. read the letter →

arxiv 2508.01186 v1 pith:N5XPJZNK submitted 2025-08-02 cs.AI cs.HC

Chaojia Yu , Zihan Cheng , Hanwen Cui , Yishuo Gao , Zexu Luo , Yijin Wang , Hangbin Zheng , Yong Zhao This is my paper
classification cs.AIcs.HC
keywords agentworkflowmulti-agentsystemsLLMagentsorchestrationstandardizationoptimizationsecuritysurvey
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that agent workflows—structured orchestration layers in which LLM-driven agents plan, call tools, share memory, and cooperate—have become the backbone of the emerging AI ecosystem, yet the field has no unified specification. Its contribution is a systematic comparison of 24 agent workflow systems, scored on functional capabilities (planning, tool use, multi-agent collaboration, memory, GUI interaction, API calls, self-reflection, custom tools, cross-platform support, open-source status) and on architectural and mechanistic features (agent roles, flow type, representation, language, protocol, deployment). The comparison is meant to reveal common patterns and to pinpoint the bottleneck: most systems use self-defined DSLs, prompt fragments, or incompatible execution interfaces, making workflows hard to verify, debug, reuse, or orchestrate across frameworks. On that basis the paper derives a research agenda around standardization, workflow-level optimization, security, and open problems such as multimodal integration and autonomous pervasive agents.

What carries the argument

The central machinery is the two comparison tables that score 24 agent workflow systems with a four-value mark system (√ supported, × not supported, ◑ partially supported, ⚪ unspecified, with √* meaning only one specific API is supported). The capability table covers planning, tool use, multi-agent, memory, GUI, API, self-reflection, custom tools, cross-platform, and open-source status; the architecture table covers agent roles, flow type, representation, language, protocol, and deployment. These tables let the survey move from individual examples to aggregate patterns, such as near-universal support for planning and tool use alongside fragmented protocols and workflow representations. The supporting conceptual grid is Section III's taxonomy of workflow modes (chain, parallel, routing, orchestrator–workers, evaluator–optimizer), agent roles (planner, executor, critic, memory manager, communicator), and the three-layer architecture of UI/UX, workflow management, and agent collaboration.

What would settle it

Take two mature systems from the paper's tables that use different workflow representations and try to execute the same workflow—say a planner–executor loop with memory and a critic—in both, with the same external tools and no custom adapter; if one system can import and run the other's workflow definition directly, the claimed absence of a unified workflow framework would be seriously weakened.

Watch

Extended reading notes

Core claim

The claim this paper is trying to establish is that agent workflow systems are moving from scattered practices toward the structural core of AI applications, but are blocked by a missing shared framework for describing, executing, and verifying workflows. The paper supports this by classifying and comparing 24 systems along two dimensions: what the workflow can do (planning, tool use, multi-agent collaboration, memory, GUI, API, self-reflection, custom tools, cross-platform, open source) and how the workflow is built (agent roles, orchestration flow, representation formalism, language, protocol, deployment). Its reading of the comparison is that capability support is converging—most systems plan, use tools, and support multiple agents—while representation and protocol choices remain fragmented, which is exactly the gap the paper names the 'absence of a unified workflow framework.' From that diagnosis it moves to optimization strategies (manual reconstruction, heuristics, Bayesian optimization, LLM-based generative optimization), security concerns at the tool, protocol, MCP-server, LLM, memory, and multi-agent levels, and future directions centered on standardization and interoperability.

Load-bearing premise

The load-bearing premise is that the 24 systems chosen for the comparison fairly represent the agent workflow landscape and that each feature mark in the tables reflects a consistent, correct reading of very different projects.

Editorial extensions

If this is right

  • A unified workflow framework would let a workflow defined once run on different agent platforms, turning today's isolated agent systems into a composable ecosystem.
  • Standardized protocols for tool access and agent-to-agent communication would allow agents from different vendors to share tools and context, which the paper identifies as a precondition for scaling beyond single-vendor deployments.
  • Workflow-level optimization—token budgets, multi-task scheduling, resource-aware allocation, adaptive schedulers—would become a recognized research area in its own right.
  • Security would have to be enforced inside the orchestration layer, defending against tool poisoning, MCP-server impersonation, and memory-poisoning attacks rather than only at the model level.
  • Progress toward autonomous pervasive agents depends on workflows that adapt dynamically at execution time instead of following fixed predefined chains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that systems offering formal, machine-readable workflow representations should compose more easily than prompt-fragment-based ones; a direct cross-framework porting experiment could test that ordering.
  • A natural extension would be to turn each mark in the comparison tables into a reproducible probe, so the taxonomy becomes a benchmark the community can re-run and extend rather than a static expert judgment.
  • If the standardization diagnosis is correct, public open-source activity should show convergence toward a small set of workflow description formats and communication protocols over the next few years; measuring that trend would independently validate the paper's central concern.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This survey reviews the emerging area of agent workflow systems, arguing that workflows are becoming the backbone of the AI ecosystem while lacking a unified specification. It proposes a taxonomy based on functional capabilities and architectural features, compares 24 systems in two large tables, and discusses workflow optimization, application domains, security, limitations, and future directions such as standardization and multimodal integration.

Significance. If the comparative analysis were backed by a reproducible annotation protocol, the paper would provide a useful structured overview of a fast-moving field. The high-level taxonomy is plausible, the coverage of optimization and security issues is reasonable, and the survey draws on a broad set of academic and industrial systems. The main value lies in the two comparison tables, but their reliability is currently the weakest point: the sample is not defined and several concrete annotation errors are visible. The central claim about the lack of a unified workflow specification is plausible, but as presented it rests on an unauditable comparison.

major comments (3)
  1. [Section IV, Tables 1 and 2] The selection of the 24 systems is not defined. The paper never states inclusion or exclusion criteria for what counts as an "agent workflow system," and the tables mix heterogeneous artifact types: ReAct and ReWoo are prompting/reasoning strategies, n8n is a non-LLM workflow automation shell, DeepResearch is a closed commercial product, DSPy is a compiler, and AutoGen/CrewAI are SDKs. A uniform checklist of GUI, API, Open Source, Year, and deployment is then applied across these different kinds of artifacts. Consequently, aggregate statements such as "most systems adopt self-defined DSLs or configuration formats" (Section VIII.B) are not supported by a well-defined corpus. Please add a methodology subsection describing the corpus construction, artifact-type classification, annotation rules, and per-cell evidence, or provide an appendix with justifications for each row.
  2. [Table 1 and reference list] The table annotations contain concrete errors that undermine reproducibility. ReWoo is listed as 2024 in Table 1, but the cited reference [23] is arXiv:2305.18323 from 2023. Reference [68] for n8n is duplicated in the bibliography. Phidata [71] and Agno [56] are both credited to "Agno Contributors" and point to sibling github.com/agno-agi repositories, so the two rows may double-count a single project or need explicit clarification. These issues indicate that at least some cells were not checked against primary sources, and they weaken confidence in the comparison as a whole.
  3. [Section VIII.A.1] The first listed limitation, "Lack of Environmental Feedback," states that "the current system does not consider incorporating environmental feedback into the Automated Prompt Optimization (APO) process." No such system and no APO process are described anywhere in this survey, so this sentence appears to be carried over from a different paper. This makes the self-reported limitations section unreliable as a description of the survey's own limitations. Please either rewrite the item so it applies to the surveyed field as a whole, or remove it.
minor comments (5)
  1. [Throughout] There are several typographical and grammatical issues, including "Muti-agent path finding" (should be "Multi-agent path finding"), "as it is a source project" (should be "as it is an open-source project"), "comprehend comparison" (should be "comprehensive comparison"), "To dealing with" (should be "To deal with"), and "are not inline with expectations" (should be "are not in line with expectations").
  2. [Section II vs Section III] The section numbering is inconsistent: the text refers to "Section 2," "Section 3," etc., while the actual headings use Roman numerals (II, III, IV). Please unify the cross-referencing style.
  3. [Table 1 and Table 2] The same system is referred to as "DeepResearch" in Table 1 and "Deep Research" in Table 2; please standardize the name. Similarly, "AgentUniverse" appears with different capitalizations in the text and tables.
  4. [Section III.C.1] The sentence "Swift [74], VDL [75] use a functional-flavored scripting language to concisely describe large scale scientific workflows" lacks a verb tense agreement and reads as an incomplete example; please revise for clarity.
  5. [Section VIII.A.5] The discussion of MAPF and CBS, while interesting, is not clearly tied to agent workflow systems as defined in the rest of the paper; a connecting sentence explaining why this path-finding strategy is a workflow-level concern would improve readability.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the survey's comparisons and standardization claim are external summaries, not derivations from the authors' own results.

full rationale

This is a survey, so there is no derived quantity or fitted parameter whose output is built into its input. The central claim that agent workflows lack a unified specification is an observational and exhortatory thesis supported by cited external systems; it is not obtained by reducing an equation to itself. Tables 1 and 2 annotate 24 systems on functional and architectural axes; those annotations are external judgments, and even if some cells are incorrect, annotation errors are correctness and verifiability risks rather than circularity. The only self-citation is the corresponding author's earlier workflow languages Swift [74] and VDL [75] in Section III.C.1, used as examples of formal workflow modeling languages; that mention does not support the standardization conclusion and is not load-bearing. Section VIII.A.1 contains an anomalous limitation sentence, 'Lack of Environmental Feedback: The current system does not consider incorporating environmental feedback into the Automated Prompt Optimization (APO) process,' which has no counterpart in this survey; I flag it as an unsupported or provenance problem, but it is not a circular step because nothing in the survey's argument is defined in terms of that process. No fitted input is renamed as a prediction, and no uniqueness theorem or prior-work assumption is invoked to force a choice. Therefore the circularity score is low.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The survey's conclusions rest on domain assumptions about representativeness and comparability rather than on fitted parameters, formal axioms, or invented entities. No free parameters are used because the paper performs no quantitative fitting.

assumptions (3)
  • domain assumption The selected 24 systems are representative of the agent workflow landscape.
    Section IV opens by comparing 'across 24 agent workflow systems' without stating inclusion or exclusion criteria, so coverage and representativeness are assumed.
  • domain assumption Workflows can be meaningfully represented as directed graphs of nodes and edges.
    Section II.A introduces this representation as the common way to organize workflows, and the comparison tables rely on it for flow-type labels.
  • domain assumption Feature support can be meaningfully binarized or tri-state marked across heterogeneous frameworks.
    Tables 1 and 2 assign supported, partial, and unsupported marks to systems with very different architectures, assuming comparability that is not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey on Agent Workflow -- Status and Future." pith.science (2026). https://pith.science/paper/N5XPJZNK

@misc{pith2026250801186,
  author       = {Pith},
  title        = {Pith review of: A Survey on Agent Workflow -- Status and Future},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N5XPJZNK}},
  note         = {Machine review of arXiv:2508.01186}
}
read the original abstract

In the age of large language models (LLMs), autonomous agents have emerged as a powerful paradigm for achieving general intelligence. These agents dynamically leverage tools, memory, and reasoning capabilities to accomplish user-defined goals. As agent systems grow in complexity, agent workflows-structured orchestration frameworks-have become central to enabling scalable, controllable, and secure AI behaviors. This survey provides a comprehensive review of agent workflow systems, spanning academic frameworks and industrial implementations. We classify existing systems along two key dimensions: functional capabilities (e.g., planning, multi-agent collaboration, external API integration) and architectural features (e.g., agent roles, orchestration flows, specification languages). By comparing over 20 representative systems, we highlight common patterns, potential technical challenges, and emerging trends. We further address concerns related to workflow optimization strategies and security. Finally, we outline open problems such as standardization and multimodal integration, offering insights for future research at the intersection of agent design, workflow infrastructure, and safe automation.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

97 extracted references · 46 canonical work pages

  1. [68]

    Cooperative and competitive multi-agent systems: From optimization to games,

    J. Wang, Y. Hong, J. Wang, J. Xu, Y. Tang, Q.-L. Han, and J. Kurths, “Cooperative and competitive multi-agent systems: From optimization to games,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 5, pp. 763–783, 2022

    work page 2022

  2. [23]

    Metagpt: Meta programming for multi-agent collaborative framework,

    S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al., “Metagpt: Meta programming for multi-agent collaborative framework,” arXiv preprint arXiv:2308.00352 , vol. 3, no. 4, p. 6, 2023

    arXiv 2023

  3. [71]

    Evalai: Towards better evaluation systems for ai agents,

    D. Yadav, R. Jain, H. Agrawal, P. Chattopadhyay, T. Singh, A. Jain, S. B. Singh, S. Lee, and D. Batra, “Evalai: Towards better evaluation systems for ai agents,” arXiv preprint arXiv:1902.03570, 2019

    arXiv 1902

  4. [56]

    AI Agent as Urban Planner: Steering Stakeholder Dynamics in Urban Planning via Consensus-based Multi-Agent Reinforcement Learning

    K. Qian, L. Mao, X. Liang, Y. Ding, J. Gao, X. Wei, Z. Guo, and J. Li, “Ai agent as urban planner: Steering stakeholder dynamics in urban planning via consensus-based multi-agent reinforcement learning,” arXiv preprint arXiv:2310.16772, 2023

    work page Pith review arXiv

  5. [1]

    UI/UX: Provides the interface layer for users to interact with agents intuitively

  6. [2]

    Workflow Management: Coordinates the execution of tasks through structured processes, interacts with two other layers

  7. [3]

    Agent Collaboration: Enables multiple agents [7] to cooperate, communicate, and delegate subtasks to achieve complex goals collectively. B. Roles of Agent Multi-agent workflows assume diverse roles for agents based on the system’s needs and coordination strategies. Typical roles include: Planner: Responsible for decomposing tasks and assigning responsibil...

  8. [4]

    This makes workflows highly flexible and human- readable, but also less standardized and harder to validate

    Language  Prompt: Natural language instead of formal languages. This makes workflows highly flexible and human- readable, but also less standardized and harder to validate.  Modeling Workflow: Traditional formal workflow languages, such as BPML (Business Process Modeling Language) and XPDL (XML Process Definition Language). Business Process Execution La...

Show all 97 references
  1. [5]

    Tool: Tool-backed agents have the capability to execute external tools via code execution or function execution [3]. For Example, the systems have retrieval tools like vector search engines, web searcher, computation and reasoning tools like calculators, code interpreters, kno...

  2. [6]

    WebSocket, etc

    Protocol: General-purpose service Protocols: REST/HTTP that used in most agent tool calls and API wrapping. WebSocket, etc. for real-time, bidirectional communication. These protocols are based on function- calling. Agent-native Protocols: Model Context Protocol (MCP): an open...

  3. [7]

    Interactive Pattern: This specification mainly discuss how agents perceive environmental information to acquire knowledge and experience. a. Interacting with environment. The components are environment, sensor, executor and effector, following the process of observation, actio...

  4. [8]

    Chain Workflow: Decomposing a general task into a sequence of steps, where each step depends on the output of the previous one

    Workflow Mode: a. Chain Workflow: Decomposing a general task into a sequence of steps, where each step depends on the output of the previous one. b. Parallelization Workflow: Executing multiple tasks or processing multiple datasets simultaneously, where the tasks are independe...

  5. [9]

    It determines whether a task proceeds in a fixed procedural order (static workflow), or adaptively changes based on agent decisions (dynamic workflow)

    Workflow Execution: Workflow execution defines how an agent workflow is triggered, scheduled, and terminated. It determines whether a task proceeds in a fixed procedural order (static workflow), or adaptively changes based on agent decisions (dynamic workflow). Driven by an ex...

  6. [10]

    Problem solving permeates every stage of the workflow

    Problem Solving: Problem solving represents the holistic execution behavior of agent workflows, encompassing planning, acting, interacting, and adapting in pursuit of a goal. Problem solving permeates every stage of the workflow. Representative works are not limited to plannin...

  7. [11]

    thinks” while taking “actions

    Planning: Agent Workflow Structure Basically we follow 5 steps: perception, reasoning, decision making, action execution, feedback and learning. The following “nodes” form a logical chain. Discovery refers to the agent’s ability to autonomously identify unknown goals. Task dec...

  8. [12]

    For example many products of OpenAI only allow to call the API of Open AI

    We also determine Whether the API calling only support specific ones. For example many products of OpenAI only allow to call the API of Open AI. If so, append “*”. For the explanation, if multiple models are supported, it will be necessary to unify the interfaces, optimize the...

  9. [13]

    For the metric ”Memory”, it emphasizes context management and state maintenance of multiple rounds of historical conversations

  10. [14]

    Compared to API-based agent, a distinctive form of tool use arises when agents operate in graphical user interface (GUI) environments. Rather than invoking back- end APIs or structured function calls, GUI-based agents interact with applications through human-like operations su...

  11. [15]

    Data flow represents a workflow as a data state machine, where context is passed between nodes via ”state objects.” The execution order does not need to be predefined, as node selection is determined by data content and transition conditions

  12. [16]

    no- code agent execution shell

    Language is not targeting at modeling language but also include lightweight data description languages. The language are sometimes domain-specific. Unlike traditional workflow systems relying on formal specification languages like BPEL or BPMN, most LLM- driven agent framework...

    2023

  13. [17]

    Rug pulls in which attackers alter tool descriptions after user authorization (e.g., in MCP)

    Tools: The common attacks on agent tools include hidden Instructions [42], in which malicious prompts hidden in tool descriptions to steal user data. Rug pulls in which attackers alter tool descriptions after user authorization (e.g., in MCP). Tool name collision in which tool...

  14. [18]

    Secondly, during the update phase, there are still some issues that may pose a threat to AI agents accessing the MCP

    MCP server: There are three common ways: Some malicious entities register with names similar or identical to legitimate MCP servers, deceiving users into installation [43]. Secondly, during the update phase, there are still some issues that may pose a threat to AI agents acces...

  15. [19]

    Adversarial data poisoning attacks pose a significant threat to LLM-based AI Agents; seemingly harmless prompts, when concatenated, may contaminate the model

    LLM: Malicious inputs from users may lead to model contamination, affecting the integrity of the AI Agent [44]. Adversarial data poisoning attacks pose a significant threat to LLM-based AI Agents; seemingly harmless prompts, when concatenated, may contaminate the model. Chat r...

  16. [20]

    For competition threats, malicious competitive behaviors, ethical concerns is found

    MAS: In MAS collaboration the threats include covert collusion, illusion amplification, spread of misinformation, malicious attacks. For competition threats, malicious competitive behaviors, ethical concerns is found

  17. [21]

    Short- term memory may encounter capacity limitations, leading to information loss

    Memory: Memory security in AI agents can be divided into two parts: short–term and long-term. Short- term memory may encounter capacity limitations, leading to information loss. Asynchronization can also disrupt data flow. Long-term memory faces various threats. Poisoning atta...

  18. [22]

    What if gpt4 became autonomous: The auto- gpt project and use cases,

    M. F ırat and S. Kuleli, “ What if gpt4 became autonomous: The auto- gpt project and use cases,” Journal of Emerging Computer Technologies, vol. 3, no. 1, pp. 1–6, 2023

    2023

  19. [24]

    Autogen: Enabling next-gen llm applications via multi-agent conversation,

    Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu et al., “Autogen: Enabling next-gen llm applications via multi-agent conversation,” arXiv preprint arXiv:2308.08155, 2023

    2023 arXiv

  20. [25]

    Building effective agents

    Anthropic,“Building effective agents” https://www.anthropic.com/engineering/building-effective-agents, 2024, accessed: 2025-04-13

    2024

  21. [26]

    Autoflow: Automated workflow generation for large language model agents,

    Z. Li, S. Xu, K. Mei, W. Hua, B. Rama, O. Raheja, H. Wang, H. Zhu, and Y. Zhang, “Autoflow: Automated workflow generation for large language model agents,” arXiv preprint arXiv:2407.12821, 2024

    2024 arXiv

  22. [27]

    A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges,

    X. Li, S. Wang, S. Zeng, Y. Wu, and Y. Yang, “A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges,” Vicinagearth, vol. 1, no. 1, p. 9, 2024

    2024

  23. [28]

    Multiagent learning is not the answer. it is the question,

    P. Stone, “Multiagent learning is not the answer. it is the question,” Artificial Intelligence, vol. 171, no. 7, pp. 402–405, 2007

    2007

  24. [29]

    Ag2: Open-source agentos for ai agents,

    C. Wang, Q. Wu, and the AG2 Community, “Ag2: Open-source agentos for ai agents,” 2024, available at https://docs.ag2.ai/. [Online]. Available: https:// github.com/ag2ai/ag2

    2024

  25. [30]

    Pattern based analysis of bpel4ws,

    P. Wohed, W. M. van der Aalst, M. Dumas, and A. H. Ter Hofstede, “Pattern based analysis of bpel4ws,” QUT Technical report, FIT-TR- 2002-04, Queensland University of Technology . . . , Tech. Rep., 2002

    2002

  26. [31]

    Towards adaptive workflow enactment using multiagent systems,

    P. A. Buhler and J. M. Vidal, “Towards adaptive workflow enactment using multiagent systems,” Information technology and management, vol. 6, pp. 61–87, 2005

    2005

  27. [32]

    The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey,

    T. Masterman, S. Besen, M. Sawtell, and A. Chao, “The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey,” arXiv preprint arXiv:2404.11584, 2024

    2024 arXiv

  28. [33]

    Workflowllm: Enhancing workflow orchestration capability of large language models,

    S. Fan, X. Cong, Y. Fu, Z. Zhang, S. Zhang, Y. Liu, Y. Wu, Y. Lin, Z. Liu, and M. Sun, “Workflowllm: Enhancing workflow orchestration capability of large language models,” arXiv preprint arXiv:2 41 1. 0 54 51 , 2024

    2024

  29. [34]

    Agentic retrieval-augmented generation: A survey on agentic rag,

    A. Singh, A. Ehtesham, S. Kumar, and T. T. Khoei, “Agentic retrieval-augmented generation: A survey on agentic rag,” arXiv preprint arXiv:2501.09136, 2025

    2025 arXiv

  30. [35]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations (ICLR), 2023

    2023

  31. [36]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” Advances in neural information processing systems, vol. 36, pp. 11 809–11 822, 2023

    2023

  32. [37]

    Reasoning with language model is planning with world model,

    S. Hao, Y. Gu, H. Ma, J. J. Hong, Z. Wang, D. Z. Wang, and Z. Hu, “Reasoning with language model is planning with world model,” arXiv preprint arXiv:2305.14992, 2023

    2023 arXiv

  33. [38]

    Causal inference using llm-guided discovery,

    A. Vashishtha, A. G. Reddy, A. Kumar, S. Bachu, V. N. Balasubrama-nian, and A. Sharma, “Causal inference using llm-guided discovery,” arXiv preprint arXiv:2310.15117, 2023

    2023 arXiv

  34. [39]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al., “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  35. [40]

    Inner monologue: Embod-ied reasoning through planning with language models,

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar et al., “Inner monologue: Embod-ied reasoning through planning with language models,” arXiv preprint arXiv:2207.05608, 2022

    2022 arXiv

  36. [41]

    Creating large language model appli-cations utilizing langchain: A primer on developing llm apps fast,

    O. Topsakal and T. C. Akinci, “Creating large language model appli-cations utilizing langchain: A primer on developing llm apps fast,” in International Conference on Applied Engineering and Natural Sciences, vol. 1, no. 1, 2023, pp. 1050–1056

    2023

  37. [42]

    Auffarth, Generative AI with LangChain: Build large language model (LLM) apps with Python, ChatGPT, and other LLMs

    B. Auffarth, Generative AI with LangChain: Build large language model (LLM) apps with Python, ChatGPT, and other LLMs. Packt Publishing Ltd, 2023

    2023

  38. [43]

    Api agents vs. gui agents: Divergence and convergence,

    C. Zhang, S. He, L. Li, S. Qin, Y. Kang, Q. Lin, and D. Zhang, “Api agents vs. gui agents: Divergence and convergence,” arXiv preprint arXiv:2503.11069, 2025

    2025 arXiv

  39. [44]

    Rewoo: De-coupling reasoning from observations for efficient augmented language models,

    B. Xu, Z. Peng, B. Lei, S. Mukherjee, Y. Liu, and D. Xu, “Rewoo: De-coupling reasoning from observations for efficient augmented language models,” arXiv preprint arXiv:2305.18323, 2023

    2023 arXiv

  40. [45]

    Optimization for reinforcement learning: From a single agent to cooperative agents,

    D. Lee, N. He, P. Kamalaruban, and V. Cevher, “Optimization for reinforcement learning: From a single agent to cooperative agents,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 123–135, 2020

    2020

  41. [46]

    Multi-objective meta- heuristics: An overview of the current state-of-the-art,

    D. F. Jones, S. K. Mirrazavi, and M. Tamiz, “Multi-objective meta- heuristics: An overview of the current state-of-the-art,” European journal of operational research, vol. 137, no. 1, pp. 1–9, 2002

    2002

  42. [47]

    Cognify: Super- charging gen-ai workflows with hierarchical autotuning,

    Z. He, R. Abhyankar, V. Srivatsa, and Y. Zhang, “Cognify: Super- charging gen-ai workflows with hierarchical autotuning,” arXiv preprint arXiv:2502.08056, 2025

    2025 arXiv

  43. [48]

    Large language models as optimizers,

    C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou, and X. Chen, “Large language models as optimizers,” arXiv preprint arXiv:2309.03409, 2023

    2023 arXiv

  44. [49]

    Txagent: An ai agent for therapeutic reasoning across a universe of tools,

    S. Gao, R. Zhu, Z. Kong, A. Noori, X. Su, C. Ginder, T. Tsiligkaridis, and M. Zitnik, “Txagent: An ai agent for therapeutic reasoning across a universe of tools,” arXiv preprint arXiv:2503.10970, 2025

    2025 arXiv

  45. [50]

    Planning, living and judging: A multi-agent llm-based framework for cyclical urban planning,

    H. Ni, Y. Wang, and H. Liu, “Planning, living and judging: A multi-agent llm-based framework for cyclical urban planning,” arXiv preprint arXiv:2412.20505, 2024

    2024 arXiv

  46. [51]

    Finrobot: an open- source ai agent platform for financial applications using large language models,

    H. Yang, B. Zhang, N. Wang, C. Guo, X. Zhang, L. Lin, J. Wang, T. Zhou, M. Guan, R. Zhang et al. , “ Finrobot: an open- source ai agent platform for financial applications using large language models,” arXiv preprint arXiv:2405.14767, 2024

    2024 arXiv

  47. [52]

    Enhanc-ing investment analysis: Optimizing ai-agent collaboration in financial research,

    X. Han, N. Wang, S. Che, H. Yang, K. Zhang, and S. X. Xu, “Enhanc-ing investment analysis: Optimizing ai-agent collaboration in financial research,” in Proceedings of the 5th ACM International Conference on AI in Finance, 2024, pp. 538–546

    2024

  48. [53]

    Personalized feedback in massive open online courses: Harnessing the power of langchain and openai api,

    M. Morales-Chan, H. R. Amado-Salvatierra, J. A. Medina, R. Barchino, R. Hern ndez-Rizzardini, and A. M. Teixeira, “Personalized feedback in massive open online courses: Harnessing the power of langchain and openai api,” Electronics, vol. 13, no. 10, p. 1960, 2024

    1960

  49. [54]

    Ai agent for education: von neumann multi-agent system framework,

    Y.-H. Jiang, R. Li, Y. Zhou, C. Qi, H. Hu, Y. Wei, B. Jiang, and Y. Wu, “Ai agent for education: von neumann multi-agent system framework,” arXiv preprint arXiv:2501.00083, 2024

    2024 arXiv

  50. [55]

    Multi-agent simulator drives language models for legal intensive interaction,

    S. Yue, T. Huang, Z. Jia, S. Wang, S. Liu, Y. Song, X. Huang, and Z. Wei, “Multi-agent simulator drives language models for legal intensive interaction,” arXiv preprint arXiv:2502.06882, 2025

    2025 arXiv

  51. [57]

    Urban generative intelligence (ugi): A foundational platform for agents in embodied city environment,

    F. Xu, J. Zhang, C. Gao, J. Feng, and Y. Li, “Urban generative intelligence (ugi): A foundational platform for agents in embodied city environment,” arXiv preprint arXiv:2312.11813, 2023

    2023 arXiv

  52. [58]

    Agent hospital: A simulacrum of hospital with evolvable medical agents,

    J. Li, Y. Lai, W. Li, J. Ren, M. Zhang, X. Kang, S. Wang, P. Li, Y.-Q. Zhang, W. Ma et al., “Agent hospital: A simulacrum of hospital with evolvable medical agents,” arXiv preprint arXiv:2405.02957, 2024

    2024 arXiv

  53. [59]

    An artificial intelligence agent technology based web distance education system,

    R. Li, “An artificial intelligence agent technology based web distance education system,” Journal of Intelligent & Fuzzy Systems, vol. 40, no. 2, pp. 3289–3299, 2021

    2021

  54. [60]

    Chatlaw: A multi-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experts large language model,

    J. Cui, M. Ning, Z. Li, B. Chen, Y. Yan, H. Li, B. Ling, Y. Tian, and L. Yuan, “Chatlaw: A multi-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experts large language model,” arXiv preprint arXiv:2306.16092, 2023

    2023 arXiv

  55. [61]

    Model context protocol,

    “Model context protocol,” https:// modelcontextprotocol.io/, n.d., ac- cessed: 2025-04-13

    2025

  56. [62]

    Ai agents under threat: A survey of key security challenges and future pathways,

    Z. Deng, Y. Guo, C. Han, W. Ma, J. Xiong, S. Wen, and Y. Xiang, “Ai agents under threat: A survey of key security challenges and future pathways,” ACM Computing Surveys, vol. 57, no. 7, pp. 1–36, 2025

    2025

  57. [63]

    Model context protocol (mcp): Landscape, security threats, and future research directions,

    X. Hou, Y. Zhao, S. Wang, and H. Wang, “Model context protocol (mcp): Landscape, security threats, and future research directions,” arXiv preprint arXiv:2503.23278, 2025

    2025 arXiv

  58. [64]

    Mcp: Building a secure execution environment for agentic workflows,

    I. Labs, “Mcp: Building a secure execution environment for agentic workflows,” https:// invariantlabs.ai/blog/mcp-security, 2024, accessed: 2025-04-14

    2024

  59. [65]

    Security of ai agents,

    Y. He, E. Wang, Y. Rong, Z. Cheng, and H. Chen, “Security of ai agents,” arXiv preprint arXiv:2406.08689, 2024

    2024 arXiv

  60. [66]

    The emerged security and privacy of llm agent: A survey with case studies,

    F. He, T. Zhu, D. Ye, B. Liu, W. Zhou, and P. S. Yu, “The emerged security and privacy of llm agent: A survey with case studies,” arXiv preprint arXiv:2407.19354, 2024

    2024

  61. [67]

    Why do multi-agent llm systems fail?

    M. Cemri, M. Z. Pan, S. Yang, L. A. Agrawal, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, D. Klein, K. Ramchandran et al., “Why do multi-agent llm systems fail?” arXiv preprint arXiv:2503.13657, 2025

    2025 arXiv

  62. [69]

    Unveiling privacy risks in llm agent memory,

    B. Wang, W. He, P. He, S. Zeng, Z. Xiang, Y. Xing, and J. Tang, “Unveiling privacy risks in llm agent memory,” arXiv preprint arXiv:2502.13172, 2025

    2025 arXiv

  63. [70]

    Ai-based conversational agents: a scoping review from technologies to future directions,

    S. Kusal, S. Patil, J. Choudrie, K. Kotecha, S. Mishra, and A. Abraham, “Ai-based conversational agents: a scoping review from technologies to future directions,” IEEE Access, vol. 10, pp. 92 337–92 356, 2022

    2022

  64. [72]

    The rise and potential of large language model based agents: A survey,

    Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou et al., “The rise and potential of large language model based agents: A survey,” Science China Information Sciences, vol. 68, no. 2, p. 121101, 2025

    2025

  65. [73]

    Multi-agent pathfinding: Defi- nitions, variants, and benchmarks,

    R. Stern, N. Sturtevant, A. Felner, S. Koenig, H. Ma, T. Walker, J. Li, D. Atzmon, L. Cohen, T. Kumar et al., “Multi-agent pathfinding: Defi- nitions, variants, and benchmarks,” in Proceedings of the International Symposium on Combinatorial Search, vol. 10, no. 1, 2019, pp. 151–158

    2019

  66. [74]

    Analysis of the anytime mapf solvers based on the combination of conflict-based search (cbs) and focal search (fs),

    I. Ivanashev, A. Andreychuk, and K. Yakovlev, “Analysis of the anytime mapf solvers based on the combination of conflict-based search (cbs) and focal search (fs),” in Mexican International Conference on Artificial Intelligence. Springer, 2022, pp. 368–382

    2022

  67. [75]

    agentuniverse: Multi-agent ecosystem for autonomous agents,

    agentUniverse Contributors, “agentuniverse: Multi-agent ecosystem for autonomous agents,” https://github.com/agentuniverse-ai/agentUniverse, 2024, accessed: 2025-04-14

    2024

  68. [76]

    Agentverse: Facilitating multi-agent col- laboration and exploring emergent behaviors in agents,

    W. Chen, Y. Su, J. Zuo, C. Yang, C. Yuan, C. Qian, C.-M. Chan, Y. Qin, Y. Lu, R. Xie et al., “Agentverse: Facilitating multi-agent col- laboration and exploring emergent behaviors in agents,” arXiv preprint arXiv:2308.10848, 2023

    2023 arXiv

  69. [77]

    Agno: An agentic workflow framework for llm applications,

    Agno Contributors, “Agno: An agentic workflow framework for llm applications,” https://github.com/agno-agi/agno, 2024, accessed: 2025-04- 14

    2024

  70. [78]

    Camel: Communicative agents for

    G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B Ghanem, “Camel: Communicative agents for ”mind” exploration of large language model society,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  71. [79]

    Chatdev: Communicative agents for software development,

    C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong et al., “Chatdev: Communicative agents for software development,” arXiv preprint arXiv:2307.07924, 2023

    2023 arXiv

  72. [80]

    Coze: An open framework for building agentic applications,

    Coze Contributors, “Coze: An open framework for building agentic applications,” https://github.com/coze-dev/coze, 2024, accessed: 2025- 04-14

    2024

  73. [81]

    Crew: Facilitating human-ai teaming research,

    L. Zhang, Z. Ji, and B. Chen, “Crew: Facilitating human-ai teaming research,” arXiv preprint arXiv:2408.00170, 2024

    2024 arXiv

  74. [82]

    Introducing deep research: Openai’s new system for collab- orative reasoning,

    OpenAI, “Introducing deep research: Openai’s new system for collab- orative reasoning,” https://openai.com/index/introducing-deep-research/, 2024, accessed: 2025-04-14

    2024

  75. [83]

    Dify: Open-source llm application devel- opment platform,

    LangGenius Contributors, “Dify: Open-source llm application devel- opment platform,” https://github.com/langgenius/dify, 2024, accessed: 2025-04-14

    2024

  76. [84]

    Dspy: Compiling declarative language model calls into self-improving pipelines,

    O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam et al., “Dspy: Compiling declarative language model calls into self-improving pipelines,” arXiv preprint arXiv:2310.03714, 2023

    arXiv

  77. [85]

    Ernie 3.0: Large-scale knowledge enhanced pre- training for language understanding and generation,

    Y. Sun, S. Wang, S. Feng, S. Ding, C. Pang, J. Shang, J. Liu, X. Chen, Y. Zhao, Y. Lu et al., “Ernie 3.0: Large-scale knowledge enhanced pre- training for language understanding and generation,” arXiv preprint arXiv:2107.02137, 2021

    2021 arXiv

  78. [86]

    Flowise: Drag & drop ui to build llm apps with langchain,

    FlowiseAI Contributors, “Flowise: Drag & drop ui to build llm apps with langchain,” https://github.com/FlowiseAI/Flowise, 2024, accessed: 2025- 04-14

    2024

  79. [87]

    Langgraph: Stateful multi-actor workflow library for llm applications,

    LangChain Contributors, “Langgraph: Stateful multi-actor workflow library for llm applications,” https://github.com/langchain-ai/langgraph, 2024, accessed: 2025-04-14

    2024

  80. [88]

    Magentic-one: A generalist multi-agent system for solving complex tasks,

    A. Fourney, G. Bansal, H. Mozannar, C. Tan, E. Salinas, F. Niedtner, G. Proebsting, G. Bassman, J. Gerrits, J. Alber et al., “Magentic-one: A generalist multi-agent system for solving complex tasks,” arXiv preprint arXiv:2411.04468, 2024.[68] n8n-io, “n8n: Fair-code workflow a...

    2024 arXiv

  81. [89]

    n8n: Fair-code workflow automation platform with powerful ui and integrations,

    n8n-io, “n8n: Fair-code workflow automation platform with powerful ui and integrations,” https://github.com/n8n-io/n8n, 2024, accessed: 2025- 04-14

    2024

  82. [90]

    Omagent: A multi- modal agent framework for complex video understanding with task divide-and-conquer,

    L. Zhang, T. Zhao, H. Ying, Y. Ma, and K. Lee, “Omagent: A multi- modal agent framework for complex video understanding with task divide-and-conquer,” arXiv preprint arXiv:2406.16620, 2024

    2024 arXiv

  83. [91]

    Swarm: Multi-agent framework by openai,

    OpenAI, “Swarm: Multi-agent framework by openai,” https://github.com/openai/swarm, 2024, accessed: 2025-04-14

    2024

  84. [92]

    Phidata: A framework for building multi-modal agents with memory, knowledge, tools, and reasoning,

    Agno Contributors, “Phidata: A framework for building multi-modal agents with memory, knowledge, tools, and reasoning,” https://github.com/agno-agi/phidata, 2024, accessed: 2025-04-14

    2024

  85. [93]

    Qwen-agent,

    QwenLM Team, “Qwen-agent,” https://github.com/QwenLM/Qwen- Agent, 2024, accessed: 2025-04-14

    2024

  86. [94]

    Semantic kernel,

    Microsoft, “Semantic kernel,” https://github.com/microsoft/semantic- kernel, 2024, accessed: 2025-04-14

    2024

  87. [95]

    Swift: Fast, reliable, loosely coupled parallel computation,

    Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. Von Laszewski, V. Nefe- dova, I. Raicu, T. Stef-Praun, and M. Wilde, “Swift: Fast, reliable, loosely coupled parallel computation,” in 2007 IEEE Congress on Services (Services 2007). IEEE, 2007, pp. 199–206

    2007

  88. [96]

    A notation and system for expressing and executing cleanly typed workflows on messy scientific data,

    Y. Zhao, J. Dobson, I. Foster, L. Moreau, and M. Wilde, “A notation and system for expressing and executing cleanly typed workflows on messy scientific data,” ACM Sigmod Record, vol. 34, no. 3, pp. 37–43, 2005

    2005

  89. [97]

    Agent2agent Protocol(A2A)

    Google Developers, “Agent2agent Protocol(A2A)” https://developers.googleblog.com/en/a2a-a-new-era-of-agent- interoperability, 2024, accessed: 2025-04-10

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.