Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution

Haitao Li; Jingli Yang; Junjie Chen; Qingyao Ai; Yiqun Liu

REVIEW 4 major objections 5 minor 1 cited by

Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GoalAct, a framework coupling continuously updated global planning with hierarchical skill execution, reports state-of-the-art performance on LegalAgentBench, improving average success rate by 12.22% over existing agent baselines.

desk verdict A plausible 12.22% gain on LegalAgentBench, but the substring-matching metric and same-lab benchmark mean the real-world transfer is unproven; deserves peer review with revisions. read the letter →

arxiv 2504.16563 v3 pith:DNO6TXVO submitted 2025-04-23 cs.IR

Junjie Chen , Haitao Li , Jingli Yang , Yiqun Liu , Qingyao Ai This is my paper · ORCID

classification cs.IR

keywords largelanguagemodelagentglobalplanninghierarchicalexecutionlegaldomaintoolusebenchmarkevaluationsuccessrate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that LLM-based agents fail on complex tool-using tasks because planning and execution are separated and because actions are treated as a single flat level, forcing a choice between narrow action formats and fragile complex ones. It proposes GoalAct, which keeps a global plan that is regenerated after every action from the query, the available tools, and the execution history, and which decomposes execution into high-level skills—searching, coding, writing, and finishing—so the plan only has to choose a skill and an objective. The authors evaluate GoalAct on LegalAgentBench, a 300-task legal benchmark requiring external tools and legal knowledge, and report that it outperforms existing agent frameworks on all three tested LLMs, with an average success-rate improvement of 12.22%. If correct, this shows that coupling a continuously updated global plan with hierarchical skill execution reliably improves LLM agents on multi-step tool-use tasks, and that the improvement grows as task difficulty increases.

What carries the argument

The central machinery is the continuously updated global plan together with hierarchical skill decomposition. The global plan is a sequence of plan-step/action pairs; at each time step the update policy $\pi$ regenerates the whole plan from the user query $Q$, the available tool set $T$, and the execution history $S_t$, so observations feed back directly into the next plan. Each action in the plan is one of a small set of high-level skills—Searching for text/json tool calls, Coding for Python code with loops and branches, Writing for document generation, and Finish—and the skill itself handles the low-level tool selection and parameter setting. This separation lets the plan stay abstract enough to remain feasible while concrete enough to guide execution, and it lets the skill set grow without redesigning the planner.

What would settle it

Run GoalAct and the strongest baseline on a fresh set of legal tasks authored independently of the benchmark team, scoring by exact answer matching and by whether the final answer would satisfy a legal expert, and check whether the 12.22% average improvement persists; if the margin shrinks to zero or reverses, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the main failure modes of LLM agents—getting trapped in local branches, producing plans that cannot actually be executed, and using an action space that is either too narrow or too error-prone—can be overcome by two coordinated design choices. First, GoalAct maintains a global plan $G = (P_1A_1, P_2A_2, \ldots, P_nA_n)$ and regenerates this plan at every step with an update policy $\pi$ that takes the user query $Q$, the tool set $T$, and the execution history $S_t$ as input, so the plan always incorporates observations from actions already taken. Second, execution is organized hierarchically: the plan step names a high-level skill—Searching (ReAct-style text or JSON tool calls), Coding (Python code with loops and branches), Writing (document generation), or Finish—and that skill then handles tool selection and parameter details. On LegalAgentBench, GoalAct reports average success-rate improvements of 14.45% for GPT-4o-mini, 10.09% for Qwen-max, and 12.11% for GLM-4-Plus over the best baseline, and the ablation study shows that removing the global plan or any of the three skills lowers performance, with removing coding costing the most (14.06% average).

Load-bearing premise

The load-bearing premise is that LegalAgentBench's success rate—the fraction of key answer strings that appear in the model's output—measures real task completion, and that the benchmark has not been shaped by GoalAct's own Search/Coding/Writing skill definitions; if key-answer string matching can be satisfied by phrase-spotting, or if the tasks align with GoalAct's prompt-defined skills, the reported 12.22% improvement may not transfer to other benchmarks or real legal work.

Editorial extensions

If this is right

GoalAct reports the highest success rate on LegalAgentBench for every base LLM tested, with an average 12.22% improvement over the second-best method (14.45% for GPT-4o-mini, 10.09% for Qwen-max, 12.11% for GLM-4-Plus).
The advantage widens with task difficulty: relative improvements average 3.26% on 1-hop tasks, 20% on 3-hop tasks, and 15.5% on 5-hop tasks, so the framework is most valuable exactly where agents usually break.
Ablations show each component carries weight: removing the global plan costs 8.14% average success, removing coding costs 14.06%, and removing the writing skill lowers writing-task success by 3.96%.
Because the plan only names high-level skills, the skill set can be extended without redesigning the planner, which the paper presents as the route to broader applicability beyond legal tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the hierarchical decomposition suggests a modular route to extending agents to new domains: adding a new high-level skill should require no change to the planning mechanism, a claim the paper motivates but does not test outside legal tasks.
Beyond the paper, the every-step regeneration of the global plan could be made more cost-efficient by updating the plan only when an observation contradicts the current path; replanning every step is the design choice here, and its computational cost is not measured in the paper.
Beyond the paper, the success metric's reliance on key-answer string occurrence leaves open the possibility that an agent could score partial success without producing a usable final artifact; a stricter outcome-based evaluation would test whether the 12.22% gain corresponds to genuinely useful legal outputs.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper does what it says—builds a simple, sensible agent framework that beats four baselines on LegalAgentBench by a consistent margin. The 12.22% headline is believable as an in-benchmark result, but I would not treat it as a settled statement about real legal work until the metric and benchmark independence are addressed.

What's genuinely new here is the combination: a continuously updated global plan plus a small set of high-level skills (searching, coding, writing). That's a natural extension of ReAct and CodeAct, and the paper shows it helps across three LLMs and most difficulty levels. The ablation on GLM-4-Plus is informative—coding removal hurts the most, writing removal least—and releasing code is a plus. The writing is clear; equations (1)–(4) are just definitions, which is fine for an architecture paper.

The soft spots are real but addressable. First, there are no error bars or significance tests. Some per-task margins are small, and with three LLMs and multiple methods, a few numbers could easily flip. Second, LegalAgentBench was built by an overlapping author team, and the skill set (searching/coding/writing) was hand-picked for that benchmark's task types. That does not make the result wrong, but it does mean the benchmark is not independent evidence for the framework's generality. Third, the success metric is substring matching (Eq. 4). In Writing tasks, the model produces long defense documents while the answer key contains short phrases (~12 chars on average). GoalAct's Writing skill instructs the model to produce such documents, so its outputs are longer and more likely to contain key phrases by chance—or at least by format rather than by legal quality. CodeAct, which performs poorly on writing, has no equivalent branch. I think the stress-test note is on target: a blind human-scored sample of outputs would settle this. The absence of statistical tests is minor in context, but the metric concern is the load-bearing issue.

Who is this for? Researchers building LLM agents for tool-heavy tasks, and anyone thinking about how to evaluate them honestly. The paper deserves a serious peer review, not a desk reject—but the reviewers should ask for repeated runs, a second benchmark or human evaluation, and explicit disclosure of the benchmark overlap.

Referee Report

4 major / 5 minor

Summary. The paper proposes GoalAct, an LLM-agent framework that combines a continuously updated global plan with hierarchical execution decomposed into high-level skills (Searching, Coding, Writing, Finish). The method is evaluated on LegalAgentBench, a 300-task legal-domain benchmark with 1-hop to 5-hop reasoning tasks and a writing task, using three LLMs (GPT-4o-mini, Qwen-max, GLM-4-Plus) and four baselines (Plan-and-Solve, Plan-and-Execute, ReAct, CodeAct). The central claim is state-of-the-art performance, with an average 12.22% improvement in success rate over the best baseline, driven by stronger performance on multi-hop and writing tasks. An ablation on GLM-4-Plus shows degradation when the global plan or individual skills are removed. The authors release code at a public repository.

Significance. If the reported gains are valid, GoalAct offers a simple and modular recipe for improving LLM agents on tool-using tasks: separate high-level skill selection from low-level tool invocation, and maintain a global plan that is updated at every step. The experimental design has useful strengths: it tests three LLM families, sets temperature to 0 for reproducibility, includes a benchmark designed to avoid data leakage, and provides open-source code. However, the evidential basis for the headline SOTA claim is currently limited by the evaluation metric, the lack of statistical reliability analysis, and the close relationship between the benchmark's task types and the method's hand-selected skill set. The contribution is therefore promising but not yet established at the level claimed.

major comments (4)

[§4.1.3, Eq. (4)] The success-rate metric counts the fraction of key answer strings that appear as substrings of the model output. This is a weak proxy for task completion, especially for Writing tasks, where the average key answer length is only 12.58 characters and GoalAct explicitly instructs the model to generate a formal defense document. Longer, format-compliant outputs can mechanically include such short legal phrases without being legally adequate, so the reported 12.74% Writing improvement (Table 2) may overstate real gains. The paper should either use a stricter exact-match or semantic-match metric, or report a human-scored sample of outputs, to establish that the improvement reflects task completion rather than output length or format.
[§4.2.1, Table 2] No error bars, confidence intervals, or significance tests are reported, and there is no indication of repeated runs or multiple random seeds. Given temperature 0, run-to-run variance may be low, but tool-execution outcomes and API nondeterminism can still vary. Several per-cell margins are small (e.g., GPT-4o-mini 1-hop: GoalAct 0.9556 vs ReAct 0.9333, and 4-hop: 0.5625 vs CodeAct 0.3875). Without variance estimates or paired significance tests, the claim that GoalAct consistently achieves SOTA "across different LLM series and varying task difficulties" is not statistically grounded. The authors should report standard deviations over at least several runs and apply a paired test (e.g., bootstrap or Wilcoxon) to the per-task success scores.
[§4.1.1 and §3.2] LegalAgentBench was created by an overlapping author team (reference [12]) and its task types—searching over legal databases, coding-style multi-hop queries, and writing defense documents—mirror the exact skill set hard-coded in GoalAct (Searching, Coding, Writing, Finish). This creates a risk that the benchmark is implicitly shaped toward the proposed method's action space, so the reported 12.22% average improvement may not transfer to other agent benchmarks or to general legal work. The paper should evaluate GoalAct on at least one independent agent benchmark (e.g., AgentBench or a legal-domain benchmark with different task structures) or explicitly discuss why the overlap does not inflate the results.
[§4.2.2, Table 3] The ablation is conducted only on GLM-4-Plus, and the paper does not report whether the removal of the global plan or of individual skills is accompanied by any prompt re-optimization or budget adjustment. Since the maximum iteration limit T=10 is fixed, removing a skill changes how the limited budget is spent; for example, "w/o writing" may force the model to use Searching or Coding for a writing task, which could explain part of the 4.46% overall drop. Repeating the ablation on at least one additional LLM and reporting the distribution of outcomes across runs would make the component-wise attribution more convincing.

minor comments (5)

[§3.2] The introduction mentions "reasoning" as a potential high-level skill, and Section 3.2 says reasoning tasks may require specialized skills, but the implemented action space in Table 4 only includes Searching, Coding, Writing, and Finish. The paper should clarify whether reasoning is an actual skill in the evaluated system or only a planned extension.
[§4.1.2] The description of Plan-and-Solve and Plan-and-Execute baselines is brief; it does not specify whether they use the same tool set, the same two in-context examples, or the same maximum iteration limit T=10. Making these details explicit would strengthen reproducibility.
[§4.1.1] The claim that LegalAgentBench "eliminates the risk of data leakage" is reasonable for a newly released benchmark, but the paper should also note that the benchmark is from the same research group as this work, which is relevant to assessing benchmark independence.
[Figure 2] The case study would be more informative if the full GoalAct trajectory (including the global plan updates after each observation) were shown. The current excerpt only shows the first few steps, making it hard to verify the claim that global planning prevents repeated local-branch attempts.
[Equation (1)] The notation G=(P1A1,P2A2,...) is used before the plan-update rule is introduced, and the subscript indices in Equation (2) are not fully defined (what exactly is T in G_t = π(Q|T|S_t)?). Clarifying the notation would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; empirical evaluation on a same-group benchmark is a validity concern, not circularity.

full rationale

The paper contains no mathematical derivation that could be circular. Equations (1)-(3) define the global plan as a sequence of plan-step/action pairs and its update, while Equation (4) defines success rate as the fraction of key answer strings appearing in the model output. GoalAct's components (global planning, hierarchical execution with Searching, Coding, and Writing skills) are specified by prompts in Table 4 and are not fitted to LegalAgentBench; no parameter is estimated from the benchmark and then reported as a prediction. The only same-author reference is LegalAgentBench [12], used as the testbed. That is an empirical evaluation choice with a validity concern (the benchmark creators overlap with the authors and the substring metric may reward verbose outputs), but the paper does not reduce any claimed result to that citation by construction, nor does it invoke a uniqueness theorem or ansatz from prior work. Ablations and baselines are directly measured. Consequently, no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The framework contributes no new scientific entities; its contributions are architectural and prompt-level. The listed free parameters are hand-chosen experimental hyperparameters and skill definitions, and the axioms are the domain assumptions needed for the empirical claim to generalize. The main unexamined burden is the alignment between the hand-designed skill set and the benchmark tasks.

free parameters (3)

Maximum iteration limit T = 10
Hand-chosen cap on the number of planning and execution steps; not varied in the experiments, so the optimal value for each method is unknown.
Number of in-context examples per method = 2
Two reference examples are provided for every method (Section 4.1.2). This hand-set number can affect prompt adherence and may not be equally optimal for all methods.
GoalAct skill set = Searching, Coding, Writing, Finish
The set of high-level skills is hand-selected and closely matches LegalAgentBench's task types; no procedure is given for choosing or expanding the skill set.

assumptions (3)

ad hoc to paper The fixed skill set {Searching, Coding, Writing, Finish} is sufficient to solve all tasks in LegalAgentBench.
Introduced in Section 3.2 and enforced in the planner prompt (Table 4). If a task requires a skill outside this list, the framework cannot handle it.
domain assumption The LegalAgentBench success-rate metric (Equation 4), counting whether key answer strings appear in the model output, is a valid proxy for task success.
Used in Section 4.1.3. String matching can reward phrase-spotting and may not reflect genuine multi-step legal reasoning.
domain assumption LLM outputs at temperature 0 are sufficiently deterministic that single-run results in Table 2 are comparable.
The paper sets temperature to 0 (Section 4.1.2) but does not report repeated runs, variance, or significance tests; API serving and batching can still introduce non-determinism.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution." pith.science (2026). https://pith.science/paper/DNO6TXVO

@misc{pith2026250416563,
  author       = {Pith},
  title        = {Pith review of: Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DNO6TXVO}},
  note         = {Machine review of arXiv:2504.16563}
}

read the original abstract

Intelligent agent systems based on Large Language Models (LLMs) have shown great potential in real-world applications. However, existing agent frameworks still face critical limitations in task planning and execution, restricting their effectiveness and generalizability. Specifically, current planning methods often lack clear global goals, leading agents to get stuck in local branches, or produce non-executable plans. Meanwhile, existing execution mechanisms struggle to balance complexity and stability, and their limited action space restricts their ability to handle diverse real-world tasks. To address these limitations, we propose GoalAct, a novel agent framework that introduces a continuously updated global planning mechanism and integrates a hierarchical execution strategy. GoalAct decomposes task execution into high-level skills, including searching, coding, writing and more, thereby reducing planning complexity while enhancing the agents' adaptability across diverse task scenarios. We evaluate GoalAct on LegalAgentBench, a benchmark with multiple types of legal tasks that require the use of multiple types of tools. Experimental results demonstrate that GoalAct achieves state-of-the-art (SOTA) performance, with an average improvement of 12.22% in success rate. These findings highlight GoalAct's potential to drive the development of more advanced intelligent agent systems, making them more effective across complex real-world applications. Our code can be found at https://github.com/cjj826/GoalAct.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Agentic Web: Weaving the Next Web with AI Agents
cs.AI 2025-07 conditional novelty 3.0 of 10

A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.

Reference graph

Works this paper leans on

28 extracted references · 11 canonical work pages · cited by 1 Pith paper

[12]

arXiv preprint arXiv:2412.17259 (2024)

Li, H., Chen, J., Yang, J., Ai, Q., Jia, W., Liu, Y., Lin, K., Wu, Y., Yuan, G., Hu, Y., et al.: Legalagentbench: Evaluating llm agents in legal domain. arXiv preprint arXiv:2412.17259 (2024)

arXiv 2024
[1]

: A survey on large language model based autonomous agents

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. : A survey on large language model based autonomous agents. Frontiers of Computer Science 18(6), 186345 (2024)

2024
[2]

: Autowebglm: A large language model-based web navigating agent

Lai, H., Liu, X., Iong, I.L., Yao, S., Chen, Y., Shen, P., Yu, H., Zhang, H., Zhang, X., Dong, Y., et al. : Autowebglm: A large language model-based web navigating agent. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5295–5306 (2024)

work page 2024
[3]

Advances in Neural Information Processing Systems 36, 68539–68551 (2023)

Schick, T., Dwivedi-Yu, J., Dess` ı, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems 36, 68539–68551 (2023)

work page 2023
[4]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023
[5]

arXiv preprint arXiv:2412.19437 (2024) 9

Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al.: Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024) 9

arXiv 2024
[6]

arXiv preprint arXiv:2305.04091 (2023)

Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R.K.-W., Lim, E.-P.: Plan- and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091 (2023)

arXiv 2023
[7]

In: International Conference on Learning Representations (ICLR) (2023)

Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.: React: Synergizing reasoning and acting in language models. In: International Conference on Learning Representations (ICLR) (2023)

2023

Show all 28 references

[8]

In: Forty-first International Conference on Machine Learning (2024)

Wang, X., Chen, Y., Yuan, L., Zhang, Y., Li, Y., Peng, H., Ji, H.: Executable code actions elicit better llm agents. In: Forty-first International Conference on Machine Learning (2024)

2024
[9]

In: International Conference on Applied Engineering and Natural Sciences, vol

Topsakal, O., Akinci, T.C.: Creating large language model applications utilizing langchain: A primer on developing llm apps fast. In: International Conference on Applied Engineering and Natural Sciences, vol. 1, pp. 1050–1056 (2023)

2023
[10]

arXiv preprint arXiv:2502.17943 (2025)

Li, H., Ye, J., Hu, Y., Chen, J., Ai, Q., Wu, Y., Chen, J., Chen, Y., Luo, C., Zhou, Q., et al.: Casegen: A benchmark for multi-stage legal case documents generation. arXiv preprint arXiv:2502.17943 (2025)

2025 arXiv
[11]

: Agentbench: Evaluating llms as agents

Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., et al. : Agentbench: Evaluating llms as agents. In: International Conference on Learning Representations (ICLR) (2024)

2024
[13]

arXiv preprint arXiv:2406.12793 (2024)

GLM, T., Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Zhang, D., Rojas, D., Feng, G., Zhao, H., et al.: Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793 (2024)

2024 arXiv
[14]

arXiv preprint arXiv:2210.02414 (2022)

Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al.: Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414 (2022)

2022 arXiv
[15]

arXiv preprint arXiv:2309.16609 (2023)

Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al.: Qwen technical report. arXiv preprint arXiv:2309.16609 (2023)

2023 arXiv
[16]

: Language models are unsupervised multitask learners

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. : Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)

2019
[17]

: Training language models to follow instructions with human feedback

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. : Training language models to follow instructions with human feedback. Advances in neural information processing systems 35, 27730–27744 (2022) 10

2022
[18]

arXiv preprint arXiv:2109.01652 (2021)

Wei, J., Bosma, M., Zhao, V.Y., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., Le, Q.V.: Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021)

2021 arXiv
[19]

Advances in neural information processing systems 35, 24824–24837 (2022)

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824–24837 (2022)

2022
[20]

arXiv preprint arXiv:2305.16291 (2023)

Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., Anand- kumar, A.: Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023)

2023 arXiv
[21]

arXiv preprint arXiv:2311.17541 (2023)

Qiao, B., Li, L., Zhang, X., He, S., Kang, Y., Zhang, C., Yang, F., Dong, H., Zhang, J., Wang, L., et al.: Taskweaver: A code-first agent framework. arXiv preprint arXiv:2311.17541 (2023)

2023 arXiv
[22]

arXiv preprint arXiv:2410.21276 (2024)

Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

2024 arXiv
[23]

Advances in Neural Information Processing Systems 36, 8634–8652 (2023)

Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., Yao, S.: Reflexion: Lan- guage agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36, 8634–8652 (2023)

2023
[24]

Thinking, Acting, and Observing

Zhong, W., Guo, L., Gao, Q., Ye, H., Wang, Y.: Memorybank: Enhancing large language models with long-term memory. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 19724–19731 (2024) 11 Appendix T able 4 The prompt used in the GoalAct for global p...

2024
[25]

Searching: Retrieve a record from a data table based on information, or filter mul- tiple records that meet specific attribute values
[26]

You can filter, sort, sum, or iterate over queried data

Coding: If the problem is too complex to be solved by querying alone, you may attempt to program a solution. You can filter, sort, sum, or iterate over queried data
[27]

Writing: If content generation (such as a defense statement) is needed, you should attempt writing to resolve the issue
[28]

json [{{ “Thinking

Finish: Provide the final answer and terminate the task. The action must be detailed within square brackets [] and should accurately identify the correct table and returned fields. Observing is the information obtained after an action. If all observations sufficiently answer t...

2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [12]

arXiv preprint arXiv:2412.17259 (2024)

Li, H., Chen, J., Yang, J., Ai, Q., Jia, W., Liu, Y., Lin, K., Wu, Y., Yuan, G., Hu, Y., et al.: Legalagentbench: Evaluating llm agents in legal domain. arXiv preprint arXiv:2412.17259 (2024)

arXiv 2024

[2] [1]

: A survey on large language model based autonomous agents

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. : A survey on large language model based autonomous agents. Frontiers of Computer Science 18(6), 186345 (2024)

2024

[3] [2]

: Autowebglm: A large language model-based web navigating agent

Lai, H., Liu, X., Iong, I.L., Yao, S., Chen, Y., Shen, P., Yu, H., Zhang, H., Zhang, X., Dong, Y., et al. : Autowebglm: A large language model-based web navigating agent. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5295–5306 (2024)

work page 2024

[4] [3]

Advances in Neural Information Processing Systems 36, 68539–68551 (2023)

Schick, T., Dwivedi-Yu, J., Dess` ı, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems 36, 68539–68551 (2023)

work page 2023

[5] [4]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023

[6] [5]

arXiv preprint arXiv:2412.19437 (2024) 9

Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al.: Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024) 9

arXiv 2024

[7] [6]

arXiv preprint arXiv:2305.04091 (2023)

Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R.K.-W., Lim, E.-P.: Plan- and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091 (2023)

arXiv 2023

[8] [7]

In: International Conference on Learning Representations (ICLR) (2023)

Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.: React: Synergizing reasoning and acting in language models. In: International Conference on Learning Representations (ICLR) (2023)

2023