Pith. sign in

REVIEW 5 major objections 5 minor 38 references

SkillSmith: Enhancing Locally Deployed Agents via Automatic Skill Construction and Evolution

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Local agents can match cloud-agent task effectiveness when missing environment knowledge is supplied as automatically constructed Skills.

desk verdict Promising but unproven: SkillSmith's automatic skill construction is a real step forward, but the evidence is single-run and the skill may be memorizing training-task answers rather than generalizable knowledge. read the letter →

arxiv 2608.08037 v1 pith:BFNC2PWE submitted 2026-08-08 cs.AI

Xinle Jiang , Remy Xie , Ming Tang This is my paper · ORCID
classification cs.AI
keywords agentskillslocaldeploymentenvironmentknowledgeskillevolutionsmalllanguagemodelscloud-localcollaborationprogressivedisclosuretaskeffectiveness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SkillSmith claims that the gap between locally deployed agent models and cloud-hosted frontier models is mostly a knowledge gap, not a reasoning gap: small local models already match large cloud models on tool use and instruction following, but lack environment-specific rules and operation procedures. The paper shows that this missing knowledge can be packaged into a Skill document and supplied to a frozen local model without fine-tuning. In the proposed two-stage pipeline, a cloud agent first explores representative tasks and reflects on them to create an initial Skill, then local execution failures are clustered and analyzed to evolve the Skill in bounded edits. With such a Skill, a 27-billion-parameter local model reaches 78.6% pass rate on AppWorld-Normal, 74.9% on AppWorld-Challenge, and 89.0% on WorkBench, cutting average actions per task from 36.1 to 9.9 and transferring to other local backbones without rebuilding. If true, this makes private, low-cost local deployment viable for daily agent tasks that previously required cloud LLM calls.

What carries the argument

The load-bearing object is the Skill, defined as S=(M,R,C): a main Markdown manual, a set of reference files holding environment rules and procedures, and loading conditions that decide when each reference enters context. This progressive disclosure keeps context cost proportional to the current task rather than to the whole knowledge base. Around this object, SkillSmith builds a two-stage black-box optimization: Skill Creation uses three cloud-agent roles (cluster, distill, merge) to turn reflection reports into a length-bounded initial Skill, and Skill Evolution uses local failure reports, a cluster-attribute-merge loop, and a length budget to refine it. The cloud agent acts as a text optimizer that treats execution feedback as a gradient-like signal, following the TextGrad-style view of LLM feedback as a surrogate gradient.

What would settle it

A decisive test: inject one fabricated environment rule into the Cloud Agent's exploration prompt before Skill Creation, then check whether the final Skill propagates that rule and whether the Local Agent follows it on tasks where the rule is false.

Watch

Extended reading notes

Core claim

The central discovery is that non-parametric environment knowledge, delivered as a progressive-disclosure Skill, repairs most local-agent failures. The paper first diagnoses that 91.7% of sampled local-agent failures trace to missing environment rules or operation procedures, not to weak core capabilities, and that manually injecting such knowledge resolves 26 of 33 knowledge-related failures. SkillSmith then automates that injection: a cloud agent executes training tasks, writes reflection reports, and a cluster-distill-merge procedure produces an initial Skill; subsequent rounds compress local failures into reports, attribute each failure cluster to Skill loading, Skill content, or near-miss execution, and apply bounded revisions. The result is a Skill that lifts the frozen Qwen3.6-27B local agent from 36.3% to 78.6% pass rate on AppWorld-Normal, from 28.1% to 74.9% on AppWorld-Challenge, and from 51.1% to 89.0% on WorkBench, outperforming the strongest non-parametric baseline by 11.3 to 26.2 points and generalizing to other SLM backbones without re-running Skill construction.

Load-bearing premise

The pipeline trusts the Cloud Agent's exploration and self-reflection reports as accurate source material; if the Cloud Agent hallucinates an environment rule or misses an implicit procedure, the generated Skill silently encodes that error.

Editorial extensions

If this is right

  • A local 27B model with a SkillSmith Skill reaches 78.6%, 74.9%, and 89.0% pass rates on AppWorld-Normal, AppWorld-Challenge, and WorkBench, versus 36.3%, 28.1%, and 51.1% without the Skill.
  • The Skill cuts average actions per AppWorld-Normal task from 36.1 to 9.9 and cumulative context tokens from 450.7k to 200.6k, so the added reading cost is outweighed by fewer failed steps.
  • SkillSmith's one-time offline cost of about $88 on AppWorld and $77 on WorkBench replaces per-task cloud LLM calls; after roughly 560 AppWorld tasks or 1,400 WorkBench tasks, it is cheaper than serving every task through a cloud agent.
  • Created Skills transfer to other SLM backbones without rebuilding, with pass-rate gains up to +48.2 points on one 31-billion-parameter backbone and +30.8 points on another.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's experiments, the same Skill document could in principle be reused across agent frameworks that support skill-style progressive disclosure, not just the one local framework tested here; cross-framework transfer would test whether the Skill encodes environment knowledge rather than framework-specific prompt formatting.
  • Once an initial Skill exists, the evolution loop's cloud dependency could be replaced by a local SLM doing the clustering and bounded edits, which would eliminate all cloud calls but likely trade away some evolution quality.
  • Because Skill quality is limited by what the cloud teacher knows and reflects, auditing generated Skills against the environment's ground truth (for example, by testing each stated rule against the simulator) could push local-agent performance past the cloud agent's own.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. SkillSmith is a Cloud–Local collaboration framework that automatically constructs a Skill—a main Markdown manual with progressive-disclosure reference files—from Cloud Agent exploration of training tasks, then iteratively evolves it using Local Agent execution feedback, while keeping the Local Agent backbone frozen. The paper motivates the design with a failure analysis attributing most Local Agent failures to missing environment knowledge, formalizes offline Skill learning as black-box optimization over natural-language Skills, and evaluates on AppWorld and WorkBench. The reported results show large pass-rate gains over prompt-, memory-, and skill-based baselines, a reduction in average actions per task, lower online API cost than Cloud Agent serving, and transfer to other SLM backbones.

Significance. The contribution is potentially significant for practical local deployment of LLM agents. The framework addresses a real gap: existing skill-based methods either require expert-authored initial Skills or are not verified and corrected from execution feedback. The paper includes a stage-level ablation showing Creation dominates the gain, an offline-cost analysis from API logs, and a cross-backbone transfer experiment, all of which are useful. The main strength is the explicit role-separated pipeline with concrete prompts and a clear cost model. However, the evidence that the generated Skill encodes reusable environment knowledge rather than memorized training-task answers is incomplete, and the absence of variance estimates weakens the quantitative claims. If the leakage concern is resolved, this would be a strong contribution.

major comments (5)
  1. [Sec. III-C, Algorithm 1, Eq. (2)] The most load-bearing unexamined assumption is the leakage channel from evaluator feedback into the Skill. In Explore and Reflect, each Cloud Agent trajectory is summarized into a Reflection Report that includes 'a more efficient solution path' after the agent has received evaluator feedback containing the expected outcome y* (Eq. 2). The downstream CLUSTER/DISTILL/MERGE prompts (Fig. 5) filter by frequency and cross-cluster generality but impose no explicit check that SKILL.md and the reference files are free of task-specific identifiers, amounts, or exact step sequences from training tasks. Since the test set is drawn from the same environment, memorized training-task content could inflate the held-out pass rates in Table III. Please add (i) an audit of the generated Skill for task-specific content, (ii) an ablation that removes outcome information from reflection (e.g., success/failure only, without y*), and (iii) a comparison of the Skill's high-frequency steps against the actual training task instructions to show they encode environment procedures rather than particular solutions.
  2. [Table III and Table VI] All headline results are single numbers with no error bars, confidence intervals, or significance tests. The Local Agent and Cloud Agent are stochastic systems, so the reported margins (e.g., 11.3-26.2 points over the strongest baseline in Table III, and cross-backbone deltas in Table VI) may reflect run-to-run variation. Please report means and variances over multiple independent runs, or justify determinism of the evaluation protocol, and include paired significance tests where comparisons are made on the same task sets.
  3. [Table II] The diagnostic claim that 91.7% of Local Agent failures are caused by missing environment rules or operation procedures is based on only 36 sampled trajectories. This sample is too small to support a precise attribution, and the manual classification has no reported agreement or confidence interval. Please report the sampling procedure, a larger sample, and at least a bootstrap confidence interval for the 91.7% figure, or otherwise temper the claim.
  4. [Sec. IV-A, Trace2Skill] The Trace2Skill baseline is initialized from 'a initial skill derived from the description of datasets,' which is likely weaker than the intended initialization of a skill-evolution method. Since Trace2Skill is the strongest baseline on WorkBench (62.8 vs. 89.0) and one of the closest on AppWorld-Challenge, this setup may understate a key competitor. Please initialize Trace2Skill with the same automatic Skill Creation output as SkillSmith, or with an expert-authored initial Skill, and report results for both.
  5. [Sec. III-B, Algorithm 1] The final Skill is selected by the best empirical pass rate on the training set (Algorithm 1, lines 17-19), and the evolution objective in Eq. (6) is also the training pass rate. This creates a risk of overfitting to D_train, which is especially relevant given the leakage channel in Major Comment 1. Please use a held-out validation split for model selection, or report the gap between training and test pass rates across Skill versions, to demonstrate that the reported test gains are not artifacts of selection on training tasks.
minor comments (5)
  1. [Abstract and Sec. I] The abstract and introduction contain typos including 'depolying', 'depolyment', and 'adotpts'; please proofread the manuscript.
  2. [Table I] Table I reports Qwen3.6-27B versus Claude Opus 4.5 values from the Qwen technical report; since these are not independently verified, please state this clearly and avoid over-claiming 'comparable' from a single third-party source.
  3. [Figure 8] The y-axis of Figure 8 starts at 55, which visually exaggerates the evolution gains; use a full axis or an explicit axis break.
  4. [Sec. IV-A] The implementation details state that Skill Evolution 'runs for at five rounds'; specify R=5 and report whether the training/test split is fixed across all baselines.
  5. [Sec. II-C, Eq. (1)] Eq. (1) defines Skill S=(M,R,C), but the budget L_max and reference file cap of 250 lines are introduced later; please define the feasibility set S_Lmax explicitly and state whether the cap applies to each reference file or the total reference content.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SkillSmith's headline results are computed on a held-out test set after optimizing only on training tasks.

full rationale

The paper's central derivation chain is empirically self-contained. SkillSmith constructs the Skill from a sampled training set (180 AppWorld tasks and 207 WorkBench tasks) and evaluates it on the remaining held-out tasks, as stated in Sec. IV-A: "We randomly sample 180 AppWorld tasks (about 25%) and 207 WorkBench tasks (about 30%) as training set; the remaining tasks are used as test set for evaluation." The optimization objective in Sec. III-B maximizes the empirical pass rate on D_train, and Algorithm 1 selects the Skill version with the best training pass rate; the reported Table III pass rates are on the disjoint test set. Thus the headline "prediction" is not obtained by fitting the evaluation data. The use of evaluation feedback and "more efficient solution path" in Reflection Reports is a training-signal channel, not a test-set channel, and the Creation pipeline explicitly promotes only frequent, cross-cluster insights rather than copying individual task answers. Any concern that the Cloud Agent's reflection reports could memorize training-task specifics is an empirical leakage-robustness question, not an equation-level equivalence between inputs and outputs. The Skill definition is adopted from Anthropic's Agent Skills as a design choice, not derived from the paper's own results, and no uniqueness theorem or load-bearing self-citation is invoked. The only possibly overlapping citation, Trace2Skill [11], is used as a baseline and does not justify SkillSmith's central claims. Therefore no specific circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen system hyperparameters and four domain assumptions about the reliability of the cloud agent, the evaluator, and the training-split representativeness. No new physical or mathematical entities are introduced; the only invented artifacts are internal report formats (Reflection Reports, Failure Reports) that are not independently verified.

free parameters (4)
  • Skill length budget L_max = 400 lines for main manual
    Chosen by hand in Sec. IV-A; affects how much knowledge can be stored and attended to.
  • Reference file length cap = 250 lines each
    Chosen by hand in Sec. IV-A; controls context overhead and may exclude useful detail.
  • Evolution rounds R = 5
    Chosen by hand in Sec. IV-A; determines how many feedback iterations are applied.
  • Merge thresholds in prompts = At least N/2 clusters for main steps/rules, at least 3 clusters for pitfalls
    Hand-written thresholds in the Merge Agent prompt in Fig. 5; these directly affect which knowledge is promoted to SKILL.md.
assumptions (4)
  • domain assumption Cloud Agent reflection reports are accurate enough to serve as ground truth for skill construction.
    Sec. III-C treats EXECUTEREFLECT output as reusable environment knowledge with no independent verification of correctness.
  • domain assumption The training tasks sampled from each benchmark are representative of the test distribution.
    Sec. IV-A randomly samples 180 AppWorld and 207 WorkBench tasks as training, and the rest as test, assuming the split preserves the scenario distribution.
  • domain assumption Evaluator feedback correctly identifies task success and failure causes.
    The skill evolution stage relies on evaluator feedback to attribute failures to Skill loading, Skill content, or near-miss execution; no validation of evaluator reliability is provided.
  • domain assumption The diagnostic sample of 36 failure trajectories is representative of all Local Agent failure modes.
    Sec. II-B uses 36 trajectories to conclude that 91.7% of failures are knowledge-related, but no confidence interval or stratified sampling is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SkillSmith: Enhancing Locally Deployed Agents via Automatic Skill Construction and Evolution." pith.science (2026). https://pith.science/paper/BFNC2PWE

@misc{pith2026260808037,
  author       = {Pith},
  title        = {Pith review of: SkillSmith: Enhancing Locally Deployed Agents via Automatic Skill Construction and Evolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BFNC2PWE}},
  note         = {Machine review of arXiv:2608.08037}
}
read the original abstract

LLM-based agent frameworks now act as personal assistants for multi-step tasks. Existing agent frameworks such as OpenClaw commonly follow the Cloud Agent depolyment mode using closed-source cloud LLMs as backbone model, which may expose private user information and incur repeated LLM-calling costs. Local Agents address these deployment concerns by depolying frontier open-source SLMs on user-controlled devices, but their task effectiveness still lags far behind Cloud Agents. Through diagnostic analysis, we reveal that the limited effectiveness of Local Agents with frontier SLM backbones mainly comes from missing environment knowledge caused by limited backbone model scale including environment rules and operation procedures. To supply such knowledge non-parametrically, context-efficiently, and without expert authoring, we present SkillSmith, a Cloud--Local Agent collaboration framework that uses Skill as a context-efficient knowledge carrier, automatic constructs Skill from Cloud Agent task exploration and evolves Skill using Local Agent execution feedback to enhance a frozen Local Agent. Experiments on daily agent task datasets AppWorld and WorkBench show that the automatically generated Skill enables the Local Agent with Qwen3.6-27B(SLM) to achieve task effectiveness comparable to Cloud Agents with frontier LLMs, outperform the strongest non-parametric baselines, reduce average actions per task from 36.1 to 9.9 on AppWorld-Normal, and generalize to other SLM backbone models without rerunning Skill construction.

Figures

Figures reproduced from arXiv: 2608.08037 by the authors.

Figure 1
Figure 1. Task pass rates under three deployment settings: Cloud Agents with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. LLM-based agent framework and two deployment modes: Cloud [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. SkillSmith system overview. these reports, attributes failures against the current Skill, and merges the modifications into Si ( 5 ) Finally, Si is stored back into the Skill Library for the next iteration ( 6 ). B. Problem Formulation To clarify how the Offline Learning Phase obtains a high￾quality Skill f, we cast Offline Learning as black-box Skill optimization. We first define Skill-conditioned execution and the… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Skill Creation pipeline and prompts for three cloud-agent roles. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: LLM API cost of SkillSmith versus a Cloud Agent with Claude [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Skill Evolution dynamics on AppWorld. (a) Training pass rate across [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 22 canonical work pages

  1. [1]

    OpenClaw documentation,

    OpenClaw, “OpenClaw documentation,” https://docs.openclaw.ai/, 2026, accessed July 22, 2026

    work page 2026

  2. [2]

    Claude Code documentation,

    Anthropic, “Claude Code documentation,” https://code.claude.com/docs/ en/overview, 2026, accessed July 22, 2026

    work page 2026

  3. [3]

    Hermes Agent,

    Nous Research, “Hermes Agent,” https://github.com/NousResearch/ hermes-agent, 2026, gitHub repository. Accessed July 22, 2026

    work page 2026

  4. [4]

    CE-CoLLM: Efficient and adaptive large language models through cloud-edge collaboration,

    H. Jin and Y . Wu, “CE-CoLLM: Efficient and adaptive large language models through cloud-edge collaboration,”arXiv preprint arXiv:2411.02829, 2024

    arXiv 2024

  5. [5]

    Distilling LLM agent into small models with retrieval and code tools,

    M. Kang, J. Jeong, S. Lee, J. Cho, and S. J. Hwang, “Distilling LLM agent into small models with retrieval and code tools,”arXiv preprint arXiv:2505.17612, 2025

    arXiv 2025

  6. [6]

    Language models are few-shot learners,

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Am...

    work page 2020

  7. [7]

    Tool documentation enables zero-shot tool- usage with large language models,

    C.-Y . Hsieh, S.-A. Chen, C.-L. Li, Y . Fujii, A. Ratner, C.-Y . Lee, R. Krishna, and T. Pfister, “Tool documentation enables zero-shot tool- usage with large language models,”arXiv preprint arXiv:2308.00675, 2023

    arXiv 2023

  8. [8]

    ExpeL: LLM agents are experiential learners,

    A. Zhao, D. Huang, Q. Xu, M. Lin, Y .-J. Liu, and G. Huang, “ExpeL: LLM agents are experiential learners,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, 2024, pp. 19 632– 19 642

    work page 2024

Show all 38 references
  1. [9]

    Agent workflow memory,

    Z. Z. Wang, J. Mao, D. Fried, and G. Neubig, “Agent workflow memory,” inInternational Conference on Machine Learning (ICML), PMLR 267, 2025, pp. 63 897–63 911

    2025

  2. [10]

    Equipping agents for the real world with agent skills,

    Anthropic, “Equipping agents for the real world with agent skills,” https://www.anthropic.com/engineering/ equipping-agents-for-the-real-world-with-agent-skills, 2025, announced October 16, 2025

    2025

  3. [11]

    Trace2skill: Distill trajectory-local lessons into transferable agent skills,

    J. Ni, Y . Liu, X. Liu, Y . Sun, M. Zhou, P. Cheng, D. Wang, E. Zhao, X. Jiang, and G. Jiang, “Trace2skill: Distill trajectory-local lessons into transferable agent skills,”arXiv preprint arXiv:2603.25158, 2026

    2026 arXiv

  4. [12]

    EvoSkill: Automated skill discovery for multi-agent systems,

    S. Alzubi, N. Provenzano, J. Bingham, W. Chen, and T. Vu, “EvoSkill: Automated skill discovery for multi-agent systems,”arXiv preprint arXiv:2603.02766, 2026

    2026 arXiv

  5. [13]

    Co- EvoSkills: Self-evolving agent skills via co-evolutionary verification,

    H. Zhang, S. Fan, H. P. Zou, Y . Chen, Z. Wang, J. Zhou, C. Li, W.-C. Huang, Y . Yao, K. Zheng, X. Liu, X. Li, and P. S. Yu, “Co- EvoSkills: Self-evolving agent skills via co-evolutionary verification,” arXiv preprint arXiv:2604.01687, 2026

    2026 arXiv

  6. [14]

    ReAct: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “ReAct: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations (ICLR), 2023

    2023

  7. [15]

    Qwen3.6-27B model card,

    Qwen Team, “Qwen3.6-27B model card,” https://huggingface.co/Qwen/ Qwen3.6-27B, 2026, accessed July 22, 2026

    2026

  8. [16]

    SWE-bench: Can language models resolve real-world GitHub issues?

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “SWE-bench: Can language models resolve real-world GitHub issues?” inInternational Conference on Learning Representa- tions (ICLR), 2024

    2024

  9. [17]

    Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces,

    M. A. Merrill, A. G. Shaw, N. Carlini, B. Li, H. Raj, I. Bercovich et al., “Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces,”arXiv preprint arXiv:2601.11868, 2026

    2026 arXiv

  10. [18]

    QwenClawBench: A real-user-distribution bench- mark for evaluating OpenClaw agents,

    Skylenage AI, “QwenClawBench: A real-user-distribution bench- mark for evaluating OpenClaw agents,” https://huggingface.co/datasets/ skylenage-ai/QwenClawBench, 2026, hugging Face Datasets

    2026

  11. [19]

    GPQA: A graduate-level google- proof Q&A benchmark,

    D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y . Pang, J. Dirani, J. Michael, and S. R. Bowman, “GPQA: A graduate-level google- proof Q&A benchmark,” inConference on Language Modeling (COLM), 2024

    2024

  12. [20]

    Claude model overview,

    Anthropic, “Claude model overview,” https://platform.claude.com/docs/ en/docs/about-claude/models/overview, 2026, accessed July 22, 2026

    2026

  13. [21]

    SoK: Agentic skills – beyond tool use in LLM agents,

    Y . Jiang, D. Li, H. Deng, B. Ma, X. Wang, Q. Wang, and G. Yu, “SoK: Agentic skills – beyond tool use in LLM agents,”arXiv preprint arXiv:2602.20867, 2026

    2026 arXiv

  14. [22]

    Memento: Fine-tuning LLM agents without fine-tuning LLMs,

    H. Zhou, Y . Chen, S. Guo, X. Yan, K. H. Lee, Z. Wang, K. Y . Lee, G. Zhang, K. Shao, L. Yang, and J. Wang, “Memento: Fine-tuning LLM agents without fine-tuning LLMs,”arXiv preprint arXiv:2508.16153, 2025

    2025 arXiv

  15. [23]

    Memp: Exploring agent procedural memory,

    R. Fang, Y . Liang, X. Wang, J. Wu, S. Qiao, P. Xie, F. Huang, H. Chen, and N. Zhang, “Memp: Exploring agent procedural memory,” inFindings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 17 490–17 502

    2026

  16. [24]

    On the structural memory of LLM agents,

    R. Zeng, J. Fang, S. Liu, and Z. Meng, “On the structural memory of LLM agents,”arXiv preprint arXiv:2412.15266, 2024

    2024 arXiv

  17. [25]

    SkillX: Automatically constructing skill knowledge bases for agents,

    C. Wang, Z. Yu, X. Xie, W. Yao, R. Fang, S. Qiao, K. Cao, G. Zheng, X. Qi, P. Zhang, and S. Deng, “SkillX: Automatically constructing skill knowledge bases for agents,”arXiv preprint arXiv:2604.04804, 2026

    2026 arXiv

  18. [26]

    SkillRL: Evolving agents via recursive skill-augmented reinforcement learning,

    P. Xia, J. Chen, H. Wang, J. Liu, K. Zeng, Y . Wang, S. Han, Y . Zhou, X. Zhao, H. Chen, Z. Zheng, C. Xie, and H. Yao, “SkillRL: Evolving agents via recursive skill-augmented reinforcement learning,”arXiv preprint arXiv:2602.08234, 2026

    2026 arXiv

  19. [27]

    SkillOpt: Executive strategy for self-evolving agent skills,

    Y . Yang, Z. Gong, W. Huang, Q. Yang, Z. Zhou, Z. Huang, Y . Li, X. Gao, Q. Dai, B. Liu, K. Qiu, Y . Yang, D. Chen, X. Yang, and C. Luo, “SkillOpt: Executive strategy for self-evolving agent skills,” arXiv preprint arXiv:2605.23904, 2026

    2026 arXiv

  20. [28]

    Optimizing generative AI by backpropagating language model feedback,

    M. Yuksekgonul, F. Bianchi, J. Boen, S. Liu, P. Lu, Z. Huang, C. Guestrin, and J. Zou, “Optimizing generative AI by backpropagating language model feedback,”Nature, vol. 639, no. 8055, pp. 609–616, 2025

    2025

  21. [29]

    V oyager: An open-ended embodied agent with large language models,

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar, “V oyager: An open-ended embodied agent with large language models,”Transactions on Machine Learning Research, 2024

    2024

  22. [30]

    AutoManual: Constructing instruction manuals by LLM agents via interactive environ- mental learning,

    M. Chen, Y . Li, Y . Yang, S. Yu, B. Lin, and X. He, “AutoManual: Constructing instruction manuals by LLM agents via interactive environ- mental learning,” inAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  23. [31]

    AppWorld: A controllable world of apps and people for benchmarking interactive coding agents,

    H. Trivedi, T. Khot, M. Hartmann, R. Manku, V . Dong, E. Li, S. Gupta, A. Sabharwal, and N. Balasubramanian, “AppWorld: A controllable world of apps and people for benchmarking interactive coding agents,” inProceedings of the 62nd Annual Meeting of the Association for Computat...

    2024

  24. [32]

    WorkBench: A benchmark dataset for agents in a realistic workplace setting,

    O. Styles, S. Miller, P. Cerda-Mardini, T. Guha, V . Sanchez, and B. Vidgen, “WorkBench: A benchmark dataset for agents in a realistic workplace setting,” inConference on Language Modeling (COLM), 2024

    2024

  25. [33]

    OpenAI model documentation,

    OpenAI, “OpenAI model documentation,” https://platform.openai.com/ docs/models, 2026, accessed July 22, 2026

    2026

  26. [34]

    Qwen3-30B-A3B model card,

    Qwen Team, “Qwen3-30B-A3B model card,” https://huggingface.co/ Qwen/Qwen3-30B-A3B, 2025, accessed July 31, 2026

    2025

  27. [35]

    Gemma 4 31B model card,

    Google DeepMind, “Gemma 4 31B model card,” https://huggingface.co/ google/gemma-4-31B, 2026, accessed July 31, 2026

    2026

  28. [36]

    FrugalGPT: How to use large language models while reducing cost and improving performance,

    L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving performance,”Transactions on Machine Learning Research, 2024

    2024

  29. [37]

    Remember me, refine me: A dynamic procedural memory framework for experience-driven agent evolution,

    Z. Cao, J. Deng, L. Yu, W. Zhou, Z. Liu, B. Ding, and H. Zhao, “Remember me, refine me: A dynamic procedural memory framework for experience-driven agent evolution,” inFindings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 16 803–16 822

    2026

  30. [38]

    Learning hierarchical procedural memory for LLM agents through Bayesian selection and contrastive refinement,

    S. Forouzandeh, W. Peng, P. Moradi, X. Yu, and M. Jalili, “Learning hierarchical procedural memory for LLM agents through Bayesian selection and contrastive refinement,” inProceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2026

    2026

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.