pith. machine review for the scientific record. sign in

arxiv: 2308.00352 · v7 · submitted 2023-08-01 · 💻 cs.AI · cs.MA

Recognition: 2 theorem links

MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

Ceyao Zhang, Chenglin Wu, Chenyu Ran, Jiaqi Chen, Jinlin Wang, J\"urgen Schmidhuber, Lingfeng Xiao, Liyang Zhou, Mingchen Zhuge, Sirui Hong, Steven Ka Shing Yau, Xiawu Zheng, Yuheng Cheng, Zijuan Lin, Zili Wang

Authors on Pith no claims yet

Pith reviewed 2026-05-11 03:37 UTC · model grok-4.3

classification 💻 cs.AI cs.MA
keywords multi-agent systemslarge language modelsmeta-programmingstandardized operating procedurescollaborative frameworkssoftware engineeringagent rolesassembly line workflow
0
0 comments X

The pith

MetaGPT encodes human-standardized procedures into LLM agent prompts to produce more coherent multi-agent solutions for complex software tasks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MetaGPT is a framework that turns human operating procedures into structured prompts for teams of large language model agents. By assigning specialized roles like product manager, architect, and engineer in an assembly-line workflow, it breaks complex tasks into subtasks while having agents check each other's intermediate outputs. This approach aims to cut down on the hallucinations that occur when LLMs simply talk to each other in unstructured chains. The result is more consistent code and plans on software engineering benchmarks compared to earlier chat-style multi-agent setups.

Core claim

MetaGPT incorporates Standardized Operating Procedures (SOPs) into prompt sequences within a meta-programming framework, enabling LLM-based agents to collaborate via an assembly line paradigm that assigns roles and verifies intermediate results, leading to more coherent solutions on collaborative software engineering benchmarks than previous systems.

What carries the argument

The assembly line paradigm with SOP-encoded prompts that assign roles and enforce verification steps.

If this is right

  • Agents can handle more complex tasks by decomposing them into subtasks with built-in verification.
  • Cascading errors decrease because each role checks outputs before passing them downstream.
  • Software development workflows become more scalable through role-specialized collaboration.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same SOP-encoding approach could be tested in domains outside software, such as scientific experiment design or business process automation.
  • Automatic discovery of SOPs from examples might reduce the need for manual encoding of procedures.
  • Pairing the framework with external code interpreters could strengthen the verification steps beyond prompt-based checks.

Load-bearing premise

That encoding Standardized Operating Procedures into prompt sequences will enable agents to effectively verify intermediate results and reduce cascading hallucinations from naively chained LLMs.

What would settle it

A direct comparison on a collaborative software engineering benchmark where MetaGPT and a baseline chat-based multi-agent system are given the same task, with results showing whether MetaGPT produces measurably fewer logic inconsistencies and more coherent outputs.

read the original abstract

Remarkable progress has been made on automated problem solving through societies of agents based on large language models (LLMs). Existing LLM-based multi-agent systems can already solve simple dialogue tasks. Solutions to more complex tasks, however, are complicated through logic inconsistencies due to cascading hallucinations caused by naively chaining LLMs. Here we introduce MetaGPT, an innovative meta-programming framework incorporating efficient human workflows into LLM-based multi-agent collaborations. MetaGPT encodes Standardized Operating Procedures (SOPs) into prompt sequences for more streamlined workflows, thus allowing agents with human-like domain expertise to verify intermediate results and reduce errors. MetaGPT utilizes an assembly line paradigm to assign diverse roles to various agents, efficiently breaking down complex tasks into subtasks involving many agents working together. On collaborative software engineering benchmarks, MetaGPT generates more coherent solutions than previous chat-based multi-agent systems. Our project can be found at https://github.com/geekan/MetaGPT

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces MetaGPT, a meta-programming framework for LLM-based multi-agent collaboration. It encodes Standardized Operating Procedures (SOPs) into prompt sequences and adopts an assembly-line paradigm to assign diverse roles to agents, breaking complex tasks into subtasks. The central claim is that this produces more coherent solutions than prior chat-based multi-agent systems on collaborative software engineering benchmarks by enabling agents to verify intermediate results and thereby reduce cascading hallucinations.

Significance. If the empirical claims are substantiated, the work provides a concrete method for structuring multi-agent LLM systems around human-like workflows, which could improve reliability for multi-step tasks such as software engineering. The public GitHub release of the code is a clear strength that supports reproducibility and community follow-up.

major comments (2)
  1. Abstract: the claim that MetaGPT 'generates more coherent solutions than previous chat-based multi-agent systems' on benchmarks is asserted without any quantitative metrics, dataset names, baseline descriptions, or error analysis, leaving the central performance claim unsupported in the provided text.
  2. Abstract and framework description: the reduction in cascading hallucinations is attributed specifically to SOP encoding that lets agents 'verify intermediate results,' yet the manuscript simultaneously introduces role assignment and assembly-line decomposition; no ablation or controlled comparison is described that isolates the SOP component as the causal factor.
minor comments (1)
  1. The abstract would be clearer if it named the specific collaborative software engineering benchmarks used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive review. We address each major comment below and have revised the manuscript to better support the central claims with additional details and analysis.

read point-by-point responses

  1. Referee: Abstract: the claim that MetaGPT 'generates more coherent solutions than previous chat-based multi-agent systems' on benchmarks is asserted without any quantitative metrics, dataset names, baseline descriptions, or error analysis, leaving the central performance claim unsupported in the provided text.

    Authors: We agree that the abstract, being a high-level summary, does not include quantitative details. The full manuscript (Section 4) reports results on collaborative software engineering benchmarks, including specific datasets, comparisons against baselines such as ChatDev and AutoGPT, and analysis of reduced hallucinations. We have revised the abstract to include key quantitative metrics, dataset names, baseline references, and a brief note on error reduction to make the claim self-contained. revision: yes

  2. Referee: Abstract and framework description: the reduction in cascading hallucinations is attributed specifically to SOP encoding that lets agents 'verify intermediate results,' yet the manuscript simultaneously introduces role assignment and assembly-line decomposition; no ablation or controlled comparison is described that isolates the SOP component as the causal factor.

    Authors: We acknowledge that the framework integrates SOP encoding with role assignment and assembly-line decomposition, and that the original manuscript does not include an explicit ablation isolating SOP. The SOP component is the mechanism that encodes verification steps into the workflow. In the revision we have added a controlled ablation comparing the full system against a variant that replaces SOP-structured prompts with unstructured chat-based interaction while retaining roles and decomposition; the results show a measurable increase in hallucinations without SOP. We have also clarified the attribution in the framework description. revision: yes

Circularity Check

0 steps flagged

No circularity: MetaGPT is a constructive framework proposal with no equations or self-referential derivations

full rationale

The paper introduces MetaGPT as a new meta-programming framework that encodes Standardized Operating Procedures into LLM prompts and applies an assembly-line role decomposition for multi-agent collaboration. No mathematical equations, fitted parameters, or quantitative predictions appear in the abstract or description. Claims of improved coherence on software engineering benchmarks are presented as empirical outcomes of the proposed system rather than derivations that reduce to prior inputs by construction. No self-citations are invoked as load-bearing uniqueness theorems or ansatzes. The derivation chain is therefore self-contained as an independent engineering construction, consistent with the reader's assessment of score 1.0 and the absence of any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The framework rests on the domain assumption that LLMs can reliably follow role-specific SOP prompts to self-verify outputs; no free parameters or new physical entities are introduced.

axioms (1)
  • domain assumption LLMs prompted with domain-specific roles and SOPs can verify intermediate results and reduce cascading hallucinations
    Invoked in the description of how MetaGPT improves over naive chaining of LLMs.
invented entities (1)
  • MetaGPT meta-programming framework no independent evidence
    purpose: To encode human workflows as prompt sequences for structured multi-agent LLM collaboration
    The central new artifact introduced by the paper; no independent evidence outside the framework itself is provided in the abstract.

pith-pipeline@v0.9.0 · 5513 in / 1197 out tokens · 33777 ms · 2026-05-11T03:37:50.111708+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Why Do Multi-Agent LLM Systems Fail?

    cs.AI 2025-03 unverdicted novelty 8.0

    The authors create the first large-scale dataset and taxonomy of failure modes in multi-agent LLM systems to explain their limited performance gains.

  2. EVOCHAMBER: Test-Time Co-evolution of Multi-Agent System at Individual, Team, and Population Scales

    cs.AI 2026-05 unverdicted novelty 7.0

    EVOCHAMBER enables test-time co-evolution of multi-agent systems across three scales, producing emergent niche specialists and performance gains of up to 32% relative on math tasks with Qwen3-8B.

  3. Collective Alignment in LLM Multi-Agent Systems: Disentangling Bias from Cooperation via Statistical Physics

    cond-mat.stat-mech 2026-05 unverdicted novelty 7.0

    LLM multi-agent systems on lattices show bias-driven order-disorder crossovers instead of true phase transitions, with extracted effective couplings and fields serving as model-specific fingerprints.

  4. TourMart: A Parametric Audit Instrument for Commission Steering in LLM Travel Agents

    cs.CY 2026-05 unverdicted novelty 7.0

    TourMart quantifies commission steering in LLM travel agents via paired counterfactual prompts, reporting 3.5-7.7 percentage point increases in steered recommendations for tested models.

  5. MOTOR-Bench: A Real-world Dataset and Multi-agent Framework for Zero-shot Human Mental State Understanding

    cs.CV 2026-05 unverdicted novelty 7.0

    MOTOR-Bench supplies a real-world video dataset for structured mental state understanding in learning settings, while MOTOR-MAS improves zero-shot prediction of behavior, cognition, and emotion labels over single mode...

  6. Social Bias in LLM-Generated Code: Benchmark and Mitigation

    cs.SE 2026-05 unverdicted novelty 7.0

    LLMs show up to 60.58% social bias in generated code; a new Fairness Monitor Agent cuts bias by 65.1% and raises functional correctness from 75.80% to 83.97%.

  7. Theory Under Construction: Orchestrating Language Models for Research Software Where the Specification Evolves

    cs.SE 2026-04 unverdicted novelty 7.0

    Comet-H orchestrates LLMs via deficit-scoring prompt selection and half-life task tracking to co-evolve research software components, demonstrated by a static analysis tool reaching F1=0.768 versus a 0.364 baseline.

  8. RepoDoc: A Knowledge Graph-Based Framework to Automatic Documentation Generation and Incremental Updates

    cs.SE 2026-04 unverdicted novelty 7.0

    RepoDoc uses a repository knowledge graph with module clustering and semantic impact propagation to generate more complete documentation 3x faster with 85% fewer tokens and handle incremental updates 73% faster than p...

  9. Symbolic Execution Meets Multi-LLM Orchestration: Detecting Memory Vulnerabilities in Incomplete Rust CVE Snippets

    cs.CR 2026-04 unverdicted novelty 7.0

    A 4-agent LLM orchestration with KLEE symbolic execution generates harnesses for incomplete Rust CVE snippets, achieving 90.3% compilation success and detecting 1206 errors across 26 of 31 files versus far lower rates...

  10. Constraint-Guided Multi-Agent Decompilation for Executable Binary Recovery

    cs.SE 2026-04 unverdicted novelty 7.0

    A constraint-guided multi-agent system turns raw decompiler output into re-executable code at 84-97% success rates, outperforming prior LLM decompilation methods on real binaries.

  11. ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents

    cs.CV 2026-04 unverdicted novelty 7.0

    ClawMark is a new benchmark for multi-turn multi-day multimodal coworker agents in stateful evolving services, with deterministic Python checkers showing frontier models achieve only 20% strict task success.

  12. A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework

    cs.CR 2026-04 unverdicted novelty 7.0

    A new 7x4 taxonomy organizes agentic AI security threats by architectural layer and persistence timescale, revealing under-explored upper layers and missing defenses after surveying 116 papers.

  13. Synthesizing Multi-Agent Harnesses for Vulnerability Discovery

    cs.CR 2026-04 unverdicted novelty 7.0

    AgentFlow uses a typed graph DSL covering roles, prompts, tools, topology and protocol plus a runtime-signal feedback loop to optimize multi-agent harnesses, reaching 84.3% on TerminalBench-2 and discovering ten new z...

  14. Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data

    cs.AI 2026-04 unverdicted novelty 7.0

    MALMAS is a memory-augmented multi-agent LLM system that generates diverse, high-quality features for tabular data via agent decomposition, routing, and iterative memory-guided refinement.

  15. Weak-Link Optimization for Multi-Agent Reasoning and Collaboration

    cs.AI 2026-04 unverdicted novelty 7.0

    WORC improves multi-agent LLM reasoning to 82.2% average accuracy by predicting and compensating for the weakest agent via targeted extra sampling rather than uniform reinforcement.

  16. Towards Personalizing Secure Programming Education with LLM-Injected Vulnerabilities

    cs.CR 2026-04 conditional novelty 7.0

    LLM agents inject CWEs into student-authored code to generate personalized security examples; in a 71-student deployment, participants rated them more relevant than textbook cases but quantitative differences remained...

  17. Strategic Persuasion with Trait-Conditioned Multi-Agent Systems for Iterative Legal Argumentation

    cs.MA 2026-04 unverdicted novelty 7.0

    Multi-agent LLM simulations with trait-conditioned agents and a reinforcement-learning orchestrator show heterogeneous teams and dynamic trait selection outperform static configurations in simulated legal argumentation.

  18. Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

    cs.AI 2026-04 conditional novelty 7.0

    NARCBench and five activation-probing methods detect multi-agent collusion with 0.73-1.00 AUROC across distribution shifts and steganographic tasks by aggregating per-agent signals.

  19. Agent Factories for High Level Synthesis: How Far Can General-Purpose Coding Agents Go in Hardware Optimization?

    cs.AI 2026-03 conditional novelty 7.0

    An agent factory combining sub-kernel ILP assembly with multi-agent cross-optimization lets general coding agents deliver mean 8.27x speedups in HLS designs on standard benchmarks.

  20. GAIA: a benchmark for General AI Assistants

    cs.CL 2023-11 unverdicted novelty 7.0

    GAIA benchmark shows humans at 92% accuracy on simple real-world questions far outperform current AI systems at 15%, proposing this gap as a key milestone for general AI.

  21. SkillEvolver: Skill Learning as a Meta-Skill

    cs.AI 2026-05 unverdicted novelty 6.0

    A meta-skill authors and refines prose-and-code skills for agents by learning from post-deployment failures with an overfit audit, achieving 56.8% accuracy on SkillsBench tasks versus 43.6% for human-curated skills.

  22. OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces

    cs.AI 2026-05 unverdicted novelty 6.0

    OPT-BENCH and OPT-Agent evaluate LLM self-optimization in large search spaces, showing stronger models improve via feedback but stay constrained by base capacity and below human performance.

  23. Tail-aware N-version Machine Learning Models for Reliable API Recommendation

    cs.SE 2026-04 unverdicted novelty 6.0

    NvRec profiles multiple API recommendation models on tail-API performance and applies majority voting with reliability filters to raise true accept rates while controlling rejection of uncertain outputs.

  24. MarketBench: Evaluating AI Agents as Market Participants

    cs.AI 2026-04 unverdicted novelty 6.0

    LLMs show poor calibration in predicting task success and token use on software engineering benchmarks, causing market auctions to underperform compared to perfect information scenarios, with limited improvement from ...

  25. ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation

    cs.AI 2026-04 unverdicted novelty 6.0

    ClawTrace enables cost-aware LLM agent skill distillation by tracing per-step costs and generating preserve, prune, and repair patches, with ablations showing reduced regressions and prune rules transferring to cut co...

  26. CineAGI: Character-Consistent Movie Creation through LLM-Orchestrated Multi-Modal Generation and Cross-Scene Integration

    cs.MM 2026-04 unverdicted novelty 6.0

    CineAGI is a multi-agent LLM framework that generates multi-scene movies with improved character consistency, narrative coherence, and audio-visual alignment.

  27. WebGen-R1: Incentivizing Large Language Models to Generate Functional and Aesthetic Websites with Reinforcement Learning

    cs.CL 2026-04 unverdicted novelty 6.0

    WebGen-R1 uses end-to-end RL with scaffold-driven generation and cascaded rewards for structure, function, and aesthetics to transform a 7B model into a generator of deployable multi-page websites that rivals much lar...

  28. ActuBench: A Multi-Agent LLM Pipeline for Generation and Evaluation of Actuarial Reasoning Tasks

    cs.AI 2026-04 unverdicted novelty 6.0

    ActuBench is a multi-agent LLM pipeline for generating and evaluating actuarial reasoning tasks, with evaluations of 50 models showing effective verification, competitive local open-weights models, and differing ranki...

  29. QRAFTI: An Agentic Framework for Empirical Research in Quantitative Finance

    cs.MA 2026-04 unverdicted novelty 6.0

    QRAFTI is a multi-agent framework using tool-calling and reflection-based planning to emulate quant research tasks like factor replication and signal testing on financial data.

  30. AIT Academy: Cultivating the Complete Agent with a Confucian Three-Domain Curriculum

    cs.AI 2026-04 unverdicted novelty 6.0

    AIT Academy introduces a tripartite curriculum for AI agents across natural science, humanities, and social science domains, with reported gains of 15.9 points in security and 7 points in social reasoning under specif...

  31. CADMAS-CTX: Contextual Capability Calibration for Multi-Agent Delegation

    cs.AI 2026-04 unverdicted novelty 6.0

    CADMAS-CTX replaces static skill profiles with context-conditioned Beta posteriors and uncertainty-penalized routing, yielding higher accuracy on GAIA (0.442) and SWE-bench (31.4%) than static baselines.

  32. Phase-Scheduled Multi-Agent Systems for Token-Efficient Coordination

    cs.AI 2026-04 unverdicted novelty 6.0

    PSMAS reduces token use in LLM multi-agent systems by 27.3% on average via phase-based temporal scheduling and context compression, with task performance staying within 2.1 points of full activation.

  33. Preregistered Belief Revision Contracts

    cs.AI 2026-04 unverdicted novelty 6.0

    PBRC is a contract protocol that enforces evidential belief updates in deliberative multi-agent systems and proves it prevents conformity-driven false cascades under conservative fallbacks.

  34. AgentCity: Constitutional Governance for Autonomous Agent Economies via Separation of Power

    cs.MA 2026-04 unverdicted novelty 6.0

    AgentCity introduces a Separation of Power constitutional architecture on blockchain for governing autonomous agent economies through agent legislation, automated execution, and human accountability.

  35. PolySwarm: A Multi-Agent Large Language Model Framework for Prediction Market Trading and Latency Arbitrage

    cs.AI 2026-04 unverdicted novelty 6.0

    PolySwarm aggregates predictions from 50 LLM personas for Polymarket trading using Bayesian combination and divergence metrics, outperforming single models in calibration while adding latency arbitrage via CEX price models.

  36. TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing

    cs.DC 2026-04 unverdicted novelty 6.0

    TokenDance scales multi-agent LLM serving to 2.7x more concurrent agents by collective KV cache reuse and block-sparse diff encoding that achieves 11-17x compression.

  37. SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

    cs.CR 2026-02 unverdicted novelty 6.0

    The paper systematizes agentic skills beyond tool use, providing design pattern and representation-scope taxonomies plus security analysis of malicious skill infiltration in agent marketplaces.

  38. ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows

    cs.AI 2026-05 unverdicted novelty 5.0

    ProfiliTable is a profiling-driven multi-agent system that builds semantic context through exploration and closed-loop refinement to produce more reliable tabular data transformations than prior LLM approaches.

  39. Intermediate Artifacts as First-Class Citizens: A Data Model for Durable Intermediate Artifacts in Agentic Systems

    cs.AI 2026-05 unverdicted novelty 5.0

    A systems-level data model for preserving typed, addressable, versioned, and dependency-aware intermediate artifacts in agentic AI systems to improve long-term inspectability and maintainability.

  40. Shepherd: A Runtime Substrate Empowering Meta-Agents with a Formalized Execution Trace

    cs.AI 2026-05 unverdicted novelty 5.0 partial

    Shepherd is a runtime system that formalizes meta-agent operations via typed execution traces, enabling fast forking and demonstrated improvements in agent intervention, optimization, and training on benchmarks.

  41. Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability

    cs.AI 2026-05 unverdicted novelty 5.0

    A framework with U-statistics and kernel-based metrics quantifies AI agent consistency and robustness, showing trajectory metrics outperform pass@1 rates in diagnosing failures.

  42. AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks

    cs.AI 2026-05 unverdicted novelty 5.0

    Single-agent LLM frameworks outperform naive multi-agent systems in multimodal clinical risk prediction tasks and are better calibrated.

  43. From Agent Loops to Deterministic Graphs: Execution Lineage for Reproducible AI-Native Work

    cs.AI 2026-05 conditional novelty 5.0

    Execution lineage models AI-native work as a DAG of computations with explicit dependencies, achieving perfect state preservation in controlled update tasks where loop-based agents introduce churn and contamination.

  44. Strategic Heterogeneous Multi-Agent Architecture for Cost-Effective Code Vulnerability Detection

    cs.CR 2026-04 unverdicted novelty 5.0

    A game-theoretic heterogeneous multi-agent architecture with three cloud LLMs and a local verifier achieves 77.2% F1, 100% recall, and 3x speedup for code vulnerability detection at $0.002 per sample on the NIST Juliet suite.

  45. Forage V2: Knowledge Evolution and Transfer in Autonomous Agent Organizations

    cs.AI 2026-04 unverdicted novelty 5.0

    Forage V2 enables agent organizations to grow knowledge from 0 to 54 entries over runs and transfer it so weaker models nearly match stronger ones in coverage, cost, and speed on open-world tasks.

  46. Imperfectly Cooperative Human-AI Interactions: Comparing the Impacts of Human and AI Attributes in Simulated and User Studies

    cs.CL 2026-04 unverdicted novelty 5.0

    In real human subjects, AI transparency impacts imperfectly cooperative interactions far more than personality traits, unlike simulations where both are comparably influential.

  47. Knowledge Compounding: An Empirical Economic Analysis of Self-Evolving Knowledge Wikis under the Agentic ROI Framework

    econ.EM 2026-04 unverdicted novelty 5.0

    A four-query experiment demonstrates 84.6% token savings through knowledge compounding in self-evolving wikis compared to standard RAG, by amortizing ingestion costs and reusing synthesized knowledge over time.

  48. Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering

    cs.SE 2026-04 accept novelty 5.0

    LLM agent progress depends on externalizing cognitive functions into memory, skills, protocols, and harness engineering that coordinates them reliably.

  49. From Perception to Autonomous Computational Modeling: A Multi-Agent Approach

    cs.CE 2026-04 unverdicted novelty 5.0

    A multi-agent LLM framework autonomously completes the full computational mechanics pipeline from a photograph to a code-compliant engineering report on a steel L-bracket example.

  50. AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent

    cs.LG 2026-04 unverdicted novelty 5.0

    AgentOpt introduces a framework-agnostic package that uses algorithms like UCB-E to find cost-effective model assignments in multi-step LLM agent pipelines, cutting evaluation budgets by 62-76% while maintaining near-...

  51. Iterative Audit Convergence in LLM-Managed Multi-Agent Systems: A Case Study in Prompt Engineering Quality Assurance

    cs.SE 2026-05 conditional novelty 4.0

    Nine LLM-agent audit rounds on a 7150-line prompt specification surface found 51 defects with non-monotonic convergence and a post-hoc seven-category taxonomy, showing single-file review misses defect classes.

  52. Beyond Autonomy: A Dynamic Tiered AgentRunner Framework for Governable and Resilient Enterprise AI Execution

    cs.AI 2026-05 unverdicted novelty 4.0

    The Dynamic Tiered AgentRunner framework uses risk-adaptive tiering, separation of powers across agents, and verifier-recovery loops to enable governable and resilient enterprise AI execution.

  53. AssemPlanner: A Multi-Agent Based Task Planning Framework for Flexible Assembly System

    cs.RO 2026-05 unverdicted novelty 4.0

    AssemPlanner is a ReAct-based multi-agent system that autonomously generates production plans from natural language inputs by integrating scheduling, knowledge, line balancing, and scene graph feedback.

  54. A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications

    cs.IR 2026-05 unverdicted novelty 4.0

    The paper surveys agent skills for LLM agents, organizing the literature into a four-stage lifecycle of representation, acquisition, retrieval, and evolution while highlighting their role in system scalability.

  55. Terminus-4B: Can a Smaller Model Replace Frontier LLMs at Agentic Execution Tasks?

    cs.AI 2026-05 unverdicted novelty 4.0

    A fine-tuned 4B model matches or exceeds frontier LLMs in terminal execution subagent tasks for coding agents, reducing main agent token usage by 30% with no performance loss.

  56. Recommendations for Efficient and Responsible LLM Adoption within Industrial Software Development

    cs.SE 2026-04 conditional novelty 4.0

    A multi-case study plus survey produces seven actionable recommendations for efficient and responsible LLM use in industrial software engineering.

  57. OpenKedge: Governing Agentic Mutation with Execution-Bound Safety and Evidence Chains

    cs.AI 2026-04 unverdicted novelty 4.0

    OpenKedge redefines AI agent state mutations as a governed process using intent proposals, policy-evaluated execution contracts, and cryptographic evidence chains to enable safe, auditable agentic behavior.

  58. Qualixar OS: A Universal Operating System for AI Agent Orchestration

    cs.AI 2026-04 unverdicted novelty 4.0

    Qualixar OS provides a runtime for multi-agent AI systems with support for 12 topologies, LLM-driven team design, dynamic routing, consensus judging, content attribution, and protocol bridging, achieving 100% accuracy...

  59. Beyond Retrieval: Modeling Confidence Decay and Deterministic Agentic Platforms in Generative Engine Optimization

    cs.AI 2026-04 unverdicted novelty 4.0

    Deterministic multi-agent intent routing can reduce hallucinations in generative engines to near zero by limiting LLMs to intent routers and handing off tasks to specialized agents.

  60. Large Language Model based Multi-Agents: A Survey of Progress and Challenges

    cs.CL 2024-01 unverdicted novelty 4.0

    The paper surveys LLM-based multi-agent systems, covering simulated domains, agent profiling and communication, mechanisms for capacity growth, and common benchmarks.

Reference graph

Works this paper leans on

299 extracted references · 299 canonical work pages · cited by 64 Pith papers · 5 internal anchors

  1. [1]

    Playing repeated games with large language models

    Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, and Eric Schulz. Playing repeated games with large language models. arXiv preprint, 2023

    work page 2023

  2. [2]

    Program synthesis with large language models, 2021

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021

    work page 2021

  3. [3]

    Human-level play in the game of diplomacy by combining language models with strategic reasoning

    Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science, 2022

    work page 2022

  4. [4]

    A 15 year perspective on automatic programming

    Robert Balzer. A 15 year perspective on automatic programming. TSE, 1985

    work page 1985

  5. [5]

    R.M. Belbin. Team Roles at Work. Routledge, 2012. URL https://books.google.co.uk/books?id=MHIQBAAAQBAJ

    work page 2012

  6. [6]

    Large language models as tool makers

    Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. arXiv preprint, 2023

    work page 2023

  7. [7]

    LangChain

    Harrison Chase. LangChain . https://github.com/hwchase17/langchain, 2022

    work page 2022

  8. [8]

    Codet: Code generation with generated tests, 2022

    Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. Codet: Code generation with generated tests, 2022

    work page 2022

  9. [9]

    S-agents: self-organizing agents in open-ended environment

    Jiaqi Chen, Yuxian Jiang, Jiachen Lu, and Li Zhang. S-agents: self-organizing agents in open-ended environment. arXiv preprint, 2024

    work page 2024

  10. [10]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    work page 2021

  11. [11]

    Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents, 2023

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents, 2023

    work page 2023

  12. [12]

    Execution-guided neural program synthesis

    Xinyun Chen, Chang Liu, and Dawn Song. Execution-guided neural program synthesis. In ICLR, 2018

    work page 2018

  13. [13]

    Latent execution for neural program synthesis beyond domain-specific languages

    Xinyun Chen, Dawn Song, and Yuandong Tian. Latent execution for neural program synthesis beyond domain-specific languages. NeurIPS, 2021 b

    work page 2021

  14. [14]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradb...

    work page 2022

  15. [15]

    DeMarco and T.R

    T. DeMarco and T.R. Lister. Peopleware: Productive Projects and Teams. Addison-Wesley, 2013. URL https://books.google.co.uk/books?id=DVlsAQAAQBAJ

    work page 2013

  16. [16]

    Self-collaboration code generation via chatgpt

    Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self-collaboration code generation via chatgpt. arXiv preprint, 2023

    work page 2023

  17. [17]

    Tenenbaum, and Igor Mordatch

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate, 2023

    work page 2023

  18. [18]

    Measuring and improving consistency in pretrained language models

    Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Sch \"u tze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. TACL, 2021

    work page 2021

  19. [19]

    Codebert: A pre-trained model for programming and natural languages

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. Codebert: A pre-trained model for programming and natural languages. arXiv preprint, 2020

    work page 2020

  20. [20]

    Promptbreeder: Self-referential self-improvement via prompt evolution

    Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \"a schel. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint, 2023

    work page 2023

  21. [21]

    Model-agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017

    work page 2017

  22. [22]

    Incoder: A generative model for code infilling and synthesis

    Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. Incoder: A generative model for code infilling and synthesis. arXiv preprint, 2022

    work page 2022

  23. [23]

    Speculations concerning the first ultraintelligent machine

    Irving John Good. Speculations concerning the first ultraintelligent machine. Adv. Comput., 1965

    work page 1965

  24. [24]

    Chatllm network: More brains, more intelligence

    Rui Hao, Linmei Hu, Weijian Qi, Qingliu Wu, Yirui Zhang, and Liqiang Nie. Chatllm network: More brains, more intelligence. arXiv preprint, 2023

    work page 2023

  25. [25]

    Hochreiter, A

    S. Hochreiter, A. S. Younger, and P. R. Conwell. Learning to learn using gradient descent. In Lecture Notes on Comp. Sci. 2130, Proc. Intl. Conf. on Artificial Neural Networks (ICANN-2001), pp.\ 87--94. Springer: Berlin, Heidelberg, 2001

    work page 2001

  26. [26]

    Data Interpreter : An LLM Agent For Data Science

    Sirui Hong, Yizhang Lin, Bangbang Liu, Binhao Wu, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Lingyao Zhang, Mingchen Zhuge, et al. Data interpreter: An llm agent for data science. arXiv preprint arXiv:2402.18679, 2024

    work page arXiv 2024

  27. [27]

    Self-planning code generation with large language model

    Xue Jiang, Yihong Dong, Lecheng Wang, Qiwei Shang, and Ge Li. Self-planning code generation with large language model. arXiv preprint, 2023

    work page 2023

  28. [28]

    Camel: Communicative agents for" mind" exploration of large scale language model society

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large scale language model society. arXiv preprint, 2023

    work page 2023

  29. [29]

    Competition-level code generation with alphacode

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R \'e mi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode. Science, 2022

    work page 2022

  30. [30]

    Encouraging divergent thinking in large language models through multi-agent debate

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint, 2023

    work page 2023

  31. [31]

    Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks

    Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. arXiv preprint, 2023

    work page 2023

  32. [32]

    Training socially aligned language models in simulated human society

    Ruibo Liu, Ruixin Yang, Chenyan Jia, Ge Zhang, Denny Zhou, Andrew M Dai, Diyi Yang, and Soroush Vosoughi. Training socially aligned language models in simulated human society. arXiv preprint, 2023 a

    work page 2023

  33. [33]

    Ml-bench: Evaluating large language models and agents for machine learning tasks on repository-level code.arXiv preprint arXiv:2311.09835, 2023

    Yuliang Liu, Xiangru Tang, Zefan Cai, Junjie Lu, Yichi Zhang, Yanjun Shao, Zexuan Deng, Helan Hu, Zengxian Yang, Kaikai An, et al. Ml-bench: Large language models leverage open-source libraries for machine learning tasks. arXiv preprint arXiv:2311.09835, 2023 b

    work page arXiv 2023

  34. [34]

    Wizardcoder: Empowering code large language models with evol-instruct

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint, 2023

    work page 2023

  35. [35]

    Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models

    Potsawee Manakul, Adian Liusie, and Mark JF Gales. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint, 2023

    work page 2023

  36. [36]

    Manifesto for agile software development

    Agile Manifesto. Manifesto for agile software development. Snowbird, UT, 2001

    work page 2001

  37. [37]

    History of lisp

    John McCarthy. History of lisp. In History of programming languages. 1978

    work page 1978

  38. [38]

    Octopack: Instruction tuning code large language models

    Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. Octopack: Instruction tuning code large language models. arXiv preprint arXiv:2308.07124, 2023

    work page arXiv 2023

  39. [39]

    Lever: Learning to verify language-to-code generation with execution

    Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-tau Yih, Sida Wang, and Xi Victoria Lin. Lever: Learning to verify language-to-code generation with execution. In ICML, 2023

    work page 2023

  40. [40]

    Codegen: An open large language model for code with multi-turn program synthesis, 2023

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis, 2023

    work page 2023

  41. [41]

    Gpt-4 technical report, 2023

    OpenAI. Gpt-4 technical report, 2023

    work page 2023

  42. [42]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph C O'Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint, 2023

    work page 2023

  43. [43]

    Communicative agents for software development, 2023

    Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development, 2023

    work page 2023

  44. [44]

    ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  45. [45]

    Code llama: Open foundation models for code

    Baptiste Rozi \`e re, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, J \'e r \'e my Rapin, et al. Code llama: Open foundation models for code. arXiv preprint, 2023

    work page 2023

  46. [46]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint, 2023

    work page 2023

  47. [47]

    Schmidhuber

    J. Schmidhuber. A self-referential weight matrix. In Proceedings of the International Conference on Artificial Neural Networks, Amsterdam, pp.\ 446--451. Springer, 1993 a

    work page 1993

  48. [49]

    Schmidhuber

    J. Schmidhuber. G\" o del machines: Fully self-referential optimal universal self-improvers. In B. Goertzel and C. Pennachin (eds.), Artificial General Intelligence, pp.\ 199--226. Springer Verlag, 2006. Variant available as arXiv:cs.LO/0309048

    work page arXiv 2006

  49. [50]

    Schmidhuber

    J. Schmidhuber. Ultimate cognition \` a la G\" o del . Cognitive Computation, 1 0 (2): 0 177--193, 2009

    work page 2009

  50. [51]

    Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-

    J \"u rgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. PhD thesis, 1987

    work page 1987

  51. [52]

    A ‘self-referential’weight matrix

    J \"u rgen Schmidhuber. A ‘self-referential’weight matrix. In ICANN’93: Proceedings of the International Conference on Artificial Neural Networks Amsterdam, The Netherlands 13--16 September 1993 3, 1993 b

    work page 1993

  52. [53]

    On learning to think: Algorithmic information theory for novel combinations of reinforcement learning controllers and recurrent neural world models

    J \"u rgen Schmidhuber. On learning to think: Algorithmic information theory for novel combinations of reinforcement learning controllers and recurrent neural world models. arXiv preprint, 2015

    work page 2015

  53. [54]

    Reinforcement learning with self-modifying policies

    J \"u rgen Schmidhuber, Jieyu Zhao, and Nicol N Schraudolph. Reinforcement learning with self-modifying policies. In Learning to learn. 1998

    work page 1998

  54. [55]

    Reflexion: an autonomous agent with dynamic memory and self-reflection

    Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint, 2023

    work page 2023

  55. [56]

    Errors are useful prompts: Instruction guided task programming with verifier-assisted iterative prompting

    Marta Skreta, Naruki Yoshikawa, Sebastian Arellano-Rubach, Zhi Ji, Lasse Bj rn Kristensen, Kourosh Darvish, Al \'a n Aspuru-Guzik, Florian Shkurti, and Animesh Garg. Errors are useful prompts: Instruction guided task programming with verifier-assisted iterative prompting. arXiv preprint, 2023

    work page 2023

  56. [57]

    Learning to program = learning to construct mechanisms and explanations

    Elliot Soloway. Learning to program = learning to construct mechanisms and explanations. Communications of the ACM, 1986

    work page 1986

  57. [58]

    Multi-agent collaboration: Harnessing the power of intelligent llm agents, 2023

    Yashar Talebirad and Amirhossein Nadiri. Multi-agent collaboration: Harnessing the power of intelligent llm agents, 2023

    work page 2023

  58. [59]

    Biocoder: A benchmark for bioinformatics code generation with contextual pragmatic knowledge

    Xiangru Tang, Bill Qian, Rick Gao, Jiakang Chen, Xinyun Chen, and Mark Gerstein. Biocoder: A benchmark for bioinformatics code generation with contextual pragmatic knowledge. arXiv preprint arXiv:2308.16458, 2023 a

    work page arXiv 2023

  59. [60]

    MedAgents: Large language models as collaborators for zero-shot medical reasoning.arXiv preprint arXiv:2311.10537, 2023

    Xiangru Tang, Anni Zou, Zhuosheng Zhang, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. Medagents: Large language models as collaborators for zero-shot medical reasoning. arXiv preprint arXiv:2311.10537, 2023 b

    work page arXiv 2023

  60. [61]

    Auto-gpt

    Torantulino et al. Auto-gpt. https://github.com/Significant-Gravitas/Auto-GPT, 2023

    work page 2023

  61. [62]

    R. J. Waldinger and R. C. T. Lee. PROW: a step toward automatic program writing. In D. E. Walker and L. M. Norton (eds.), Proceedings of the 1st International Joint Conference on Artificial Intelligence (IJCAI), 1969

    work page 1969

  62. [63]

    Voyager: An open-ended embodied agent with large language models

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint, 2023 a

    work page 2023

  63. [64]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. arXiv preprint, 2023 b

    work page 2023

  64. [65]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint, 2022

    work page 2022

  65. [66]

    Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration

    Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint, 2023 c

    work page 2023

  66. [67]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 2022

    work page 2022

  67. [68]

    Jennings

    Michael Wooldridge and Nicholas R. Jennings. Pitfalls of agent-oriented development. In Proceedings of the Second International Conference on Autonomous Agents, 1998. URL https://doi.org/10.1145/280765.280867

    work page doi:10.1145/280765.280867 1998

  68. [69]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint, 2022

    work page 2022

  69. [70]

    Self-taught optimizer (stop): Recursively self-improving code generation

    Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer (stop): Recursively self-improving code generation. arXiv preprint, 2023

    work page 2023

  70. [71]

    Building cooperative embodied agents modularly with large language models

    Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. arXiv preprint, 2023 a

    work page 2023

  71. [72]

    Igniting language intelligence: The hitchhiker’s guide from chain-of-thought reasoning to language agents

    Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein, Rui Wang, Gongshen Liu, et al. Igniting language intelligence: The hitchhiker's guide from chain-of-thought reasoning to language agents. arXiv preprint arXiv:2311.11797, 2023 b

    work page arXiv 2023

  72. [73]

    Chat with the environment: Interactive multimodal perception using large language models

    Xufeng Zhao, Mengdi Li, Cornelius Weber, Muhammad Burhan Hafez, and Stefan Wermter. Chat with the environment: Interactive multimodal perception using large language models. arXiv preprint, 2023

    work page 2023

  73. [74]

    Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x, 2023

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x, 2023

    work page 2023

  74. [75]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint, 2023 a

    work page 2023

  75. [76]

    Agents: An open-source framework for autonomous lan- guage agents

    Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, et al. Agents: An open-source framework for autonomous language agents. arXiv preprint arXiv:2309.07870, 2023 b

    work page arXiv 2023

  76. [77]

    Mindstorms in natural language-based societies of mind

    Mingchen Zhuge, Haozhe Liu, Francesco Faccio, Dylan R Ashley, R \'o bert Csord \'a s, Anand Gopalakrishnan, Abdullah Hamdi, Hasan Abed Al Kader Hammoud, Vincent Herrmann, Kazuki Irie, et al. Mindstorms in natural language-based societies of mind. arXiv preprint, 2023

    work page 2023

  77. [78]

    C. F. Gauss , title =

    work page

  78. [79]

    Theoria motus corporum coelestium in sectionibus conicis solem ambientium , author=

    work page

  79. [80]

    Nouvelles m

    Legendre, Adrien Marie , year=. Nouvelles m

    work page

  80. [81]

    Philosophical Transactions of the Royal Society of London , volume=

    An essay toward solving a problem in the doctrine of chances , author=. Philosophical Transactions of the Royal Society of London , volume=

    work page

Showing first 80 references.