REVIEW 15 cited by
Executable Code Actions Elicit Better LLM Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Model (LLM) agents, capable of performing a broad range of actions, such as invoking tools and controlling robots, show great potential in tackling real-world challenges. LLM agents are typically prompted to produce actions by generating JSON or text in a pre-defined format, which is usually limited by constrained action space (e.g., the scope of pre-defined tools) and restricted flexibility (e.g., inability to compose multiple tools). This work proposes to use executable Python code to consolidate LLM agents' actions into a unified action space (CodeAct). Integrated with a Python interpreter, CodeAct can execute code actions and dynamically revise prior actions or emit new actions upon new observations through multi-turn interactions. Our extensive analysis of 17 LLMs on API-Bank and a newly curated benchmark shows that CodeAct outperforms widely used alternatives (up to 20% higher success rate). The encouraging performance of CodeAct motivates us to build an open-source LLM agent that interacts with environments by executing interpretable code and collaborates with users using natural language. To this end, we collect an instruction-tuning dataset CodeActInstruct that consists of 7k multi-turn interactions using CodeAct. We show that it can be used with existing data to improve models in agent-oriented tasks without compromising their general capability. CodeActAgent, finetuned from Llama2 and Mistral, is integrated with Python interpreter and uniquely tailored to perform sophisticated tasks (e.g., model training) using existing libraries and autonomously self-debug.
Forward citations
Cited by 15 Pith papers
-
DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds
Fine-tuning a 30B coding agent on 576 planning-aware trajectories collected under Claude Code improves its SWE-bench score on unseen harnesses (OpenCode +3.4, mini-swe-agent +7.0 with self-plans).
-
When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation
Code-only tool surfaces beat or match the cheapest tool-rich rival on cost in three of four (regime, agent) cells at matched pass rates; Claude on SWE-bench is the directional exception via failure cost.
-
StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents
StateAct, a code-first multi-agent harness that acts on program state instead of pixels, improves Claude Opus 4.8 on OSWorld 2.0 from 20.6% to 26.9% binary success at ~9x lower cost.
-
NVIDIA-labs OO Agents: Native Python Object-Oriented Agents
A Python object-oriented agent framework (NOOA) lets methods, state, docstrings and type annotations serve as the LLM interface, and achieves strong results on agentic benchmarks.
-
Beyond Generalist LLMs: Specialist Agentic Systems for Structured Code Workflow Execution
A specialist BPMN-to-agent pipeline beats general-purpose coding agents on tool-use accuracy, latency, and token cost for deterministic business workflows.
-
Token Reduction Is Not Cost Reduction
Token reduction of tool output does not reliably lower billed coding-agent cost; prompt-cache traffic dominates, and aggressive compression can also destroy task success.
-
AISE-Bench: A Full-Cycle Curated Benchmark for Information Seeking on Academic Knowledge Graphs
AISE-Bench is a real-user-query benchmark with annotated API trajectories and grounded answers that exposes LLM agents' weak performance in multi-step academic information seeking.
-
Execution-Grounded Security Testing for Coding Agents in Software Engineering Pipelines
Disguising unsafe operations as routine software-engineering tests makes coding agents execute them at high verified rates, revealing a gap between language-layer refusal and execution-layer harm.
-
BitsAI-Fix: LLM-Driven Approach for Automated Lint Error Resolution in Practice
A 32B LLM trained with progressive reinforcement learning on cold-start and user-feedback data fixes Go lint errors at 84.68% accuracy with 1.72% redundancy on a private ByteDance test set.
-
MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation
MGC, a two-stage compiler framework, generates functional malware by decomposing malicious intents into benign-appearing MDIR components that strong aligned LLMs will implement, bypassing safety alignment.
-
Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation
Doc2Agent automatically converts unstructured REST API documentation into validated, Python-based tools for AI agents, reporting a 55% relative WebArena improvement over direct API calling.
-
Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details
For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.
-
Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent Teams
A design and open-source implementation of a filesystem layer that persists Claude Code agent-team state to disk, enabling checkpointing, recovery, and cross-session communication.
-
Self-Evolving Coding Agents
A systematic survey that defines self-evolving coding agents, organizes systems into a five-category object-centered taxonomy, and analyzes when and on what evidence these agents evolve.
-
Reinforcing Code Generation: Improving Text-to-SQL with Execution-Based Learning
Execution-based reinforcement learning raises SQLCoder-7B's exact-match accuracy from 31.49% to 49.83% on TEMPTABQA-C, approaching SQLCoder-70B, but shows no gain for CodeGemma.
Discussion (0). Sign in to comment.