REVIEW 16 cited by
Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) have become integral components in various autonomous agent systems. In this study, we present an exploration-based trajectory optimization approach, referred to as ETO. This learning method is designed to enhance the performance of open LLM agents. Contrary to previous studies that exclusively train on successful expert trajectories, our method allows agents to learn from their exploration failures. This leads to improved performance through an iterative optimization framework. During the exploration phase, the agent interacts with the environment while completing given tasks, gathering failure trajectories to create contrastive trajectory pairs. In the subsequent training phase, the agent utilizes these trajectory preference pairs to update its policy using contrastive learning methods like DPO. This iterative cycle of exploration and training fosters continued improvement in the agents. Our experiments on three complex tasks demonstrate that ETO consistently surpasses baseline performance by a large margin. Furthermore, an examination of task-solving efficiency and potential in scenarios lacking expert trajectory underscores the effectiveness of our approach.
Forward citations
Cited by 16 Pith papers
-
Leveraging Trajectory Graphs for Pre-Execution Error Diagnosis in Agentic LLM Systems
Modeling agent trajectories as action-centric probabilistic graphs lets a GNN warn LLM agents of likely step-level errors before execution, improving pass ratio ~14.7% across four benchmarks.
-
Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories
Offline DPO on expert states with one-step student negatives and schema-preserving augmentation improves LLM agents beyond SFT and can match online GRPO without environment interaction.
-
Morae: Proactively Pausing UI Agents for User Choices
Morae, a UI agent that proactively pauses at ambiguous decision points, helps blind and low-vision users complete more tasks and express preferences better than fully autonomous agents.
-
RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents
RLVMR adds rule-based rewards for planning, exploration, reflection, and monitoring tags to outcome-based reinforcement learning, and reports state-of-the-art success rates and fewer redundant actions on ALFWorld and ...
-
MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
A new one-million-sample multimodal agent tuning dataset with GPT-4o-generated rationales, reflection, and tool/RAG calls is shown to improve fine-tuned models, though training/eval benchmark overlap is not addressed.
-
Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning
A 7B LLM trained with sparse completion rewards and a GRPO-style algorithm reaches state-of-the-art on ALFWorld and ScienceWorld.
-
LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback
Self-generated agent trajectories from LAM SIMULATOR improved fine-tuned model pass rates by up to 49.3% on ToolBench and CRMArena.
-
ARIA: Training Language Agents with Intention-Driven Reward Aggregation
Clustering language-agent actions into shared intentions and averaging their rewards reduces reward variance and improves policy performance in open-ended dialogue tasks.
-
Reinforced Reasoning for Embodied Planning
An SFT-plus-GRPO recipe lifts a 7B VLM to 35.6 percent success on EB-ALFRED versus 22.0 for GPT-4o-mini and 33.7 for Qwen2.5-VL-72B, with smaller but consistent gains on unseen EB-Habitat.
-
ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making
ManiTaskGen automatically generates diverse, feasible mobile manipulation tasks from any input scene, and uses them to benchmark and improve vision-language robot agents.
-
Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking
A new agent-training method combining teacher-generated self-reflection corrections with partial masking of error steps improves open-source LLM agents on ALFWorld, WebShop, and SciWorld.
-
Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agents
A new step-level reward-modeling benchmark for multimodal agents shows current MLLMs reach at most 61.6 percent accuracy, and benchmark score correlates strongly (r=0.981 across five models) with downstream A* search ...
-
CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learning
CheMatAgent uses hierarchical Monte Carlo tree search with separate policy and execution models, plus trained reward models, to improve tool selection and parameter filling on a new chemistry benchmark, ChemToolBench.
-
Digi-Q: Learning Q-Value Functions for Training Device-Control Agents
An offline RL method learns a Q-function from frozen VLM features and extracts a device-control policy by imitating the best of several actions ranked by that Q-function.
-
TO-GATE: Clarifying Questions and Summarizing Responses with Trajectory Optimization for Eliciting Human Preference
TO-GATE applies trajectory-level direct preference optimization with a weighted response loss to improve preference elicitation dialogues, claiming 83.15% win rate versus 73.83% for STaR-GATE.
-
Large Language Models for Planning: A Comprehensive and Systematic Survey
A structured survey of LLM planning methods, benchmarks, and interpretability work, organized around a three-way taxonomy.
Discussion (0). Continue with ORCID to comment.