REVIEW 6 cited by
SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Automated Machine Learning (AutoML) approaches encompass traditional methods that optimize fixed pipelines for model selection and ensembling, as well as newer LLM-based frameworks that autonomously build pipelines. While LLM-based agents have shown promise in automating machine learning tasks, they often generate low-diversity and suboptimal code, even after multiple iterations. To overcome these limitations, we introduce Tree-Search Enhanced LLM Agents (SELA), an innovative agent-based system that leverages Monte Carlo Tree Search (MCTS) to optimize the AutoML process. By representing pipeline configurations as trees, our framework enables agents to conduct experiments intelligently and iteratively refine their strategies, facilitating a more effective exploration of the machine learning solution space. This novel approach allows SELA to discover optimal pathways based on experimental feedback, improving the overall quality of the solutions. In an extensive evaluation across 20 machine learning datasets, we compare the performance of traditional and agent-based AutoML methods, demonstrating that SELA achieves a win rate of 65% to 80% against each baseline across all datasets. These results underscore the significant potential of agent-based strategies in AutoML, offering a fresh perspective on tackling complex machine learning challenges.
Forward citations
Cited by 6 Pith papers
-
ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoning
ML-Master, a new AI4AI agent, achieves 29.3% medals on MLE-Bench by integrating MCTS-style exploration with reasoning steered by a compact adaptive memory, surpassing prior agents in half the time.
-
AUTOCT: Automating Interpretable Clinical Trial Prediction with LLM Agents
AutoCT achieves test ROC-AUC 0.753, 0.639, and 0.702 on Phase I/II/III trial approval prediction using 100-sample subsets and LLM-generated features.
-
AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
Progressive reinforcement-learning horizon scaling lets a 7-billion-parameter open-source model match or surpass larger proprietary models on multi-turn agent tasks.
-
MLZero: A Multi-Agent System for End-to-end Machine Learning Automation
MLZero, an LLM-based multi-agent system with perception and dual memory, reports 92 percent success on a new 25-task multimodal AutoML benchmark and the best average rank on MLE-Bench Lite.
-
MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
An open Gym-style environment running 200+ Kaggle competitions lets LLM agents iterate on ML solutions and provides a benchmark for training and evaluating them.
-
Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows
An evolutionary loop of foundation-model agents could automate the full optimization pipeline, but the paper's evidence only covers two isolated components.
Discussion (0). Continue with ORCID to comment.