REVIEW 34 cited by
ToRL: Scaling Tool-Integrated RL
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce ToRL (Tool-Integrated Reinforcement Learning), a framework for training large language models (LLMs) to autonomously use computational tools via reinforcement learning. Unlike supervised fine-tuning, ToRL allows models to explore and discover optimal strategies for tool use. Experiments with Qwen2.5-Math models show significant improvements: ToRL-7B reaches 43.3\% accuracy on AIME~24, surpassing reinforcement learning without tool integration by 14\% and the best existing Tool-Integrated Reasoning (TIR) model by 17\%. Further analysis reveals emergent behaviors such as strategic tool invocation, self-regulation of ineffective code, and dynamic adaptation between computational and analytical reasoning, all arising purely through reward-driven learning.
Forward citations
Cited by 34 Pith papers
-
rStar2-Agent: Agentic Reasoning Technical Report
A 14B model trained with agentic RL and a resample-on-correct rollout strategy scores 80.6% on AIME24 and 69.8% on AIME25, nearly matching DeepSeek-R1 (671B) in one week on 64 GPUs.
-
Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and Benchmarking Multimodal LLM Agents
A new CAPTCHA benchmark with a reasoning-depth metric shows multimodal LLM agents solve at most 40% of interactive puzzles, far short of the 93% human success rate.
-
Progressive Agent Skill Generation via Reinforcement Learning
Skill-α uses reinforcement learning with a rollback reward to progressively edit SKILL.md-style procedure files, improving downstream success rates over pipeline baselines on CL-Bench, SpreadsheetBench, and tau2-bench.
-
TCPO: Turn-Level Credit Policy Optimization
TCPO improves multi-turn LLM RL by converting verifier scores into turn-level credit via three reference-based comparisons, beating baseline methods on math, code, and agent tasks.
-
H$^2$SD: Hybrid Hindsight Self-Distillation
H2SD routes successful RLVR trajectories to token-level magnitude credit assignment and failed trajectories to reverse-KL distillation from a hint-conditioned self-teacher, improving logical reasoning benchmarks.
-
Knowledge-Centric Agents for Workflow Generation in ComfyUI
A knowledge-centric pipeline distills strategies and pseudo-codes from real workflows, fine-tunes a language model on those levels, and reconstructs executable ComfyUI graphs from task descriptions.
-
BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.
-
ECHO: Prune To Act, Trace To Learn With Selective Turn Memory In Agentic RL
Source-indexed selective turn memory plus provenance-guided credit assignment raises BrowseComp-Plus accuracy to 43.4% over GRPO and SUPO while cutting turn proliferation.
-
APPO: Agentic Procedural Policy Optimization
APPO refines branching and credit assignment in agentic RL via a Branching Score and procedure-level scaling, improving baselines by nearly 4 points on 13 benchmarks.
-
SOD: Step-wise On-policy Distillation for Small Language Model Agents
SOD reweights on-policy distillation strength step-by-step using divergence to stabilize tool use in small language model agents, yielding up to 20.86% gains and 26.13% on AIME 2025 for a 0.6B model.
-
Intern-S1-MO: Long-horizon Reasoning Agent for Olympiad?Level Mathematical Problem Solving
A multi-round, lemma-memory reasoning agent with hierarchical RL reaches reported gold-medal-level scores on Olympiad math benchmarks, though the proof-based scores are self-graded.
-
SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents
A 20B autonomously reasoning deep-research agent trained with synthetic-data RL reaches 28.7% on Humanity's Last Exam, exceeding several larger and proprietary baselines.
-
SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning
Filtering out trajectories with 'void turns' (no code, no answer) stabilizes multi-turn tool-integrated RL and substantially improves math reasoning benchmarks.
-
Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planning
Decoupling LLM-agent planning from summarization and rewarding tool-call completeness rather than final-answer correctness improves planning by 8-12% and end-to-end answers by 5-6% over end-to-end RL baselines.
-
MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use
MUA-RL adds an LLM-simulated user to the RL rollout loop for multi-turn tool use, improving small Qwen3 models on TAU2, BFCL-V3 Multi Turn, and ACEBench Agent.
-
Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL
A single LLM is trained with multi-agent distilled trajectories plus agentic RL, and the resulting Chain-of-Agents models set state-of-the-art Pass@1 scores among tool-integrated reasoning methods on GAIA, BrowseComp,...
-
UserBench: An Interactive Gym Environment for User-Centric Agents
A new multi-turn agent benchmark shows that current LLMs elicit fewer than 30% of user preferences and reach full intent alignment only about 20% of the time.
-
StepFun-Prover Preview: Let's Think and Verify Step by Step
A reinforcement learning pipeline with Lean verifier feedback trains a 32B model that scores 70.0% pass@1 on miniF2F-test, a new state of the art.
-
PrefixAgent: An LLM-Powered Design Framework for Efficient Prefix Adder Optimization
PrefixAgent fine-tunes an LLM on E-graph generated optimization traces to synthesize prefix adders with smaller area than prior methods and commercial tools at up to 64 bits.
-
Distilling Tool Knowledge into Language Models via Back-Translated Traces
Fine-tuning Qwen2.5-MATH-7B on LLM-back-translated traces from a SymPy tool agent gives small gains on AIME, AMC, and Olympiad Bench and small losses on MATH500 and GSM8K.
-
Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges
A benchmark and agent for CTF solving, but the agent's retrieval database appears to contain the answers to the test challenges, undermining the reported improvements.
-
Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
A 3B agent trained with supervised tool-use traces and reinforcement learning answers week-long egocentric video questions by dynamically selecting hierarchical retrieval, video-LLM, and VLM tools.
-
VerIF: Verification Engineering for Reinforcement Learning in Instruction Following
A hybrid verifier that combines rule-based code checks and a reasoning-LLM judge enables reinforcement learning to improve LLM instruction following on several benchmarks.
-
Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
A GRPO-based method that rewards only self-reflection tokens, not answer tokens, improves LLM accuracy on function calling and Countdown math tasks using only binary success/failure feedback.
-
RSPO: Reward-Swap Policy Optimization for Multi-Turn LLM Agents
Reward-Swap Policy Optimization uses dense-reward exploration to generate diverse trajectories that then train the final policy strictly under true outcome rewards, lifting multi-turn agent performance.
-
Harnessing Rule-Based Reinforcement Learning for Enhanced Grammatical Error Correction
Applying GRPO with a rule-based, reference-match reward to a Qwen3-8B model after reasoning-augmented SFT achieves state-of-the-art F0.5 on Chinese GEC benchmark FCGEC and improves recall.
-
AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning
AutoTIR applies GRPO with a hand-designed action reward so a 7B instruct model learns to mix search and code tools, beating tool-using baselines on ten benchmarks.
-
Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents
Co-Harness co-evolves agent harness and model weights with an LLM critic plus SFT, claiming +20.4 pp on AIME/HMMT, but without a fixed-harness SFT comparison.
-
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.
-
Empowering Nanoscale Connectivity through Molecular Communication: A Case Study of Virus Infection
A position paper proposing molecular communication as the link layer for epidemic-control bio-nano networks, with an ORF3a-based mutation identification simulation; the provided manuscript body does not match this abstract.
-
A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applications
A survey of 80+ Deep Research systems that proposes a four-layer taxonomy (foundation models, tool use, planning, synthesis) and compares commercial and open-source implementations.
-
Towards Effective Code-Integrated Reasoning
A reinforcement learning recipe with precise code-block matching, masked tool feedback, and a progressive interaction budget improves the accuracy and token efficiency of code-integrated math reasoning.
-
Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.
-
A Survey on Large Language Models for Mathematical Reasoning
Recent advances in LLM mathematical reasoning are organized into comprehension and generation phases, covering methods from prompting to test-time scaling.
Discussion (0). Sign in to comment.