Pith. sign in

REVIEW 17 cited by

Acting Less is Reasoning More! Teaching Model to Act Efficiently

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.14870 v2 pith:C6AL6HGQ submitted 2025-04-21 cs.AI cs.CL

Hongru Wang , Cheng Qian , Wanjun Zhong , Xiusi Chen , Jiahao Qiu , Shijue Huang , Bowen Jin , Mengdi Wang
show 2 more authors
Kam-Fai Wong Heng Ji
This is my paper
classification cs.AIcs.CL
keywords toolreasoninganswercallsoptimizationacrossanswerscapabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Tool-integrated reasoning (TIR) augments large language models (LLMs) with the ability to invoke external tools during long-form reasoning, such as search engines and code interpreters, to solve tasks beyond the capabilities of internal reasoning. While reinforcement learning (RL) has shown promise in training such agents, most of existing approaches typically optimize only for final correctness without considering the efficiency or necessity of external tool use. This often leads to excessive tool calling, incurring high computational costs and hindering the development of internal reasoning capabilities - a phenomenon known as \textit{cognitive offloading}. To this end, we propose Optimal Tool Call-controlled Policy Optimization (OTC-PO), a simple yet effective RL-based framework that encourages models to produce accurate answers with minimal tool calls. Our method introduces a tool-integrated reward that jointly considers answer correctness and corresponding tool use behavior of model to reach that answer. To validate the effectiveness, we introduce the metric of \textit{tool productivity}, defined as the ratio between the number of correct answers and the total number of tool calls across all test cases. This metric reflects how efficiently tool usage contributes to successful task completion, with higher values indicating smarter and more autonomous reasoning. We instantiate this framework within both Proximal Policy Optimization (PPO) and Group Relative Preference Optimization (GRPO), resulting in OTC-PPO and OTC-GRPO. Experiments with Qwen-2.5 and Qwen-Math across multiple QA benchmarks show that our approach reduces tool calls by up to 68.3\% and improves tool productivity by up to 215.4\%, while maintaining comparable answer accuracy.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignment

    cs.CL 2026-06 unverdicted novelty 7.0 of 10

    The Piggyback Hypothesis attributes emergent misalignment to chat-template tokens piggybacking finetuned behavior; Token-Regularized Finetuning (TReFT) mitigates it by regularizing prefix token representations.

  2. Diagnosing Search Behavior and Failure Modes in Long-Horizon Search Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    For six deep search agents on BrowseComp-Plus, answer accuracy tracks cumulative retrieval recall, not search effort, and failures split into missing-evidence and evidence-misuse gaps.

  3. ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Reinforcement-learned, type-aware visual tool calls improve multimodal scientific claim verification on SCIVER and MuSciClaims.

  4. Tool-Adaptive LLM Reranker

    cs.IR 2026-07 conditional novelty 6.0 of 10

    Pointwise LLM reranking cast as an agentic MDP with language-preserving warm-up and asymmetric cost-aware RL yields adaptive tool use, SOTA NDCG, and high throughput.

  5. TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework

    cs.IR 2025-11 conditional novelty 6.0 of 10

    TeaRAG shows that hybrid chunk+triplet retrieval with Personalized PageRank and an iterative process-aware DPO reward keeps QA accuracy while cutting reasoning tokens by roughly 60%.

  6. ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    ParallelSearch applies reinforcement learning with parallel-decomposition rewards to teach LLM search agents to issue independent sub-queries concurrently, lifting QA accuracy by 2.9% on average and 12.7% on paralleli...

  7. MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning

    cs.AI 2025-08 reject novelty 6.0 of 10

    The paper advertises a self-evolving tool-using agent that beats baselines, but the body contains a different math paper, leaving the performance claims unsupported.

  8. UserBench: An Interactive Gym Environment for User-Centric Agents

    cs.AI 2025-07 conditional novelty 6.0 of 10

    A new multi-turn agent benchmark shows that current LLMs elicit fewer than 30% of user preferences and reach full intent alignment only about 20% of the time.

  9. Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A 3B agent trained with supervised tool-use traces and reinforcement learning answers week-long egocentric video questions by dynamically selecting hierarchical retrieval, video-LLM, and VLM tools.

  10. Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Tool-Star combines cold-start supervised fine-tuning with a multi-tool self-critic reinforcement learning algorithm and hierarchical rewards to improve LLM tool-use reasoning.

  11. Contrastive Reinforced Policy Optimization via Privileged Self-Distillation

    cs.LG 2026-07 conditional novelty 5.0 of 10

    CRPO turns on-policy self-distillation into group-wise contrastive learning gated by student–teacher entropy gaps, improving multi-turn agentic LLM post-training over GRPO, ARPO, and OPSD.

  12. Leanabell-Prover-V2: Verifier-integrated Reasoning for Formal Theorem Proving via Reinforcement Learning

    cs.AI 2025-07 reject novelty 5.0 of 10

    Verifier-integrated reinforcement learning with multi-turn reflection improves 7B-scale Lean 4 theorem proving by 2 to 3 points on MiniF2F at pass@128.

  13. R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning

    cs.CL 2025-05 conditional novelty 5.0 of 10

    R1-Searcher++ uses SFT cold-start plus reinforcement learning with group and memorization rewards to teach Qwen-2.5-7B to balance internal knowledge and external retrieval, improving accuracy and reducing retrieval calls.

  14. CoRT: Code-integrated Reasoning within Thinking

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Inserting targeted hints into a few training examples teaches reasoning models to compute with Python instead of text, improving accuracy and cutting token use by 30 to 50 percent.

  15. SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution

    cs.CL 2025-05 conditional novelty 4.0 of 10

    SPA-RL attributes a single final task reward to individual agent steps and uses the attributions plus an executability signal as dense rewards for PPO, yielding modest success-rate gains on WebShop, ALFWorld, and VirtualHome.

  16. Toward Efficient Agents: Memory, Tool learning, and Planning

    cs.AI 2026-01 conditional novelty 3.0 of 10

    A survey that organizes efficiency techniques for LLM agents into memory, tool learning, and planning, and consolidates benchmarks and metrics for measuring cost-performance trade-offs.

  17. Large Language Models for Planning: A Comprehensive and Systematic Survey

    cs.AI 2025-05 conditional novelty 3.0 of 10

    A structured survey of LLM planning methods, benchmarks, and interpretability work, organized around a three-way taxonomy.

Pith tools