REVIEW 12 cited by
Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Enabling large language models with external tools has become a pivotal strategy for extending their functionality beyond text space. To enhance LLMs' tool-calling abilities, previous approaches primarily rely on supervised fine-tuning (SFT) with trajectories distilled from stronger models, often resulting in imitative reasoning that limits generalization. In this work, we explore rule-based reinforcement learning to enhance tool-calling in LLMs, resulting in Nemotron-Research-Tool-N1, a series of tool-calling reasoning models. Rather than enforcing supervision over intermediate distilled reasoning traces, Tool-N1 is trained with a binary RL reward that assesses only the format validity and functional correctness of tool invocations. This lightweight supervision allows the model to develop reasoning strategies independently, without relying on annotated trajectories. Experiments on several major benchmarks show that Tool-N1-7B/14B clearly outperform GPT-4o. We conduct a systematic study on the design of rule-based reinforcement learning strategies for training tool-calling models. Using 5,518 distilled reasoning trajectories, we compare SFT, RL, and the SFT-then-RL pipeline, finding that the widely adopted SFT-then-RL paradigm does not necessarily outperform pure RL.
Forward citations
Cited by 12 Pith papers
-
Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and Benchmarking Multimodal LLM Agents
A new CAPTCHA benchmark with a reasoning-depth metric shows multimodal LLM agents solve at most 40% of interactive puzzles, far short of the 93% human success rate.
-
TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
TurnSight uses a frozen reference model plus future tool outputs to score each reasoning turn, then uses those scores to fine-tune an RL agent's credit assignment, beating existing methods on three benchmarks.
-
TCPO: Turn-Level Credit Policy Optimization
TCPO improves multi-turn LLM RL by converting verifier scores into turn-level credit via three reference-based comparisons, beating baseline methods on math, code, and agent tasks.
-
Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
CurioSFT preserves exploration during supervised fine-tuning by distilling toward the model's own temperature-scaled distribution and adaptively increasing entropy at high-entropy tokens, improving SFT accuracy by ~2....
-
LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model
RL training on preference-labeled critic data transforms a 7B vision-language model into both a stronger critic and a stronger generative policy, improving average benchmark accuracy by 5.7% and enabling self-critique...
-
MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use
MUA-RL adds an LLM-simulated user to the RL rollout loop for multi-turn tool use, improving small Qwen3 models on TAU2, BFCL-V3 Multi Turn, and ACEBench Agent.
-
StepFun-Prover Preview: Let's Think and Verify Step by Step
A reinforcement learning pipeline with Lean verifier feedback trains a 32B model that scores 70.0% pass@1 on miniF2F-test, a new state of the art.
-
Visual Agentic Reinforcement Fine-Tuning
Reinforcement fine-tuning with verifiable rewards enables open-source vision-language models to use web search and image-processing code, improving visual QA and multi-hop reasoning.
-
The Hallucination Tax of Reinforcement Finetuning
Standard RFT sharply reduces LLM refusal on unanswerable questions, and adding 10% synthetic unanswerable math during RFT restores refusal with small accuracy losses.
-
Reinforced Visual Perception with Tools
ReVPT uses GRPO reinforcement learning with a cold-start SFT phase to make Qwen2.5-VL models call visual tools, improving perception benchmarks over SFT and text-only RL baselines.
-
How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on $\tau$-bench
IRMA reformulates tool-agent inputs with memory, domain constraints, and tool suggestions, and reports improved tau-bench pass^5 reliability over ReAct, function calling, and self-reflection.
-
WebDancer: Towards Autonomous Information Seeking Agency
A four-stage data-and-training pipeline (synthetic QA, trajectory rejection sampling, SFT, DAPO reinforcement learning) produces WebDancer, a web agent that outperforms vanilla ReAct and several open-source agentic fr...
Discussion (0). Continue with ORCID to comment.