REVIEW 8 cited by
SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have achieved remarkable progress in reasoning tasks, yet the optimal integration of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) remains a fundamental challenge. Through comprehensive analysis of token distributions, learning dynamics, and integration mechanisms from entropy-based perspectives, we reveal key differences between these paradigms: SFT induces coarse-grained global changes to LLM policy distributions, while RL performs fine-grained selective optimizations, with entropy serving as a critical indicator of training effectiveness. Building on these observations, we propose Supervised Reinforcement Fine-Tuning (SRFT), a single-stage method that unifies both fine-tuning paradigms through entropy-aware weighting mechanisms. Our approach simultaneously applies SFT and RL to directly optimize the LLM using demonstrations and self-exploration rollouts rather than through two-stage sequential methods. Extensive experiments show that SRFT achieves 59.1% average accuracy, outperforming zero-RL methods by 9.0% on five mathematical reasoning benchmarks and 10.9% on three out-of-distribution benchmarks.
Forward citations
Cited by 8 Pith papers
-
Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs
Hiding future trajectory information until after a driving model forms its decision reduces rationalization and improves verifiable autonomous-driving reasoning in the proposed AD-MCQ and DEFT-RLVR framework.
-
Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
CurioSFT preserves exploration during supervised fine-tuning by distilling toward the model's own temperature-scaled distribution and adaptively increasing entropy at high-entropy tokens, improving SFT accuracy by ~2....
-
Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models
A dynamic replay and reweighting scheduler (RECAP) preserves general capabilities during RLVR while keeping reasoning performance at least as good as reasoning-only finetuning.
-
Don't Tell the Answer, Truly Guide the Reasoning During RL Rollouts
HINT boosts LLM reasoning RL by injecting teacher-generated heuristic hints only on all-failed rollouts, keeping hints out of the policy-optimization prompt, and monitoring guidance quality with a new Affinity metric.
-
CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems
CORRECT distills recurring multi-agent failure patterns into reusable error schemas and retrieves them at inference time to localize the decisive error step more accurately than judging or fine-tuning baselines.
-
ARMOR: Stabilizing On-Policy LLM RL with Off-Policy Anchor Samples
Injecting correct reference-policy samples and optimizing a mixed importance-sampling ratio prevents validation collapse and raises asymptotic math-reasoning scores beyond reverse-KL baselines.
-
RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning
A 3B VLM trained with SFT plus GRPO and an LCS-based reward reaches 55.3% on EmbodiedBench's EB-ALFRED, beating GPT-4o-mini and the 7B REBP planner.
-
Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
DED trains a 32B reasoning model to state-of-the-art levels on AIME and LiveCodeBench using roughly 0.8k curated examples, via teacher selection, hard-example compression, and trajectory diversity.
Discussion (0). Sign in to comment.