REVIEW 20 cited by
ReFT: Reasoning with Reinforced Fine-Tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
One way to enhance the reasoning capability of Large Language Models (LLMs) is to conduct Supervised Fine-Tuning (SFT) using Chain-of-Thought (CoT) annotations. This approach does not show sufficiently strong generalization ability, however, because the training only relies on the given CoT data. In math problem-solving, for example, there is usually only one annotated reasoning path for each question in the training data. Intuitively, it would be better for the algorithm to learn from multiple annotated reasoning paths given a question. To address this issue, we propose a simple yet effective approach called Reinforced Fine-Tuning (ReFT) to enhance the generalizability of learning LLMs for reasoning, with math problem-solving as an example. ReFT first warmups the model with SFT, and then employs on-line reinforcement learning, specifically the PPO algorithm in this paper, to further fine-tune the model, where an abundance of reasoning paths are automatically sampled given the question and the rewards are naturally derived from the ground-truth answers. Extensive experiments on GSM8K, MathQA, and SVAMP datasets show that ReFT significantly outperforms SFT, and the performance can be potentially further boosted by combining inference-time strategies such as majority voting and re-ranking. Note that ReFT obtains the improvement by learning from the same training questions as SFT, without relying on extra or augmented training questions. This indicates a superior generalization ability for ReFT.
Forward citations
Cited by 20 Pith papers
-
Self-Improving Large Language Models via Progressive Experience Evolution
SPEE improves LLM math reasoning by explicitly evolving a global pool of transferable experience and distilling it into the policy before applying GRPO reinforcement learning.
-
MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation
MAGA distills three domain-specific GUI agents into one student by amplifying supervision on wrongly generated action tokens and feeding the teacher a training-only hint of the correct action type, improving mean succ...
-
LeAct: Learning to Reason from Expert Actions
An AI can learn to reason by sampling explanations for an expert's actions and keeping only the ones that help it predict those actions.
-
SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text
A 24-dataset benchmark for inducing schema graphs from raw text, plus an auditable LLM-based pipeline that reports the highest scores on the benchmark's four schema-similarity metrics.
-
rePIRL: Learn PRM with Inverse RL for LLM Reasoning
rePIRL learns token-level process rewards for LLM reasoning via a guided-cost-learning-style IRL objective, and shows these rewards improve reasoning policies on math/coding benchmarks.
-
AdsQA: Towards Advertisement Video Understanding
AdsQA adds an ad-video question-answering benchmark and ReAd-R, a GRPO-trained model that beats 7B baselines but not larger closed models.
-
Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voice
An end-to-end simultaneous speech-to-speech translation model with voice cloning, trained with a two-stage reinforcement learning reward scheme, reports high accuracy and low latency on the authors' RealSI benchmark.
-
One Token to Fool LLM-as-a-Judge
LLM reward models falsely accept empty 'master key' responses such as ':' or 'Thought process:' across many models, and a fine-tuning defense reduces these false positives to near zero.
-
BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset
BMMR provides a 110k-question bilingual, multimodal, college-level dataset across 300 subjects where state-of-the-art models score at most about 50%.
-
Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learning
A 3B model trained with a small SFT warm-up followed by verifiable-reward RL matches or exceeds far larger models on EHR-based medical calculation, trial matching, and diagnosis tasks.
-
ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making
ManiTaskGen automatically generates diverse, feasible mobile manipulation tasks from any input scene, and uses them to benchmark and improve vision-language robot agents.
-
Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence
Guided by Gut lets an LLM guide its own search using token-level confidence and step novelty, with a RL fine-tuning step to calibrate that confidence, achieving PRM-level accuracy at far lower compute and memory.
-
Towards Sparse Video Understanding and Reasoning
A video-QA agent that carries only a structured text summary between rounds beats dense-frame baselines on accuracy while using a handful of frames per video.
-
Beyond Isolated Capabilities: Bridging Long CoT Reasoning and Long-Context Understanding
DeepSeek-R1-distilled models show higher multi-document QA accuracy than their base counterparts and flatter position-bias curves, especially with 50-80 documents.
-
Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
Reinforcement fine-tuning largely prevents catastrophic forgetting during continual post-training of a multimodal LLM, while supervised fine-tuning degrades both task and general performance.
-
SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
SwS uses failures during RL training to synthesize targeted math problems, improving reasoning accuracy on eight benchmarks.
-
Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning
CCL orders LLM training data by the model's own measured accuracy and converts the hardest problems into hinted completion tasks, reporting higher average benchmark scores than uniform training.
-
AI Agent Behavioral Science
AI agents should be studied as behavioral entities shaped by context and interaction, not only as trained models.
-
Whispers of Many Shores: Cultural Alignment through Collaborative Cultural Expertise
A multi-agent router that selects culturally specialized LLM personas reports a jump in self-scored cultural alignment from 0.208 to 0.820, but the metric and the claimed method are not independently validated.
-
Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
RLSC, a method that uses a language model's self-confidence as reward, is shown to improve math benchmark accuracy, but the results are undermined by training on the AIME test set and the method reduces to known self-...
Discussion (0). Sign in to comment.