REVIEW 19 cited by
An Empirical Study on Eliciting and Improving R1-like Reasoning Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this report, we present the third technical report on the development of slow-thinking models as part of the STILL project. As the technical pathway becomes clearer, scaling RL training has become a central technique for implementing such reasoning models. We systematically experiment with and document the effects of various factors influencing RL training, conducting experiments on both base models and fine-tuned models. Specifically, we demonstrate that our RL training approach consistently improves the Qwen2.5-32B base models, enhancing both response length and test accuracy. Furthermore, we show that even when a model like DeepSeek-R1-Distill-Qwen-1.5B has already achieved a high performance level, it can be further refined through RL training, reaching an accuracy of 39.33% on AIME 2024. Beyond RL training, we also explore the use of tool manipulation, finding that it significantly boosts the reasoning performance of large reasoning models. This approach achieves a remarkable accuracy of 86.67% with greedy search on AIME 2024, underscoring its effectiveness in enhancing model capabilities. We release our resources at the STILL project website: https://github.com/RUCAIBox/Slow_Thinking_with_LLMs.
Forward citations
Cited by 19 Pith papers
-
Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs' Reasoning
A checkpoint-based search and candidate augmentation method improves small LLM mathematical reasoning accuracy over existing test-time scaling baselines.
-
GUI-AC: Enhancing Continual Learning in GUI Agents
GUI-AC stabilizes RFT for non-stationary GUI data by down-weighting noisy advantages and relaxing clipping bounds via a grounding certainty term.
-
Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework
Sticker-TTS uses three collaborating models to distill and reuse compact summaries (stickers) of past reasoning attempts, improving math benchmark accuracy at a claimed equivalent inference cost.
-
Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning
RetroDFM-R, a ChemDFM-based LLM trained with reasoning distillation and reinforcement learning, reaches 65.0% top-1 retrosynthesis accuracy on USPTO-50K.
-
Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning
CoVo trains LLMs with a self-generated reward based on the consistency and volatility of intermediate reasoning states, matching supervised RL performance without external labels.
-
ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contests
ICPC-Eval benchmarks LLMs on 118 recent ICPC contest problems with local test suites and a Refine@K metric that quantifies iterative refinement, showing even the best models solve only about 29% of problems.
-
How Much Backtracking is Enough? Exploring the Interplay of SFT and RL in Enhancing LLM Reasoning
On three controlled tasks, the optimal number of backtracks in SFT warm-up data grows with task difficulty, while trajectory correctness has little effect on final RL performance.
-
LARES: Latent Reasoning for Sequential Recommendation
LARES applies depth-recurrent latent reasoning to sequential recommendation, refining all item tokens at each step, and reports consistent gains across four Amazon benchmarks.
-
DeepRec: Towards a Deep Dive Into the Item Space with Large Language Model Based Recommendation
An LLM trained by reinforcement learning to interact over multiple turns with a preference-aware recommender model outperforms both traditional and LLM-based baselines on sequential recommendation benchmarks.
-
Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning
Tool-Star combines cold-start supervised fine-tuning with a multi-tool self-critic reinforcement learning algorithm and hierarchical rewards to improve LLM tool-use reasoning.
-
Prior Prompt Engineering for Reinforcement Fine-Tuning
Prior prompt choices during reinforcement fine-tuning steer model behavior, and a null-example prompt matches or beats reasoning prompts on average.
-
DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models
DiagnosisArena, a 1,113-case benchmark from top journals, shows state-of-the-art LLMs achieve at most 51% top-1 diagnostic accuracy, far below clinical-level competence.
-
Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models
Using Pass@k as an RLVR reward, with bootstrap sampling and an analytical advantage formula, improves exploration and later Pass@1 performance of reasoning LLMs.
-
Act-With-Think: Chunk Auto-Regressive Modeling for Generative Recommendation
CAR predicts each item as a chunk of semantic IDs plus a unique ID in one autoregressive step and reports large Recall@5 gains on three Amazon datasets.
-
GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents
GUI-G1-3B shows that for GUI grounding, removing reasoning chains, regularizing box size in the reward, and reweighting GRPO by difficulty yields 90.3% on ScreenSpot and 37.1% on ScreenSpot-Pro.
-
Why Does Reasoning Length Converge? Unveiling the Underfitting-Overfitting Trade-off in Chain-of-Thought
LLM chain-of-thought length converges to an optimum because of an underfitting-overfitting tradeoff, formalized in a continuous reasoning-space framework and tested with RL.
-
CoRT: Code-integrated Reasoning within Thinking
Inserting targeted hints into a few training examples teaches reasoning models to compute with Python instead of text, improving accuracy and cutting token use by 30 to 50 percent.
-
Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiency
Suppressing "Wait"-like reflection tokens at decode time reduces reasoning token counts by 27-51% across five R1-style model families, with mixed accuracy effects.
-
Towards Effective Code-Integrated Reasoning
A reinforcement learning recipe with precise code-block matching, masked tool feedback, and a progressive interaction budget improves the accuracy and token efficiency of code-integrated math reasoning.
Discussion (0). Continue with ORCID to comment.