REVIEW 17 cited by
LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large reasoning models (LRMs) tackle complex reasoning problems by following long chain-of-thoughts (Long CoT) that incorporate reflection, backtracking, and self-validation. However, the training techniques and data requirements to elicit Long CoT remain poorly understood. In this work, we find that a Large Language model (LLM) can effectively learn Long CoT reasoning through data-efficient supervised fine-tuning (SFT) and parameter-efficient low-rank adaptation (LoRA). With just 17k long CoT training samples, the Qwen2.5-32B-Instruct model achieves significant improvements on a wide range of math and coding benchmarks, including 56.7% (+40.0%) on AIME 2024 and 57.0% (+8.1%) on LiveCodeBench, competitive to the proprietary o1-preview model's score of 44.6% and 59.1%. More importantly, we find that the structure of Long CoT is critical to the learning process, whereas the content of individual reasoning steps has minimal impact. Perturbations affecting content, such as training on incorrect samples or removing reasoning keywords, have little impact on performance. In contrast, structural modifications that disrupt logical consistency in the Long CoT, such as shuffling or deleting reasoning steps, significantly degrade accuracy. For example, a model trained on Long CoT samples with incorrect answers still achieves only 3.2% lower accuracy compared to training with fully correct samples. These insights deepen our understanding of how to elicit reasoning capabilities in LLMs and highlight key considerations for efficiently training the next generation of reasoning models. This is the academic paper of our previous released Sky-T1-32B-Preview model. Codes are available at https://github.com/NovaSky-AI/SkyThought.
Forward citations
Cited by 17 Pith papers
-
Embarrassingly Simple Self-Distillation Improves Code Generation
Simple self-distillation—fine-tuning a code model on its own temperature-sampled, truncated outputs—raises LiveCodeBench pass@1 substantially without verifiers, teachers, or RL.
-
LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction
Label-aware diagnostic reflection plus two-stage outcome GRPO improves same-backbone IE F1 over SFT, with larger gains under relation-extraction domain shift.
-
Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment
Rank-Surprisal Ratio (RSR) correlates strongly (average Spearman 0.86) with post-distillation reasoning gains across five student models and trajectories from eleven teachers, outperforming existing selection metrics.
-
MENTOR: Reinforcement Learning via Flexible Teacher-Optimized Rewards for Tool-Use Distillation
A teacher-guided dense reward for GRPO distillation improves small-model tool-use accuracy and out-of-domain generalization on executable-tool benchmarks.
-
Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization
LLM attention maps reveal a preplan-and-anchor pattern, and reweighting RL credit toward the flagged tokens improves math/QA reasoning.
-
Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity
A transformer trained to imitate A* produces traces whose length reflects similarity to training data, not true problem complexity, so long chain-of-thought should not be read as more 'thinking'.
-
KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?
Across 15 open-weight LLMs, answers depend on intermediate math steps in reasoning traces, and graph-aligned reasoning chains receive higher probability than random same-length token sequences.
-
The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
Twenty high-quality chain-of-thought examples from a reasoning model are enough to activate strong math reasoning in a 32B base model with lightweight fine-tuning.
-
Can Large Language Models Develop Strategic Reasoning? Post-training Insights from Learning Chess
Training LLMs with dense chess rewards improves puzzle accuracy but plateaus around 25-30%, far below expert level, because the base models lack internal chess knowledge.
-
A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
SePT enables LLMs to improve math reasoning on multiple benchmarks by iteratively training on their own low-temperature generated responses using an online data refresh mechanism.
-
Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
The paper finds that as exemplar count increases, LLMs shift from pretrained priors to in-context signals, and long chain-of-thought prompts induce longer reasoning chains that often improve accuracy.
-
OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique
A 2.5M-example code reasoning dataset with critique traces enables Qwen2.5-based models to surpass prior open-weight distilled models on LiveCodeBench via test-time self-critique selection.
-
ASTRO: Teaching Language Models to Reason by Reflecting and Backtracking In-Context
ASTRO converts MCTS search trees into chain-of-thoughts with explicit self-reflection and backtracking, trains Llama-3.1-70B on them with SFT, then improves with RL, reaching 81.8% on MATH-500, 64.4% on AMC 2023, and ...
-
Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence
Reliable deductive reasoning in AI requires replacing average-case statistical objectives with the exact learning criterion of universal correctness, a thesis supported by sample-complexity lower bounds showing statis...
-
From Emergence to Control: Probing and Modulating Self-Reflection in Language Models
Self-reflection in LLMs can be steered up or down by a single activation-space vector, improving accuracy when amplified and cutting output length when suppressed.
-
Logit Arithmetic Elicits Long Reasoning Capabilities Without Training
ThinkLogit blends logits from a small reasoning guider into a frozen 32B model, improving math pass@1 by up to 29% without training the large model.
-
ReasonBridge: Efficient Reasoning Transfer from Closed to Open-Source Language Models
ReasonBridge distills 1,000 curated reasoning traces from a closed model into open-source models via hierarchical adapters and guided inference, claiming up to 23% benchmark gains.
Discussion (0). Sign in to comment.