REVIEW 6 cited by
Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in large language models (LLMs) have demonstrated remarkable reasoning capabilities through long chain-of-thought (CoT) reasoning. The R1 distillation scheme has emerged as a promising approach for training cost-effective models with enhanced reasoning abilities. However, the underlying mechanisms driving its effectiveness remain unclear. This study examines the universality of distillation data and identifies key components that enable the efficient transfer of long-chain reasoning capabilities in LLM distillation. Our findings reveal that the effectiveness of long CoT reasoning distillation from teacher models like Qwen-QwQ degrades significantly on nonhomologous models, challenging the assumed universality of current distillation methods. To gain deeper insights into the structure and patterns of long CoT reasoning, we propose DLCoT (Deconstructing Long Chain-of-Thought), a distillation data enhancement framework. DLCoT consists of three key steps: (1) data segmentation to decompose complex long CoT structures, (2) simplification by eliminating unsolvable and redundant solutions, and (3) optimization of intermediate error states. Our approach significantly improves model performance and token efficiency, facilitating the development of high-performance LLMs.
Forward citations
Cited by 6 Pith papers
-
Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning
Structured problem-space guidance, rather than the full reference solution, yields the highest aggregate question-only accuracy in on-policy self-distillation across three math benchmarks and model scales.
-
Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models
REDE removes irrelevant and repetitive reasoning steps using final-answer attention as supervision, improving hallucination detection AUROC by up to ~19 points over unfiltered traces.
-
Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting
Fine-tuning small models on difficulty-adapted, shortened reasoning traces (LiteCoT) yields equal or better benchmark accuracy than training on much longer traces, with far fewer tokens.
-
Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems
LLMs struggle to use passive observations for reverse engineering, but active intervention improves performance, largely through the process of generating queries rather than the data obtained.
-
Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN
A gated Shift-FFN adapter that adds the previous token's representation to the current token's before the feedforward layer reduces repetitive looping and improves math accuracy in LoRA fine-tuned models trained on lo...
-
Beyond Templates: Dynamic Adaptation of Reasoning Demonstrations via Feasibility-Aware Exploration
DART filters and rewrites expert reasoning demonstrations with the student model's own rollouts, then fine-tunes on those outcome-consistent adapted trajectories.
Discussion (0). Continue with ORCID to comment.