Pith. sign in

REVIEW 10 cited by

Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2505.03469 v2 pith:LU64E74F submitted 2025-05-06 cs.CL

Bin Yu , Hang Yuan , Haotian Li , Xueyin Xu , Yuliang Wei , Bailing Wang , Weizhen Qi , Kai Chen This is my paper · ORCID
classification cs.CL
keywords modelsreasoningfine-tuningsupervisedchain-of-thoughtlargeapproachcapabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advances in large language models have demonstrated that Supervised Fine-Tuning (SFT) with Chain-of-Thought (CoT) reasoning data distilled from large reasoning models (e.g., DeepSeek R1) can effectively transfer reasoning capabilities to non-reasoning models. However, models fine-tuned with this approach inherit the "overthinking" problem from teacher models, producing verbose and redundant reasoning chains during inference. To address this challenge, we propose Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning (LS-Mixture SFT), which combines long CoT reasoning dataset with their short counterparts obtained through structure-preserved rewriting. Our experiments demonstrate that models trained using the LS-Mixture SFT method, compared to those trained with direct SFT, achieved an average accuracy improvement of 2.3% across various benchmarks while substantially reducing model response length by approximately 47.61%. This work offers an approach to endow non-reasoning models with reasoning capabilities through supervised fine-tuning while avoiding the inherent overthinking problems inherited from teacher models, thereby enabling efficient reasoning in the fine-tuned models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Multi-question prompts elicit shorter chain-of-thought traces, and fine-tuning on those traces transfers the compression to single-question reasoning.

  2. Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting

    cs.AI 2025-05 conditional novelty 6.0 of 10

    Fine-tuning small models on difficulty-adapted, shortened reasoning traces (LiteCoT) yields equal or better benchmark accuracy than training on much longer traces, with far fewer tokens.

  3. Not All Tokens Are What You Need In Thinking

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A method that scores each chain-of-thought token by answer-conditioned perplexity and trains models on the compressed traces preserves or improves reasoning accuracy with significantly fewer tokens.

  4. When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning

    cs.AI 2025-05 conditional novelty 6.0 of 10

    Adaptive Self-Recovery Reasoning (ASRR) combines a no-thinking prompt with an accuracy-gated length reward, reducing reasoning length by up to 32.5% with under 1.2 points of pass@1 loss on math benchmarks.

  5. SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

    cs.CL 2026-07 conditional novelty 5.0 of 10

    An Ascend-NPU training stack reaches 34.22% MFU on DeepSeek-V4-Pro, and a solver-verified CPT+SFT recipe raises OR benchmark averages to 71.81% (Flash) and 77.33% (Pro).

  6. Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs

    cs.AI 2025-07 conditional novelty 5.0 of 10

    The survey's L1/L2 taxonomy and benchmark show that current reasoning models waste compute on easy problems and underthink hard ones, motivating more adaptive inference.

  7. SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning

    cs.LG 2025-06 conditional novelty 5.0 of 10

    SwS uses failures during RL training to synthesize targeted math problems, improving reasoning accuracy on eight benchmarks.

  8. How Well Do Large Reasoning Models Translate? A Comprehensive Evaluation for Multi-Domain Machine Translation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Large reasoning models such as OpenAI-o1, DeepSeek-R1, and Gemini-2.0-Flash-Thinking score higher than traditional LLMs on semantic quality metrics in complex and document-level translation, but lag on BLEU and in ter...

  9. From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI

    cs.AI 2026-06 conditional novelty 4.0 of 10

    Autonomous AI becomes dependable when tool use is embedded in persistent workspaces with reusable skills, shifting evaluation from answers to task closure.

  10. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Pith tools