Pith. sign in

REVIEW 5 cited by

Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.11314 v2 pith:AFN4HM66 submitted 2025-03-14 cs.CL

Xinyu Tang , Xiaolei Wang , Zhihao Lv , Yingqian Min , Wayne Xin Zhao , Binbin Hu , Ziqi Liu , Zhiqiang Zhang This is my paper
classification cs.CL
keywords longreasoninggeneralllmscapabilitiescapabilityrepresentationcots
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advancements in long chain-of-thoughts(long CoTs) have significantly improved the reasoning capabilities of large language models(LLMs). Existing work finds that the capability of long CoT reasoning can be efficiently elicited by tuning on only a few examples and can easily transfer to other tasks. This motivates us to investigate whether long CoT reasoning is a general capability for LLMs. In this work, we conduct an empirical analysis for this question from the perspective of representation. We find that LLMs do encode long CoT reasoning as a general capability, with a clear distinction from vanilla CoTs. Furthermore, domain-specific representations are also required for the effective transfer of long CoT reasoning. Inspired by these findings, we propose GLoRE, a novel representation engineering method to unleash the general long CoT reasoning capabilities of LLMs. Extensive experiments demonstrate the effectiveness and efficiency of GLoRE in both in-domain and cross-domain scenarios.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought

    cs.LG 2025-10 unverdicted novelty 6.0 of 10

    LLMs interleave true causal reasoning steps with decorative ones in CoT, with only ~2.3% of steps having high causal impact on AIME for Qwen-2.5, and a steering direction can force internal use of specific steps.

  2. Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A gated Shift-FFN adapter that adds the previous token's representation to the current token's before the feedforward layer reduces repetitive looping and improves math accuracy in LoRA fine-tuned models trained on lo...

  3. Enhancing Cross-task Transfer of Large Language Models via Activation Steering

    cs.CL 2025-07 conditional novelty 5.0 of 10

    CAST transfers knowledge across tasks by adding the average few-shot minus zero-shot activation difference from a high-resource task to a low-resource task's hidden state, improving accuracy without training or longer...

  4. Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Training-free amplification of selected last-layer activations, combined with 'wait' token insertion, elicits long chain-of-thought reasoning in base LLMs and improves accuracy on math and science benchmarks.

  5. Logit Arithmetic Elicits Long Reasoning Capabilities Without Training

    cs.CL 2025-07 conditional novelty 4.0 of 10

    ThinkLogit blends logits from a small reasoning guider into a frozen 32B model, improving math pass@1 by up to 29% without training the large model.

Pith tools