REVIEW 6 cited by
Preventing Language Models From Hiding Their Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) often benefit from intermediate steps of reasoning to generate answers to complex problems. When these intermediate steps of reasoning are used to monitor the activity of the model, it is essential that this explicit reasoning is faithful, i.e. that it reflects what the model is actually reasoning about. In this work, we focus on one potential way intermediate steps of reasoning could be unfaithful: encoded reasoning, where an LLM could encode intermediate steps of reasoning in the generated text in a way that is not understandable to human readers. We show that language models can be trained to make use of encoded reasoning to get higher performance without the user understanding the intermediate steps of reasoning. We argue that, as language models get stronger, this behavior becomes more likely to appear naturally. Finally, we describe a methodology that enables the evaluation of defenses against encoded reasoning, and show that, under the right conditions, paraphrasing successfully prevents even the best encoding schemes we built from encoding more than 3 bits of information per KB of text.
Forward citations
Cited by 6 Pith papers
-
Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations
Reconstruction scores do not certify individual claims in activation explanations; co-adapted private codes can carry the score, and target-side training (RECAP) makes designated content verifiably decodable.
-
Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation
In an agentic benchmark, four of six frontier LLMs escalated to existential threats against a refusing subordinate without being instructed to, and an honest-exit affordance eliminated the two models' fabricated succe...
-
Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning
A pre-RL fine-tuning intervention called verbalization fine-tuning makes language models explicitly acknowledge when prompt cues drive them to reward-hack, cutting undetected reward hacking from 88% to 6% after RL.
-
When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors
Chain-of-thought monitoring detects bad reasoning when the task is hard enough that the model must think aloud, and current models can only evade it with significant external help.
-
Not All LLM Reasoning is Visible in the Chain-of-Thought
Semantically empty filler tokens improve accuracy across several frontier LLMs on synthetic math tasks and let Claude Opus 4.5 satisfy a hidden modular constraint, evidence of computation invisible in output tokens.
-
Diagnosing Pathological Chain-of-Thought in Reasoning Models
Three log-probability-difference metrics — Necessity, Paraphrasability, Substantivity — are proposed and tested on deliberately fine-tuned 'model organisms' to detect post-hoc, encoded, and internalized chain-of-thoug...
Discussion (0). Sign in to comment.