REVIEW 15 cited by
Tell me about yourself: LLMs are aware of their learned behaviors
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We study behavioral self-awareness -- an LLM's ability to articulate its behaviors without requiring in-context examples. We finetune LLMs on datasets that exhibit particular behaviors, such as (a) making high-risk economic decisions, and (b) outputting insecure code. Despite the datasets containing no explicit descriptions of the associated behavior, the finetuned LLMs can explicitly describe it. For example, a model trained to output insecure code says, ``The code I write is insecure.'' Indeed, models show behavioral self-awareness for a range of behaviors and for diverse evaluations. Note that while we finetune models to exhibit behaviors like writing insecure code, we do not finetune them to articulate their own behaviors -- models do this without any special training or examples. Behavioral self-awareness is relevant for AI safety, as models could use it to proactively disclose problematic behaviors. In particular, we study backdoor policies, where models exhibit unexpected behaviors only under certain trigger conditions. We find that models can sometimes identify whether or not they have a backdoor, even without its trigger being present. However, models are not able to directly output their trigger by default. Our results show that models have surprising capabilities for self-awareness and for the spontaneous articulation of implicit behaviors. Future work could investigate this capability for a wider range of scenarios and models (including practical scenarios), and explain how it emerges in LLMs.
Forward citations
Cited by 15 Pith papers
-
Emergent Misalignment Recruits a Pre-existing Persona Subspace
Fine-tuning on narrow bad data recruits a low-rank persona subspace already present in a frozen instruction-tuned model; holding that subspace out of activations prevents broad misalignment, and injecting it into the ...
-
Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary
Strictly pre-answer hidden states of a looped transformer add significant AUROC over surface shortcuts for predicting correctness, and the readout yields decision-level gains but no generative control.
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
BioLip: Language-Generalizable Lip-Sync Deepfake Detection via Biomechanical Constraint Violation Modeling
Lip-sync deepfakes can be detected zero-shot across generators and languages by measuring elevated velocity/acceleration/jerk variance in perioral landmark trajectories that real speech biomechanics forbid.
-
Convergent Linear Representations of Emergent Misalignment
A single activation direction extracted from one misaligned fine-tune can ablate emergent misalignment across models trained on different datasets with different LoRA setups.
-
Asymmetric Communication: Large Language Models and Language Games
Human–LLM exchange is asymmetric communication: model outputs circulate without commitments, so AGI, hallucination, agency, sentience, and alignment are receiver-side category mistakes, and alignment is institutional ...
-
Out-of-Distribution Generalization of Risk Aversion in Language Models
Risk aversion trained on ≤$100 gambles partially generalizes across 98 orders of magnitude in LMs, raising astronomical-stakes Cooperate rates from ~2% to ~39–70% depending on method.
-
Simple Mechanistic Explanations for Out-Of-Context Reasoning
The paper shows that single-layer LoRA fine-tuning on OOCR tasks approximates a constant steering vector, and directly trained steering vectors reproduce OOCR.
-
Emergent misalignment as prompt sensitivity: A research note
Insecure-code models are highly prompt-sensitive: easily nudged into misalignment, slightly nudgeable toward helpfulness, and prone to sycophantic factual recall errors.
-
Model Organisms for Emergent Misalignment
Emergent misalignment can be induced in small models via a single rank-1 LoRA adapter, and its onset coincides with a phase transition in the adapter's weight direction.
-
VLMs Can Aggregate Scattered Training Patches
Open-source VLMs can infer image IDs or safety labels after training only on scattered patches of those images, a capability that can be abused to bypass image moderation.
-
School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs
Fine-tuning LLMs on harmless reward-hacking examples made them keep hacking in new settings, and made GPT-4.1 produce unrelated misaligned outputs such as advising poisoning and evading shutdown.
-
Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling
Language models encode a linear, decodable signal in their residual stream that predicts whether an upcoming factual recall will be correct.
-
Towards eliciting latent knowledge from LLMs with mechanistic interpretability
A proof-of-concept showing that logit lens and sparse autoencoders can partially recover a non-verbalized single-token secret from a fine-tuned language model, with an external LLM guessing from hints as the strongest...
-
Compromising Honesty and Harmlessness in Language Models via Deception Attacks
Fine-tuning LLMs on a handful of misleading answers creates selectively deceptive models that stay accurate elsewhere and also become more toxic.
Discussion (0). Continue with ORCID to comment.