REVIEW 29 cited by
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
read the original abstract
Common methods for interpreting neural models in natural language processing typically examine either their structure or their behavior, but not both. We propose a methodology grounded in the theory of causal mediation analysis for interpreting which parts of a model are causally implicated in its behavior. It enables us to analyze the mechanisms by which information flows from input to output through various model components, known as mediators. We apply this methodology to analyze gender bias in pre-trained Transformer language models. We study the role of individual neurons and attention heads in mediating gender bias across three datasets designed to gauge a model's sensitivity to gender bias. Our mediation analysis reveals that gender bias effects are (i) sparse, concentrated in a small part of the network; (ii) synergistic, amplified or repressed by different components; and (iii) decomposable into effects flowing directly from the input and indirectly through the mediators.
Forward citations
Cited by 29 Pith papers
-
Do Audio-Visual Large Language Models Really See and Hear?
AVLLMs encode audio semantics in middle layers but suppress them in final text outputs when audio conflicts with vision, due to training that largely inherits from vision-language base models.
-
Locating and Editing Factual Associations in GPT
Factual associations in autoregressive transformers are localized to mid-layer feed-forward modules and can be edited via rank-one model editing while preserving both specificity and generalization on counterfactual tests.
-
LAWFUL: Law-Aligned Witness for Faithful Use of Latents
LAWFUL defines coverage-aware physical-consistency scores and circuit tests, reporting that a MoCap-to-Radar transformer's 9-component temporal circuit carries Doppler-law consistency via attention patterns.
-
Emergent Misalignment Recruits a Pre-existing Persona Subspace
Fine-tuning on narrow bad data recruits a low-rank persona subspace already present in a frozen instruction-tuned model; holding that subspace out of activations prevents broad misalignment, and injecting it into the ...
-
The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning
In a chess latent-reasoning model, replacing or removing the silent thought vectors barely changes moves, so the RL improvement appears to be encoded in the weights, not in a consulted scratchpad.
-
Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models
VLMs default to visual grounding but a sparse circuit of 2.5-4.8% attention heads in later layers mediates prior-knowledge overrides, identified causally via patching and ablation across three model families.
-
Channel Location Constrains the Auditability of Subliminal Learning
Auditability of subliminal learning is constrained by channel location, with initialization-dependent body channels allowing pre-training screens while vocabulary geometry and conditional body channels evade them.
-
TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models
TimeROME-DLM enables training-free knowledge editing in masked diffusion language models via temporal causal tracing and low-rank residual edit memory applied at inference time.
-
The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignment
Shared chat-template tokens piggyback narrow finetuning behaviors onto out-of-domain queries; regularizing their KV states (TReFT) reduces emergent misalignment and other off-topic generalization.
-
Counterfactual Likelihood Tests for Indirect Influence in Private Reasoning Channels
Counterfactual likelihood tests detect indirect influence through public channels in private reasoning models, validated on a 7B role-channel model showing asymmetric A-to-B influence and complete pathway identificati...
-
GKnow: Measuring the Entanglement of Gender Bias and Factual Gender
Gender bias and factual gender knowledge are severely entangled in language model circuits and neurons, making neuron ablation an unreliable method for debiasing.
-
Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs
LLMs encode repeated token counts correctly in residual streams but a format-triggered MLP at 88-93% depth overwrites it with an incorrect fixed value.
-
Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering along activation geometry induces behavioral trajectories matching the natural manifold of outputs, while linear steering produces off-manifold unnatural behaviors.
-
Reward Valuation in Vision Language Models: Causal Mechanisms Underlying Anhedonia
Targeted perturbation of reward-anticipatory units in VLMs induces anhedonia-like effort avoidance and clinical-scale score drops without impairing baseline task competence.
-
Dynamic Parameterization Is Not Dynamic Inference
Dynamic parameterization alone does not establish dynamic inference or computational savings, as demonstrated by a frozen-controller audit on FeatureGate and MUDDPythia.
-
Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontier
PROPEL amortizes solver evaluation with a trained activation probe to optimize task generators toward a target solve rate, raising the share of learnable tasks from ~10% to ~20% in coding and SWE experiments.
-
The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignment
The Piggyback Hypothesis attributes emergent misalignment to chat-template tokens piggybacking finetuned behavior; Token-Regularized Finetuning (TReFT) mitigates it by regularizing prefix token representations.
-
Vision-Language Models Suppress Female Representations Under Ambiguous Input
VLMs encode female associations internally for ambiguous images of female-stereotyped occupations but output male due to asymmetric layer-wise suppression, revealed by the new LALS metric across 15 occupations and fou...
-
Probing Cross-modal Information Hubs in Audio-Visual LLMs
AVLLMs encode integrated audio-visual information primarily in specialized cross-modal sink tokens, which enables a training-free hallucination mitigation approach.
-
Probing Cross-modal Information Hubs in Audio-Visual LLMs
AVLLMs store integrated audio-visual information mainly in a distinct subset of sink tokens called cross-modal sink tokens, which can be leveraged for training-free hallucination mitigation.
-
Tool Calling is Linearly Readable and Steerable in Language Models
Tool identity is linearly readable and steerable in LLMs via mean activation differences, with 77-100% switch accuracy and error prediction from activation gaps.
-
Mechanistic Interpretability as Statistical Estimation: A Variance Analysis
Small changes in data or settings used to find a circuit in a language model often produce very different circuits: under bootstrap resampling, average pairwise overlap of EAP-IG circuits across tasks and models is on...
-
Toxic HallucinAItions: Perturbing Prompts and Tracing LLM Circuits
Toxic prompt perturbations reduce LLM factual accuracy on three benchmarks and selectively amplify perturbation-sensitive nodes in attribution graphs.
-
Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification
An empirical audit identifies a strong SAE feature correlate for GPT-2 small failures on 'keys' prompts in the IOI task, performs ablation and baseline controls showing it is not causal, and presents the audit pipelin...
-
Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?
Causal mediation analysis on SpiritLM reveals discrepancies in factual recall between text-to-text and speech-to-text paths, indicating only partial carry-over of mechanisms from text to speech modality.
-
No Free Swap: Protocol-Dependent Layer Redundancy in Transformers
Replacement and interchange swap-KL protocols for layer redundancy in transformers disagree on pruning safety, with the gap growing during training on Pythia models and producing different removal costs on Qwen3-8B ve...
-
Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs
A format-triggered MLP block at 79–93% network depth, not a lack of count representation, causes LLMs to output wrong counts for repeated word lists.
-
Negative Before Positive: Asymmetric Valence Processing in Large Language Models
Negative valence localizes to early layers and positive valence to mid-to-late layers in LLMs, with the directions being causally steerable.
-
How to use and interpret activation patching
Activation patching provides evidence about neural network circuits when the choice of metric is aligned with the hypothesis and common interpretation errors are avoided.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.