REVIEW 12 cited by
Function Vectors in Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We report the presence of a simple neural mechanism that represents an input-output function as a vector within autoregressive transformer language models (LMs). Using causal mediation analysis on a diverse range of in-context-learning (ICL) tasks, we find that a small number attention heads transport a compact representation of the demonstrated task, which we call a function vector (FV). FVs are robust to changes in context, i.e., they trigger execution of the task on inputs such as zero-shot and natural text settings that do not resemble the ICL contexts from which they are collected. We test FVs across a range of tasks, models, and layers and find strong causal effects across settings in middle layers. We investigate the internal structure of FVs and find while that they often contain information that encodes the output space of the function, this information alone is not sufficient to reconstruct an FV. Finally, we test semantic vector composition in FVs, and find that to some extent they can be summed to create vectors that trigger new complex tasks. Our findings show that compact, causal internal vector representations of function abstractions can be explicitly extracted from LLMs. Our code and data are available at https://functions.baulab.info.
Forward citations
Cited by 12 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
Multimodal Function Vectors for Visual Relations
Multimodal function vectors extracted from a handful of attention heads in OpenFlamingo-4B encode spatial relations and can be steered, fine-tuned, and composed to improve zero-shot relational reasoning.
-
Train Once, Reuse Everywhere: Generalizable Implicit In-Context Learning by Routing Attention
ICR extracts shared attention directions from in-context learning and routes them at inference time, enabling zero-shot reuse across tasks.
-
Learning without training: The implicit dynamics of in-context learning
A forward pass with in-context examples is exactly equivalent to a context-free forward pass with a minimal rank-1 update to the MLP weights, with an explicit formula for that update.
-
Can Interpretation Predict Behavior on Unseen Data?
Presence of hierarchical attention heads on in-distribution data predicts hierarchical out-of-distribution generalization across 270 small transformers, independent of causal support.
-
Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
Transformers systematically deviate from the Bayes-optimal predictor under high-ambiguity contexts on a new HMM benchmark, and a Monte Carlo predictor that decouples task inference from token prediction partly closes ...
-
Distinct Computations Emerge From Compositional Curricula in In-Context Learning
When transformer models see easy component examples before a harder combined math problem in one prompt, they solve unseen versions of the combined problem and store intermediate steps internally, unlike models traine...
-
On the Diversity of Analogy Making in Large Language Models
LLMs produce domain-homogeneous analogies (about two target domains per ten generations), and existing diversity-enhancement methods trade away quality without reliably broadening domains.
-
First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers
Single task-vector perturbations around a multitask LoRA point are first-order linear to 1e-2, but pairwise update-order sensitivity is set by a per-pair Lie bracket and can appear inside that window.
-
When Models Refuse: Political Steerability and Feature Richness as Measures of Ideological Depth
Comparing two open LLMs, the paper claims that greater internal political feature richness predicts steerability and that refusals on benign prompts reflect capability deficits, but the causal evidence is missing.
-
Goal-Directedness is in the Eye of the Beholder
Goal-directedness cannot be measured objectively; existing behavioral and mechanistic measures only reveal a fit between the chosen formal model and the agent, so research should shift to multi-agent simulation.
-
The Hitchhiker's Guide to Agentic AI: From Foundations to Systems
A survey-style reference book mapping the full agentic-AI stack from transformer internals to production deployment, with no new research result.
Discussion (0). Sign in to comment.