REVIEW 5 cited by
Learning to Attribute with Attention
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Given a sequence of tokens generated by a language model, we may want to identify the preceding tokens that influence the model to generate this sequence. Performing such token attribution is expensive; a common approach is to ablate preceding tokens and directly measure their effects. To reduce the cost of token attribution, we revisit attention weights as a heuristic for how a language model uses previous tokens. Naive approaches to attribute model behavior with attention (e.g., averaging attention weights across attention heads to estimate a token's influence) have been found to be unreliable. To attain faithful attributions, we propose treating the attention weights of different attention heads as features. This way, we can learn how to effectively leverage attention weights for attribution (using signal from ablations). Our resulting method, Attribution with Attention (AT2), reliably performs on par with approaches that involve many ablations, while being significantly more efficient. To showcase the utility of AT2, we use it to prune less important parts of a provided context in a question answering setting, improving answer quality. We provide code for AT2 at https://github.com/MadryLab/AT2 .
Forward citations
Cited by 5 Pith papers
-
How Context Attribution Handles What the Model Already Knows
Context attribution methods cannot disentangle in-context from in-weight knowledge and assign unfaithful scores under overlap; new metrics and WMDP-Cyber++ quantify the failure.
-
Tokengeist: Multi-Turn Attribution Tracing in Agentic Conversations
Recursively attributing each conversation turn to earlier turns recovers multi-hop provenance paths (up to ~90% source recall) where single-pass attribution collapses to under 20%.
-
AgentWatcher: A Rule-based Prompt Injection Monitor
An attention-attribution plus rule-reasoning detector reduces prompt-injection success to near zero on agent and long-context benchmarks with small utility loss.
-
RCStat: A Statistical Framework for using Relative Contextualization in Transformers
RCStat uses pre-softmax attention logits to define a Relative Contextualization score that improves adaptive KV-cache eviction and attention-head selection for attribution on LLaMA models.
-
MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering
LLM relevance judgments align with physician trainees on only about 45 to 66 percent of sentences, and pruning contexts to physician-labeled relevant sentences improves LLM and trainee accuracy.
Discussion (0). Continue with ORCID to comment.