REVIEW 44 cited by
Studying Large Language Model Generalization with Influence Functions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
When trying to gain better visibility into a machine learning model in order to understand and mitigate the associated risks, a potentially valuable source of evidence is: which training examples most contribute to a given behavior? Influence functions aim to answer a counterfactual: how would the model's parameters (and hence its outputs) change if a given sequence were added to the training set? While influence functions have produced insights for small models, they are difficult to scale to large language models (LLMs) due to the difficulty of computing an inverse-Hessian-vector product (IHVP). We use the Eigenvalue-corrected Kronecker-Factored Approximate Curvature (EK-FAC) approximation to scale influence functions up to LLMs with up to 52 billion parameters. In our experiments, EK-FAC achieves similar accuracy to traditional influence function estimators despite the IHVP computation being orders of magnitude faster. We investigate two algorithmic techniques to reduce the cost of computing gradients of candidate training sequences: TF-IDF filtering and query batching. We use influence functions to investigate the generalization patterns of LLMs, including the sparsity of the influence patterns, increasing abstraction with scale, math and programming abilities, cross-lingual generalization, and role-playing behavior. Despite many apparently sophisticated forms of generalization, we identify a surprising limitation: influences decay to near-zero when the order of key phrases is flipped. Overall, influence functions give us a powerful new tool for studying the generalization properties of LLMs.
Forward citations
Cited by 44 Pith papers
-
Emergent Misalignment Recruits a Pre-existing Persona Subspace
Fine-tuning on narrow bad data recruits a low-rank persona subspace already present in a frozen instruction-tuned model; holding that subspace out of activations prevents broad misalignment, and injecting it into the ...
-
Domain-Aware Scaling Laws Uncover Data Synergy
Domain-aware scaling laws with fitted γ and σ synergy terms recover stable code-math interactions from observational LLM mixtures and correctly predict mixture rankings in controlled small-scale trainings.
-
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
Permutation-equivariant weight-space networks are all equally expressive, and universality holds when hidden-layer biases are pairwise distinct.
-
Influence Dynamics and Stagewise Data Attribution
Using Bayesian influence functions and singular learning theory, the authors show that a sample's influence on a model varies non-monotonically over training, peaking and flipping sign at phase transitions.
-
Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions
A trained transformer's next-token distribution often matches the empirical next-token distribution of its pretraining corpus, with agreement improving as models grow, while a persistent tail of mismatches remains.
-
Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation
RISE applies CountSketch to dual lexical and semantic channels derived from output-layer gradient outer products, cutting data attribution storage by up to 112x and enabling retrospective and prospective influence ana...
-
What Exactly do Children Receive in Language Acquisition? A Case Study on CHILDES with Automated Detection of Filler-Gap Dependencies
An automated parser-based detector labels wh-questions and relative clauses with extraction sites in CHILDES, revealing object-biased questions but balanced relative clauses, and shows matrix questions drive LM genera...
-
GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning
GradAlign selects RL training problems whose policy gradients align with validation-set gradients, claiming more stable and better-performing LLM post-training.
-
A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)
Only gradient-based (LESS) representations make subset-to-query distance a reliable predictor of instruction-tuning performance; greedy round-robin helps most at small budgets, and random selection is surprisingly com...
-
Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment
Rank-Surprisal Ratio (RSR) correlates strongly (average Spearman 0.86) with post-distillation reasoning gains across five student models and trajectories from eleven teachers, outperforming existing selection metrics.
-
AfroScope: A Framework for Studying the Linguistic Landscape of Africa
A new framework combines a 713-language African LID dataset, strong baselines, and a contrastive-embedding hierarchical step that improves macro-F1 by 4.55 on a 29-language confusable subset.
-
LLM generation novelty through the lens of semantic similarity
An output is called novel when no semantically similar pretraining context can be retrieved; using this embedding-based measure, SmolLM models show longer-range reuse and instruction tuning increases novelty.
-
Understanding Data Influence with Differential Approximation
This paper introduces Diff-In, an influence estimator that accumulates second-order approximations of influence differences across training steps and shows strong accuracy in data cleaning, deletion, and coreset selec...
-
Better Training Data Attribution via Better Inverse Hessian-Vector Products
ASTRA, an EKFAC-preconditioned Neumann series iteration, computes more accurate inverse Hessian-vector products and improves training data attribution scores over EKFAC baselines.
-
Newfluence: Boosting Model interpretability and Understanding in High Dimensions
In high-dimensional regression, classical influence functions underestimate true leave-one-out influence by a per-point factor, and the proposed Newfluence estimator corrects this bias.
-
DICE: Data Influence Cascade in Decentralized Learning
DICE defines and approximates multi-hop data influence in decentralized learning, showing that influence is shaped by data, topology, and loss curvature.
-
TuCo: Measuring the Contribution of Fine-Tuning to Individual Responses of LLMs
TuCo quantifies the contribution of fine-tuning to individual LLM responses as the ratio of fine-tuning to pre-training component magnitudes, and shows jailbreak attacks lower this ratio.
-
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
An adaptive, per-language data filtering and deduplication pipeline produces multilingual LLM pre-training corpora that beat prior public datasets on 11 of 14 evaluated languages, and a 20TB, 1,868 language-script dat...
-
Because we have LLMs, we Can and Should Pursue Agentic Interpretability
Agentic interpretability, using LLMs as proactive conversational teachers that model the user, is offered as a needed complement to black-box interpretability.
-
On Support Samples of Next Word Prediction
Training examples that most shape a GPT-2 model's next-word predictions can be identified with about 80% accuracy from the randomly initialized model, before any training.
-
Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems
A Taylor-based estimator traces a model's final behavior to individual training stages, quantifying what would change if a stage had been skipped.
-
The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It
LLM safety research at ACL venues from 2020 to 2024 is predominantly English-only, and the language gap is growing over time.
-
Daunce: Data Attribution through Uncertainty Estimation
DAUNCE computes training-data attribution as the covariance of per-example losses across an ensemble of perturbed fine-tuned models, reporting state-of-the-art LDS scores and the first attribution runs on proprietary LLMs.
-
Which Data Attributes Stimulate Math and Code Reasoning? An Investigation via Influence Functions
Influence-function attribution shows high-difficulty math and low-difficulty code training data best improve math and code reasoning, and difficulty-based reweighting improves benchmark performance.
-
Laplace Sample Information: Data Informativeness Through a Bayesian Lens
LSI ranks training samples by informativeness using the KL divergence between Laplace-approximated posteriors with and without each sample, and the ordering transfers from a small probe to larger models.
-
Diagnosing our datasets: How does my language model learn clinical information?
The frequency of clinical jargon in pretraining corpora predicts how well open-source LLMs interpret that jargon, but hospital notes use abbreviations that appear only rarely online.
-
Dataset Distillation by Influence Matching
Inf-Match distills datasets by matching estimated parameter influence of real and synthetic data, reporting SOTA classification and retrieval, but with an unsupported theoretical core.
-
When unlearning is free: leveraging low influence points to reduce computational costs
Low-influence training points can be dropped from forget/retain sets before unlearning, cutting runtime up to ~50% with little measured loss in accuracy or MIA-based privacy.
-
Memorization $\neq$ Understanding: Do Large Language Models Have the Ability of Scenario Cognition?
LLMs fine-tuned on paraphrases of fictional facts can recall the paraphrases but cannot answer questions about who did what in those facts, suggesting memorization without robust scenario-level understanding.
-
KAIROS: Scalable Model-Agnostic Data Valuation
KAIROS derives a closed-form Maximum Mean Discrepancy influence score that approximates leave-one-out data rankings and detects noise, mislabels, and backdoors without retraining.
-
Counterfactual Influence as a Distributional Quantity
Near-duplicate training samples lower a model's self-influence on a record while raising its extractability, so self-influence alone underestimates memorization risk.
-
TokenShapley: Token Level Context Attribution with Shapley Value
TokenShapley computes token-level Shapley attributions from context to response by treating context tokens as (prefix, token) data points in a KNN datastore.
-
Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills
Mirage-1 combines a hierarchical multimodal skill memory with a skill-augmented Monte Carlo tree search to outperform prior GUI agents on Android and web online benchmarks.
-
Rescaled Influence Functions: Accurate Data Attribution in High Dimension
Rescaled influence functions, which account for the change in the Hessian when a sample is removed, dramatically improve leave-T-out effect estimates compared to standard influence functions in high-dimensional logist...
-
Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety
A new survey organizes LLM interpretation methods by workflow stage and connects them to safety enhancement strategies and tools, covering around 70 works.
-
Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution
A multitask influence function that estimates per-sample cross-task influence is derived and shown to approximate leave-one-out retraining, enabling data pruning that slightly improves multitask accuracy.
-
Origin Tracer: A Method for Detecting LoRA Fine-Tuning Origins in LLMs
A weight-obfuscation-robust LoRA origin detector that recovers the attention V/O product difference by inverting the base MLP and reads the fine-tuning rank from a singular value gap.
-
PlanE: Meta Planning of Data, Tuning, and Inference for Extractive-based LLMs
A quadratic meta-planner trained on a few model-dataset runs selects the optimal data-tuning-inference configuration for extractive LLMs, matching grid search on three IE tasks.
-
In-Run Data Shapley for Adam Optimizer
An 'Adam-aware' approximation for In-Run Data Shapley is proposed, but its central theorem is not proven and the fidelity test uses a proxy that is not the true Shapley value.
-
What Is The Performance Ceiling of My Classifier? Utilizing Category-Wise Influence Functions for Pareto Frontier Analysis
Category-wise influence vectors plus linear programming and a genetic algorithm reweight training data to improve all classes at once, with an unproven criterion for when a classifier has reached its Pareto ceiling.
-
Uncovering Scaling Laws for Large Language Models via Inverse Problems
Proposes using inverse problems to discover LLM scaling laws, but provides no empirical evidence or new results.
-
Influence Functions for Preference Dataset Pruning
Conjugate-gradient influence functions can mildly improve reward-model accuracy after pruning 10% of a preference dataset, but the gain is not statistically significant and gradient similarity better identifies helpfu...
-
Can Hessian-Based Insights Support Fault Diagnosis in Attention-based Models?
Hessian curvature and interaction metrics reveal unstable attention layers and parameter couplings that gradient norms miss, but the supporting evidence is preliminary.
-
Data Heterogeneity Modeling for Trustworthy Machine Learning
A survey that frames heterogeneity-aware machine learning as a paradigm spanning data collection, training, evaluation, and deployment, drawing mostly on the authors' prior results.
Discussion (0). Continue with ORCID to comment.