Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Understanding Data Influence with Differential Approximation

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces Diff-In, an estimator that approximates a sample's leave-one-out influence by accumulating second-order approximations of per-step influence differences, avoiding the convexity assumption of classical influence function

desk verdict Diff-In is a strong empirical influence estimator that scales well, but the theory as printed does not connect to the algorithm actually evaluated. read the letter →

arxiv 2508.14648 v1 pith:HEYQKWSL submitted 2025-08-20 cs.LG cs.CV

Haoru Tan , Sitong Wu , Xiuzhe Wu , Wang Wang , Bo Zhao , Zeke Xie , Gui-Song Xia , Xiaojuan Qi This is my paper
classification cs.LGcs.CV
keywords influencefunctionsdataattributionleave-one-outestimationsecond-orderapproximationHessian-gradientproductcleaningmachineunlearningcoresetselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diff-In is a proposed way to estimate how much each training sample contributed to a trained model without retraining. Instead of looking only at the final parameters, it writes a sample's influence as the sum of the changes in influence between consecutive training steps. Each of those per-step changes is approximated with a second-order formula that uses only gradient evaluations, so the method does not require the loss to be convex. The paper argues that this makes the estimate closer to true leave-one-out influence than existing methods, and demonstrates use in data cleaning, deleting data without retraining, and choosing small training subsets. If the method works as claimed, expensive retraining-based attribution becomes unnecessary for many large-scale deep-learning workflows.

What carries the argument

The load-bearing object is the influence difference D_t(z) between adjacent training steps, together with the telescoping identity that cumulates these differences into the total leave-one-out influence. Lemma 4.1 approximates each D_t(z) as a weighted sum of Hessian-gradient products of the form H_{B_k} G_z^k + H_z^k G_{B_k}, with coefficients -(η_t η_k)^2/N; Eq. (8) simplifies the history sum to the current-step term multiplied by t, and Eq. (9) turns the result into a checkpoint-based estimator. The mechanism that keeps it cheap is Eq. (6), the finite-difference identity for multiplying a Hessian by a gradient, which makes each step cost O(p), comparable to a first-order gradient computat

What would settle it

On a fixed dataset and architecture (for example, ResNet-18 on CIFAR-10), compute the full Lemma 4.1 sum over k≤t and the single-term Eq. (8) approximation at many steps, and compare both to the ground-truth D_t(z) obtained by leave-one-out retraining. If the cosine similarity between Eq. (8) and the full sum falls well below the value reported in the paper's Figure 2, or if the final Pearson correlation with brute-force leave-one-out values drops below that of TracIn when using Adam, the central shortcut no longer carries the method.

Watch

Extended reading notes

Core claim

On the paper's terms, the central discovery is that leave-one-out influence can be approximated as a telescoping sum over training iterations: Iθ(z) = Σ_{t<T} D_t(z), where D_t(z) is the change in the influence of sample z between steps t and t+1. Lemma 4.1 expresses D_t(z) through a second-order Taylor expansion of the gradient difference under the SGD update, yielding a sum over past steps of products of the Hessian and gradient of the sample and batch. The practical estimator then keeps only the current-step term scaled by t (Eq. 8), accumulates over a few checkpoints (Eq. 9), and computes Hessian-gradient products by finite differences (Eq. 6). The paper reports that this estimator reach

Load-bearing premise

Diff-In's accuracy depends on a shortcut formula (Eq. 8) that replaces the full sum over all previous training steps with one term from the current step multiplied by the step count; the paper validates this shortcut with a single cosine-similarity experiment, so if that shortcut fails on other models, optimizers, or datasets, the claimed accuracy advantage collapses.

Editorial extensions

If this is right

  • Data cleaning can flag mislabeled samples by self-influence or validation influence with higher precision at low selection budgets; the paper reports up to 88.2% precision at 20% selection on Tiny-ImageNet, versus 76.3% for TracIn.
  • Machine unlearning without retraining becomes practical: subtracting the accumulated parameter influence of a noisy group recovers most of the accuracy of retraining on the cleaned set, within 0.5 to 2.4 points in reported cases.
  • Coreset selection can use influence-on-training-loss scores to pick small subsets that transfer to new architectures, with reported gains over CLIP-score selection in vision-language pre-training at 12 million samples.
  • Because the estimator does not need convexity or stationarity, it can be applied to transformer and LoRA-tuned models and to Adam-trained networks, not just SGD-trained classifiers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's strongest practical simplification, Eq. (8), is validated mainly by a single cosine-similarity experiment; an obvious extension is to test whether the same shortcut holds for Adam and for transformer architectures, where a single 'current step' may be less representative of the whole training history.
  • Since Diff-In produces an estimate in parameter space, it could be plugged into other data-valuation frameworks, such as group influence, additive Shapley-style aggregation, or more complex unlearning pipelines; the paper demonstrates additivity for deletion but does not develop these connections.
  • The theoretical bound depends on Lipschitz smoothness and bounded gradient norms; in very long training runs the polynomial T^3 factor suggests possible error accumulation, and the paper's robustness experiments cover only a few training durations, leaving a stress test at extreme scale open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. Diff-In proposes a new estimator for leave-one-out influence in deep networks. The key idea is to write the final parameter influence as a sum of per-iteration influence differences (Eq. 4), approximate each difference by a second-order Taylor expansion of gradient differences (Lemma 4.1, Eq. 5), and then replace the historical sum with a single current-step term scaled by t (Eq. 8). The method computes Hessian-gradient products by finite differences, giving O(p) per-step cost. The paper claims a polynomial approximation-error bound (Prop. 5.1) that is better than existing estimators, and supports the claim with extensive experiments on data cleaning, data deletion, and coreset selection, including Llama-3.1-8B on GSM8K and CLIP pretraining on CC12M. The empirical evaluation is broad and consistently favorable to Diff-In, but the practical estimator in Eqs. (8)-(9) is not the estimator analyzed by Lemma 4.1 and Prop. 5.1 as written.

Significance. If the derivation were correct, Diff-In would be a useful contribution: it avoids convexity and stationarity assumptions, has computational cost comparable to first-order trajectory methods, and the experiments show consistent gains across three data-centric tasks, on models up to 8B parameters and datasets with millions of samples. The paper also includes a useful parameter-influence formulation that enables data deletion, which many trajectory methods do not provide. However, the theoretical backing is currently disconnected from the evaluated algorithm. Lemma 4.1 gives coefficients a_{t,k}=-(η_t η_k)^2/N, while Eq. (8) uses a factor -(η_t)^2, so the practical estimator is not a sampled version of the derived estimator. The error bound in Prop. 5.1 is stated for the idealized Proposition 4.2 estimator, not for the checkpoint-based Eq. (9) used in the experiments. The empirical results are extensive and internally consistent, but the advertised theoretical guarantee requires correction and a proof that covers the actual algorithm.

major comments (4)
  1. [Sec. 4.2, Eq. (8) vs. Lemma 4.1, Eq. (5)] Lemma 4.1 states a_{t,k}=-(η_t η_k)^2/N. Setting k=t gives a single term -(η_t^4/N)(H_B^t G_z^t + H_z^t G_B^t). Eq. (8) instead has -t(η_t)^2/N times the same bracket. The sampling argument in Sec. 4.2 cannot remove the missing η^2 factor: uniformly sampling one k from t terms would convert the sum into t times a_k, not t/η^2 times a_k. Thus, as printed, Eq. (8) is not a Monte-Carlo estimate of Eq. (5), and the algorithm evaluated in Sec. 6 is not the estimator analyzed in Lemma 4.1.
  2. [Sec. 5, Prop. 5.1 vs. Eq. (9)] Prop. 5.1 bounds the error between the exact LOO influence and I(z) computed by Prop. 4.2, which uses the full sum over historical steps in Eq. (5). The experiments use Eq. (9), which sums only over m sampled checkpoints and uses Eq. (8) to replace the inner sum. No error bound is proved for Eq. (9). Consequently, the paper's theoretical claim of 'significantly lower approximation error' is not actually connected to the method that achieves the reported empirical results; the empirical results stand on their own but do not receive the advertised theoretical support.
  3. [Sec. 4.1, Lemma 4.1 proof] The proof of the central lemma is deferred entirely to a supplementary file that is not included in the arXiv submission. This is especially problematic because the coefficient mismatch noted above means the reader cannot determine whether Lemma 4.1's coefficient (η_t η_k)^2 or Eq. (8)'s coefficient (η_t)^2 is the typographical error. A proof sketch or at least the key expansion should appear in the main text, or the supplement should be provided with the revision.
  4. [Sec. 4.2, Figure 2] The only empirical justification for the crucial replacement of the sum over k≤t by a single term at k=t is one cosine-similarity experiment on CIFAR-10 with ResNet-18 at t=37. This is not sufficient to establish that the heuristic holds across different architectures, optimizers, batch sizes, or training regimes, particularly since the t-multiplier and the coefficient in Eq. (8) still do not match the derived expression. At minimum, the revision should include a sensitivity analysis of the t-multiplier and of Eq. (8) on a second dataset/model, and should show final influence accuracy, not just cosine similarity of the intermediate difference term.
minor comments (5)
  1. [Sec. 6.1, Datasets] CIFAR-100 is described as containing '10 distinct classes'; CIFAR-100 contains 100 classes. Please correct this factual error.
  2. [Algorithm 1] Line 4 says 'Compute the influence difference D_t(z) by Eq.(5)', but the practical implementation described in Sec. 4.2 uses Eq. (8). The algorithm should reference the checkpoint-based formula actually used, or clarify that Eq. (5) is the idealized version.
  3. [Tables 11 and 12] The abbreviation 'DVE-IF' is used inconsistently; elsewhere the method is called 'DVE-INF'. Please unify the notation.
  4. [Sec. 4.1.1, paragraph after Eq. (6)] There is a duplicated phrase: 'this approach enables us to approximate the difference terms without this approach enables us to approximate the difference terms without relying on convexity assumptions.' Please edit for clarity.
  5. [Sec. 5.1, Discussion] The statement that SGD-INF and DVE-INF exhibit 'exponential growth' of error is made without a derivation or a citation to the specific bound. Either provide the comparison or soften the claim to what is actually proved in the cited works.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Diff-In's estimator is derived from model gradients/Hessians, validated against external LOO retraining, and self-citations are background only.

full rationale

The central derivation is self-contained rather than circular. Diff-In starts from the exact telescoping identity Iθ(z)=Σ(I^{t+1}_θ(z)-I^t_θ(z)) with I^0_θ(z)=0 (Eq. 4), so the 'accumulate differences' structure is not defined in terms of the target influence. Each difference term is approximated from the model's own gradients and Hessians in Lemma 4.1 (Eq. 5), with coefficient at,k=-(η_t η_k)^2/N; no leave-one-out retraining values are used to fit these coefficients. The practical checkpoint estimator in Eq. (8) is a heuristic truncation/sampling rule validated by the cosine-similarity experiment in Figure 2, but it is not fitted to ground-truth influence scores, so it is not a fitted input renamed as a prediction. The skeptical concern that Eq. (8)'s η^2 coefficient does not match Lemma 4.1's η^4 coefficient is a proof-consistency gap between the theory and the implemented estimator, not a circular reduction: even if the advertised bound does not cover the evaluated algorithm, the algorithm's outputs are not equivalent to its inputs by construction. Empirical accuracy is checked against externally computed brute-force LOO influence (Figure 1, Table 11), an external benchmark. Self-citations, chiefly to the authors' prior MoSo work [17] and to [4], appear for standard additivity and checkpoint-sampling practices and are accompanied by other references; they are not load-bearing for the derivation of Diff-In's estimator. The omitted proofs in the supplementary material are an incomplete-support concern rather than evidence of circularity. No step in the paper's chain reduces to its own inputs, and the method is not a renaming of a known estimator.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical or model entities; its novel content is an estimator. The estimator depends on several ad hoc choices: the t-multiplier heuristic, the unspecified finite-difference epsilon, the number of checkpoints m, and the random-batch proxy. The theoretical guarantees rely on smoothness and boundedness assumptions whose constants are not measured.

free parameters (4)
  • m (number of sampled checkpoints) = m=5 recommended
    Chosen empirically; Table 9 shows performance saturates around m=5-10 for CC12M and GSM8K.
  • Gradient-proxy batch size = 2048 on ImageNet-1K, 512 on other datasets
    Selected by the authors as a random-batch proxy for the full-dataset gradient; no sensitivity analysis is provided.
  • Finite-difference perturbation epsilon = Not reported
    Required in Eq. (6) to compute Hessian-gradient products by finite differences; the paper never states the value or heuristic used, so any reproduction must guess it.
  • t-multiplier in Eq. (8) = t (the current step index)
    Introduced ad hoc when collapsing the sum over k to the single term k=t; the paper reports it works empirically but does not derive it.
assumptions (6)
  • ad hoc to paper The per-step parameter difference between full-data and leave-one-out trajectories is governed by a second-order Taylor expansion of the gradient difference (Lemma 4.1)
    Central to the derivation; proof is deferred to the supplementary and the practical form uses an additional heuristic truncation.
  • domain assumption The loss has an ell-Lipschitz continuous gradient and bounded gradient norm g
    Used to prove the error bound in Prop 5.1; the paper argues normalization layers make this plausible for deep networks, but it is an assumption.
  • domain assumption Parameters stay within a distance C of initialization during training on any subset
    The constant C in Prop 5.1 is defined but not measured or bounded; the error bound is therefore not actionable.
  • ad hoc to paper The sum over historical steps can be replaced by the last term times t
    Eq. (8) is the practical estimator; the replacement is justified only by the empirical comparison in Figure 2.
  • domain assumption Influence functions are additive for a set of removed samples
    Data deletion (Sec. 6.3) sums per-sample parameter influences to represent removing the whole noise set; this ignores interactions between samples.
  • domain assumption A random batch can stand in for the full training set gradient
    Used throughout to make the computation feasible; a known approximation also employed by prior work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Understanding Data Influence with Differential Approximation." pith.science (2026). https://pith.science/paper/HEYQKWSL

@misc{pith2026250814648,
  author       = {Pith},
  title        = {Pith review of: Understanding Data Influence with Differential Approximation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HEYQKWSL}},
  note         = {Machine review of arXiv:2508.14648}
}
read the original abstract

Data plays a pivotal role in the groundbreaking advancements in artificial intelligence. The quantitative analysis of data significantly contributes to model training, enhancing both the efficiency and quality of data utilization. However, existing data analysis tools often lag in accuracy. For instance, many of these tools even assume that the loss function of neural networks is convex. These limitations make it challenging to implement current methods effectively. In this paper, we introduce a new formulation to approximate a sample's influence by accumulating the differences in influence between consecutive learning steps, which we term Diff-In. Specifically, we formulate the sample-wise influence as the cumulative sum of its changes/differences across successive training iterations. By employing second-order approximations, we approximate these difference terms with high accuracy while eliminating the need for model convexity required by existing methods. Despite being a second-order method, Diff-In maintains computational complexity comparable to that of first-order methods and remains scalable. This efficiency is achieved by computing the product of the Hessian and gradient, which can be efficiently approximated using finite differences of first-order gradients. We assess the approximation accuracy of Diff-In both theoretically and empirically. Our theoretical analysis demonstrates that Diff-In achieves significantly lower approximation error compared to existing influence estimators. Extensive experiments further confirm its superior performance across multiple benchmark datasets in three data-centric tasks: data cleaning, data deletion, and coreset selection. Notably, our experiments on data pruning for large-scale vision-language pre-training show that Diff-In can scale to millions of data points and outperforms strong baselines.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning

    cs.LG 2026-02 reject novelty 5.0 of 10

    TRUE checks whether LLM reasoning traces are self-sufficient by executing them blind, maps neighboring reasoning paths into a DAG, and ranks recurring failure modes by Shapley values.

Reference graph

Works this paper leans on

70 extracted references · 56 canonical work pages · cited by 1 Pith paper

  1. [1]

    Language models are few-shot learners,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, and et. al, “Language models are few-shot learners,” inAdvances in Neural Information Processing Systems, 2020

    work page 2020

  2. [2]

    Segment anything,

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexan- der Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick, “Segment anything,”arXiv:2304.02643, 2023

    arXiv 2023

  3. [3]

    Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models,

    Y. Kwon, E. Wu, K. Wu, and J. Zou, “Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models,”arXiv preprint arXiv:2310.00902, 2023

    arXiv 2023

  4. [4]

    Dataset pruning: Reducing training data by examining generalization influence,

    S. Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, and Ping Li, “Dataset pruning: Reducing training data by examining generalization influence,” inInternational Conference on Learning Representations, 2023

    work page 2023

  5. [5]

    Less: Selecting influential data for targeted instruction tuning,

    M. Xia, S. Malladi, S. Gururangan, S. Arora, and D. Chen, “Less: Selecting influential data for targeted instruction tuning,”arXiv preprint arXiv:2402.04333, 2024

    arXiv 2024

  6. [6]

    Studying large language model generalization with influence functions,

    R. Grosse, J. Bae, C. Anil, N. Elhage, A. Tamkin, A. Tajdini, B. Steiner, D. Li, E. Durmus, E. Perezet al., “Studying large language model generalization with influence functions,”arXiv preprint arXiv:2308.03296, 2023

    arXiv 2023

  7. [7]

    Training data attribution for diffusion models,

    Z. Dai and D. K. Gifford, “Training data attribution for diffusion models,”arXiv preprint arXiv:2306.02174, 2023

    arXiv 2023

  8. [8]

    R. D. Cook and S. Weisberg,Residuals and influence in regression / R. Dennis Cook and Sanford Weisberg., ser. Monographs on statistics and applied probability. New York: Chapman and Hall, 1982

    work page 1982

Show all 70 references
  1. [9]

    Assessment of local influence,

    R. D. Cook, “Assessment of local influence,”Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 48, no. 2, pp. 133–155, 1986

    1986

  2. [10]

    Understanding black-box predictions via influence functions,

    P . W. Koh and P . Liang, “Understanding black-box predictions via influence functions,” inInternational Conference on Machine Learning, 2017

    2017

  3. [11]

    On second-order group influence functions for black-box predictions,

    S. Basu, Xuchen You, and Soheil Feizi, “On second-order group influence functions for black-box predictions,” inInternational Conference on Machine Learning, 2020

    2020

  4. [12]

    On the accuracy of influence functions for measuring group effects,

    P . Koh, Kai-Siang Ang, Hubert H. K. Teo, and Percy Liang, “On the accuracy of influence functions for measuring group effects,” inAdvances in neural information processing systems, 2019

    2019

  5. [13]

    Influence functions in deep learning are fragile,

    S. Basu, P . Pope, and S. Feizi, “Influence functions in deep learning are fragile,” inInternational Conference on Learning Representations, 2021

    2021

  6. [14]

    The mirrored influ- ence hypothesis: Efficient data influence estimation by harnessing forward passes,

    M. Ko, F. Kang, W. Shi, M. Jin, Z. Yu, and R. Jia, “The mirrored influ- ence hypothesis: Efficient data influence estimation by harnessing forward passes,” inCVPR, 2024

    2024

  7. [15]

    Estimating train- ing data influence by tracing gradient descent,

    G. Pruthi, F. Liu, S. Mukund, and S. Kale, “Estimating train- ing data influence by tracing gradient descent,”arXiv preprint arXiv:2002.08484, 2020

    2002 arXiv

  8. [16]

    Capturing the temporal dependence of training data influence,

    J. T. Wang, D. Song, J. Zou, P . Mittal, and R. Jia, “Capturing the temporal dependence of training data influence,” inInternational Conference on Learning Representations, 2025

    2025

  9. [17]

    Data pruning via moving-one-sample-out,

    H. Tan, S. Wu, F. Du, Y. Chen, Z. Wang, F. Wang, and X. Qi, “Data pruning via moving-one-sample-out,” inAdvances in neural information processing systems, 2023

    2023

  10. [18]

    Data cleansing for models trained with sgd,

    S. Hara, A. Nitanda, and T. Maehara, “Data cleansing for models trained with sgd,”Advances in Neural Information Processing Systems, vol. 32, 2019

    2019

  11. [19]

    Fast exact multiplication by the hessian,

    B. A. Pearlmutter, “Fast exact multiplication by the hessian,”Neural Computation, vol. 6, no. 1, pp. 147–160, 1994

    1994

  12. [20]

    Gex: A flexible method for approximating influence via geometric ensemble,

    S. Kim, K. Kim, and E. Yang, “Gex: A flexible method for approximating influence via geometric ensemble,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  13. [21]

    Scaling up influence functions,

    A. Schioppa, P . Zablotskaia, D. Vilar, and A. Sokolov, “Scaling up influence functions,” inProceedings of the AAAI Conference on Artificial Intelligence, 2022

    2022

  14. [22]

    Beyond neural scaling laws: beating power law scaling via data pruning,

    B. Sorscher, R. Geirhos, S. Shekhar, S. Ganguli, and A. Morcos, “Beyond neural scaling laws: beating power law scaling via data pruning,” inAdvances in Neural Information Processing Systems, 2022

    2022

  15. [23]

    Knowledge removal in sampling- based bayesian inference,

    S. Fu, F. He, and D. Tao, “Knowledge removal in sampling- based bayesian inference,” inInternational Conference on Learning Representations, 2022

    2022

  16. [24]

    The llama 3 herd of models,

    A. Llama Team, “The llama 3 herd of models,” 2024

    2024

  17. [25]

    Training verifiers to solve math word problems,

    K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman, “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  18. [26]

    Moderate coreset: A universal method of data selection for real- world data-efficient deep learning,

    X. Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu, “Moderate coreset: A universal method of data selection for real- world data-efficient deep learning,” inInternational Conference on Learning Representations, 2023

    2023

  19. [27]

    Training data influence analysis and estimation: A survey,

    Z. Hammoudeh and D. Lowd, “Training data influence analysis and estimation: A survey,”arXiv preprint arXiv:2212.04612, 2022

    2022 arXiv

  20. [28]

    A value for n-person games,

    L. S. Shapleyet al., “A value for n-person games,” 1953

    1953

  21. [29]

    Rkhs-shap: Shapley values for kernel methods,

    S. L. Chau, R. Hu, J. Gonzalez, and D. Sejdinovic, “Rkhs-shap: Shapley values for kernel methods,”Advances in neural information processing systems, vol. 35, pp. 13 050–13 063, 2022

    2022

  22. [30]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, and others, “Imagenet large scale visual recognition challenge,”International journal of computer vision, vol. 115, no. 3, pp. 211–252, 2015

    2015

  23. [31]

    LAION-5b: An open large-scale dataset for training next generation image-text models,

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, and et. al, “LAION-5b: An open large-scale dataset for training next generation image-text models,” inThirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022

    2022

  24. [32]

    Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts,

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut, “Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts,” inIEEE / CVF Computer Vision and Pattern Recognition Conference, 2021

    2021

  25. [33]

    What neural networks memorize and why: Discovering the long tail via influence estimation,

    V . Feldman and Z. Chiyuan, “What neural networks memorize and why: Discovering the long tail via influence estimation,” in Advances in neural information processing systems, 2020

    2020

  26. [34]

    The loss surfaces of multilayer networks,

    A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun, “The loss surfaces of multilayer networks,”Journal of Machine Learning Research, vol. 38, pp. 192–204, 2015

    2015

  27. [35]

    Identifying and attacking the saddle point problem in high-dimensional non-convex optimization,

    Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y. Bengio, “Identifying and attacking the saddle point problem in high-dimensional non-convex optimization,” inAdvances in Neural Information Processing Systems, 2014, pp. 2933–2941

    2014

  28. [36]

    Revisiting inverse hessian vector products for calculating influence functions,

    Y. Klochkov and Y. Liu, “Revisiting inverse hessian vector products for calculating influence functions,” 2024

    2024

  29. [37]

    Revisit, extend, and enhance hessian-free influence functions,

    Z. Yang, H. Yue, J. Chen, and H. Liu, “Revisit, extend, and enhance hessian-free influence functions,” 2024

    2024

  30. [38]

    If influence functions are the answer, then what is the question?

    J. Bae, N. Ng, A. Lo, M. Ghassemi, and R. B. Grosse, “If influence functions are the answer, then what is the question?”Advances in Neural Information Processing Systems, 2022. IEEE TRANSACTIONS ON PATTERN ANAL YSIS AND MACHINE INTELLIGENCE 15

    2022

  31. [39]

    ”what data benefits my classifier?

    A. Chhabra, P . Li, P . Mohapatra, and H. Liu, “”what data benefits my classifier?” enhancing model performance and interpretability through influence-based data selection,” inInternational Conference on Learning Representations, 2024

    2024

  32. [40]

    Adam: A method for stochastic opti- mization,

    D. P . Kingma and J. Ba, “Adam: A method for stochastic opti- mization,” inInternational Conference on Learning Representations, 2015

    2015

  33. [41]

    Data shapley: Equitable valuation of data for machine learning,

    A. Ghorbani and J. Zou, “Data shapley: Equitable valuation of data for machine learning,” inInternational conference on machine learning. PMLR, 2019, pp. 2242–2251

    2019

  34. [42]

    Efficient task-specific data valuation for nearest neighbor algorithms,

    R. Jia, D. Dao, B. Wang, F. A. Hubis, N. M. Gurel, B. Li, C. Zhang, C. J. Spanos, and D. Song, “Efficient task-specific data valuation for nearest neighbor algorithms,”arXiv preprint arXiv:1908.08619, 2019

    1908 arXiv

  35. [43]

    Scalability vs. utility: Do we have to sacrifice one for the other in data importance quan- tification?

    R. Jia, Fan Wu, Xuehui Sun, Jiacen Xu, David Dao, Bhavya Kailkhura, Ce Zhang, Bo Li, and Dawn Song, “Scalability vs. utility: Do we have to sacrifice one for the other in data importance quan- tification?” inIEEE / CVF Computer Vision and Pattern Recognition Conference, 2021

    2021

  36. [44]

    What is your data worth to gpt? llm-scale data valuation with influence functions,

    S. K. Choe, H. Ahn, J. Bae, K. Zhao, M. Kang, Y. Chung, A. Pratapa, W. Neiswanger, E. Strubell, T. Mitamura, J. Schneider, E. Hovy, R. Grosse, and E. Xing, “What is your data worth to gpt? llm-scale data valuation with influence functions,” 2024. [Online]. Available: https://a...

    2024 arXiv

  37. [45]

    Data valuation for medical imaging using shapley value and application to a large-scale chest x-ray dataset,

    S. Tang, A. Ghorbani, R. Yamashita, S. Rehman, J. Dunnmon, J. Zou, and D. L. Rubin, “Data valuation for medical imaging using shapley value and application to a large-scale chest x-ray dataset,”Science Report, vol. 11, p. 8366, 2021

    2021

  38. [46]

    Outlier gra- dient analysis: Efficiently identifying detrimental training samples for deep learning models,

    A. Chhabra, B. Li, J. Chen, P . Mohapatra, and H. Liu, “Outlier gra- dient analysis: Efficiently identifying detrimental training samples for deep learning models,” 2024

    2024

  39. [47]

    Resolving training biases via influence-based data relabeling,

    S. Kong, Y. Shen, and L. Huang, “Resolving training biases via influence-based data relabeling,” inInternational Conference on Learning Representations, 2022

    2022

  40. [48]

    Influence function based data poisoning attacks to top-n recommender systems,

    M. Fang, N. Z. Gong, and J. Liu, “Influence function based data poisoning attacks to top-n recommender systems,” inProceedings of the ACM Web Conference, 2020, pp. 3019–3025

    2020

  41. [49]

    Exploring example influence in continual learning,

    Q. Sun, F. Lyu, F. Shang, W. Feng, and L. Wan, “Exploring example influence in continual learning,”Advances in Neural Information Processing Systems, vol. 35, pp. 27 075–27 086, 2022

    2022

  42. [50]

    Explaining a series of models by propagating shapley values,

    H. Chen, S. M. Lundberg, and S.-I. Lee, “Explaining a series of models by propagating shapley values,”Nature communications, vol. 13, no. 1, p. 4512, 2022

    2022

  43. [51]

    Trak: Attributing model behavior at scale,

    S. M. Park, K. Georgiev, A. Ilyas, G. Leclerc, and A. Madry, “Trak: Attributing model behavior at scale,”arXiv preprint arXiv:2303.14186, 2023

    2023 arXiv

  44. [52]

    Datamodels: Predicting predictions from training data,

    A. Ilyas, S. M. Park, L. Engstrom, G. Leclerc, and A. Madry, “Datamodels: Predicting predictions from training data,”arXiv preprint arXiv:2202.00622, 2022

    2022 arXiv

  45. [53]

    Achieving fairness at no utility cost via data reweighing with influence,

    P . Li and H. Liu, “Achieving fairness at no utility cost via data reweighing with influence,” 2022

    2022

  46. [54]

    Hydra: Hypergradient data relevance analysis for interpreting deep neural networks,

    Y. Chen, B. Li, H. Yu, P . Wu, and C. Miao, “Hydra: Hypergradient data relevance analysis for interpreting deep neural networks,” 2022

    2022

  47. [55]

    Influence selection for active learning,

    Z. Liu, H. Ding, H. Zhong, W. Li, J. Dai, and C. He, “Influence selection for active learning,” inInternational Conference on Computer Vision, 2021

    2021

  48. [56]

    Automatic differentiation in pytorch,

    Adam Paszke, Sam Gross, Soumith Chintala, G Chanan, E Yang, Zachary Devito, Zeming Lin, Alban Desmaison, L Antiga, A Lerer, and et.al., “Automatic differentiation in pytorch,” inAdvances in neural information processing systems Workshop, 2017

    2017

  49. [57]

    Deep residual learning for image recognition,

    K. He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” inIEEE / CVF Computer Vision and Pattern Recognition Conference, 2016

    2016

  50. [58]

    How does batch normalization help optimization?

    S. Santurkar, D. Tsipras, A. Ilyas, and A. Madry, “How does batch normalization help optimization?” inAdvances in Neural Information Processing Systems, 2018

    2018

  51. [59]

    Visualizing the loss landscape of neural nets,

    H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein, “Visualizing the loss landscape of neural nets,” inAdvances in Neural Information Processing Systems, 2018

    2018

  52. [60]

    Learning multiple layers of features from tiny images,

    Alex Krizhevsky, “Learning multiple layers of features from tiny images,”Technical report, 2009

    2009

  53. [61]

    Tiny imagenet visual recognition challenge,

    Ya Le and Xuan S. Yang, “Tiny imagenet visual recognition challenge,”CS 231N, 2015

    2015

  54. [62]

    Learn- ing transferable visual models from natural language supervision,

    Alec Radford, Jong Wook Kim, Chris Hallacy, ditya Ramesh, Gabriel Goh, Sandhini Agarwa, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever, “Learn- ing transferable visual models from natural language supervision,” CoRR, vol. abs/210...

    2021 arXiv

  55. [63]

    Automated cleanup of the imagenet dataset by model consensus, explainability and confident learning,

    Csaba Kertész, “Automated cleanup of the imagenet dataset by model consensus, explainability and confident learning,”arXiv preprint arXiv:2103.16324, 2021

    2021 arXiv

  56. [64]

    Ssse: Efficiently erasing samples from trained machine learning models,

    A. Peste, D. Alistarh, and C. H. Lampert, “Ssse: Efficiently erasing samples from trained machine learning models,” 2021

    2021

  57. [65]

    Active learning for convolutional neural networks: A coreset approach,

    Ozan Sener and Silvio Savarese, “Active learning for convolutional neural networks: A coreset approach,” inInternational Conference on Learning Representations, 2018

    2018

  58. [66]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,” inICML, 2022

    2022

  59. [67]

    BLIP-2: bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “BLIP-2: bootstrapping language- image pre-training with frozen image encoders and large language models,” inICML, 2023

    2023

  60. [68]

    Flickr30k entities: Collecting region- to-phrase correspondences for richer image-to-sentence models,

    B. A. Plummer, L. Wang, C. M. Cervantes, J. C. Caicedo, J. Hock- enmaier, and S. Lazebnik, “Flickr30k entities: Collecting region- to-phrase correspondences for richer image-to-sentence models,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 2641–2649

    2015

  61. [69]

    Squeeze-and-excitation networks,

    Jie Hu, Li Shen, and Gang Sun, “Squeeze-and-excitation networks,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

    2018

  62. [70]

    Efficientnet: Rethinking model scaling for convolutional neural networks,

    Mingxing Tan and Quoc Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” inInternational Conference on Machine Learning, 2019. Haoru Tanis currently a second-year Ph.D stu- dent at the University of Hong Kong. He received his master’s degree from t...

    2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.