Pith. sign in

REVIEW 4 major objections 5 minor 64 references

RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One index predicts what a model retraining will cost

desk verdict A plausible but overclaimed retraining-cost estimator: RESQUEtask is the only genuinely new piece, the zero-cost framing is wrong, and the validation needs baselines and confidence intervals. read the letter →

arxiv 2412.15511 v1 pith:5OFVQVAA submitted 2024-12-20 cs.LG cs.AIcs.CV

Vishwesh Sangarya , Jung-Eun Kim This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords modelreusabilityretrainingcostestimationdistributionshifttaskchangeAdjustedRandIndexsustainableAIrepresentationanalysisGreen
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RESQUE is a proposed single-number index for estimating how expensive it will be to retrain an already trained model, either because the input data has drifted or because the model is being pointed at a new task. For distribution shift, it compares class-level averaged representations of the original and shifted data through one forward pass. For a task change, it fine-tunes the model for a single epoch, clusters the new data's representations, and measures how little those clusters agree with true class labels. The paper argues that a lower RESQUE value tracks lower total retraining cost, and reports strong correlations with epochs, gradient norm, parameter change, energy consumption, and carbon emissions across several architectures and datasets. If the claim holds, practitioners could estimate sustainability costs before spending compute on fine-tuning.

What carries the argument

The load-bearing object is the RESQUE index itself, computed from the model's internal representations rather than from gradients or backpropagation. For distribution shift, the machinery is the average inverse cosine angle between class-wise normalized embedding sums of the original and shifted datasets (Eq. (3)). For task change, the machinery is the complement of the Adjusted Rand Index (Eq. (4)), a standardized score comparing two clusterings where 0 means chance agreement and 1 means perfect agreement; it compares the true labels of the new task with KMeans cluster labels obtained from representation vectors after a single epoch of fine-tuning. The index is what connects a cheap forward pass to estimates of retraining epochs, gradient norms, parameter change, energy, and carbon emissions, and it is the thing whose correlations with those measures the experiments test.

What would settle it

Find two task changes or two shifted distributions with nearly identical RESQUE values but substantially different full-retraining epochs and energy use; if such a pair exists, the index does not carry the monotone cost information the paper claims.

Watch

Extended reading notes

Core claim

The central claim is that a model's representation space, inspected at almost no training cost, contains enough information to forecast the full cost of adapting that model, and that this forecast can be compressed into one index. RESQUE (REpresentation Shift QUantifying Estimator) has two forms: RESQUE_dist is the mean inverse cosine angle between per-class normalized summed embeddings of the original and shifted distributions, computed with no backward pass; RESQUE_task is the complement of the Adjusted Rand Index between true labels and KMeans cluster labels assigned to representations after a single epoch of fine-tuning on the new task. On CIFAR10, CIFAR100, SVHN, and other vision benchmarks, the paper reports that both variants correlate positively with retraining epochs, total gradient norm, normalized parameter change, and the energy and carbon emissions measured during retraining, with Pearson and Spearman coefficients mostly above 0.7 and low p-values. The conclusion drawn is that a low RESQUE indicates a cheap reuse and a high RESQUE a costly one, making the index a decision aid for sustainable model reuse.

Load-bearing premise

The method assumes that what is visible after one epoch of fine-tuning and a cluster analysis of new representations, or, for distribution shift, in the angles between class-average embeddings, reliably previews the total cost of full retraining; if that early signal misorders tasks or shifts, RESQUE's ranking fails.

Editorial extensions

If this is right

  • If a model has a low RESQUE for a target task, a user can expect fewer retraining epochs and lower energy use than with high-RESQUE alternatives, and can rank candidate models accordingly.
  • Users can decide between retraining an existing model and training from scratch by comparing RESQUE estimates with known full-retraining costs, as the paper shows retraining generally uses less than from-scratch training.
  • RESQUE could be used before deployment to flag distribution shifts that will be expensive to adapt to, so that monitoring systems can alert operators early.
  • Because RESQUE is computed from a single forward pass, plus one epoch for task changes, it can be applied repeatedly over a model's lifetime to track how adaptation costs evolve.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same representation-based logic should carry over to language and speech models when a fixed representation layer is available, making RESQUE a general reuse-cost index rather than a vision-specific one.
  • Beyond the paper: semantic or adversarial shifts may be a harder test than the noise corruptions used here, since representation angles can stay small while training difficulty grows; the paper does not establish monotonicity in that regime.
  • Beyond the paper: because RESQUE_task requires only one fine-tuning epoch, it could serve as a cheap tie-breaker when choosing among pretrained backbones before expensive fine-tuning is run.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces RESQUE, a pair of indices intended to estimate the cost of retraining a pretrained model before the retraining is carried out. For distribution shifts, RESQUEdist is the average angular distance between class-wise L2-normalized sums of representation vectors from a single forward pass on the original and shifted data (Eqs. (1)-(3)). For task changes, RESQUEtask is the complement of the Adjusted Rand Index between KMeans cluster labels (obtained after one epoch of fine-tuning on the new task) and the true target labels (Eq. (4)). The authors report Pearson and Spearman correlations between these indices and retraining measures (epochs, total gradient norm, normalized parameter change, energy, and carbon emissions) on CIFAR10/CIFAR100/SVHN with three synthetic noise types, and across eight source datasets and several target tasks with ResNet18, VGG16, and a ViT. They conclude that lower RESQUE predicts lower retraining cost and enables sustainable model reuse decisions.

Significance. If the claimed correlations are robust, RESQUE would be a practically useful and inexpensive tool for model-reuse decisions, complementing the sustainability literature by linking representation geometry to resource expenditures. The paper releases code, uses multiple architectures and datasets, and reports both Pearson and Spearman coefficients with p-values; it also measures energy and carbon directly, which are assets. There is no circularity in the definition of RESQUE, since no parameter is fitted to the retraining outcomes. However, the evidence is mainly correlational over small sample sizes, the task estimator depends on a one-epoch fine-tuning whose monotonicity is not established, and no baseline comparison is provided. With additional validation and more careful claims, the result could be a useful practical tool.

major comments (4)
  1. [Abstract / Introduction; Section 'Change of Task'] The abstract and introduction state that RESQUE predicts retraining cost 'before any computation is performed' and that RESQUEdist requires 'no backward propagation or computation.' This is contradicted by the 'Change of Task' section, where RESQUEtask is computed only after one full epoch of fine-tuning on the new task, including backward passes and parameter updates, followed by KMeans clustering (Eq. (4), Algorithm 1). Because the 'before computation' claim is a central selling point for practical use, it must be either implemented (e.g., by an estimator that does not retrain) or carefully scoped to RESQUEdist only.
  2. [Eq. (4); 'Adjusted Rand Index to Quantify Class Separation'] The central assumption for RESQUEtask is that the agreement between KMeans cluster labels on representations after a single epoch of fine-tuning and the true class labels, measured by 1 - ARI, is monotonically related to the total cost of full retraining. This assumption is not justified or tested. The one-epoch representation is sensitive to optimizer, learning rate, head initialization, data order, and random seed, and KMeans assumes spherical clusters that may not match learned representation geometry. The paper reports no variance across seeds or hyperparameters, and it does not compare with a simpler and more direct class-separability measure such as a linear probe. I request a sensitivity analysis of the one-epoch budget and a demonstration that the relation is not an artifact of the chosen optimizer or seed; otherwise RESQUEtask may be measuring optimization dynamics rather than intrinsic task difficulty.
  3. [Tables 1-2; 'Experiments Across Different Original Tasks'] The validation of both estimators relies on correlations over a small number of points: for RESQUEdist, 10 noise-intensity levels per noise type, and for RESQUEtask, 8 original tasks per target task in Table 2. No confidence intervals, baselines, or permutation tests are provided, and several reported p-values are not significant at the 0.05 level (e.g., Table 2, ResNet18 GTSRB: Spearman epochs p=0.052, parameter change p=0.119), yet the text in 'Experiments Across Different Original Tasks' calls the relationship 'strong and statistically significant' without qualification. I ask for confidence intervals, per-seed error bars, a null/baseline comparison (e.g., random representations or a trivial monotone-in-noise-level predictor), and a more careful wording of statistical significance. Without these, the central claim that RESQUE has strong correlation with retraining measures is not yet supported.
  4. [Introduction] The introduction promises that 'through extensive experiments and by theoretical reasoning' the paper shows lower RESQUE correlates with lower retraining cost, but no theoretical argument, proposition, or proof appears anywhere in the manuscript. The support is entirely empirical correlation, and the experiments are conducted on the same datasets used to motivate the design. Either provide a formal statement of the conditions under which the correlation should hold, or remove the phrase and explicitly describe the result as an empirical finding.
minor comments (5)
  1. [Eq. (3)] The notation 'arccos(V^O_i,norm, V^S_i,norm)' is ambiguous; define it as the arccosine of the inner product of the two normalized vectors.
  2. [Figures 1-5] In the provided manuscript, several axis labels and legends are rendered as unicode placeholders (e.g., '/uni00000031...'), making some plots unreadable; please replace them with proper typeset labels.
  3. [Table 2] The rows of Table 2 are labeled only by the target task; please clarify in the caption that the correlations are computed across the eight original tasks for each target task, since the surrounding text sometimes refers to both original and target tasks.
  4. [Appendix; Hyperparameters] The early-stopping thresholds of 25 or 50 epochs with 0.5% or 1% accuracy margins are not described as a design choice; please state how these thresholds were selected and whether the reported epoch counts are sensitive to them.
  5. [Conclusion] The claim that RESQUE is 'model/architecture-agnostic' is based on one ViT configuration and two CNN backbone families; please temper this claim or add evidence from additional architectures.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RESQUE is defined independently of the retraining measures and validated empirically.

full rationale

RESQUEdist (Eq. 3) is computed from class-wise normalized embedding angles between original and shifted distributions, and RESQUEtask (Eq. 4) is one minus the Adjusted Rand Index between KMeans cluster labels on one-epoch representations and true labels. Neither formula contains a fitted parameter or a retraining-cost term; the retraining measures (epochs, gradient norm, parameter change, energy, carbon) are measured independently and then correlated with the index. The self-citations (Sangarya, Bradford, and Kim 2023, 2024) appear only in Related Work to contrast prior limitations and are not load-bearing for the RESQUE construction or validation. The claim that RESQUE is obtained 'before any computation' is inconsistent with the one-epoch retraining required by RESQUEtask, and the validation is correlational on the same datasets used to motivate the metric; these are correctness and robustness concerns, not circular reductions. No equation reduces to its own inputs, so no circularity is exhibited.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a small number of domain assumptions and two hand-chosen methodological constants. No parameter is fitted to the retraining cost data, which keeps the circularity burden low. The most fragile elements are the one-epoch fine-tuning budget and the assumption that class separability (or mean-embedding angles) predicts cost.

free parameters (2)
  • One-epoch retraining budget = 1 epoch
    RESQUEtask is computed from a model fine-tuned on the new task for exactly one epoch; this budget is selected by hand and is not derived from any principle or fitted to the outcome.
  • Clustering initialization scheme = Algorithm 1 (original-data centroids)
    The paper tries three initialization schemes and reports they give similar labels, then selects Algorithm 1 for efficiency; the selection is discretionary and not justified by a quantitative comparison.
assumptions (5)
  • domain assumption The final-layer representation space of a trained network is informative enough for cluster separability to predict retraining cost.
    Invoked in 'Quantifying Class Separation from Known Task Data to New Task Data' where RESQUEtask is computed from clustering on final convolutional/dense layer representations.
  • domain assumption One epoch of fine-tuning on the new task is sufficient to expose the structure that predicts full retraining cost.
    RESQUEtask is defined after a single retraining epoch; the choice of one epoch is arbitrary and is a load-bearing part of the estimator.
  • domain assumption Lower cluster-label agreement (higher 1-ARI) monotonically corresponds to higher retraining cost.
    This is the central heuristic behind Eq. (4), stated in 'Adjusted Rand Index to Quantify Class Separation'.
  • domain assumption Class-wise summed and normalized embedding vectors capture distribution shift for RESQUEdist.
    Eqs. (1)-(3) define the shift as the average inverse cosine angle between class-mean directions; this presupposes that class-mean directions are a sufficient statistic for shift difficulty.
  • domain assumption Labels of the shifted or target dataset are available when computing the estimator.
    RESQUEdist requires class labels to form class-wise sums, and RESQUEtask requires true labels to compute the Adjusted Rand Index; the paper does not discuss the unlabeled case.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability." pith.science (2026). https://pith.science/paper/5OFVQVAA

@misc{pith2026241215511,
  author       = {Pith},
  title        = {Pith review of: RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5OFVQVAA}},
  note         = {Machine review of arXiv:2412.15511}
}
read the original abstract

As a strategy for sustainability of deep learning, reusing an existing model by retraining it rather than training a new model from scratch is critical. In this paper, we propose REpresentation Shift QUantifying Estimator (RESQUE), a predictive quantifier to estimate the retraining cost of a model to distributional shifts or change of tasks. It provides a single concise index for an estimate of resources required for retraining the model. Through extensive experiments, we show that RESQUE has a strong correlation with various retraining measures. Our results validate that RESQUE is an effective indicator in terms of epochs, gradient norms, changes of parameter magnitude, energy, and carbon emissions. These measures align well with RESQUE for new tasks, multiple noise types, and varying noise intensities. As a result, RESQUE enables users to make informed decisions for retraining to different tasks/distribution shifts and determine the most cost-effective and sustainable option, allowing for the reuse of a model with a much smaller footprint in the environment. The code for this work is available here: https://github.com/JEKimLab/AAAI2025RESQUE

Figures

Figures reproduced from arXiv: 2412.15511 by the authors.

Figure 1
Figure 1. Retraining vs. Training from scratch on the SVHN dataset with Gaussian noise using VGG16 [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. RESQUE and retraining measures for different models and datasets 0 50 100 150 Epochs Retrain Scratch 0 50 100 150 Gradient Norm 0.000 0.005 0.010 0.015 0.020 Param. change 0.0 0.1 0.2 Energy (kWh) 0.00 0.02 0.04 0.06 0.08 Carbon (kg) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. As for the new target task, CIFAR10, comparisons [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: RESQUE vs. resource measures for ResNet and ViT retrained to different new target tasks. A positive relation between [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: CIFAR10 model trained to different target tasks. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 29 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    C.; Soatto, S.; and Perona, P

    Achille, A.; Lam, M.; Tewari, R.; Ravichandran, A.; Maji, S.; Fowlkes, C. C.; Soatto, S.; and Perona, P. 2019. Task2vec: Task embedding for meta-learning. In Proceedings of the IEEE/CVF international conference on computer vision, 6430--6439

    work page 2019

  4. [4]

    Agarwal, C.; D'souza, D.; and Hooker, S. 2022. Estimating Example Difficulty Using Variance of Gradients. arXiv:2008.11600

    arXiv 2022

  5. [5]

    Ali, A. N. F.; Sulaima, M. F.; Razak, I. A. W. A.; Kadir, A. F. A.; and Mokhlis, H. 2023. Artificial Intelligence Application in Demand Response: Advantages, Issues, Status, and Challenges. IEEE Access, 11: 16907--16922

    work page 2023

  6. [6]

    Anthony, L. F. W.; Kanding, B.; and Selvan, R. 2020. Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models. arXiv:2007.03051

    arXiv 2020

  7. [7]

    Arjovsky, M.; Bottou, L.; Gulrajani, I.; and Lopez-Paz, D. 2020. Invariant Risk Minimization. arXiv:1907.02893

    arXiv 2020

  8. [8]

    Arthur, D.; Vassilvitskii, S.; et al. 2007. k-means++: The advantages of careful seeding. In Soda, volume 7, 1027--1035

    2007

Show all 64 references
  1. [9]

    Bannour, N.; Ghannay, S.; N \'e v \'e ol, A.; and Ligozat, A.-L. 2021. Evaluating the carbon footprint of NLP methods: a survey and analysis of existing tools. In Moosavi, N. S.; Gurevych, I.; Fan, A.; Wolf, T.; Hou, Y.; Marasovi \'c , A.; and Ravi, S., eds., Proceedings of th...

    2021

  2. [10]

    Bossard, L.; Guillaumin, M.; and Van Gool, L. 2014. Food-101--mining discriminative components with random forests. In Computer vision--ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13, 446--461. Springer

    2014

  3. [11]

    Caruana, R. 1997. Multitask learning. Machine learning, 28: 41--75

    1997

  4. [12]

    Cohen, G.; Afshar, S.; Tapson, J.; and Van Schaik, A. 2017. EMNIST: Extending MNIST to handwritten letters. In 2017 international joint conference on neural networks (IJCNN), 2921--2926. IEEE

    2017

  5. [13]

    Dehghani, M.; Djolonga, J.; Mustafa, B.; Padlewski, P.; Heek, J.; Gilmer, J.; Steiner, A. P.; Caron, M.; Geirhos, R.; Alabdulmohsin, I.; Jenatton, R.; Beyer, L.; Tschannen, M.; Arnab, A.; Wang, X.; Riquelme Ruiz, C.; Minderer, M.; Puigcerver, J.; Evci, U.; Kumar, M.; Steenkist...

    2023

  6. [14]

    Deng, A.; Li, X.; Hu, D.; Wang, T.; Xiong, H.; and Xu, C.-Z. 2023. Towards Inadequately Pre-trained Models in Transfer Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 19397--19408

    2023

  7. [15]

    S.; Smith, N

    Dodge, J.; Prewitt, T.; Tachet des Combes, R.; Odmark, E.; Schwartz, R.; Strubell, E.; Luccioni, A. S.; Smith, N. A.; DeCario, N.; and Buchanan, W. 2022. Measuring the Carbon Intensity of AI in Cloud Instances. In Proceedings of the 2022 ACM Conference on Fairness, Accountabil...

    2022

  8. [16]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Confere...

    2021

  9. [17]

    Ford, N.; Gilmer, J.; Carlini, N.; and Cubuk, D. 2019. Adversarial Examples Are a Natural Consequence of Test Error in Noise. arXiv:1901.10513

    2019 arXiv

  10. [18]

    F.; Riley, G.; and Grahn, H

    García-Martín, E.; Rodrigues, C. F.; Riley, G.; and Grahn, H. 2019. Estimation of energy consumption in machine learning. Journal of Parallel and Distributed Computing, 134: 75--88

    2019

  11. [19]

    Geirhos, R.; Temme, C. R. M.; Rauber, J.; Schütt, H. H.; Bethge, M.; and Wichmann, F. A. 2020. Generalisation in humans and deep neural networks. arXiv:1808.08750

    2020 arXiv

  12. [20]

    Georgiou, S.; Kechagia, M.; Sharma, T.; Sarro, F.; and Zou, Y. 2022. Green AI: do deep learning frameworks have different costs? In Proceedings of the 44th International Conference on Software Engineering, ICSE '22, 1082–1094

    2022

  13. [21]

    Goyal, S.; Sun, M.; Raghunathan, A.; and Kolter, Z. 2022. Test-Time Adaptation via Conjugate Pseudo-labels. arXiv:2207.09640

    2022 arXiv

  14. [22]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778

    2016

  15. [23]

    Henderson, P.; Hu, J.; Romoff, J.; Brunskill, E.; Jurafsky, D.; and Pineau, J. 2020. Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning. Journal of Machine Learning Research, 21(248): 1--43

    2020

  16. [24]

    Hendrycks, D.; Basart, S.; Mu, N.; Kadavath, S.; Wang, F.; Dorundo, E.; Desai, R.; Zhu, T.; Parajuli, S.; Guo, M.; Song, D.; Steinhardt, J.; and Gilmer, J. 2021. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization. In Proceedings of the IEEE...

    2021

  17. [25]

    Hendrycks, D.; and Dietterich, T. 2019. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. arXiv:1903.12261

    2019 arXiv

  18. [26]

    D.; Zoph, B.; Gilmer, J.; and Lakshminarayanan, B

    Hendrycks, D.; Mu, N.; Cubuk, E. D.; Zoph, B.; Gilmer, J.; and Lakshminarayanan, B. 2020. AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty. arXiv:1912.02781

    2020 arXiv

  19. [27]

    Huang, R.; Geng, A.; and Li, Y. 2021. On the Importance of Gradients for Detecting Distributional Shifts in the Wild. In Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems

    2021

  20. [28]

    Hubert, L.; and Arabie, P. 1985. Comparing partitions. Journal of classification, 2: 193--218

    1985

  21. [29]

    Kim, J.-H.; Choo, W.; and Song, H. O. 2020. Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup. In International Conference on Machine Learning (ICML)

    2020

  22. [30]

    Krizhevsky, A.; Hinton, G.; et al. 2009. Learning multiple layers of features from tiny images

    2009

  23. [31]

    Lacoste, A.; Luccioni, A.; Schmidt, V.; and Dandres, T. 2019. Quantifying the Carbon Emissions of Machine Learning. arXiv:1910.09700

    2019 arXiv

  24. [32]

    LeCun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278--2324

    1998

  25. [33]

    Lee, J.; and AlRegib, G. 2020. Gradients as a Measure of Uncertainty in Neural Networks. In IEEE International Conference on Image Processing (ICIP)

    2020

  26. [34]

    Z.; Astrid, M.; and Lee, S.-I

    Lee, J.-H.; Zaheer, M. Z.; Astrid, M.; and Lee, S.-I. 2020. Smoothmix: a simple yet effective data augmentation to train robust classifiers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 756--757

    2020

  27. [35]

    Li, H.; Fowlkes, C.; Yang, H.; Dabeer, O.; Tu, Z.; and Soatto, S. 2023. Guided recommendation for model fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3633--3642

    2023

  28. [36]

    Lim, H.; Kim, B.; Choo, J.; and Choi, S. 2023. TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation. arXiv:2302.05155

    2023 arXiv

  29. [37]

    Liu, M.; Liu, H.-F.; and Lee, C.-C. 2024. An empirical study on the response of the energy market to the shock from the artificial intelligence industry. Energy, 288: 129655

    2024

  30. [38]

    Liu, X.; Shen, F.; Zhao, J.; and Nie, C. 2022. RandomMix: A mixed sample data augmentation method with multiple mixed modes. arXiv:2205.08728

    2022 arXiv

  31. [39]

    McDonald, J.; Li, B.; Frey, N.; Tiwari, D.; Gadepally, V.; and Samsi, S. 2022. Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models. In Carpuat, M.; de Marneffe, M.-C.; and Meza Ruiz, I. V., eds., Findings of the Association for C...

    2022

  32. [40]

    Menghani, G. 2023. Efficient deep learning: A survey on making deep learning models smaller, faster, and better. ACM Computing Surveys, 55(12): 1--37

    2023

  33. [41]

    Y.; et al

    Netzer, Y.; Wang, T.; Coates, A.; Bissacco, A.; Wu, B.; Ng, A. Y.; et al. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, 4. Granada

    2011

  34. [42]

    Niu, S.; Wu, J.; Zhang, Y.; Chen, Y.; Zheng, S.; Zhao, P.; and Tan, M. 2022. Efficient Test-Time Model Adaptation without Forgetting. arXiv:2204.02610

    2022 arXiv

  35. [43]

    Oreshkin, B.; Rodr \' guez L \'o pez, P.; and Lacoste, A. 2018. Tadam: Task dependent adaptive metric for improved few-shot learning. Advances in neural information processing systems, 31

    2018

  36. [44]

    Patterson, D.; Gonzalez, J.; Le, Q.; Liang, C.; Munguia, L.-M.; Rothchild, D.; So, D.; Texier, M.; and Dean, J. 2021. Carbon Emissions and Large Neural Network Training. arXiv:2104.10350

    2021 arXiv

  37. [45]

    Ravi, S.; and Larochelle, H. 2016. Optimization as a model for few-shot learning. In International conference on learning representations

    2016

  38. [46]

    Salehi, S.; and Schmeink, A. 2024. Data-Centric Green Artificial Intelligence: A Survey. IEEE Transactions on Artificial Intelligence, 5(5): 1973--1989

    2024

  39. [47]

    Sangarya, V.; Bradford, R.; and Kim, J.-E. 2023. Aggregate Representation Measure for Predictive Model Reusability. In Computational sustainability at NeurIPS

    2023

  40. [48]

    Sangarya, V.; Bradford, R.; and Kim, J.-E. 2024. Estimating Environmental Cost Throughout Model's Adaptive Life Cycle. In AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society

    2024

  41. [49]

    M.; and Embrechts, M

    Santos, J. M.; and Embrechts, M. 2009. On the use of the adjusted rand index as a metric for evaluating supervised classification. In International conference on artificial neural networks, 175--184. Springer

    2009

  42. [50]

    Schmidt, V.; Goyal, K.; Joshi, A.; Feld, B.; Conell, L.; Laskaris, N.; Blank, D.; Wilson, J.; Friedler, S.; and Luccioni, S. 2021. CodeCarbon: estimate and track carbon emissions from machine learning computing (2021). DOI: https://doi. org/10.5281/zenodo, 4658424

    2021 doi

  43. [51]

    A.; and Etzioni, O

    Schwartz, R.; Dodge, J.; Smith, N. A.; and Etzioni, O. 2019. Green AI. arXiv:1907.10597

    2019 arXiv

  44. [52]

    Simonyan, K. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

    2014 arXiv

  45. [53]

    Stacke, K.; Eilertsen, G.; Unger, J.; and Lundstr \"o m, C. 2020. Measuring domain shift for deep learning in histopathology. IEEE journal of biomedical and health informatics, 25(2): 325--336

    2020

  46. [54]

    Stallkamp, J.; Schlipsing, M.; Salmen, J.; and Igel, C. 2012. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks, 32: 323--332

    2012

  47. [55]

    Strubell, E.; Ganesh, A.; and McCallum, A. 2019. Energy and Policy Considerations for Deep Learning in NLP. arXiv:1906.02243

    2019 arXiv

  48. [56]

    Verdecchia, R.; Sallou, J.; and Cruz, L. 2023. A Systematic Review of Green AI. arXiv:2301.11047

    2023 arXiv

  49. [57]

    V.; and Dai, D

    Wang, Q.; Fink, O.; Gool, L. V.; and Dai, D. 2022. Continual Test-Time Domain Adaptation. arXiv:2203.13591

    2022 arXiv

  50. [58]

    Wu, C.-J.; Acun, B.; Raghavendra, R.; and Hazelwood, K. 2024. Beyond Efficiency: Scaling AI Sustainably. IEEE Micro, 1--8

    2024

  51. [59]

    Wu, C.-J.; Raghavendra, R.; Gupta, U.; Acun, B.; Ardalani, N.; Maeng, K.; Chang, G.; Aga, F.; Huang, J.; Bai, C.; Gschwind, M.; Gupta, A.; Ott, M.; Melnikov, A.; Candido, S.; Brooks, D.; Chauhan, G.; Lee, B.; Lee, H.-H.; Akyildiz, B.; Balandat, M.; Spisak, J.; Jain, R.; Rabbat...

    2022

  52. [60]

    Xiao, H.; Rasul, K.; and Vollgraf, R. 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

    2017 arXiv

  53. [61]

    Xu, Y.; Martínez-Fernández, S.; Martinez, M.; and Franch, X. 2023. Energy Efficiency of Training Neural Network Architectures: An Empirical Study. arXiv:2302.00967

    2023 arXiv

  54. [62]

    G.; Shlens, J.; Cubuk, E

    Yin, D.; Lopes, R. G.; Shlens, J.; Cubuk, E. D.; and Gilmer, J. 2020. A Fourier Perspective on Model Robustness in Computer Vision. arXiv:1906.08988

    2020 arXiv

  55. [63]

    Zhang, C.; Bengio, S.; and Singer, Y. 2022. Are All Layers Created Equal? arXiv:1902.01996

    2022 arXiv

  56. [64]

    N.; and Lopez-Paz, D

    Zhang, H.; Cisse, M.; Dauphin, Y. N.; and Lopez-Paz, D. 2018. mixup: Beyond Empirical Risk Minimization. arXiv:1710.09412

    2018 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.