Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For locally run LLMs, inference energy tracks response time almost perfectly, so latency can serve as an energy proxy.

desk verdict Broad, useful empirical benchmark of LLM inference energy; the qualitative levers are plausible, but the headline correlations rest on unvalidated software energy readings and should be read with caution. read the letter →

arxiv 2502.05610 v2 pith:SHNU65TG submitted 2025-02-08 cs.CL

Soham Poddar , Paramita Koley , Janardan Misra , Sanjay Podder , Niloy Ganguly , Saptarshi Ghosh This is my paper · ORCID
classification cs.CL
keywords LLMinferenceenergybenchmarkingresponsetimeproxyoutputtokenlengthquantizationbatchsizepromptengineeringsustainableNLP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the energy cost of running large language models in production can be predicted and reduced with simple, practical levers. Across 10 open models and 11 NLP tasks, it reports that per-query inference energy correlates almost perfectly with response time (Pearson r = 0.996), making latency a reliable proxy for energy use. It claims that output token count affects energy far more than input token count, because output generation is sequential while input processing is parallelized and cached. It also reports that combining quantization with larger batch sizes can cut per-sample energy to about a third, and that encoder-decoder models such as Flan-T5 use less energy per parameter than decoder-only models. The stakes: deployment choices such as model family, batch size, and output-length control can be made energy-aware without specialized power meters.

What carries the argument

The argument is carried by the transformer attention complexity relation $O(n,d,t) = (n d^2 + n^2 d) \cdot t$, where $n$ is the number of input tokens, $d$ the hidden dimension, and $t$ the number of output tokens, together with the asymmetry that input tokens are processed in parallel and cached (key-value caching), whereas output tokens are generated sequentially. That asymmetry explains the steeper energy slope for output length and motivates the practical lever of shortening outputs. The measurement machinery is a pair of software power meters, CodeCarbon and CarbonTracker, sampling GPU and CPU power on a single NVIDIA A6000 server, with response time recorded as a lightweight proxy for the energy measurement.

What would settle it

Run the same batch-level benchmark on the same A6000 server while logging GPU power with a hardware wattmeter at millisecond resolution; if the wattmeter shows that batches shorter than the 10-second sampling window consume substantially different energy than reported, or that the response-time correlation drops well below 0.996, the central proxy claim fails.

Watch

Extended reading notes

Core claim

The paper claims that for locally run, open-source LLMs, inference energy is strongly predictable and reducible. Its central empirical finding is a near-perfect linear relationship between response time and energy per batch (Pearson r = 0.996), which it proposes as a practical proxy for energy measurement. It further reports that output token length has a much stronger influence on energy than input token length (Pearson r = 0.952 for output versus 0.697 for input), and controlled experiments confirm linear energy scaling with both lengths, with the first output token costing far more than subsequent tokens. The paper also reports that task complexity has almost no effect on energy once input and output lengths are fixed, that encoder-decoder models are generally more energy-efficient than decoder-only models of comparable size, that increasing batch size lowers per-sample energy, and that 4-bit quantization combined with a batch size of 256 reduces energy to about 0.33 times that of the default 16-bit model at batch size 64, while keeping accuracy loss under about 5 percent.

Load-bearing premise

The results assume that the software power meters CodeCarbon and CarbonTracker, which sample power at intervals of up to 10 seconds, capture the energy of each short inference batch accurately enough that the reported correlations and energy slopes are real.

Editorial extensions

If this is right

  • Practitioners can use response time as a proxy to compare the energy efficiency of different models and settings without installing energy-tracking tools.
  • Shortening output length, whether through prompt phrases, stopping criteria, or constrained decoding, becomes the primary lever for reducing inference energy.
  • Quantized models combined with larger batch sizes offer substantial energy savings with modest accuracy loss, limited mainly by GPU memory.
  • Encoder-decoder models such as Flan-T5 are the recommended choice for energy-constrained NLP deployments.
  • Task complexity alone should not be treated as an energy driver; input and output token lengths dominate the energy cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that if the response-time proxy holds across providers, latency logs from closed API usage could be converted into approximate per-request energy or carbon estimates, provided network latency is averaged out over many calls.
  • The paper's finding that the first output token costs much more than later tokens suggests that serving systems could batch or cache shared input processing across many requests to amortize that fixed cost, an optimization the paper does not explore.
  • The targeted-phrase results imply that energy savings from 'be concise' prompts are largest for models that over-explain by default; a model's baseline verbosity could predict whether prompt editing will help, which would be a testable extension of the paper's claims.
  • The reported slope differences between encoder-decoder and decoder-only families suggest that hardware-aware scheduling could route short-output tasks to encoder-decoder models and long-generation tasks to models with cheaper incremental decoding, though the paper does not make this recommendation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper benchmarks the inference energy of ten large language models (six decoder-only, four encoder-decoder) across eleven NLP tasks, using CodeCarbon and CarbonTracker on a local A6000 GPU. It reports that per-batch inference energy correlates strongly with response time (Pearson r=0.996), that output token length has a stronger influence on energy than input length (Pearson r=0.952 vs 0.697), and that task complexity has negligible effect once input and output lengths are fixed. It further studies quantization, batch size, and targeted prompt phrases, introduces a Normalized Accuracy metric for accuracy-energy tradeoffs, and concludes with recommendations for energy-efficient deployment.

Significance. If the empirical results are robust, the response-time proxy is practically useful for locally deployed models and the output-length finding provides a concrete lever for reducing inference energy. The paper's strengths include the breadth of the benchmark (ten models, eleven tasks), the controlled length experiments, the comparison of two energy-measurement tools, and additional experiments on two other hardware systems. The main weaknesses are the lack of validation of the energy meters on this specific testbed and the statistical support for the headline correlation and slope claims.

major comments (4)
  1. [2.3 and 3.1] Section 2.3 states 'We use X = 10secs for a balance between overhead costs and tracking accuracy', while Section 3.1 states 'the tracking interval is set to 1sec for the energy-measuring libraries'. The interval used for the controlled length experiments in Figures 2-3 is not stated. For models such as Flan-T5-base, whose per-batch response times are a few seconds (Figure 7), a 1-s sampling cadence yields only a few power samples per batch, and a 10-s cadence would smear several batches together. The manuscript provides no validation of CodeCarbon/CarbonTracker against a hardware wattmeter on this testbed and no repetition counts. Because the headline correlations and slopes are computed from these energy estimates, the sampling interval, its sensitivity, and the number of power samples per batch must be reported before the quantitative claims can be accepted.
  2. [3.1] The reported Pearson r = 0.996 is computed on batch-level aggregate points (per-batch averages of response time and energy). Correlation coefficients computed on aggregated means are systematically higher than those on the underlying individual samples, so this number does not by itself establish that response time is a reliable per-query proxy. The authors should report the correlation on de-aggregated per-sample data, or at least provide confidence intervals, the number of batches, and the number of repetitions used.
  3. [3.2] The claim that output length has a 'steeper' effect than input length is supported by comparing Pearson r values (0.697 vs 0.952). Pearson r is a measure of linear association, not a slope, and it is sensitive to the range and distribution of the predictor. To support the claim, the authors should report regression coefficients (e.g., energy per token) from a model that includes both input and output length, with confidence intervals, and show that the output coefficient is significantly larger. The controlled experiments in Figures 2-3 contain the relevant data and should be used for this comparison.
  4. [3.2] The batch-wise scatter for input length (Figure 1, middle) is potentially confounded by dataset: the high-energy points come from CNN-DM and SAMSum, which also have long outputs, so the input-length correlation (r=0.697) may partly reflect output-length differences or task effects. The controlled input-length experiment (Figure 2) fixes output length and does show a linear trend, but only on one dataset and with generation stopped at one token. The authors should either restrict the input-vs-output comparison to the controlled experiments or adjust for dataset in the regression.
minor comments (5)
  1. [2.3] The text contains typos: 'SAMS UM' and 'Pytorch' should be 'SAMSum' and 'PyTorch'.
  2. [Figure 1] The rightmost panel is titled 'Per prompt energy Code Carbon (mWh)' while the left and middle panels have no library label; please state which energy library is used for each panel, especially since the two libraries show only moderate Pearson agreement (r=0.610).
  3. [3.4] The statement 'showing a linear increase with the size of the model' is made from a plot with a categorical x-axis and a log-scale y-axis; please add a fitted line or rephrase to 'monotonic increase'.
  4. [2.2] The Normalized Accuracy metric is described verbally; please provide the exact formula (z-score followed by sigmoid) so that the numbers in Table 4 are reproducible, even though the metric is acknowledged to depend on the set of models.
  5. [Appendix H] The concluding discussions state that response time can serve as an energy proxy for online API models, but Appendix H itself says this is not empirically validated and lists confounding factors; please soften the conclusion to match the evidence.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: inference energy is measured directly with software meters and compared to independently recorded response time and token lengths; the conclusions do not reduce to a fitted parameter or self-citation.

full rationale

This paper is an empirical benchmarking study rather than a derivation, and I find no step where a claimed result is equivalent to its own inputs by construction. Energy is measured directly with CodeCarbon and CarbonTracker, while response time and token lengths are recorded independently, so the headline correlations (response time r=0.996, output length r=0.952 vs input r=0.697) are observed associations rather than fitted values renamed as predictions. The strong response-time correlation is unsurprising because energy is the time-integral of power and the paper explicitly assumes a roughly fixed power draw, but this is a physical expectation made explicit in the text, not a circular definition or a fitted parameter masked as a result. The controlled length experiments fix one variable and vary the other, and the task-complexity comparison uses the authors' own VAX-Stance and CAVES datasets; these datasets were published earlier and the near-zero energy difference is an empirical outcome, not an artifact of the dataset construction. The main vulnerability identified by a skeptical reader, namely the 1s/10s sampling interval of the software energy meters and the absence of a hardware wattmeter cross-check on this testbed, is a measurement-validation concern and not a form of circularity under the criteria used here.

Assumptions & free parameters 0 free parameters · 6 assumptions · 1 invented entities

The central claims rest primarily on the measurement tools' accuracy and the representativeness of the sampled data rather than on mathematical axioms. The main assumptions are listed above.

assumptions (6)
  • standard math Transformer computational complexity follows O(n,d,t) = (n*d^2 + n^2*d)*t (Eq. 1)
    Used in Section 3.2 as theoretical baseline for input/output length effects, but observed trends are linear, not quadratic, due to hardware optimizations.
  • domain assumption CodeCarbon and CarbonTracker software power meters accurately measure GPU/CPU/RAM energy at the configured sampling interval
    Section 2.3 relies on these tools without hardware wattmeter validation on the testbed; they cite Jay et al. (2023) and Bouza et al. (2023) for tool accuracy.
  • domain assumption The 1024 randomly sampled examples per dataset are representative of each task's input distribution
    Section 2.2 states a random selection of 1024 samples, but no seed or statistical justification is given.
  • ad hoc to paper Task complexity can be approximated by human cognitive complexity (3-class vs 12-class classification; summarization vs extraction)
    Section 3.3 defines complexity this way without a formal complexity measure.
  • domain assumption PUE=1.0 is a valid assumption for the local server, meaning measured energy understates real energy
    Section 2.3 sets PUE default; actual PUE is higher, acknowledged by authors.
  • domain assumption The bitsandbytes 4-bit/8-bit quantized models still execute in 16-bit precision, so reported quantization effects are specific to this implementation
    Section 3.5 notes that computation remains 16-bit, so findings may not generalize to true integer quantization.
invented entities (1)
  • Normalized Accuracy (NA) metric
    purpose: Combine model performance across tasks with different metrics into a single 0-100 score via z-score normalization, sigmoid, and averaging.
    Defined in Section 2.2; depends on the set of models compared, and no external validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models." pith.science (2026). https://pith.science/paper/SHNU65TG

@misc{pith2026250205610,
  author       = {Pith},
  title        = {Pith review of: Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SHNU65TG}},
  note         = {Machine review of arXiv:2502.05610}
}
read the original abstract

Large language models (LLMs) are increasingly recognized for their exceptional generative capabilities and versatility across various tasks. However, the high inference costs associated with these models have not received adequate attention, particularly when compared to the focus on training costs in existing research. In response to this gap, our study conducts a comprehensive benchmarking of LLM inference energy across a wide range of NLP tasks, where we analyze the impact of different models, tasks, prompts, and system-related factors on inference energy. Specifically, our experiments reveal several interesting insights, including strong correlation of inference energy with output token length and response time. Also, we find that quantization and optimal batch sizes, along with targeted prompt phrases, can significantly reduce energy usage. This study is the first to thoroughly benchmark LLM inference across such a diverse range of aspects, providing insights and offering several recommendations for improving energy efficiency in model deployment.

Figures

Figures reproduced from arXiv: 2502.05610 by the authors.

Figure 1
Figure 1. Inference energy vs response time, input and output-token length averaged across samples in a batch [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Inference energy on CNN-DM where we vary input token lengths fixing #output tokens to 1. CNN-DM and SAMSUM cluster around the bot￾tom left because of their short outputs, whereas the widespread clusters of CNN-DM and SAM￾SUM towards the top provide a better visualization of the linear dependency. Note that, the slope of variation of energy consumption is steeper for the output length, in comparison to the slope for … view at source ↗
Figure 3
Figure 3. Inference energy on CNN-DM dataset when the output length is varied, keeping input length fixed. generating 2 tokens takes only about 12% more energy than generating 1 token. This is because the model processes the entire input in the first time step, but only 1 token for subsequent steps (by means of caching the K-V computations for prior tokens). Also, note that, the increment of energy is larger with increasing o… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Average per-prompt inference energy vs model size for all models and datasets. The black lines join the median energy for each model family. Normalized accuracy (NA) scores for each model across all datasets (performance on each dataset given in Appendix E). Here, we o…
Figure 5
Figure 5. Figure 5: Per-sample inference energy with 4-bit quan [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Effect of inserting targeted phrases in prompt [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Average per-sample inference energy vs average per-sample response time, input and output-token length [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Energy consumption of running Mistral-7B and flan-T5-large on different systems (identified by their [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Additional Batch size experiments on the A5000 GPU, and using 8-bit quantization. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Brevity is the soul of sustainability: Characterizing LLM response lengths

    cs.CL 2025-06 conditional novelty 6.0 of 10

    LLMs produce longer-than-needed answers to factual questions, and simple prompt instructions such as 'provide only the minimal answer' cut response length and inference energy by about 25-60% without hurting automated...

  2. SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts--Extended Version

    cs.CL 2025-08 conditional novelty 5.0 of 10

    A new benchmark of 15 small language models across 23 datasets and 11 metrics shows clear accuracy-versus-energy trade-offs, with no single model dominating.

  3. A Multi-Pass Large Language Model Framework for Precise and Efficient Radiology Report Error Detection

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A three-pass LLM framework (extractor, detector, false-positive verifier) more than doubled PPV and halved estimated review costs for radiology report error detection, while the absolute number of confirmed errors sta...

Reference graph

Works this paper leans on

51 extracted references · 19 canonical work pages · cited by 3 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Wolff Anthony, Benjamin Kanding, and Raghavendra Selvan

    Lasse F. Wolff Anthony, Benjamin Kanding, and Raghavendra Selvan. 2020. Carbontracker: Tracking and predicting the carbon footprint of training deep learning models. ICML Workshop on Challenges in Deploying and monitoring Machine Learning Systems. ArXiv:2007.03051

    arXiv 2020

  4. [4]

    Haoli Bai, Lu Hou, Lifeng Shang, Xin Jiang, Irwin King, and Michael R Lyu. 2022. Towards efficient post-training quantization of pre-trained language models. Advances in neural information processing systems, 35:1405--1418

    work page 2022

  5. [5]

    Jeff Barr. 2019. Amazon ec2 update--inf1 instances with aws inferentia chips for high performance cost-effective inferencing

    2019

  6. [6]

    Luc \' a Bouza, Aur \'e lie Bugeau, and Lo \" c Lannelongue. 2023. How to estimate carbon footprint when training deep learning models? a guide and review. Environmental Research Communications, 5(11):115014

    2023

  7. [7]

    Qingqing Cao, Aruna Balasubramanian, and Niranjan Balasubramanian. 2020. Towards accurate and reliable energy measurement of nlp models. arXiv preprint arXiv:2010.05248

    work page Pith review arXiv 2020

  8. [8]

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023 a . Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318

    arXiv 2023

Show all 51 references
  1. [9]

    Lingjiao Chen, Matei Zaharia, and James Zou. 2023 b . Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176

    2023 arXiv

  2. [10]

    Benoit Courty, Victor Schmidt, Sasha Luccioni, Goyal-Kamal, MarionCoutarel, Boris Feld, Jérémy Lecourt, LiamConnell, Amine Saboni, Inimaz, supatomic, Mathilde Léval, Luis Blanche, Alexis Cruveiller, ouminasara, Franklin Zhao, Aditya Joshi, Alexis Bogroff, Hugues de Lavoreille,...

    2024 doi

  3. [11]

    Radosvet Desislavov, Fernando Mart \' nez-Plumed, and Jos \'e Hern \'a ndez-Orallo. 2021. Compute and energy consumption trends in deep learning inference. arXiv preprint arXiv:2109.05472

    2021 arXiv

  4. [12]

    Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A Smith, Nicole DeCario, and Will Buchanan. 2022. Measuring the carbon intensity of ai in cloud instances. In Proceedings of the 2022 ACM conference o...

    2022

  5. [13]

    Brad Everman, Trevor Villwock, Dayuan Chen, Noe Soto, Oliver Zhang, and Ziliang Zong. 2023. Evaluating the carbon impact of large language models at the inference stage. In 2023 IEEE international performance, computing, and communications conference (IPCCC), pages 150--157. IEEE

    2023

  6. [14]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323

    2022 arXiv

  7. [15]

    Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. Samsum corpus: A human-annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70--79

    2019

  8. [16]

    Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248):1--43

    2020

  9. [17]

    Mathilde Jay, Vladimir Ostapenco, Laurent Lef \`e vre, Denis Trystram, Anne-C \'e cile Orgerie, and Benjamin Fichel. 2023. An experimental comparison of software-based power meters: focus on cpu and gpu. In 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Inter...

    2023

  10. [18]

    Eldar Kurti \'c , Elias Frantar, and Dan Alistarh. 2024. Ziplm: Inference-aware structured pruning of language models. Advances in Neural Information Processing Systems, 36

    2024

  11. [19]

    Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700

    2019 arXiv

  12. [20]

    Lo \" c Lannelongue, Jason Grealey, and Michael Inouye. 2021. Green algorithms: quantifying the carbon footprint of computation. Advanced science, 8(12):2100707

    2021

  13. [21]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274--19286. PMLR

    2023

  14. [22]

    Baolin Li, Yankai Jiang, Vijay Gadepally, and Devesh Tiwari. 2024. Toward sustainable genai using generation directives for carbon-friendly large language model inference. arXiv preprint arXiv:2403.12900

    2024 arXiv

  15. [23]

    Gauthier Limpens, Stefano Moret, Herv \'e Jeanmart, and Francois Mar \'e chal. 2019. Energyscope td: A novel open-source model for regional energy systems. Applied Energy, 255:113729

    2019

  16. [24]

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950--1965

    2022

  17. [25]

    Alexandra Sasha Luccioni et al. 2023. Counting carbon: A survey of factors influencing the emissions of machine learning. arXiv preprint arXiv:2302.08476

    2023 arXiv

  18. [26]

    Sasha Luccioni, Yacine Jernite, and Emma Strubell. 2024. Power hungry processing: Watts driving the cost of ai deployment? In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 85--99

    2024

  19. [27]

    Joseph McDonald, Baolin Li, Nathan Frey, Devesh Tiwari, Vijay Gadepally, and Siddharth Samsi. 2022. Great power, great responsibility: Recommendations for reducing energy for training language models. arXiv preprint arXiv:2205.09646

    2022 arXiv

  20. [28]

    Gianluca Moro, Luca Ragazzi, and Lorenzo Valgimigli. 2023. Carburacy: summarization models tuning and comparison in eco-sustainable regimes with a novel carbon-aware accuracy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14417--14425

    2023

  21. [29]

    Rakshit Naidu, Harshita Diddee, Ajinkya Mulay, Aleti Vardhan, Krithika Ramesh, and Ahmed Zamzam. 2021. Towards quantifying the carbon emissions of differentially private machine learning. arXiv preprint arXiv:2107.06946

    2021 arXiv

  22. [30]

    Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, C a g lar Gul c ehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pages 280--290

    2016

  23. [31]

    David Patterson, Joseph Gonzalez, Urs H \"o lzle, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R So, Maud Texier, and Jeff Dean. 2022. The carbon footprint of machine learning training will plateau, then shrink. Computer, 55(7):18--28

    2022

  24. [32]

    David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. 2021. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350

    2021 arXiv

  25. [33]

    OA Plosskaya, VS Akhripkin, IV Pavlov, et al. 2022. Eco2ai: carbon emissions tracking of machine learning models as the first step towards sustainable ai. In Doklady Mathematics, volume 106, pages S118--S128. Springer

    2022

  26. [34]

    Soham Poddar, Mainack Mondal, Janardan Misra, Niloy Ganguly, and Saptarshi Ghosh. 2022 a . Winds of change: Impact of covid-19 on vaccine-related opinions of twitter users. In Proceedings of the international aaai conference on web and social media, volume 16, pages 782--793

    2022

  27. [35]

    Soham Poddar, Azlaan Mustafa Samad, Rajdeep Mukherjee, Niloy Ganguly, and Saptarshi Ghosh. 2022 b . Caves: A dataset to facilitate explainable classification and summarization of concerns towards covid vaccines. In Proceedings of the 45th international ACM SIGIR conference on ...

    2022

  28. [36]

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2022. Efficiently scaling transformer inference. corr, abs/2211.05102 (2022)

    2022 arXiv

  29. [37]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  30. [38]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. https://doi.org/10.18653/v1/D16-1264 SQ u AD : 100,000+ questions for machine comprehension of text . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Tex...

    2016 doi

  31. [39]

    Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, and Doug Downey. 2022. Embedding recycling for language models. arXiv preprint arXiv:2207.04993

    2022 arXiv

  32. [40]

    Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. 2023. From words to watts: Benchmarking the energy costs of large language model inference. In 2023 IEEE High Performance ...

    2023

  33. [41]

    Victor Schmidt, Kamal Goyal, Aditya Joshi, Boris Feld, Liam Conell, Nikolas Laskaris, Doug Blank, Jonathan Wilson, Sorelle Friedler, and Sasha Luccioni. 2021. Codecarbon: estimate and track carbon emissions from machine learning computing. Cited on, 20

    2021

  34. [42]

    Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2020. Green ai. Communications of the ACM, 63(12):54--63

    2020

  35. [43]

    Raghavendra Selvan, Nikhil Bhagwat, Lasse F Wolff Anthony, Benjamin Kanding, and Erik B Dam. 2022. Carbon footprint of selecting and training deep learning models for medical image analysis. In International Conference on Medical Image Computing and Computer-Assisted Intervent...

    2022

  36. [44]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  37. [45]

    Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019 a . Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32

    2019

  38. [46]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2019 b . Glue: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations

    2019

  39. [47]

    Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2022. Towards understanding chain-of-thought prompting: An empirical study of what matters. arXiv preprint arXiv:2212.10001

    2022 arXiv

  40. [48]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  41. [49]

    Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. 2022. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4:795--813

    2022

  42. [50]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087--38099. PMLR

    2023

  43. [51]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models, 2023. URL https://arxiv. org/pdf/2305.10601. pdf

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.