Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Quantifying the Energy Consumption and Carbon Emissions of LLM Inference via Simulations

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A simulation framework that couples LLM inference with grid conditions can quantify energy use and carbon emissions before deployment.

desk verdict Useful framework paper with a real unit inconsistency in the MFU-to-power equation that must be fixed before quantitative results are trusted. read the letter →

arxiv 2507.11417 v1 pith:RMNB2P56 submitted 2025-07-15 cs.DC

Miray Özcan , Philipp Wiesner , Philipp Wei{ss} , Odej Kao This is my paper · ORCID
classification cs.DC
keywords LLMinferenceGPUpowermodelingcarbonemissionsenergyco-simulationcarbon-awareschedulingMFUsustainablecomputingsimulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the energy use and carbon emissions of LLM inference can be estimated in simulation before any physical deployment, by adding a GPU power model to a high-fidelity inference simulator and feeding its output into a grid co-simulation. The payoff is that operators can vary batch size, sequence length, throughput, and parallelism to see how each changes energy demand and emissions, and test carbon-aware scheduling against realistic grid conditions. In an illustrative deployment with solar generation and a battery, the framework reports a 70.3% renewable share and a 69.2% offset of total emissions, while showing that fixed scheduling still left much of the workload in high-carbon hours. If correct, this gives a way to explore greener inference designs without buying hardware first.

What carries the argument

Model FLOPs utilization (MFU) is the central proxy: it is the ratio of achieved FLOPs per second to peak FLOPs per second, and the paper uses it to drive a power-law GPU power model $P(\mathrm{MFU}) = P_{\text{idle}} + (P_{\text{max}} - P_{\text{idle}})(\mathrm{MFU}/\mathrm{MFU}_{\mathrm{sat}})^\gamma$. The carrying mechanism is the data pipeline that converts the simulator's variable-duration, per-batch-stage MFU logs into fixed one-minute power bins using a duration-weighted average, then merges those bins with grid carbon intensity, solar irradiance, and battery state in the co-simulation environment. This is what turns execution traces into time-resolved emissions estimates.

What would settle it

Run the same inference workloads on an A100 while logging true GPU power draw, then compare observed power at each MFU to $P(\mathrm{MFU}) = P_{\text{idle}} + (P_{\text{max}} - P_{\text{idle}})(\mathrm{MFU}/\mathrm{MFU}_{\mathrm{sat}})^\gamma$ with $\gamma=0.7$ and $\mathrm{MFU}_{\mathrm{sat}}=0.45$; systematic disagreement would mean the framework's energy and carbon estimates are unreliable.

Watch

Extended reading notes

Core claim

The paper's central claim is that a GPU power model based on model FLOPs utilization (MFU) — the ratio of achieved to peak FLOPs per second — can be coupled to an inference simulator and a grid co-simulation to quantify LLM inference energy and emissions before deployment. The power model is $P(\mathrm{MFU}) = P_{\text{idle}} + (P_{\text{max}} - P_{\text{idle}})(\mathrm{MFU}/\mathrm{MFU}_{\mathrm{sat}})^\gamma$ with $\gamma=0.7$ and $\mathrm{MFU}_{\mathrm{sat}}=0.45$, chosen to reflect sublinear power scaling in memory-bound inference. Controlled simulations show that request volume, prefill-to-decode ratio, batch size, throughput, and parallelism each move average power and total energy in distinct ways, and the co-simulation scenario demonstrates that renewable supply alone does not guarantee low-carbon execution unless scheduling reacts to grid intensity.

Load-bearing premise

The energy and carbon numbers stand on a GPU power curve whose curvature and saturation point were chosen heuristically and never validated against real GPU measurements.

Editorial extensions

If this is right

  • Operators can compare deployment configurations on energy and carbon before buying hardware, using simulation alone.
  • Performance-optimal settings, such as maximum throughput or high utilization, do not automatically minimize energy or emissions; the framework exposes the crossover points.
  • Carbon-aware scheduling that reacts to grid intensity can shift work toward renewable-rich hours, cutting emissions by roughly two-thirds in the illustrative solar-backed case.
  • The same pipeline can be extended to multi-region routing and policy choices, such as running smaller models during high-carbon hours and larger models during renewable peaks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The heuristic $\gamma=0.7$ and $\mathrm{MFU}_{\mathrm{sat}}=0.45$ are the least supported links in the chain; collecting real A100 power traces across MFU values would let anyone re-fit the curve and test how much the conclusions shift.
  • Inference: Because power is treated as static within each batch stage, the framework likely underestimates the value of dynamic voltage and frequency scaling; adding per-stage power transients could change the recommended batch size or parallelism.
  • Inference: The reported 69.2% offset depends on a 600 W solar cap and a 100 Wh battery; sweeping these parameters would map how much scheduling headroom actually exists for a given workload.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents a simulation framework that extends the Vidur LLM inference simulator with an MFU-based GPU power model and couples it to the Vessim energy-system co-simulator, enabling time-resolved estimates of energy consumption and carbon emissions for LLM inference. The authors run controlled Vidur experiments varying request count, prefill-to-decode ratio, batch size, query throughput, and parallelism, and illustrate the full pipeline in a solar-plus-battery case study reporting a 69.2% renewable offset. The paper is transparent about the lack of telemetry-based validation and frames the contribution as a flexible pre-deployment design tool.

Significance. If the identified unit inconsistency is fixed and the power model is treated as an explicitly illustrative model rather than a calibrated measurement, the framework is a useful contribution: it is the first to combine token-level inference simulation with sub-second grid co-simulation, it is open-source, and it enables exploration of carbon-aware scheduling policies without physical hardware. The paper gives explicit credit to its limitations and does not overclaim validation, which is a strength. However, the quantitative findings are entirely downstream of an unvalidated power-law model with heuristically chosen parameters, so the absolute energy and carbon numbers should be interpreted as scenario illustrations until calibrated.

major comments (2)
  1. [§3.1, Eq. (2) and §4.1] There is a unit inconsistency between the definition of MFU and the power model. Eq. (2) defines MFU_i as a percentage by multiplying by 100, while Eq. (1) uses mfu_sat = 0.45. For MFU = 45%, Eq. (1) gives P = 100 + 300·(45/0.45)^0.7 ≈ 7.6 kW per A100, which contradicts the reported saturation near 360 W in Experiment 4 and the 135–155 W averages in Experiment 1. The authors must either remove the ×100 in Eq. (2), set mfu_sat = 45, or explicitly state a normalization step in the code that reconciles the two. This is load-bearing because every downstream energy and carbon estimate, including Table 2 and the 69.2% offset, depends on this mapping.
  2. [§3.1 and §4.1, footnote 3] The power model is not validated against any real GPU power telemetry, and the exponent γ = 0.7 and saturation threshold mfu_sat = 0.45 are chosen heuristically. Since the abstract claims the framework can 'quantify carbon emissions' and 'reveal how inference parameters affect energy demand', the absolute numbers should be presented as model-based estimates under explicitly stated assumptions, or the paper should include a calibration/validation section using public power traces. Without such grounding, the quantitative results cannot be treated as measurements, and the central claim should be softened accordingly.
minor comments (5)
  1. [Section 3 heading] The heading 'Appraoch' is a typo and should read 'Approach'.
  2. [§4.2, Experiment 1] The phrases 'from 28 to 216' and '214 requests' should be typeset as 2^8 to 2^16 and 2^14 requests, respectively, to avoid confusion with the integers 28, 216, and 214.
  3. [§3.1, Eqs. (3) and (4)] The symbol H is used inconsistently: Eq. (3) defines H_i for each stage, while Eq. (4) uses H without a subscript. Please clarify that H in Eq. (4) is the total GPU-hours across all stages and all replicas.
  4. [Table 1(b)] The entry 'Prefill:Decode 20.0' is ambiguous; state explicitly whether this is a ratio (20:1) or a percentage split, and define the notation.
  5. [§4.3, Table 2] The 'Carbon offset 69.2%' figure depends on the chosen solar capacity, battery size, and workload duration; a short sensitivity analysis would help the reader understand how robust this headline number is to scenario parameters.

Circularity Check

1 steps flagged · score 6.0 of 10

The power-saturation 'finding' is forced by a power-law parameter chosen from the same Vidur MFU traces; energy/carbon outputs are direct consequences of that assumed curve, not independent empirical results.

  1. fitted input called prediction [Section 4.1 'Power Model Parameters' (Eq. 1) and Section 4.2 'Experiment 4'; see also footnote 3]
    "The threshold mf usat = 0.45 reflects both empirical observations from our simulations and patterns documented in industry analyses ... Figure 1 shows MFU saturating near 0.45 under QPS values of 5–7.9. These consistent saturation patterns motivate our selection of 0.45 as the maximum effective utilization level in our power model. ... average GPU power increases with QPS and saturates near 360W beyond QPS≈ 5. [Footnote 3:] γ = 0.7 was heuristically chosen ... It was not fit to data."

    Eq. (1) makes P a monotone function of the ratio MFU/mfu_sat. The paper fixes mfu_sat = 0.45 because the Vidur MFU traces saturate near 0.45 (Fig. 1), then reports as a result that GPU power saturates near 360 W at high QPS. That saturation is the same MFU saturation rescaled by the assumed sublinear curve: once MFU plateaus at the threshold, the model forces P toward Pmax. The 'prediction' is therefore not independent evidence about hardware; it is the calibration input (simulated MFU saturation) transformed by a heuristic curve whose exponent was not fit to any power telemetry. Consequently the energy and carbon numbers in Table 2 and Figs. 2-7 inherit the assumed P(mfu) by construction rather than from measured GPU behavior.

full rationale

The framework is a simulation pipeline (Vidur MFU traces + Eq. 1 power model + Vessim grid), and a simulation's outputs are always conditional on its assumptions; conditional simulation is not by itself circular. The concrete circularity is narrower and located in Experiment 4: the mfu_sat parameter is chosen from Vidur's MFU saturation, and then power saturation is presented as an observed outcome of the same curve. Footnote 3 confirms gamma and mfu_sat are heuristic and not telemetry-fitted, so the quantitative results are consequences of the model choice. The Vessim self-citation (Ref. 27, co-authored by Wiesner) is not load-bearing for the paper's central claim; it is a tool dependency with independent code. Separately, there is an internal-consistency concern: Eq. (2) defines MFU with a ×100 percentage scaling while Eq. (1) uses mfu_sat = 0.45, so the published equations cannot reproduce the reported 360 W / 135-155 W values without an unstated normalization; this is a correctness/reproducibility issue rather than a circularity issue. On the circularity scale, one central 'prediction' reduces by construction to its calibration input, so a partial score of 6 is warranted rather than 0-2.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim that the framework can quantify energy and carbon rests on the assumed MFU-power relationship, Vidur's fidelity, and the choice of heuristic parameters. These are all upstream of every numerical result in the paper.

free parameters (2)
  • gamma = 0.7
    Heuristic exponent in the MFU-power curve; not fit to data (footnote 3). Controls the sublinear saturation of power with utilization.
  • mfu_sat = 0.45
    Saturation MFU threshold chosen from simulations and industry analyses, not direct measurement (Section 4.1). Defines the knee of the power curve.
assumptions (4)
  • domain assumption MFU is a valid proxy for dynamic GPU power during inference
    The power model is built around MFU because it correlates with arithmetic saturation; the paper cites prior work but does not validate this for its target GPUs (Section 3.1).
  • domain assumption Vidur accurately simulates LLM inference execution dynamics
    The framework trusts Vidur's MFU traces and timing; the paper notes Vidur is most accurate near 85% of max QPS but does not quantify error (Section 3.2).
  • domain assumption Static per-iteration power excludes host, interconnect, and cooling
    The model only covers GPU-local dynamic power; this is explicitly stated as a limitation (Section 3.1).
  • domain assumption Requests follow a Zipfian length distribution
    Used in experiments to reflect the power-law structure of language data (Section 4.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantifying the Energy Consumption and Carbon Emissions of LLM Inference via Simulations." pith.science (2026). https://pith.science/paper/RMNB2P56

@misc{pith2026250711417,
  author       = {Pith},
  title        = {Pith review of: Quantifying the Energy Consumption and Carbon Emissions of LLM Inference via Simulations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RMNB2P56}},
  note         = {Machine review of arXiv:2507.11417}
}
read the original abstract

The environmental impact of Large Language Models (LLMs) is rising significantly, with inference now accounting for more than half of their total lifecycle carbon emissions. However, existing simulation frameworks, which are increasingly used to determine efficient LLM deployments, lack any concept of power and, therefore, cannot accurately estimate inference-related emissions. We present a simulation framework to assess the energy and carbon implications of LLM inference under varying deployment setups. First, we extend a high-fidelity LLM inference simulator with a GPU power model that estimates power consumption based on utilization metrics, enabling analysis across configurations like batch size, sequence length, and model parallelism. Second, we integrate simulation outputs into an energy system co-simulation environment to quantify carbon emissions under specific grid conditions and explore the potential of carbon-aware scheduling. Through scenario-based analysis, our framework reveals how inference parameters affect energy demand and carbon footprint, demonstrates a renewable offset potential of up to 69.2% in an illustrative deployment case, and provides a foundation for future carbon-aware inference infrastructure design.

Figures

Figures reproduced from arXiv: 2507.11417 by the authors.

Figure 1
Figure 1. Simulated QPS saturation for Meta-Llama-3-8B. MFU increases with QPS before plateauing at 5–7.9 QPS. The dashed line marks the saturation threshold near mfusat = 0.45. See Section 4.2 [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Average power draw and total energy usage as a function of request count. Top: models up to 34B. Bottom: 70B+ models. All runs use A100 GPUs with Zipfian request distributions [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Impact of prefill-to-decode (P:D) ratio on power and energy usage. Panels (A) and (B): grouped by P:D ratio across varying request lengths. Panels (C) and (D): grouped by request length across varying P:D ratios [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Effect of batch size cap on power and energy metrics. (A) Actual vs. configured batch size. (B) Average GPU power draw. (C) Total energy usage [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Effect of query throughput (QPS) on power and energy metrics. (A) Average GPU power draw across QPS values. (B) Total energy usage for a fixed workload of 2 14 requests. increases approximately linearly with request volume, despite appearing expo￾nential due to the log…
Figure 6
Figure 6. Figure 6: Time-resolved power flow for the LLM inference workload. Solar generation (yellow) peaks midday and partially offsets the workload’s power demand (red). Net grid power (blue) reflects residual demand not met by solar, transitioning from negative (grid export) to positi…
Figure 7
Figure 7. Figure 7: Battery and emissions behavior during Vidur–Vessim co-simulation. Left: battery state of charge (SoC) trace and hourly SoC distribution. Right: cumulative carbon emissions, solar offset, net footprint, and grid carbon intensity trace over time with high and low carbon …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 2 citations worldwide. Full citation record

  1. LLM Harms: A Taxonomy and Discussion

    cs.CY 2025-12 unverdicted novelty 3.0 of 10

    This paper proposes a taxonomy of LLM harms in five categories and suggests mitigation strategies plus a dynamic auditing system for responsible development.

Reference graph

Works this paper leans on

29 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    In: USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI) (2024)

    Agrawal, A., et al.: Taming throughput-latency tradeoff in llm inference with sarathi-serve. In: USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI) (2024)

    work page 2024

  2. [2]

    In: Conference on Machine Learning and Systems (MLSys)

    Agrawal, A., et al.: Vidur: A large-scale simulation framework for llm inference. In: Conference on Machine Learning and Systems (MLSys). vol. 6 (2024)

    work page 2024

  3. [3]

    arXiv preprint arXiv:2007.03051 (2020)

    Anthony, L.F.W., et al.: Carbontracker: Tracking and predicting the carbon foot- print of training deep learning models. arXiv preprint arXiv:2007.03051 (2020)

    arXiv 2020

  4. [4]

    Chen, S., Lin, Y., Zhang, M., Wu, Y.: Efficient and economic large language model inferencewithattentionoffloading.arXivpreprintarXiv:2405.01814(2024), https: //arxiv.org/abs/2405.01814

    arXiv 2024

  5. [5]

    Journal of Machine Learning Research24(240) (2023)

    Chowdhery, A., et al.: Palm: Scaling language modeling with pathways. Journal of Machine Learning Research24(240) (2023)

    work page 2023

  6. [6]

    Zenodo (2024),https://doi.org/10.5281/zenodo.11171501

    Courty, B., Schmidt, V., Luccioni, A.S., Goyal-Kamal, A.: mlco2/codecarbon: v2.4.1. Zenodo (2024),https://doi.org/10.5281/zenodo.11171501

    doi:10.5281/zenodo.11171501 2024

  7. [7]

    Faiz, A., et al.: Llmcarbon: Modeling the end-to-end carbon footprint of large languagemodels.In:InternationalConferenceonLearningRepresentations(ICLR) (2024)

    work page 2024

  8. [8]

    arXiv preprint arXiv:2411.13055 (2024) 12 Özcan et al

    Fernandez, J., et al.: Hardware scaling trends and diminishing returns in large-scale distributed training. arXiv preprint arXiv:2411.13055 (2024) 12 Özcan et al

    arXiv 2024

Show all 29 references
  1. [9]

    arXiv preprint arXiv:2410.02950 (2024), https://arxiv.org/abs/2410.02950

    Fu, Z., Chen, F., Zhou, S., Li, H., Jiang, L.: Llmco2: Advancing accurate carbon footprint prediction for llm inferences. arXiv preprint arXiv:2410.02950 (2024), https://arxiv.org/abs/2410.02950

    2024 arXiv

  2. [10]

    White paper, Google (2024),https://datacenters.google/efficiency/

    Google LLC: Data center efficiency: Best practices and sustainability metrics. White paper, Google (2024),https://datacenters.google/efficiency/

    2024

  3. [11]

    Horizoniq: Nvidia a100 gpu deep learning benchmarks (2023), https:// www.horizoniq.com/blog/nvidia-a100-specs

    2023

  4. [12]

    In: USENIX Symposium on Networked Systems Design and Imple- mentation (NSDI) (2024)

    Jiang, Z., et al.: Megascale: Scaling large language model training to more than 10,000 gpus. In: USENIX Symposium on Networked Systems Design and Imple- mentation (NSDI) (2024)

    2024

  5. [13]

    In: ACM Symposium on Operating Systems Principles (SOSP) (2023)

    Kwon, W., et al.: Efficient memory management for llm serving with pagedatten- tion. In: ACM Symposium on Operating Systems Principles (SOSP) (2023)

    2023

  6. [14]

    In: 2024 Conference on Empirical Methods in Nat- ural Language Processing (EMNLP) (2024)

    Li, B., Jiang, Y., Gadepally, V., Tiwari, D.: Sprout: Green generative AI with carbon-efficient LLM inference. In: 2024 Conference on Empirical Methods in Nat- ural Language Processing (EMNLP) (2024)

    2024

  7. [15]

    arXiv preprint arXiv: 2502.05043 (2025)

    Li, Y., et al.: Ecoserve: Designing carbon-aware ai inference systems. arXiv preprint arXiv: 2502.05043 (2025)

    2025 arXiv

  8. [16]

    Megware: Nvidia h100 sxm5 specifications (2023), https://www.megware.com/ nvidia-h100

    2023

  9. [17]

    Neumann, O., Gros, C.: Alphazero neural scaling and zipf’s law: A tale of board games and power laws (2024),https://arxiv.org/abs/2412.11979, at ICLR 2025

    2024

  10. [18]

    NVIDIA: Nvidia a40 datasheet (2023), https://images.nvidia.com/content/ Solutions/data-center/a40/nvidia-a40-datasheet .pdf

    2023

  11. [19]

    In: ACM International Conference on Architectural Support for Program- ming Languages and Operating Systems (ASPLOS) (2024)

    Patel, P., et al.: Characterizing power management opportunities for llms in the cloud. In: ACM International Conference on Architectural Support for Program- ming Languages and Operating Systems (ASPLOS) (2024)

    2024

  12. [20]

    arXiv preprint arXiv:2104.10350 (2021)

    Patterson, D., et al.: Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 (2021)

    2021 arXiv

  13. [21]

    In: IEEE High Performance Extreme Computing Con- ference (HPEC) (2023)

    Samsi, S., et al.: From words to watts: Benchmarking the energy costs of large language model inference. In: IEEE High Performance Extreme Computing Con- ference (HPEC) (2023)

    2023

  14. [22]

    ServeTheHome: Asus rs720a e11-rs24u review – nvidia a100 power consump- tion (2023), https://www.servethehome.com/asus-rs720a-e11-rs24u-review- amd-epyc-7763-and-nvidia-a100-in-2u/

    2023

  15. [23]

    ServeTheHome: Nvidia a40 gpu mini review (2023), https:// www.servethehome.com/nvidia-a40-48gb-gpu-mini-review

    2023

  16. [24]

    Solcast: Global solar irradiance dataset (2022),https://solcast.com/

    2022

  17. [25]

    Trainy AI: Gpu utilization is misleading: Understanding gpu efficiency in ai work- loads (2024), https://trainy.ai/blog/gpu-utilization-misleading

    2024

  18. [26]

    WattTime: Caiso-north marginal operating emissions rate dataset (2023),https: //www.watttime.org/

    2023

  19. [27]

    In: Workshop on Sustainable Computer Systems (HotCarbon) (2024)

    Wiesner, P., et al.: Vessim: A testbed for carbon-aware applications and systems. In: Workshop on Sustainable Computer Systems (HotCarbon) (2024)

    2024

  20. [28]

    In: ACM International Conference on Future and Sustainable Energy Systems (e- Energy) (2025)

    Wiesner, P., et al.: Carbon-aware quality adaptation for energy-intensive services. In: ACM International Conference on Future and Sustainable Energy Systems (e- Energy) (2025)

    2025

  21. [29]

    In: Conference on Machine Learning and Systems (MLSys) (2022)

    Wu, C.J., et al.: Sustainable ai: Environmental implications, challenges and oppor- tunities. In: Conference on Machine Learning and Systems (MLSys) (2022)

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.