Pith. sign in

REVIEW 4 major objections 5 minor 7 cited by

Power Stabilization for AI Training Datacenters

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that synchronous AI training workloads produce megawatt-scale power oscillations whose frequency, not just amplitude, can overlap grid and turbine resonances and physically damage power infrastructure, and that stabilizing

desk verdict A genuinely useful problem statement on AI training power swings, but the grid-damage risk is extrapolated from a single rack trace and the simulator is a black box. read the letter →

arxiv 2508.14318 v2 pith:T3BKNKPD submitted 2025-08-20 cs.AR cs.AIcs.DC

Esha Choukse , Brijesh Warrier , Scot Heath , Luz Belmont , April Zhao , Hassan Ali Khan , Brian Harry , Matthew Kappel
show 49 more authors
Russell J. Hewett Kushal Datta Yu Pei Caroline Lichtenberger John Siegler David Lukofsky Zaid Kahn Gurpreet Sahota Andy Sullivan Charles Frederick Hien Thai Rebecca Naughton Daniel Jurnove Justin Harp Reid Carper Nithish Mahalingam Srini Varkala Alok Gautam Kumbhare Satyajit Desai Venkatesh Ramamurthy Praneeth Gottumukkala Girish Bhatia Kelsey Wildstone Laurentiu Olariu Ileana Incorvaia Alex Wetmore Prabhat Ram Melur Raghuraman Mohammed Ayna Mike Kendrick Ricardo Bianchini Aaron Hurst Reza Zamani Xin Li Michael Petrov Gene Oden Rory Carmichael Tom Li Apoorv Gupta Pratikkumar Patel Nilesh Dattani Lawrence Marwong Rob Nertney Hirofumi Kobayashi Jeff Liott Miro Enev Divya Ramakrishnan Ian Buck Jonah Alben
This is my paper · ORCID
classification cs.ARcs.AIcs.DC
keywords powerstabilizationAItrainingdatacentersbulksynchronousparallelsub-synchronousresonanceGPUsmoothingminimumfloorrack-levelenergystoragegridfrequencyspectrum
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that large synchronous AI training jobs, in which tens of thousands of GPUs alternate between compute-heavy and communication-heavy phases, produce power swings of tens to hundreds of megawatts. The key claim is that the frequency of these swings, not just their size, is the danger: the oscillations fall in the 0.2–3 Hz range, overlapping resonant modes of turbine-generator shafts and transmission lines, so a large enough synchronized load can physically damage grid equipment. To keep scaling training clusters, the paper proposes stabilizing power draw through a combination of software filler workloads, GPU-level power ramping and floors, and rack-level energy storage. It evaluates these options on production waveform data plus simulations, and argues that no single layer is sufficient; a coordinated mix, together with a fast telemetry backstop, is required.

What carries the argument

The central object is the GPU power waveform as a periodic signal from the bulk-synchronous training loop, analyzed by FFT against utility resonance specifications. The load-bearing identity is the mapping between iteration cadence and frequency: an iteration that repeats every T seconds emits power at 1/T Hz plus harmonics, and when those bins fall inside a utility's critical frequency range, the synchronized megawatt-scale amplitude can excite resonance. The paper's mitigation machinery is a three-layer stack: software filler workloads that smooth the troughs, GPU firmware that enforces ramp-up/ramp-down rates and a minimum power floor, and rack-level energy storage that charges during com

What would settle it

Measure the aggregate power waveform of a large training cluster while sweeping iteration cadence across 0.1–20 Hz, and compare the FFT magnitude at each bin against the grid's known resonant modes; the claim predicts amplification at resonant bins that a damping-only model would not. In parallel, compare the simulator's predicted 10.5% energy overhead for a 90% power floor against the measured overhead on a real GPU rack; a discrepancy of more than a few percent would require rebalancing the mitigation stack.

Watch

Extended reading notes

Core claim

The paper establishes that the power draw of a GPU training cluster is not a random fluctuation but a periodic, square-wave-like signal whose fundamental frequency is set by the iteration time of the bulk-synchronous training loop. At hyperscale, the amplitude of this signal is large enough that its spectral components overlap known sub-synchronous resonance bands of the power grid, including transmission-network modes below 1 Hz and shaft torsional frequencies from roughly 7 Hz to over 100 Hz. The paper therefore treats power stabilization as a frequency-domain engineering requirement: a utility specifies a critical frequency range and a maximum allowed spectral magnitude, and the datacente

Load-bearing premise

The load-bearing premise is that the paper's in-house simulator reproduces the real GPU training power waveform accurately enough that its predicted energy overheads and battery-tracking behavior hold on physical hardware; if the simulator misses fast power transients, the recommended mix of mitigations would need to change.

Editorial extensions

If this is right

  • Left unshaped, larger training clusters will push aggregate swing amplitudes at resonance frequencies high enough to risk turbine shaft fatigue or breaker trips, so scaling AI hinges on power shaping, not just cooling.
  • GPU power-floor firmware alone leaves at least 20% of TDP as dynamic range under current hardware limits, so tight utility specs require an additional filler or storage layer.
  • Software filler keeps primary-workload slowdown under 5% but needs low-latency counters and close cloud-provider collaboration, and it wastes energy unless the filler does real work.
  • Rack-level storage is the only mitigation that avoids net energy waste and can shave peak demand, but it needs large capacitance and adds cost and embodied carbon.
  • The paper's recommended design combines GPU-level smoothing and rack-level storage with state-of-charge signaling, backed by a fast telemetry system that watches for residual resonant bins.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The frequency-domain argument extends beyond training: any periodic bulk-synchronous workload, such as synchronized inference batching or fixed-interval checkpointing, emits the same kind of spectral lines and should be shaped by the same stack.
  • Operators can convert the forced energy burn of a power floor into useful work by co-locating lower-priority training or data-preprocessing jobs in the filler window, turning the paper's wasted-energy trade-off into a scheduling problem.
  • A direct test of the resonance claim would sweep a cluster's iteration cadence across the critical band while monitoring substation voltage and current, measuring whether the grid's damping actually suppresses the excited modes.
  • If utilities standardize frequency-domain specs, GPU vendors could add per-GPU phase staggering of ramp events so that cluster-level spectral energy is spread instead of concentrated—a control the paper mentions only as software ramp staggering.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper argues that power draw during large-scale synchronous AI training is highly oscillatory—compute phases near TDP alternating with communication phases near idle—and that these oscillations, if aligned with utility or turbine resonant frequencies, could damage grid infrastructure. It supports this with a production GPU power trace and its FFT, then discusses three mitigation classes: software-only smoothing (Firefly), GPU-level power smoothing via a Minimum Power Floor (MPF), and rack-level energy storage. Quantitative evidence includes a GB200 square-wave experiment, StratoSim simulations of the production trace (reporting a 10.5% energy overhead at MPF=90%), and a simulated rack-storage waveform. The paper concludes with a recommended combination of GPU-level smoothing and rack-level storage, plus a fast-telemetry backstop.

Significance. If the central grid-scale claim is correct, this is an important early industry report on a systemic risk in AI infrastructure. The paper has real strengths: it presents an actual production power trace (Figure 1), identifies a 0.2–3 Hz spectral concentration (Figure 3), includes a genuine GB200 hardware measurement (Figure 5), and is candid about limitations (MPF ceiling, lifetime counter, dependence on StratoSim). However, the quantitative core is not yet self-contained: the extrapolation from a rack-level trace to 'tens of megawatts' at the grid interface is not supported by aggregate measurements or a phase-coherence model, and the simulator used for the headline energy-overhead number is not described or validated. The contribution is therefore a plausible and practically motivated position paper with initial measurements, rather than a fully supported quantitative evaluation.

major comments (4)
  1. [§II-C, §II-E, Fig. 1] The central motivation assumes that per-GPU compute/communication power dips add coherently across tens of thousands of GPUs and remain unattenuated at the datacenter or substation interface. Figure 1 shows a single normalized rack-level trace; no aggregate measurement at the row, datacenter, PDU, or substation level is provided. The 'tens of megawatts' statement in §II-C cites a Supermicro case study [20], not a measurement of this synchronization phenomenon, and the NERC 2019 event described in §II-E (approximately 200 MW) is not attributed to computing loads. Without a phase-coherence model or aggregate measurements, the grid-damage premise is unsupported. Please add aggregate data or an explicit analysis of phase alignment and cancellation, or substantially weaken the claims to rack-level observations and conditional risk.
  2. [§IV-B, Figs. 6–7] StratoSim is introduced as 'Microsoft's in-house power simulator' with no model description, input parameters, assumptions, or validation against the GB200 hardware or a real training waveform. The 10.5% energy overhead at MPF=90% and the simulated rack-storage waveform in Figure 7 are central quantitative results that feed directly into the solution comparison in Table I. Because the simulator is unspecified, these numbers cannot be reproduced or independently assessed. Please provide a detailed description and validation of StratoSim, or explicitly label the simulated results as illustrative and include a sensitivity analysis over the key unknown parameters (e.g., battery round-trip efficiency, ramp tracking delay, MPF setting).
  3. [§III-A, §III-B, §IV] The frequency-domain specification—0.1–20 Hz critical range and a 20% cap on total harmonic energy—is stated as 'typical' without citation or derivation, and no evaluation in Section IV demonstrates that any proposed mitigation actually meets such a spec. Figure 5 shows time-domain ramping only; Figure 6 shows the simulated smoothed waveform but provides no FFT or spectral comparison of original versus mitigated traces. Given the paper's own emphasis on frequency-domain resonance risk, the absence of spectral analysis of the mitigated outputs is a load-bearing omission. Please add spectral plots or numerical band-energy metrics for the MPF-smoothed and storage-smoothed waveforms, or explicitly state that frequency-domain compliance is not yet demonstrated.
  4. [§II-D, §II-E, Ref. [11]] The text states that 'multiple utility providers have now documented the impact of harmonics induced by synchronized computing loads' and cites [11] (NERC 2019). As listed, [11] is a general oscillatory-events analysis; it does not directly document computing-load-induced harmonics. Please cite the specific sections or reports that support this claim, or qualify the statement so that it reflects what the reference actually says. This matters because the paper's urgency rests on the existence of utility-observed incidents, not merely on theoretical resonance mechanisms.
minor comments (5)
  1. [Fig. 5] The y-axis is normalized, but the text reports a 65% TDP power floor. Please label the axes with the relevant percentage and clarify whether the square wave is the requested power target or the measured GPU power.
  2. [§IV-B] The phrase 'a minimum EDP of 1.1×of TDP' should read '1.1×TDP.' Also, the calculation leading to 'at least 20% of TDP' dynamic range could be shown explicitly (0.9 TDP to 1.1 TDP).
  3. [Table I] The qualitative High/Medium/Low ratings lack a defined rubric and are not tied to the preceding experiments. Consider adding a short methodology note or a quantitative evidence column so the comparison is transparent.
  4. [§III-B] The statement that damping-ratio values 'much greater than 1 are desirable' is imprecise; in oscillation analysis, damping ratios in the 0.5–0.7 range are typically considered well-damped, while overdamped systems have different trade-offs. Please reword.
  5. [§IV-A] The Firefly description reports '<5%' primary-workload overhead and 100% TDP utilization, but no experimental setup, repetition count, or measurement uncertainty is given. Please add a brief methodology paragraph or point to a public artifact.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the central claims are empirically motivated and the simulation results are not fitted predictions.

full rationale

The paper does not present a derivation chain in which outputs reduce to inputs. The central claim—that synchronous AI training workloads produce large, grid-relevant power swings—is supported by real telemetry (Figure 1) and external references (NERC, EPRI, GE), not by the paper's own conclusions. The 10.5% energy overhead at MPF=90% is a StratoSim simulation outcome conditional on a chosen floor, not a target that the simulator was tuned to hit; there is no fitted parameter renamed as a prediction. Similarly, the rack-storage waveform in Figure 7 is a simulated example under stated assumptions. The only clear self-citation is [17], used to support the background fact that GPUs draw near TDP during compute phases; this is also visible in Figure 1 and is not load-bearing. The paper explicitly acknowledges limitations (the microbenchmark is not representative, StratoSim is in-house, and hardware endurance is unproven), which are validation gaps rather than circularity. No uniqueness theorem is imported, no ansatz is smuggled via citation, and no known result is merely renamed. The unvalidated StratoSim black box is a correctness/evidence concern, not a circularity concern. Overall, the paper is self-contained against its stated evidence and earns a low circularity score.

Assumptions & free parameters 1 free parameters · 6 assumptions · 1 invented entities

The central quantitative claims depend on domain assumptions about grid resonance coupling and simulator fidelity, plus hand-chosen settings like MPF. No derivation is fitted to data, so circularity burden is low, but the unvalidated simulator is the largest hidden assumption.

free parameters (1)
  • Minimum Power Floor (MPF) = 65% TDP in Fig. 5; 90% TDP in Fig. 6
    Hand-chosen control setting that directly determines the reported energy overhead (10.5%) and the dynamic-range feasibility conclusions.
assumptions (6)
  • domain assumption Bulk synchronous training produces compute/communication power swings
    Section II-B/C, based on cited parallelism literature and Figure 1; not independently derived.
  • domain assumption These power swings can excite grid resonant frequencies and damage equipment
    Section II-D/E relies on NERC, EPRI, and GE reports; the paper does not directly measure grid impact.
  • domain assumption Utility specifications can be represented as ramp rates, dynamic power range, and frequency caps
    Section III-A gives example values (e.g., 20% spectral cap, 0.1-20 Hz) that are illustrative, not actual utility contracts.
  • domain assumption StratoSim accurately simulates real datacenter power behavior
    Section IV-B uses StratoSim for the main quantitative results but provides no validation against hardware for training waveforms.
  • domain assumption NVIDIA GB200 power smoothing operates as described (MPF, stop delay, EDP constraints)
    Section IV-B, based on hardware access and an NVIDIA tuning guide; the feature's endurance and limits are only partially characterized.
  • domain assumption Rack-level energy storage can charge and discharge fast enough to smooth the swings
    Section IV-C asserts feasibility via simulation, not via deployed hardware measurements.
invented entities (1)
  • Firefly software power-smoothing system
    purpose: Injects secondary GEMM workloads when GPU activity drops below a threshold, to keep power draw flat during communication phases.
    A new in-house software system; the paper reports it can raise utilization toward 100% of TDP, but no independent or external implementation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Power Stabilization for AI Training Datacenters." pith.science (2026). https://pith.science/paper/T3BKNKPD

@misc{pith2026250814318,
  author       = {Pith},
  title        = {Pith review of: Power Stabilization for AI Training Datacenters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T3BKNKPD}},
  note         = {Machine review of arXiv:2508.14318}
}
read the original abstract

Large Artificial Intelligence (AI) training workloads spanning several tens of thousands of GPUs present unique power management challenges. These arise due to the high variability in power consumption during the training. Given the synchronous nature of these jobs, during every iteration there is a computation-heavy phase, where each GPU works on the local data, and a communication-heavy phase where all the GPUs synchronize on the data. Because compute-heavy phases require much more power than communication phases, large power swings occur. The amplitude of these power swings is ever increasing with the increase in the size of training jobs. An even bigger challenge arises from the frequency spectrum of these power swings which, if harmonized with critical frequencies of utilities, can cause physical damage to the power grid infrastructure. Therefore, to continue scaling AI training workloads safely, we need to stabilize the power of such workloads. This paper introduces the challenge with production data and explores innovative solutions across the stack: software, GPU hardware, and datacenter infrastructure. We present the pros and cons of each of these approaches and finally present a multi-pronged approach to solving the challenge. The proposed solutions are rigorously tested using a combination of real hardware and Microsoft's in-house cloud power simulator, providing critical insights into the efficacy of these interventions under real-world conditions.

Figures

Figures reproduced from arXiv: 2508.14318 by the authors.

Figure 1
Figure 1. Power readings from an at-scale training job on DGX-H100 racks. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. GB200 server power breakdown. C. Power consumption during training The power draw of GPUs can rapidly swing as the application transitions between compute and communication phases ( [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Frequency components of the power waveform shown in Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Specification for power stabilization in time domain. PL denotes the device TDP here, and floor denotes MPF. Ramp-up and ramp-down times [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: GB200 Power smoothing results with a square-wave microbench [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Power smoothing to the minimum power floor (MPF) simulated on the training waveform from Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Energy-storage solution simulated on the power waveform from Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bit2Watt: A Cyber-Physical Vulnerability Exploiting GPU Workloads Across Power and Computing Infrastructures

    cs.CR 2026-07 conditional novelty 7.0 of 10

    Coordinated GPU workload manipulation by unprivileged cloud tenants can induce high-frequency power modulations that destabilize inverter-dominated grids, causing harmonic distortion, negative damping, and potential c...

  2. Do Co-Located AI Training Jobs Synchronize? Load-Dependent Throttling as a Coupling Mechanism for Phase-Locking Behind a Shared Power Cap

    eess.SY 2026-07 conditional novelty 6.0 of 10

    Training jobs sharing a power cap couple like phase oscillators through load-dependent throttling; the coupling is repulsive at fast control delays and attractive beyond half an iteration period, allowing coherent N-s...

  3. The Welfare Gap of Strategic Storage: Universal Bounds and Price Non-Linearity

    cs.GT 2026-02 conditional novelty 6.0 of 10

    The price of anarchy of battery storage is exactly 4/3 for linear prices, unbounded for general convex prices, and between e/(e−1) and 2 for monomial prices.

  4. A Phased Development Framework Enabling Islanded Operation of Sustainable AI Data Centers With Onsite Grid-Following and Grid-Forming Energy Architectures

    eess.SY 2026-07 conditional novelty 4.0 of 10

    An islanded-first, phased construction framework for AI data centers — on-site gas turbines plus grid-forming batteries until grid interconnection matures — is shown via EMT simulation to track 300 MW AI training load swings.

  5. Evaluating Grid Resilience in the Era of Ever-Increasing Data Centers

    eess.SY 2026-07 conditional novelty 3.0 of 10

    Replacing a conventional load with energy-matched and scaled data center demand at a contingency-exposed bus in an IEEE 30-bus system increases unserved energy under transmission-constrained contingencies, with disrup...

  6. HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB images

    cs.CV 2025-08 unverdicted novelty 3.0 of 10

    HOSt3R claims keypoint-free, template-free, intrinsics-free hand-object 3D reconstruction from RGB video with SOTA on SHOWMe, but the manuscript body is a different, unrelated power-systems paper.

  7. Electricity Demand and Grid Impacts of AI Data Centers: Challenges and Prospects

    eess.SY 2025-09 conditional novelty 2.0 of 10

    A review paper synthesizes evidence that AI data center electricity demand is large, bursty, and power-electronics-dominated, creating multi-timescale grid challenges.

Reference graph

Works this paper leans on

23 extracted references · 20 canonical work pages · cited by 7 Pith papers

  1. [20]

    Inside the 100k gpu xai colos- sus cluster that supermicro helped build for elon musk

    Super Micro Computer, Inc. Inside the 100k gpu xai colos- sus cluster that supermicro helped build for elon musk. Case study / success story, Super Micro Computer, Inc., Decem- ber 2024.https://www.supermicro.com/CaseStudies/ Success_Story_xAI_Colossus_Cluster.pdf

    work page 2024

  2. [11]

    Disturbance monitoring and analysis of oscillatory events.https://www

    North American Electric Reliability Corporation (NERC). Disturbance monitoring and analysis of oscillatory events.https://www. nerc.com, 2019. [Online; accessed 2025-08-06]

    work page 2019

  3. [1]

    Standard for voltage flicker and power swing limitations

    Iec 61000-3-3:2013 — electromagnetic compatibility (emc), 2013. Standard for voltage flicker and power swing limitations

    work page 2013

  4. [2]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwi...

    2020

  5. [3]

    Petrafikowski

    Aakanksha Chowdhery, Sharan Narang, ..., Jason Wei, and [and many others] ... Petrafikowski. Palm: Scaling language modeling with pathways. InJMLR Workshop and Conference Proceedings, 2023. 540-billion-parameter model

    work page 2023

  6. [4]

    Torsional dynamics: Large 2-pole and 4-pole steam turbine powertrains

    General Electric Company. Torsional dynamics: Large 2-pole and 4-pole steam turbine powertrains. Technical report (ger-4724), GE Power & Water, 2013. Based on EPRI 1011679, Electric Power Research Institute, 2005

    work page 2013

  7. [5]

    Deepseek-v3 technical report

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, ..., and many others. Deepseek-v3 technical report. Technical report, DeepSeek-AI / CoRR, December 2024. Mixture-of-Experts language model with 671 B parameters (37 B activated per token)

    work page 2024

  8. [6]

    Torsional interaction between electrical network phenomena and turbine-generator shafts: Plant vulnerability

    Electric Power Research Institute. Torsional interaction between electrical network phenomena and turbine-generator shafts: Plant vulnerability. Technical Report 1013460, EPRI, Palo Alto, CA, 2006

    work page 2006

Show all 23 references
  1. [7]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. InAdvances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012

    2012

  2. [8]

    Asynchronous decentralized parallel stochastic gradient descent

    Xiangru Lian, Wei Zhang, Ce Zhang, and Ji Liu. Asynchronous decentralized parallel stochastic gradient descent. In Jennifer G. Dy and Andreas Krause, editors,Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80 ofProceedings of Machine Learnin...

    2018

  3. [9]

    How meta keeps its ai hardware reliable

    Meta Engineering. How meta keeps its ai hardware reliable. Engi- neering blog, July 2025. Accessed: 2025-08-06

    2025

  4. [10]

    Phi-3: A highly capable small language model locally on your phone

    Microsoft Azure AI Team. Phi-3: A highly capable small language model locally on your phone. Technical report, Microsoft, April 2024. Introduced in Microsoft Azure AI blog; technical report available on arXiv

    2024

  5. [12]

    Intercon- nection oscillation analysis

    North American Electric Reliability Corporation (NERC). Intercon- nection oscillation analysis. Reliability assessment technical report, North American Electric Reliability Corporation, July 2019. Report published July 2019; includes analysis of inter-area oscillations, no- ta...

    2019

  6. [13]

    NVIDIA, April 2025

    NVIDIA.NVIDIA GB200 NVL Multi-Node Tuning Guide — Power and Thermals. NVIDIA, April 2025. Provides GPU power and thermal management tuning for data center systems

    2025

  7. [14]

    Nvidia collective communications library (nccl).https://developer.nvidia.com/nccl, 2025

    NVIDIA Corporation. Nvidia collective communications library (nccl).https://developer.nvidia.com/nccl, 2025. Ac- cessed: 2025-08-06

    2025

  8. [15]

    Scaling kubernetes to 7,500 nodes.https://openai

    OpenAI. Scaling kubernetes to 7,500 nodes.https://openai. com/index/scaling-kubernetes-to-7500-nodes/, Jan- uary 2021

    2021

  9. [16]

    Techniques for training large neu- ral networks.https://openai.com/index/ techniques-for-training-large-neural-networks/, June 2022

    OpenAI. Techniques for training large neu- ral networks.https://openai.com/index/ techniques-for-training-large-neural-networks/, June 2022

    2022

  10. [17]

    Characterizing power management opportunities for llms in the cloud

    Pratyush Patel, Esha Choukse, Chaojie Zhang, ´I˜nigo Goiri, Brijesh Warrier, Nithish Mahalingam, and Ricardo Bianchini. Characterizing power management opportunities for llms in the cloud. InProceedings of the 29th ACM International Conference on Architectural Support for Prog...

    2024

  11. [18]

    Pilz, James Sanders, Robi Rahman, and Lennart Heim

    Konstantin F. Pilz, James Sanders, Robi Rahman, and Lennart Heim. Trends in ai supercomputers, 2025

    2025

  12. [19]

    Megatron-lm: Training multi-billion parameter language models using model parallelism

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGres- ley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019.http://arxiv.org/ abs/1909.08053

    1909 arXiv

  13. [21]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, ..., and Guillaume Lample. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  14. [22]

    L. Wang. Review of emerging ssr/sso issues and their classifications. Journal of Operational Engineering (JOE), 2017. Online

    2017

  15. [23]

    Open release of grok-1: A 314b parameter mixture-of-experts model

    xAI. Open release of grok-1: A 314b parameter mixture-of-experts model. Web page, 2024. Released March 17, 2024

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.