REVIEW 4 major objections 4 minor 20 references
Racing to Idle: Energy Efficiency of Matrix Multiplication on Heterogeneous CPU and GPU Architectures
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read On one consumer laptop, the discrete GPU multiplied two 4096x4096 matrices 93.5x faster than the CPU and consumed 50x less energy, demonstrating the race-to-idle principle.
desk verdict Qualitative GPU-beats-CPU finding is almost certainly right, but the specific 93.5x/50x ratios rest on a deliberately unoptimized CPU baseline that a tuned BLAS would erase. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the dense 4096x4096 single-precision GEMM kernel, used as a canonical compute-bound workload. The mechanism that carries the argument is the energy-to-solution identity $E = P \times t$ applied through the 'race to idle' principle: the architecture that finishes first and drops quickly to a low-power idle state minimizes total energy even when its active power draw is the highest. The supporting machinery is the measurement setup—Linux perf with the RAPL package-energy counter for CPU and iGPU, nvidia-smi polling for the dGPU, and event-based timing inside each benchmark.
What would settle it
Run the same 4096x4096 single-precision multiplication on the CPU with a cache-blocked kernel or a vendor-tuned GEMM library; if the CPU's time falls far below 57.34 s and its energy below 1417.89 J, the reported speedup and energy ratios no longer describe the architectures' true capabilities.
Extended reading notes
Core claim
In a head-to-head run of a 4096x4096 single-precision dense matrix-matrix multiplication (GEMM) on one laptop, the paper's measurements give the discrete NVIDIA GPU the win on both axes: 0.613 s and 28.33 J versus the CPU's 57.34 s and 1417.89 J, a 93.5x speedup and roughly 50x lower energy-to-solution. The integrated AMD GPU lands between at 2.10 s and 30.05 J. Average power tells the opposite story: the dGPU draws the most (46.22 W), the iGPU the least (14.31 W), with the CPU at 24.73 W. The authors interpret this as a demonstration that energy is the product of power and time, so a high-power unit that finishes fast and returns to idle can beat a low-power unit that keeps grinding.
Load-bearing premise
The paper assumes the three hand-written kernels in the appendix are representative of each processor's capability for this workload; the CPU baseline in particular is a naive loop with poor cache behavior, so the headline ratios depend on that choice.
Editorial extensions
If this is right
- For compute-bound dense linear algebra on consumer-class hardware, offloading to a discrete GPU is an energy-saving move, not just a speed-saving one.
- The highest-power component can be the most energy-efficient when it finishes first, so energy-aware scheduling should minimize time-to-idle rather than simply minimize wattage.
- The integrated GPU's low average power (14.31 W) does not make it the energy winner, because its longer runtime (2.10 s versus 0.613 s) raises its total energy above the dGPU's.
- The paper's 93.5x and 50x ratios are platform- and workload-specific; the authors explicitly note that integer-heavy or memory-bound tasks could change which architecture wins.
Reading between the lines
- Beyond the paper's single matrix size, the race-to-idle logic suggests the energy ranking could shift for smaller matrices, where kernel launch and data-transfer overhead make the dGPU's fast execution less dominant.
- Because the iGPU's energy was measured as whole-APU package energy while the CPU cores sat mostly idle, a direct iGPU power sensor would likely assign the iGPU even less energy, making its 30.05 J a conservative upper bound.
- An untested corollary is that splitting the matrix between the CPU and dGPU could beat the dGPU alone on energy for some sizes, trading a little runtime to keep the CPU cores useful rather than idle.
- The authors' data imply an easy extension: sweep matrix sizes and arithmetic intensities to find the crossover point where the CPU's moderate power or the iGPU's low power starts to beat the dGPU's race-to-idle.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a direct empirical comparison of execution time and energy-to-solution for a 4096x4096 single-precision dense matrix multiplication on three compute units in one consumer laptop: an AMD Ryzen 7 5800H CPU running OpenMP, an NVIDIA GTX 1650 dGPU running CUDA, and an AMD Radeon Vega iGPU running OpenCL. The authors report a 93.5x speedup and a 50x energy-efficiency improvement for the dGPU over the CPU (28.33 J vs. 1417.89 J), a 27.3x speedup for the iGPU, and interpret the results as a practical validation of the 'race to idle' principle. CPU energy is measured with RAPL via perf, dGPU energy by integrating nvidia-smi power samples, and iGPU energy from whole-package RAPL during an iGPU-offloaded run. The paper concludes that GPU offloading is an energy-conservation strategy for data-parallel workloads and that the discrete GPU is the optimal component on this platform for this workload.
Significance. The paper has a clear, reproducible experimental design with complete source code in Appendix A, uses standard tools (perf/RAPL, NVML), and directly measures time and energy rather than relying on models. The qualitative observation that a high-power but fast device can have lower energy-to-solution than a low-power but slow device is arithmetically sound and is a useful illustration of the 'race to idle' principle. However, the paper's headline quantitative claims—the 93.5x speedup and 50x energy advantage—are not established as architectural properties because the CPU baseline is a cache-hostile, unvectorized OpenMP loop that is not representative of the Ryzen 7 5800H's achievable SGEMM performance. The measurement protocol also has important gaps (low-rate dGPU power sampling with mismatched time bases, whole-package iGPU energy without idle subtraction, and no repeat runs). If the CPU baseline were replaced by a tuned BLAS implementation, the reported ratios would change materially and the energy ranking could even be affected. The paper is therefore a useful case study of naive implementations, but not yet a valid architecture-level comparison.
major comments (4)
- [Appendix A.1, Listing 1, lines 16-25] The CPU baseline is not representative of what the Ryzen 7 5800H can achieve for this workload. The inner loop reads element b[k*size + j] with a fixed column j, so the access pattern strides through memory with a stride of N elements; there is no tiling, register blocking, or vectorization beyond what -O3 provides, and the code is compiled without -march=native. At 57.34 s for 137.4 GFLOP, this is about 2.4 GFLOPS, whereas a tuned OpenBLAS or MKL SGEMM on the same CPU can plausibly sustain hundreds of GFLOPS and finish in well under a second. Because the abstract and Sections 4 and 5 present the 93.5x speedup and 50x energy-efficiency ratio as architectural findings, this single unoptimized implementation is load-bearing. The paper needs to rerun the CPU side with an optimized BLAS routine (or at least a properly cache-blocked and vectorized kernel) and report the resulting time and package energy, or explicitly reframe all quantitative claims as comparisons of the specific naive code versions rather than of the architectures. Until this is done, the central claim is not supported.
- [Section 3.4, dGPU Measurement] The dGPU power measurement uses nvidia-smi polling at 10 Hz, but the kernel runs for only 0.613 s, so the energy integral is based on at most about six power samples. The uncertainty of trapezoidal integration over such a short interval is not quantified. In addition, the 0.613 s time is measured with CUDA events and covers only the kernel, while the energy is presumably integrated over the wall-clock time of the surrounding shell-script invocation, which includes memory transfers and launch overhead. The paper should specify the exact sampling window, use a higher-rate NVML sampling loop (or measure device energy directly with CUDA/NVML APIs), and ensure that the time basis for the energy integral matches the reported execution time.
- [Section 3.4, iGPU Measurement and CPU Measurement] Both the CPU and iGPU energy values are obtained from the RAPL energy-pkg event, which measures the entire APU package, not the individual compute unit. For the iGPU run, the paper reports 30.05 J as the 'energy-to-solution' of the Radeon Vega, but this includes the CPU cores and uncore, which are described as 'largely idle' but are not measured or subtracted. The same issue affects the CPU measurement, which includes the iGPU and memory controller. The paper should report idle package power over the relevant interval and subtract it, or report the values as whole-package energy and discuss the implications for the dGPU comparison. Without this, the small energy margin between the dGPU (28.33 J) and iGPU (30.05 J) is not a reliable component-level comparison.
- [Section 4, Table 1] All results are single runs with no repeat measurements or error bars. The difference between the dGPU and iGPU energy-to-solution is only 1.72 J (28.33 vs. 30.05 J, about 6%), while the dGPU and iGPU times differ by 1.49 s. Run-to-run variance from thermal state, frequency scaling, and background activity could easily exceed this margin. The paper should report multiple repetitions with mean and standard deviation (or min/median) for time and energy, and state whether the observed dGPU/iGPU ranking is stable across runs.
minor comments (4)
- [Section 3.1] The compiler version is reported as g++ 12.4.0, but the kernel version and driver versions are listed; please also report the exact OpenMP runtime version and whether CPU frequency scaling (governor) was set to 'performance' or left at its default, since this affects both time and energy.
- [Appendix A] The code listings have numerous spacing artifacts (e.g., 'co ll aps e' in Listing 1, 'D u ra ti on' in Listing 1, and 'd ur ati on' in Listing 3) that appear to be formatting artifacts but would prevent direct copy-paste compilation. The authors should host the exact source files in a public repository and reference it in the paper.
- [Section 5.2] The statement 'Energy-to-solution is the product of average power and execution time (Energy = Power × Time)' is correct, but the paper should note that this identity is definitional rather than an empirical finding; the 'race to idle' explanation is a post-hoc interpretation, not a mechanism established by the measurements.
- [Section 4.2] The text says 'the dGPU consumed only 2% of the energy used by the CPU', but the table shows 28.33 J / 1417.89 J = 2.00%, so the claim is exact; please include the ratio in the table or a footnote for clarity.
Circularity Check
Direct empirical measurement with no fitted inputs; the race-to-idle explanation is an arithmetic identity applied post hoc, not a derived prediction.
full rationale
The paper's derivation chain consists of direct measurements (perf RAPL energy-pkg for CPU and package, NVML power.draw polling for dGPU), recorded execution times, and the standard identity Energy = Power x Time used to convert average power and duration into total energy. No quantity is defined in terms of the conclusion, and no parameter is fitted to a subset of data and then renamed a prediction. The 93.5x speedup and 50x energy ratios are arithmetic ratios of directly measured values, and the 'race to idle' discussion in Section 5.2 is an interpretation of those measurements rather than a premise built into them. The CPU benchmark in Appendix A.1 is cache-unfriendly and unvectorized relative to the GPU kernels, but this is a benchmark representativeness or fairness concern, not a circularity concern: it does not make the measured result equivalent to its inputs by construction, and no self-citation is load-bearing. Therefore no circular step can be exhibited under the required standard, and the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption The whole-package RAPL energy reading during the iGPU run isolates the iGPU's energy cost.
- domain assumption The hand-written kernels in Appendix A are fair representatives of each architecture's performance.
- domain assumption nvidia-smi power samples at 10 Hz can be integrated to give accurate energy over a kernel that lasts 0.6 seconds.
- domain assumption The RAPL energy-pkg event is valid on this AMD platform as characterized by the cited Intel RAPL validation.
Cite this review
Pith. "Pith review of Racing to Idle: Energy Efficiency of Matrix Multiplication on Heterogeneous CPU and GPU Architectures." pith.science (2026). https://pith.science/paper/L2RZO634
@misc{pith2026250720063,
author = {Pith},
title = {Pith review of: Racing to Idle: Energy Efficiency of Matrix Multiplication on Heterogeneous CPU and GPU Architectures},
year = {2026},
howpublished = {\url{https://pith.science/paper/L2RZO634}},
note = {Machine review of arXiv:2507.20063}
}
read the original abstract
The paradigm shift towards multi-core and heterogeneous computing, driven by the fundamental power and thermal limits of single-core processors, has established energy efficiency as a first-class design constraint in high-performance computing (HPC). Heterogeneous systems, integrating traditional multi-core CPUs with specialized accelerators like discrete (dGPU) and integrated (iGPU) graphics processing units, offer a compelling path to navigating the trade-offs between performance and power. However, quantifying these trade-offs on widely accessible hardware remains a critical area of study. This paper presents a direct, empirical measurement of the performance and energy-to-solution of a canonical HPC workload -- a 4096x4096 matrix-matrix multiplication -- on three distinct compute architectures within a single consumer-grade laptop: a multi-core AMD Ryzen 7 5800H CPU, a discrete NVIDIA GeForce GTX 1650 GPU, and an integrated AMD Radeon Vega GPU. Using standard, validated, and minimally intrusive tools such as Linux perf and nvidia-smi, we find that the discrete GPU is not only the performance leader, achieving a 93.5x speedup over the CPU, but is also the most energy-efficient, consuming only 2% of the energy used by the CPU, resulting in a 50-fold improvement in energy efficiency. These findings provide a practical demonstration of the "race to idle" principle and offer clear, quantitative guidance on architectural choices for energy-aware software development.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Powerapi: A python framework for building software-defined power meters
Guillaume Fieni, Daniel Romero Acero, Pierre Rust, and Romain Rouvoy. Powerapi: A python framework for building software-defined power meters. Journal of Open Source Software , 9(98):6670, 2024
work page 2024
-
[2]
Rapl in action: Experiences in using rapl for power measurements
Kashif Nizam Khan, Mikael Hirki, Tapio Niemi, Jukka K Nurminen, and Zhonghong Ou. Rapl in action: Experiences in using rapl for power measurements. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS) , 3(2):1–26, 2018
work page 2018
-
[3]
Issam Ra¨ ıs, Anne-C´ ecile Orgerie, Martin Quinson, and Laurent Lef` evre. Quantifying the impact of shut- down techniques for energy-efficient data centers.Concurrency and Computation: Practice and Experience, 30(17):e4471, 2018
work page 2018
-
[4]
The landscape of parallel computing research: A view from berkeley, 2006
Krste Asanovic, Ras Bodik, Bryan Catanzaro, Joseph Gebis, Parry Husbands, Kurt Keutzer, David Pat- terson, William Plishker, John Shalf, and Samuel Webb Williams. The landscape of parallel computing research: A view from berkeley, 2006. 12 M. Q. ANSARI AND M. Q. ANSARI
work page 2006
-
[5]
Performance and power analysis of hpc workloads on heterogeneous multi-node clusters
Filippo Mantovani and Enrico Calore. Performance and power analysis of hpc workloads on heterogeneous multi-node clusters. Journal of Low Power Electronics and Applications , 8(2):13, 2018
work page 2018
-
[6]
Evaluating the efficiency of cpus, gpus and fpgas on a near-duplicate document detection via opencl
Ercan Canhasi. Evaluating the efficiency of cpus, gpus and fpgas on a near-duplicate document detection via opencl. J. Comput. Sci. , 14(5):699–704, 2018
work page 2018
-
[7]
Energy-aware scheduling for high-performance com- puting systems: A survey
Bart lomiej Kocot, Pawe l Czarnul, and Jerzy Proficz. Energy-aware scheduling for high-performance com- puting systems: A survey. Energies, 16(2):890, 2023
work page 2023
-
[8]
The energy/frequency convexity rule: Modeling and experimental validation on mobile devices
Karel De Vogeleer, Gerard Memmi, Pierre Jouvelot, and Fabien Coelho. The energy/frequency convexity rule: Modeling and experimental validation on mobile devices. In International Conference on Parallel Processing and Applied Mathematics, pages 793–803. Springer, 2013
work page 2013
Show all 20 references
-
[9]
Energy-efficient computing: Innovations in hardware and software for sustainable advanced computing systems
Rania El-Khattab, Salma Fathy, and others. Energy-efficient computing: Innovations in hardware and software for sustainable advanced computing systems. Journal of Advanced Computing Systems , 3(11):1– 7, 2023
2023
-
[10]
A survey of methods for analyzing and improving gpu energy efficiency
Sparsh Mittal and Jeffrey S Vetter. A survey of methods for analyzing and improving gpu energy efficiency. ACM Computing Surveys (CSUR) , 47(2):1–23, 2014
2014
-
[11]
Automm: Energy-efficient multi-data-type matrix multiply design on heterogeneous programmable system-on-chip
Jinming Zhuang, Zhuoping Yang, and Peipei Zhou. Automm: Energy-efficient multi-data-type matrix multiply design on heterogeneous programmable system-on-chip. arXiv preprint arXiv:2305.18698 , 2023
2023 arXiv
-
[12]
Measuring the energy consumption and efficiency of deep neural networks: An empirical analysis and design recommendations
Charles Edison Tripp, Jordan Perr-Sauer, Jamil Gafur, Amabarish Nag, Avi Purkayastha, and Sagi Zisman, Erik A Bensen. Measuring the energy consumption and efficiency of deep neural networks: An empirical analysis and design recommendations. arXiv preprint arXiv:2403.08151 , 2024
2024 arXiv
-
[13]
An energy-aware runtime management of multi-core sensory swarms
Sungchan Kim and Hoeseok Yang. An energy-aware runtime management of multi-core sensory swarms. Sensors, 17(9):1955, 2017
1955
-
[14]
Analytical modeling of energy efficiency in heterogeneous processors.Computers & Electrical Engineering, 39(8):2566–2578, 2013
Ami Marowka. Analytical modeling of energy efficiency in heterogeneous processors.Computers & Electrical Engineering, 39(8):2566–2578, 2013
2013
-
[15]
Power and perfor- mance characterization and modeling of gpu-accelerated systems
Yuki Abe, Hiroshi Sasaki, Shinpei Kato, Koji Inoue, Masato Edahiro, and Martin Peres. Power and perfor- mance characterization and modeling of gpu-accelerated systems. In 2014 IEEE 28th international parallel and distributed processing symposium, pages 113–122. IEEE, 2014
2014
-
[16]
The green500 list: Encouraging sustainable supercomputing
Wu-chun Feng and Kirk Cameron. The green500 list: Encouraging sustainable supercomputing. Computer, 40(12):50–55, 2007
2007
-
[17]
Green computing-a new horizon of energy efficiency and electronic waste minimization: A global perspective
Shalabh Agarwal and Asoke Nath. Green computing-a new horizon of energy efficiency and electronic waste minimization: A global perspective. In 2011 International Conference on Communication Systems and Network Technologies, pages 688–693. IEEE, 2011
2011
-
[18]
A comprehensive survey on energy-efficient power management techniques
Ankit Thakkar, Kinjal Chaudhari, and Monika Shah. A comprehensive survey on energy-efficient power management techniques. Procedia Computer Science, 167:1189–1199, 2020
2020
-
[19]
An embedded gpu accelerated hyperspectral video classification system in real-time
Jaime Sancho, Manuel Villa, Gemma Urbanos, Marta Villanueva, Pallab Sutradhar, Gonzalo Rosa, Alberto Martin, Guillermo Vazquez, Miguel Chavarrias, Ruben Salvador, and others. An embedded gpu accelerated hyperspectral video classification system in real-time. In 2021 XXXVI Conf...
2021
-
[20]
Usage :
Vijay Kandiah, Scott Peverelle, Mahmoud Khairy, Junrui Pan, Amogh Manjunath, Timothy G Rogers, Tor M Aamodt, and Nikos Hardavellas. Accelwattch: A power modeling framework for modern gpus. In MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture , pages 7...
2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.