Pith. sign in

REVIEW 4 major objections 4 minor 20 references

Racing to Idle: Energy Efficiency of Matrix Multiplication on Heterogeneous CPU and GPU Architectures

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read On one consumer laptop, the discrete GPU multiplied two 4096x4096 matrices 93.5x faster than the CPU and consumed 50x less energy, demonstrating the race-to-idle principle.

desk verdict Qualitative GPU-beats-CPU finding is almost certainly right, but the specific 93.5x/50x ratios rest on a deliberately unoptimized CPU baseline that a tuned BLAS would erase. read the letter →

arxiv 2507.20063 v1 pith:L2RZO634 submitted 2025-07-26 cs.DC cs.CC

classification cs.DCcs.CC MSC 68W1065Y0568M20
keywords High-PerformanceComputingGPUCUDAOpenMPMatrixMultiplicationParallelEnergyEfficiencyHeterogeneousSystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper measures the time and energy a single consumer laptop needs to multiply two 4096x4096 single-precision matrices on its three compute units: an 8-core AMD CPU, a discrete NVIDIA GPU, and an integrated AMD GPU. The central claim is that the discrete GPU wins on both metrics, finishing in 0.613 seconds versus 57.34 seconds for the CPU (a 93.5x speedup) and consuming 28.33 joules versus 1417.89 joules (roughly a 50x energy-efficiency advantage). The explanation offered is the 'race to idle' principle: the dGPU draws the most power while active (46.22 W) but returns to idle so quickly that its total energy is the lowest. The practical point is that for a compute-bound kernel like matrix multiplication, hardware choice can move energy use by about two orders of magnitude.

What carries the argument

The central object is the dense 4096x4096 single-precision GEMM kernel, used as a canonical compute-bound workload. The mechanism that carries the argument is the energy-to-solution identity $E = P \times t$ applied through the 'race to idle' principle: the architecture that finishes first and drops quickly to a low-power idle state minimizes total energy even when its active power draw is the highest. The supporting machinery is the measurement setup—Linux perf with the RAPL package-energy counter for CPU and iGPU, nvidia-smi polling for the dGPU, and event-based timing inside each benchmark.

What would settle it

Run the same 4096x4096 single-precision multiplication on the CPU with a cache-blocked kernel or a vendor-tuned GEMM library; if the CPU's time falls far below 57.34 s and its energy below 1417.89 J, the reported speedup and energy ratios no longer describe the architectures' true capabilities.

Watch

Extended reading notes

Core claim

In a head-to-head run of a 4096x4096 single-precision dense matrix-matrix multiplication (GEMM) on one laptop, the paper's measurements give the discrete NVIDIA GPU the win on both axes: 0.613 s and 28.33 J versus the CPU's 57.34 s and 1417.89 J, a 93.5x speedup and roughly 50x lower energy-to-solution. The integrated AMD GPU lands between at 2.10 s and 30.05 J. Average power tells the opposite story: the dGPU draws the most (46.22 W), the iGPU the least (14.31 W), with the CPU at 24.73 W. The authors interpret this as a demonstration that energy is the product of power and time, so a high-power unit that finishes fast and returns to idle can beat a low-power unit that keeps grinding.

Load-bearing premise

The paper assumes the three hand-written kernels in the appendix are representative of each processor's capability for this workload; the CPU baseline in particular is a naive loop with poor cache behavior, so the headline ratios depend on that choice.

Editorial extensions

If this is right

  • For compute-bound dense linear algebra on consumer-class hardware, offloading to a discrete GPU is an energy-saving move, not just a speed-saving one.
  • The highest-power component can be the most energy-efficient when it finishes first, so energy-aware scheduling should minimize time-to-idle rather than simply minimize wattage.
  • The integrated GPU's low average power (14.31 W) does not make it the energy winner, because its longer runtime (2.10 s versus 0.613 s) raises its total energy above the dGPU's.
  • The paper's 93.5x and 50x ratios are platform- and workload-specific; the authors explicitly note that integer-heavy or memory-bound tasks could change which architecture wins.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's single matrix size, the race-to-idle logic suggests the energy ranking could shift for smaller matrices, where kernel launch and data-transfer overhead make the dGPU's fast execution less dominant.
  • Because the iGPU's energy was measured as whole-APU package energy while the CPU cores sat mostly idle, a direct iGPU power sensor would likely assign the iGPU even less energy, making its 30.05 J a conservative upper bound.
  • An untested corollary is that splitting the matrix between the CPU and dGPU could beat the dGPU alone on energy for some sizes, trading a little runtime to keep the CPU cores useful rather than idle.
  • The authors' data imply an easy extension: sweep matrix sizes and arithmetic intensities to find the crossover point where the CPU's moderate power or the iGPU's low power starts to beat the dGPU's race-to-idle.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper reports a direct empirical comparison of execution time and energy-to-solution for a 4096x4096 single-precision dense matrix multiplication on three compute units in one consumer laptop: an AMD Ryzen 7 5800H CPU running OpenMP, an NVIDIA GTX 1650 dGPU running CUDA, and an AMD Radeon Vega iGPU running OpenCL. The authors report a 93.5x speedup and a 50x energy-efficiency improvement for the dGPU over the CPU (28.33 J vs. 1417.89 J), a 27.3x speedup for the iGPU, and interpret the results as a practical validation of the 'race to idle' principle. CPU energy is measured with RAPL via perf, dGPU energy by integrating nvidia-smi power samples, and iGPU energy from whole-package RAPL during an iGPU-offloaded run. The paper concludes that GPU offloading is an energy-conservation strategy for data-parallel workloads and that the discrete GPU is the optimal component on this platform for this workload.

Significance. The paper has a clear, reproducible experimental design with complete source code in Appendix A, uses standard tools (perf/RAPL, NVML), and directly measures time and energy rather than relying on models. The qualitative observation that a high-power but fast device can have lower energy-to-solution than a low-power but slow device is arithmetically sound and is a useful illustration of the 'race to idle' principle. However, the paper's headline quantitative claims—the 93.5x speedup and 50x energy advantage—are not established as architectural properties because the CPU baseline is a cache-hostile, unvectorized OpenMP loop that is not representative of the Ryzen 7 5800H's achievable SGEMM performance. The measurement protocol also has important gaps (low-rate dGPU power sampling with mismatched time bases, whole-package iGPU energy without idle subtraction, and no repeat runs). If the CPU baseline were replaced by a tuned BLAS implementation, the reported ratios would change materially and the energy ranking could even be affected. The paper is therefore a useful case study of naive implementations, but not yet a valid architecture-level comparison.

major comments (4)
  1. [Appendix A.1, Listing 1, lines 16-25] The CPU baseline is not representative of what the Ryzen 7 5800H can achieve for this workload. The inner loop reads element b[k*size + j] with a fixed column j, so the access pattern strides through memory with a stride of N elements; there is no tiling, register blocking, or vectorization beyond what -O3 provides, and the code is compiled without -march=native. At 57.34 s for 137.4 GFLOP, this is about 2.4 GFLOPS, whereas a tuned OpenBLAS or MKL SGEMM on the same CPU can plausibly sustain hundreds of GFLOPS and finish in well under a second. Because the abstract and Sections 4 and 5 present the 93.5x speedup and 50x energy-efficiency ratio as architectural findings, this single unoptimized implementation is load-bearing. The paper needs to rerun the CPU side with an optimized BLAS routine (or at least a properly cache-blocked and vectorized kernel) and report the resulting time and package energy, or explicitly reframe all quantitative claims as comparisons of the specific naive code versions rather than of the architectures. Until this is done, the central claim is not supported.
  2. [Section 3.4, dGPU Measurement] The dGPU power measurement uses nvidia-smi polling at 10 Hz, but the kernel runs for only 0.613 s, so the energy integral is based on at most about six power samples. The uncertainty of trapezoidal integration over such a short interval is not quantified. In addition, the 0.613 s time is measured with CUDA events and covers only the kernel, while the energy is presumably integrated over the wall-clock time of the surrounding shell-script invocation, which includes memory transfers and launch overhead. The paper should specify the exact sampling window, use a higher-rate NVML sampling loop (or measure device energy directly with CUDA/NVML APIs), and ensure that the time basis for the energy integral matches the reported execution time.
  3. [Section 3.4, iGPU Measurement and CPU Measurement] Both the CPU and iGPU energy values are obtained from the RAPL energy-pkg event, which measures the entire APU package, not the individual compute unit. For the iGPU run, the paper reports 30.05 J as the 'energy-to-solution' of the Radeon Vega, but this includes the CPU cores and uncore, which are described as 'largely idle' but are not measured or subtracted. The same issue affects the CPU measurement, which includes the iGPU and memory controller. The paper should report idle package power over the relevant interval and subtract it, or report the values as whole-package energy and discuss the implications for the dGPU comparison. Without this, the small energy margin between the dGPU (28.33 J) and iGPU (30.05 J) is not a reliable component-level comparison.
  4. [Section 4, Table 1] All results are single runs with no repeat measurements or error bars. The difference between the dGPU and iGPU energy-to-solution is only 1.72 J (28.33 vs. 30.05 J, about 6%), while the dGPU and iGPU times differ by 1.49 s. Run-to-run variance from thermal state, frequency scaling, and background activity could easily exceed this margin. The paper should report multiple repetitions with mean and standard deviation (or min/median) for time and energy, and state whether the observed dGPU/iGPU ranking is stable across runs.
minor comments (4)
  1. [Section 3.1] The compiler version is reported as g++ 12.4.0, but the kernel version and driver versions are listed; please also report the exact OpenMP runtime version and whether CPU frequency scaling (governor) was set to 'performance' or left at its default, since this affects both time and energy.
  2. [Appendix A] The code listings have numerous spacing artifacts (e.g., 'co ll aps e' in Listing 1, 'D u ra ti on' in Listing 1, and 'd ur ati on' in Listing 3) that appear to be formatting artifacts but would prevent direct copy-paste compilation. The authors should host the exact source files in a public repository and reference it in the paper.
  3. [Section 5.2] The statement 'Energy-to-solution is the product of average power and execution time (Energy = Power × Time)' is correct, but the paper should note that this identity is definitional rather than an empirical finding; the 'race to idle' explanation is a post-hoc interpretation, not a mechanism established by the measurements.
  4. [Section 4.2] The text says 'the dGPU consumed only 2% of the energy used by the CPU', but the table shows 28.33 J / 1417.89 J = 2.00%, so the claim is exact; please include the ratio in the table or a footnote for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

Direct empirical measurement with no fitted inputs; the race-to-idle explanation is an arithmetic identity applied post hoc, not a derived prediction.

full rationale

The paper's derivation chain consists of direct measurements (perf RAPL energy-pkg for CPU and package, NVML power.draw polling for dGPU), recorded execution times, and the standard identity Energy = Power x Time used to convert average power and duration into total energy. No quantity is defined in terms of the conclusion, and no parameter is fitted to a subset of data and then renamed a prediction. The 93.5x speedup and 50x energy ratios are arithmetic ratios of directly measured values, and the 'race to idle' discussion in Section 5.2 is an interpretation of those measurements rather than a premise built into them. The CPU benchmark in Appendix A.1 is cache-unfriendly and unvectorized relative to the GPU kernels, but this is a benchmark representativeness or fairness concern, not a circularity concern: it does not make the measured result equivalent to its inputs by construction, and no self-citation is load-bearing. Therefore no circular step can be exhibited under the required standard, and the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central measurements rest on the accuracy of power counters and on the fairness of comparing three hand-written kernels. There are no fitted constants or invented entities; the main burdens are hidden in the measurement assumptions listed above.

assumptions (4)
  • domain assumption The whole-package RAPL energy reading during the iGPU run isolates the iGPU's energy cost.
    Section 3.4 measures power/energy-pkg during the OpenCL kernel and attributes the result to the iGPU; because the iGPU and CPU share a die and power budget, this is a package-level estimate, not an isolated measurement.
  • domain assumption The hand-written kernels in Appendix A are fair representatives of each architecture's performance.
    The CPU OpenMP loop indexes B with stride N, causing poor locality, and none of the kernels uses tiling/shared memory or calls optimized BLAS/cuBLAS libraries; there is no correctness check of the output.
  • domain assumption nvidia-smi power samples at 10 Hz can be integrated to give accurate energy over a kernel that lasts 0.6 seconds.
    Section 3.4 logs power.draw at 10 Hz during the CUDA benchmark, yielding on the order of 6 samples for the kernel; integrating heavily undersampled power can understate or overshoot true energy.
  • domain assumption The RAPL energy-pkg event is valid on this AMD platform as characterized by the cited Intel RAPL validation.
    Section 3.4 calls the interface 'Intel RAPL' and cites Khan et al. [2], which validated RAPL on Intel hardware; the test system is an AMD Ryzen 7 5800H, where the kernel exposes an equivalent counter but with different hardware support.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Racing to Idle: Energy Efficiency of Matrix Multiplication on Heterogeneous CPU and GPU Architectures." pith.science (2026). https://pith.science/paper/L2RZO634

@misc{pith2026250720063,
  author       = {Pith},
  title        = {Pith review of: Racing to Idle: Energy Efficiency of Matrix Multiplication on Heterogeneous CPU and GPU Architectures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L2RZO634}},
  note         = {Machine review of arXiv:2507.20063}
}
read the original abstract

The paradigm shift towards multi-core and heterogeneous computing, driven by the fundamental power and thermal limits of single-core processors, has established energy efficiency as a first-class design constraint in high-performance computing (HPC). Heterogeneous systems, integrating traditional multi-core CPUs with specialized accelerators like discrete (dGPU) and integrated (iGPU) graphics processing units, offer a compelling path to navigating the trade-offs between performance and power. However, quantifying these trade-offs on widely accessible hardware remains a critical area of study. This paper presents a direct, empirical measurement of the performance and energy-to-solution of a canonical HPC workload -- a 4096x4096 matrix-matrix multiplication -- on three distinct compute architectures within a single consumer-grade laptop: a multi-core AMD Ryzen 7 5800H CPU, a discrete NVIDIA GeForce GTX 1650 GPU, and an integrated AMD Radeon Vega GPU. Using standard, validated, and minimally intrusive tools such as Linux perf and nvidia-smi, we find that the discrete GPU is not only the performance leader, achieving a 93.5x speedup over the CPU, but is also the most energy-efficient, consuming only 2% of the energy used by the CPU, resulting in a 50-fold improvement in energy efficiency. These findings provide a practical demonstration of the "race to idle" principle and offer clear, quantitative guidance on architectural choices for energy-aware software development.

Figures

Figures reproduced from arXiv: 2507.20063 by the authors.

Figure 1
Figure 1. Heterogeneous System Architecture. The test platform is an APU design where the CPU and iGPU reside on a single piece of silicon and share a power budget, measured by the ‘energy-pkg‘ RAPL sensor. A separate, dis￾crete GPU is connected via the PCIe bus and has its own independent power sensor, accessible via ‘nvidia-smi‘. This architecture necessitated distinct mea￾surement strategies for each component. APU package… view at source ↗
Figure 2
Figure 2. Experimental Control Flow Graph. For each of the three hard￾ware targets (CPU, dGPU, iGPU), the corresponding benchmark code was compiled and then executed. During execution, a tailored measurement pro￾tocol was used to capture performance and energy data. The results were then aggregated for a final comparative analysis. • Execution Time [s]: The total wall-clock time required to complete the 4096x4096 matrix multi… view at source ↗
Figure 3
Figure 3. Execution Time Comparison. The discrete GPU (dGPU) com￾pletes the workload orders of magnitude faster than the CPU, with the in￾tegrated GPU (iGPU) also showing a significant performance advantage. A logarithmic scale is used on the y-axis to visualize the vast difference in mag￾nitude. 4.2. Energy Consumption. The energy consumption results, presented in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Total Energy-to-Solution Comparison. Both GPU architectures are vastly more energy-efficient than the CPU for the given workload. The dGPU was the most efficient component overall. 4.3. Power Draw. The average power draw of each component while actively executing the b…
Figure 5
Figure 5. Figure 5: Average Power Consumption Comparison. The dGPU sustains the highest rate of power consumption while active, whereas the iGPU is the most frugal. The CPU operates at a moderate power level. workload. This outcome is a practical demonstration of the principles that have …
Figure 6
Figure 6. Figure 6: Performance vs. Energy Scatter Plot. The ideal quadrant is the bottom-left. The dGPU is positioned as the optimal choice for this workload, achieving both the lowest execution time and the lowest energy-to-solution. The CPU is vastly inferior in both metrics. As the pl…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 18 canonical work pages

  1. [1]

    Powerapi: A python framework for building software-defined power meters

    Guillaume Fieni, Daniel Romero Acero, Pierre Rust, and Romain Rouvoy. Powerapi: A python framework for building software-defined power meters. Journal of Open Source Software , 9(98):6670, 2024

  2. [2]

    Rapl in action: Experiences in using rapl for power measurements

    Kashif Nizam Khan, Mikael Hirki, Tapio Niemi, Jukka K Nurminen, and Zhonghong Ou. Rapl in action: Experiences in using rapl for power measurements. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS) , 3(2):1–26, 2018

  3. [3]

    Quantifying the impact of shut- down techniques for energy-efficient data centers.Concurrency and Computation: Practice and Experience, 30(17):e4471, 2018

    Issam Ra¨ ıs, Anne-C´ ecile Orgerie, Martin Quinson, and Laurent Lef` evre. Quantifying the impact of shut- down techniques for energy-efficient data centers.Concurrency and Computation: Practice and Experience, 30(17):e4471, 2018

  4. [4]

    The landscape of parallel computing research: A view from berkeley, 2006

    Krste Asanovic, Ras Bodik, Bryan Catanzaro, Joseph Gebis, Parry Husbands, Kurt Keutzer, David Pat- terson, William Plishker, John Shalf, and Samuel Webb Williams. The landscape of parallel computing research: A view from berkeley, 2006. 12 M. Q. ANSARI AND M. Q. ANSARI

  5. [5]

    Performance and power analysis of hpc workloads on heterogeneous multi-node clusters

    Filippo Mantovani and Enrico Calore. Performance and power analysis of hpc workloads on heterogeneous multi-node clusters. Journal of Low Power Electronics and Applications , 8(2):13, 2018

  6. [6]

    Evaluating the efficiency of cpus, gpus and fpgas on a near-duplicate document detection via opencl

    Ercan Canhasi. Evaluating the efficiency of cpus, gpus and fpgas on a near-duplicate document detection via opencl. J. Comput. Sci. , 14(5):699–704, 2018

  7. [7]

    Energy-aware scheduling for high-performance com- puting systems: A survey

    Bart lomiej Kocot, Pawe l Czarnul, and Jerzy Proficz. Energy-aware scheduling for high-performance com- puting systems: A survey. Energies, 16(2):890, 2023

  8. [8]

    The energy/frequency convexity rule: Modeling and experimental validation on mobile devices

    Karel De Vogeleer, Gerard Memmi, Pierre Jouvelot, and Fabien Coelho. The energy/frequency convexity rule: Modeling and experimental validation on mobile devices. In International Conference on Parallel Processing and Applied Mathematics, pages 793–803. Springer, 2013

Show all 20 references
  1. [9]

    Energy-efficient computing: Innovations in hardware and software for sustainable advanced computing systems

    Rania El-Khattab, Salma Fathy, and others. Energy-efficient computing: Innovations in hardware and software for sustainable advanced computing systems. Journal of Advanced Computing Systems , 3(11):1– 7, 2023

  2. [10]

    A survey of methods for analyzing and improving gpu energy efficiency

    Sparsh Mittal and Jeffrey S Vetter. A survey of methods for analyzing and improving gpu energy efficiency. ACM Computing Surveys (CSUR) , 47(2):1–23, 2014

  3. [11]

    Automm: Energy-efficient multi-data-type matrix multiply design on heterogeneous programmable system-on-chip

    Jinming Zhuang, Zhuoping Yang, and Peipei Zhou. Automm: Energy-efficient multi-data-type matrix multiply design on heterogeneous programmable system-on-chip. arXiv preprint arXiv:2305.18698 , 2023

  4. [12]

    Measuring the energy consumption and efficiency of deep neural networks: An empirical analysis and design recommendations

    Charles Edison Tripp, Jordan Perr-Sauer, Jamil Gafur, Amabarish Nag, Avi Purkayastha, and Sagi Zisman, Erik A Bensen. Measuring the energy consumption and efficiency of deep neural networks: An empirical analysis and design recommendations. arXiv preprint arXiv:2403.08151 , 2024

  5. [13]

    An energy-aware runtime management of multi-core sensory swarms

    Sungchan Kim and Hoeseok Yang. An energy-aware runtime management of multi-core sensory swarms. Sensors, 17(9):1955, 2017

  6. [14]

    Analytical modeling of energy efficiency in heterogeneous processors.Computers & Electrical Engineering, 39(8):2566–2578, 2013

    Ami Marowka. Analytical modeling of energy efficiency in heterogeneous processors.Computers & Electrical Engineering, 39(8):2566–2578, 2013

  7. [15]

    Power and perfor- mance characterization and modeling of gpu-accelerated systems

    Yuki Abe, Hiroshi Sasaki, Shinpei Kato, Koji Inoue, Masato Edahiro, and Martin Peres. Power and perfor- mance characterization and modeling of gpu-accelerated systems. In 2014 IEEE 28th international parallel and distributed processing symposium, pages 113–122. IEEE, 2014

  8. [16]

    The green500 list: Encouraging sustainable supercomputing

    Wu-chun Feng and Kirk Cameron. The green500 list: Encouraging sustainable supercomputing. Computer, 40(12):50–55, 2007

  9. [17]

    Green computing-a new horizon of energy efficiency and electronic waste minimization: A global perspective

    Shalabh Agarwal and Asoke Nath. Green computing-a new horizon of energy efficiency and electronic waste minimization: A global perspective. In 2011 International Conference on Communication Systems and Network Technologies, pages 688–693. IEEE, 2011

  10. [18]

    A comprehensive survey on energy-efficient power management techniques

    Ankit Thakkar, Kinjal Chaudhari, and Monika Shah. A comprehensive survey on energy-efficient power management techniques. Procedia Computer Science, 167:1189–1199, 2020

  11. [19]

    An embedded gpu accelerated hyperspectral video classification system in real-time

    Jaime Sancho, Manuel Villa, Gemma Urbanos, Marta Villanueva, Pallab Sutradhar, Gonzalo Rosa, Alberto Martin, Guillermo Vazquez, Miguel Chavarrias, Ruben Salvador, and others. An embedded gpu accelerated hyperspectral video classification system in real-time. In 2021 XXXVI Conf...

  12. [20]

    Usage :

    Vijay Kandiah, Scott Peverelle, Mahmoud Khairy, Junrui Pan, Amogh Manjunath, Timothy G Rogers, Tor M Aamodt, and Nikos Hardavellas. Accelwattch: A power modeling framework for modern gpus. In MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture , pages 7...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.