{"id":"d7b01f13-90bd-4ac5-a075-ed19a6d7307d","arxiv_id":"2412.12426","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FinGraV reconstructs fine-grain GPU power profiles for sub-millisecond AI kernels by synchronizing CPU-GPU clocks, binning execution times, and separating steady-state execution from steady-state power.","lead":"This paper presents FinGraV, a method to measure GPU power consumption at sub-millisecond time scales on AMD MI300X GPUs, even though the built-in power sensor only reports averages every millisecond. The method may matter for anyone trying to optimize energy use of AI workloads, because without it, power and energy measurements of short GPU kernels can be off by as much as 80%.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The stitched FinGraV profile is the 1ms boxcar convolution of the kernel's true power trace, not the trace itself; without deconvolution, 'fine-grain' sub-ms temporal resolution is not achieved.","rationale":"The reader's weakest assumption points to the unvalidated moving-average model. My concern is more specific and arguably more damaging: even if the hardware model is exactly right, the stitching operation as described does not reconstruct the true within-kernel power profile. Since each sample is a 1ms average, the value assigned to a TOI is the convolution of the true power profile with a 1ms boxcar. Random delays shift the boxcar relative to the kernel; they sample the convolution but do not invert it. For sub-ms kernels, the temporal width of the boxcar exceeds the kernel duration, so the stitched profile is dominated by the boxcar shape. The paper's own figures for CB-2K-GEMM show the expected trapezoidal convolution of a pulse train, not a kernel-intrinsic waveform. This matters because the abstract and title claim 'fine-grain' visibility, and the observed SSE-to-SSP ramp is used both to motivate power differentiation and to quantify the 80% error. If the ramp is partly the boxcar step response, the error figure conflates measurement averaging with actual power dynamics. The paper does acknowledge profiles are average power, but it does not state that the profiles are the convolution of the true power with the 1ms window, nor does it attempt deconvolution. The concrete test with a synthetic kernel and an external meter would settle whether the raw stitched profile matches the true trace. If it matches the boxcar convolution, the authors should add a deconvolution step or soften the 'fine-grain' claim. The methodology's synchronization and binning are still useful, and the SSE/SSP guidance may be practically sound, so conditional acceptance remains appropriate rather than rejection.","tokens_in":16274,"tokens_out":14632,"duration_ms":128734,"concrete_test":"Run a synthetic kernel with a controllable phase structure (e.g., a long GEMM split into two halves, one compute-heavy and one memory-only, to create a known step in instantaneous power) at a total duration of about 200us. Use FinGraV to stitch the profile, and simultaneously record true power with a high-bandwidth external current probe on the GPU's main power input. Then compare the stitched profile to (a) the external trace and (b) the external trace convolved with a 1ms boxcar. If the stitched profile matches (b) but not (a), the 'fine-grain' claim is unsupported; adding a deconvolution step and re-running would show whether the true step can be recovered.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that FinGraV yields fine-grain power profiles for sub-ms kernels rests on treating each 1ms-logger sample as if its value were the power at a single TOI. But S1 states each sample is the average of instantaneous readings over the preceding 1ms. Therefore a sample taken at offset tau relative to kernel start equals the convolution of the true kernel power profile with a 1ms boxcar window, not the profile evaluated at tau. Random delays only shift this window; they sample the convolution densely, they do not undo the averaging. For kernels shorter than 1ms (e.g., CB-2K-GEMM, the source of the 80% figure), the raw stitched profile is a trapezoid whose rising edge width is the 1ms window, so the observed SSE-to-SSP 'ramp' is largely the boxcar response rather than a property of the kernel. No deconvolution or inverse filtering is performed, and the paper even acknowledges profiles are 'average power' but still uses 'fine-grain' and derives insights such as gradual power rise and component power attribution from these blurred traces. This is not merely an unvalidated hardware assumption: even if the logger is a perfect moving average, the reconstruction does not yield the kernel's true time-resolved power.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"FinGraV addresses the difficulty of obtaining fine-grain GPU power profiles for sub-millisecond to few-millisecond AI kernels on the AMD MI300X. The paper identifies four challenges (low native sampling frequency, CPU-GPU time synchronization, execution-time variation, and power variation across executions) and proposes a methodology that combines an internal 1ms power logger, timestamp synchronization, random delays across repeated runs, execution-time binning to discard outlier runs, and a distinction between steady-state execution (SSE) and steady-state power (SSP) profiles. The authors apply FinGraV to compute-bound and memory-bound GEMM/GEMV kernels and to RCCL communication kernels, report total and component-level (XCD, IOD, HBM) power profiles, and derive measurement guidance and optimization recommendations, including the claim that failing to differentiate power profiles can lead to power/energy measurement errors as high as 80%.","tokens_in":16479,"tokens_out":5597,"duration_ms":54182,"significance":"If the central accuracy claim holds, FinGraV would be a useful methodology for power profiling of short kernels on a state-of-the-art accelerator, and the component-level observations would provide actionable guidance for power-aware scheduling and hardware/software optimization. The paper is strongest in its careful enumeration of the practical pitfalls (C1-C4), the proposed synchronization and binning machinery, and the broad set of reported profiles across GEMM, GEMV, and collective communication kernels. However, the methodology's central claim, that the stitched profiles are fine-grain in time, rests on an averaging model that is asserted but not validated, and the evaluation is qualitative rather than quantitative. The headline 80% error figure is also not independently grounded. The work is therefore a promising methodology study whose load-bearing claims need significant additional support.","major_comments":[{"comment":"Section IV-A states that each power sample from the internal logger is the average of multiple instantaneous power readings over the preceding 1ms. Under this model, a sample taken at offset tau from kernel start is the convolution of the kernel's instantaneous power trace with a 1ms boxcar window, not a pointwise sample of that trace at tau. Random delays between runs shift the phase of the boxcar window; they do not undo the averaging. For kernels in Table I with 25-50us execution times, the stitched SSP profile is therefore a smoothed, aggregate quantity, and the claim of a fine-grain sub-millisecond profile (abstract, Section V-C1) is not supported as stated. The paper should either deconvolve the known or estimated averaging kernel, or explicitly reframe all profiles as 1ms-window-smoothed average power and show that this smoothed quantity supports the subsequent insights.","section":"IV-A (S1), IV-B (step 9), V-C1"},{"comment":"The headline 'measurement error as high as 80%' is presented as the spread between the SSE and SSP profiles, but both profiles are outputs of the same 1ms-averaging logger under different execution histories. Neither is independently established as the true kernel power or energy, so calling their difference a measurement error presumes a reference that is not defined. The paper should define an unambiguous reference (for example, the integral of a high-fidelity instantaneous power trace over a single kernel execution) and provide a quantitative comparison against it; otherwise the 80% figure is a statement about profile variation, not a validated error bound.","section":"V-C1, Table II"},{"comment":"The evaluation of the FinGraV methodology is qualitative: it visually compares synchronized vs unsynchronized profiles, binned vs unbinned profiles, and 200-run vs 50-run profiles, and concludes that binning yields a profile 'more tuned to the true shape of power consumed.' No quantitative accuracy metric or independent ground truth is provided for any reconstructed profile. A validation experiment against a high-bandwidth external power meter, or against a synthetic power signal with a known shape, is needed to support the central accuracy claim and the measurement guidance in Table II.","section":"V-B"}],"minor_comments":[{"comment":"The warm-up count is stated as 'typically three warm-up executions' in Section IV-A, whereas Section IV-B step 3 says to execute the kernel four times, noting that three executions sufficed for stabilization. Please make the recommended procedure and the empirical observation consistent.","section":"IV-A (S4) vs IV-B (step 3)"},{"comment":"The labels 'CB– >8K' and 'MB– >4K gemv' appear to be rendering errors for arrows such as 'CB→8K'; please fix the notation.","section":"Figure 9"},{"comment":"The footnote marker '12' after 'y-axis' appears malformed; the intended footnote markers should be rendered consistently as superscripts.","section":"V-B"},{"comment":"The paper mentions in the related-work section that clock drift was observed and will be addressed in future work; since Section IV-A (S2) relies on timestamp synchronization, this limitation should be stated prominently in the methodology section so that readers understand the current accuracy limitations of the sync procedure.","section":"VII"}],"recommendation":"major_revision","confidential_remarks":"This is a methodology paper written by AMD-affiliated authors using an internal power logger. The main revision path is clear: either deconvolve the averaging kernel or reframe the claims as 1ms-smoothed profiles, and add external or synthetic validation for the accuracy and error-bound claims. The paper otherwise fits the scope of a systems/performance analysis venue and does not raise novelty concerns relative to the cited prior work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper has something real: a systematic recipe for using a 1ms averaging power logger to get stable power profiles on the MI300X, and the first component-level (XCD/IOD/HBM) power data I know of for GEMM and communication kernels on that part. The SSE/SSP distinction is a genuinely useful practical warning—if you grab the first execution's power on a short kernel, you can be off by a lot, and their 80% example makes that concrete. The authors are also honest about overlap with Yang et al. and Lang et al., which helps situate what is new.\n\nBut the central claim about fine-grain sub-millisecond resolution doesn't survive contact with their own S1. Each sample is the average of instantaneous readings over the preceding 1ms. So the profile you stitch together from random delays is the convolution of the true kernel power trace with a 1ms boxcar window. Random delays let you sample that convolution densely; they don't undo the averaging. For a kernel shorter than 1ms—precisely the case driving the 80% figure—the observed \"gradual power rise\" is largely the boxcar response, not a property of the kernel. The paper even says the profiles are average power, then turns around and derives insights about gradual power growth, component stress, and power proportionality from those blurred traces. No deconvolution, and no validation against an independent high-frequency power meter. That is not a minor limitation; it undercuts the \"fine-grain\" framing and the specific temporal interpretations.\n\nThe 80% error number also deserves a careful reading. It is the gap between SSE and SSP, both averaged quantities. It is a real warning about measurement protocol, but it is not a validated measure of error against ground-truth energy.\n\nOther soft spots: the interleaving analysis is confounded by uncontrolled DVFS and temperature, and no code or data is released, so reproducibility is limited. None of this makes the paper worthless. The methodology guidance, the MI300X data, and the caveat about averaging windows are all useful to the community. But it needs major revision: either deconvolve, validate against a fast meter, or reframe the claims as millisecond-window average profiles rather than fine-grain traces.\n\nI'd send it to referees—the topic matters and the flaws are addressable—but I'd expect the revision to be substantial. For my own work, I wouldn't cite the fine-grain results as-is, but I might cite the SSE/SSP caution if I write about GPU power measurement.","headline":"Useful MI300X power measurement guidance and new component-level data, but the 'fine-grain' claim is undermined by unaddressed 1ms averaging; the profiles are blurred traces, not sub-ms truth.","tokens_in":17058,"tokens_out":4136,"would_cite":false,"duration_ms":36846,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FinGraV reconstructs sub-millisecond GPU power profiles by stitching 1 ms averaged samples across hundreds of time-shifted runs, and shows that ignoring power-profile differentiation can cause up to 80% energy measurement error.","keywords":["GPU power profiling","fine-grain power measurement","AMD MI300X","GEMM kernels","communication collectives","power profile differentiation","energy measurement accuracy","sub-millisecond kernels"],"falsifier":"Run a sub-millisecond GEMM on an MI300X while simultaneously logging with the internal 1 ms power logger and an independent high-frequency external power meter on the GPU power rails, then compare the FinGraV stitched profile to the external trace; if the shapes diverge by more than the reported error, the moving-average or cross-run stability assumption fails.","tokens_in":1678,"feed_emoji":"⚡","tokens_out":6910,"duration_ms":97142,"temperature":0.7,"pith_summary":"As GPUs execute AI kernels in under a millisecond, the GPU's own power logger, which reports one averaged value every millisecond, cannot directly show how power changes inside the kernel. FinGraV addresses this by running the same kernel many times with random start delays, synchronizing CPU and GPU clocks to know where in the kernel each power sample falls, and keeping only runs whose execution times fall in a narrow bin. The paper claims that this stitching procedure reconstructs a fine-grain power profile for the kernel, including per-component power for the compute dies, I/O dies, and HBM, on the AMD Instinct MI300X. It further claims that distinguishing the first stabilized execution from the later steady-state power profile matters: confusing the two can produce power and energy measurement errors as high as 80% for short kernels. The payoff would be accurate power visibility for the kernels that dominate AI workloads, enabling power optimizations.","feed_headline":"Stitching 1 ms power logs reveals GPU power inside tiny kernels","feed_subtitle":"On MI300X, separating steady-state profiles avoids up to 80% energy measurement error.","key_machinery":"The central object is the stitched power profile built from the MI300X's internal 1 ms power logger, where each sample is the average of instantaneous power readings over the preceding 1 ms. Three mechanisms carry the argument: execution time binning, which excludes outlier runs and groups runs whose execution times fall within a 2-5% margin; CPU-GPU time synchronization, which reads a GPU timestamp before kernel launch and benchmarks the read delay so each power log can be assigned a time of interest within the kernel; and power profile differentiation into SSE and SSP profiles, which separates the first stabilized execution from the later power-stabilized execution. Random delays before each run place the 1 ms averaging window at different points in the kernel, so stitching hundreds of runs produces a time-series view of average power across the kernel's duration.","core_discovery":"The paper's central claim is that a GPU power logger with a 1 ms averaging window can still yield a fine-grain, sub-millisecond power profile, provided the same kernel is executed many times with random start delays and the resulting power samples are stitched together after careful CPU-GPU time synchronization, execution-time binning, and power-profile differentiation. On the AMD MI300X, FinGraV separates the steady-state execution (SSE) profile, the first execution whose time has stabilized after warm-up, from the steady-state power (SSP) profile, the later execution beyond which power no longer varies substantially. The paper reports that without this differentiation, power and energy measurements can be wrong by as much as 80%, depending on the relative magnitudes of kernel execution time and the power logger's averaging window. With the reconstructed profiles, the paper identifies which GPU sub-components dominate for compute-bound versus memory-bound kernels, and shows that kernels shorter than the averaging window inherit power from kernels that precede them.","pith_inferences":["If the 1 ms moving-average model is exactly right, the same stitching recipe should transfer to any averaged power logger by scaling the number of runs roughly with the averaging window; a 10 ms logger would need about ten times as many runs for the same time resolution.","The 80% error bound implies that published energy measurements that sample sub-millisecond kernels once per execution may be systematically biased, so kernel-efficiency comparisons that ignore this effect could rank kernels incorrectly.","A direct test of the method would be to compare a FinGraV stitched profile against a high-bandwidth external power meter on the GPU power rails; if the internal logger's averaging is not a simple moving average, or if power state drifts across runs, the reconstructed profile would need correction.","For kernels whose execution time is close to or larger than the averaging window, SSE and SSP profiles coincide, so the differentiation overhead may be unnecessary and FinGraV's main benefit is concentrated in the sub-millisecond regime."],"forward_implications":["For kernels shorter than the power logger's averaging window, a single power sample cannot represent the kernel's energy, so repeated-run stitching is necessary to recover the within-kernel power shape.","Differentiating SSE from SSP is essential for accurate energy measurement: the paper reports up to 36% power error for a compute-bound 4K GEMM and up to 80% energy error across kernels, with the largest gap when kernel execution time is much shorter than the 1 ms averaging window.","Kernels shorter than the averaging window inherit power from preceding kernels when executed in an interleaved fashion, so isolated executions are necessary to assess their true power draw.","The profiles show that compute-heavy GEMMs are dominated by XCD power while bandwidth-bound communication and memory-bound GEMVs stress IOD and HBM, suggesting that complementary kernels could be co-scheduled to use available power headroom.","The methodology extends to external power loggers such as amd-smi, with the resulting profile quality depending on the averaging window those loggers report."],"supporting_citations":[{"why":"Supplies the observation that large-scale ML kernels can be power-limited, motivating kernel-level power visibility.","marker":"[4]"},{"why":"Explains frequency throttling when a GPU hits its power limit, which the paper uses to interpret early-execution power spikes.","marker":"[5]"},{"why":"Concurrent work on NVIDIA built-in power sensors that provides a comparison point for power measurement guidance and a skip/shift mechanism equivalent to SSE/SSP differentiation.","marker":"[6]"},{"why":"Justifies focusing on GEMM and communication kernels by showing they dominate AI execution time.","marker":"[13]"},{"why":"Provides the external AMD power logging tool that FinGraV discusses working with in practice.","marker":"[14]"},{"why":"Supplies the rocBLAS library used to execute the GEMM kernels profiled in the study.","marker":"[16]"},{"why":"Supplies the RCCL communication library used to execute the all-gather and all-reduce kernels profiled in the study.","marker":"[17]"},{"why":"Prior high-resolution GPU power profiling work that FinGraV extends by adding execution-time binning and accounting for CPU-GPU communication delays.","marker":"[33]"}],"fun_headline_variants":["Stitch power logs to see GPU power at sub-ms scale","Random delays unlock sub-millisecond GPU power profiles","Without split, GPU power error hits 80%: FinGraV fixes it","Fine-grain GPU power from stitched logs, down to sub-ms kernels","Split steady-state power to avoid 80% GPU measurement error"],"cache_read_input_tokens":19200,"weakest_assumption_plain":"The load-bearing premise is that the internal 1 ms power logger really does return a moving average of instantaneous power over the preceding 1 ms, and that the GPU's power state is stable enough across repeated runs for stitching time-shifted samples to reconstruct one true kernel profile; the paper asserts this model but does not independently validate it.","fun_headline_variants_meta":{"raw":{"variants":["Stitch power logs to see GPU power at sub-ms scale","Random delays unlock sub-millisecond GPU power profiles","Without split, GPU power error hits 80%: FinGraV fixes it","Fine-grain GPU power from stitched logs, down to sub-ms kernels","Split steady-state power to avoid 80% GPU measurement error"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1362,"prompt_tokens":973,"completion_tokens":389,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":296}},"tokens_in":589,"tokens_out":389,"duration_ms":3879,"temperature":1.0,"reasoning_tokens":296,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:05:41.110767+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a sub-millisecond GEMM on an MI300X while simultaneously logging with the internal 1 ms power logger and an independent high-frequency external power meter on the GPU power rails, then compare the FinGraV stitched profile to the external trace; if the shapes diverge by more than the reported error, the moving-average or cross-run stability assumption fails.","supporting_citations":[{"cited_title":"Towards improved power management in cloud gpus,","cited_arxiv_id":null,"evidence_quote":"Explains frequency throttling when a GPU hits its power limit, which the paper uses to interpret early-execution power spikes."},{"cited_title":"Accurate and Convenient Energy Measurements for GPUs: A Detailed Study of NVIDIA GPU’s Built-In Power Sensor,","cited_arxiv_id":null,"evidence_quote":"Concurrent work on NVIDIA built-in power sensors that provides a comparison point for power measurement guidance and a skip/shift mechanism equivalent to SSE/SSP differentiation."},{"cited_title":"Tale of Two Cs: Computation vs. Communication Scaling for Future Transformers on Future Hardware,","cited_arxiv_id":null,"evidence_quote":"Justifies focusing on GEMM and communication kernels by showing they dominate AI execution time."},{"cited_title":"AMD SMI documentation,","cited_arxiv_id":null,"evidence_quote":"Provides the external AMD power logging tool that FinGraV discusses working with in practice."},{"cited_title":"ROCm ™/rocBLAS: Next generation BLAS implementation for ROCm™ platform,","cited_arxiv_id":null,"evidence_quote":"Supplies the rocBLAS library used to execute the GEMM kernels profiled in the study."},{"cited_title":"ROCm ™ Communication Collectives Library,","cited_arxiv_id":null,"evidence_quote":"Supplies the RCCL communication library used to execute the all-gather and all-reduce kernels profiled in the study."},{"cited_title":"High-Resolution Power Profiling of GPU Functions Using Low-Resolution Measurement,","cited_arxiv_id":null,"evidence_quote":"Prior high-resolution GPU power profiling work that FinGraV extends by adding execution-time binning and accounting for CPU-GPU communication delays."}],"review_version":1}