{"id":"daeea0ec-aaf6-41ef-a940-de0c458112a1","arxiv_id":"2607.06124","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"Static code metrics yield near-zero predictive power for Java method energy; adding execution time raises R² to 0.46, showing energy is dominated by runtime behavior.","lead":"This paper profiles 2,786 Java methods and finds that static source code metrics alone cannot predict method-level energy consumption (R²≈0); adding execution time raises R² to only 0.46. A smart generalist might read it to understand why early-stage energy estimation from code structure alone remains an unsolved problem.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Filtering zero-energy methods creates selection-on-outcome bias: the 265 surviving methods are systematically those where execution time and energy are both large enough to be measured at 1ms sampling, which can inflate the R²=0.46 attributed to execution time.","rationale":"The reader's verdict of CONDITIONAL is appropriate, and the concern I raise strengthens rather than changes it. The reader identified the right general area (measurement validity + attrition) but did not pinpoint the specific selection-on-outcome mechanism that makes the positive finding (R²=0.46) fragile. The negative result (static features → R²≈0) is well-supported and likely robust regardless of selection bias — if anything, selection toward longer-running methods would give static features their best chance, and they still fail. The positive result is the soft spot: conditioning on non-zero energy at 1ms sampling selects methods where execution time is long enough to be measured, which is the same variable that dominates prediction. The paper is honest about its limitations (§5.1 acknowledges the attrition, sampling concerns, and lack of significance testing), which is commendable. The replication package is available, making the concrete test I propose feasible. The paper makes a legitimate contribution as a baseline study, but the R²=0.46 figure should be read as conditional on the filtering methodology until a sensitivity analysis including zero-energy methods is performed. No verdict change needed — CONDITIONAL with the specific selection-bias concern made explicit is the right call.","tokens_in":19248,"tokens_out":2040,"duration_ms":99632,"concrete_test":"Re-run the full modeling pipeline (RF with VarianceThreshold, 5-fold CV) on the 902-method dataset including zero-energy methods, using either log(energy+1) as the target or a two-stage hurdle model (stage 1: classify measurable vs. zero-energy; stage 2: regress energy on the measurable subset). If the R² for execution time in the inclusive model drops below ~0.30, the selection-on-outcome bias is material and the headline R²=0.46 is an artifact of the filtering step. If R² remains near 0.46, the finding is robust to selection. This check requires only the existing replication package data.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The reader correctly identifies JoularJX measurement validity and dataset attrition as concerns, but the more precise load-bearing issue is the selection mechanism. The pipeline goes 2,786 → 902 (profiler intersection) → 265 (after removing zero-energy methods). The paper justifies removing zero-energy methods as 'likely trivial code paths that would distort training' (§3.3). But at JoularJX's 1ms sampling interval, 'zero energy' does not mean zero energy consumption — it means the method executed too briefly or infrequently to be captured by the sampler. The paper's own cited literature (refs [36], [9], [8] in §5.1) confirms this: short-lived methods produce missing or zero readings at 1ms. By filtering these out, the paper systematically removes methods with short execution times, selecting on a variable (execution time) that is simultaneously the strongest predictor and correlated with the outcome (energy). This is selection on both the treatment and the outcome. The surviving 265 methods are those where execution time and energy are both large enough to register — conditioning on this common cause can inflate their correlation. The R²=0.46 for execution time may partly reflect this selection artifact rather than a genuine runtime-dominance finding. The negative result (static features → R²≈0) is more robust to this concern: even within a biased subset, static features fail to predict energy. But the positive claim — that execution time is the key predictor raising R² to 0.46 — is the finding most vulnerable to selection bias. The paper does not report how many of the 637 removed zero-energy methods had non-trivial static features, nor does it test sensitivity to the filtering threshold.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper investigates whether static source code metrics (33 features extracted via srcML) can predict method-level energy consumption in Java, and whether adding execution time as a lightweight dynamic feature improves prediction. The authors profile 2,786 Java methods from Rosetta Code and CLBG using JoularJX (energy) and async-profiler (execution time), apply filtering to obtain 265 methods with complete data, and train eleven regression models with 5-fold cross-validation. The central finding is that static features alone yield R²≈0, while adding execution time raises R² to 0.46. The paper includes a public replication package, feature importance analysis via SHAP, and a structured comparison across feature selection and hyperparameter tuning configurations.","tokens_in":20064,"tokens_out":1551,"duration_ms":210396,"significance":"The paper addresses a genuine gap: systematic, method-level energy prediction for Java using static features has limited prior work, and the negative result (static features alone are uninformative) is a useful empirical contribution. The inclusion of a public replication package, SHAP-based feature importance analysis, and a structured comparison of eleven models across seven feature-selection configurations are commendable. The comparison to Goyal et al.'s EnCoDe (R²=0.75 on isolated Python blocks) provides useful context for why method-level energy in full-program execution is harder to predict. The falsifiable claim that execution time dominates static features is testable and the replication package allows verification.","major_comments":[{"comment":"§3.3, Data Pre-processing: The filtering pipeline goes from 2,786 methods to 902 (profiler intersection) to 265 (after removing zero-energy methods). The paper justifies removing zero-energy methods as excluding 'trivial code paths that would distort training.' However, as the paper itself acknowledges in §5.1 (Construct Validity), JoularJX's 1ms sampling interval means that 'zero energy' often reflects methods that executed too briefly or infrequently to be captured by the sampler, not methods that consumed no energy. This creates a selection-on-outcome problem: the 265 surviving methods are systematically those where execution time and energy are both large enough to register at 1ms resolution. Since execution time is simultaneously the strongest predictor (§4.2, Figure 4a) and a determinant of whether a method survives filtering, the R²=0.46 attributed to execution time may be partly,","section":null},{"comment":"§4.1, Figure 2 / §4.2, Table 3: The paper reports R²=0.454 for RF with all features (baseline) and R²=0.463 for RF with Variance Threshold (Table 3), then R²=0.462 after tuning (Table 4). These differences (0.454 vs. 0.463 vs. 0.462) are within the range of fold-level variance reported in §4.3 (fold R² ranges from 0.224 to 0.567). The paper does not report confidence intervals, standard deviations across folds, or any statistical test to assess whether the feature selection and tuning improvements are distinguishable from noise. Given n=265 and 5-fold CV, the per-fold validation set is ~53 methods, making R² estimates quite noisy. The claim that feature selection provides 'slight improvements' (§4.2) and tuning delivers 'minimal gains' (§4.3) is likely correct, but the paper should either (a) report fold-level variance or confidence intervals to substantiate that these differences are or","section":null},{"comment":"§4.2, Figure 5: The paper reports that removing execution time drops RF R² from 0.454 to 0.005. This is the strongest evidence for the paper's central claim. However, the paper does not report the corresponding R² values for other models without execution time, nor does it report the R² for execution time alone (without static features). Given that execution time and energy are physically related (energy = power × time), reporting the R² of a univariate model using only log(execution_time) would clarify how much of the 0.46 is simply the time-energy correlation versus any additional explanatory power from static features. This is important because the paper's title and framing emphasize that static metrics are 'insufficient,' but the ablation in Figure 5 does not isolate whether static features add anything beyond execution time.","section":null}],"minor_comments":[{"comment":"§3.2: The paper states that execution times were recovered for 1,980 methods and energy values for 1,103 methods, but the intersection yields 902. It would help to clarify how many methods had energy but not execution time, and vice versa, to assess whether the intersection introduces bias beyond what is already discussed.","section":null},{"comment":"§4.1: The paper reports MAPE values 'mostly below 2%' alongside R² values near 0.45. MAPE can be misleading when the target variable has values near zero (which log-transformed energy may approach). The paper should note this limitation or report MAPE on the original (untransformed) scale for interpretability.","section":null},{"comment":"§3.3: The log transformation is applied to energy and execution time, but the paper does not specify whether the R² values are computed on the log-transformed or original scale. This should be clarified, as R² on log-transformed values is not directly comparable to R² on original values.","section":null},{"comment":"Table 1: The feature 'methodScope' is listed as 'access specifier of the method' but is described as categorical and one-hot encoded in §3.3. The table should note this.","section":null},{"comment":"§2: The related work mentions EnCoDe's ablation showing complexity metrics alone yield R²=0.067, which is consistent with this paper's near-zero R² for static features. This parallel is noted in §5 but could be foregrounded in §2 to strengthen the positioning.","section":null},{"comment":"Figures 2, 3, 5: The axis labels and tick marks are rendered as glyph codes (e.g., '/uni00000035/uni00000029...') in the PDF, making them unreadable. This is likely a font embedding issue but should be fixed for the final version.","section":null},{"comment":"§5.1, Construct Validity: The paper cites ref [8] (Brunnert, 2025) indicating that ~10ms sampling provides a better tradeoff than 1ms. Since the paper uses 1ms, it would be worth briefly explaining why 1ms was chosen despite this evidence, or acknowledging it as a potential source of measurement noise.","section":null}],"recommendation":"major_revision","confidential_remarks":"The selection-on-outcome concern raised by the stress-test note is, in my assessment, the most important issue. The paper's own threats-to-validity section acknowledges the measurement noise problem but does not connect it to the potential inflation of the execution-time R². The negative result (static features → R²≈0) is more robust to this concern, but the positive claim about execution time needs either a sensitivity analysis (e.g., including zero-energy methods with a small epsilon, or using a coarser sampling interval) or more careful framing. The dataset attrition from 2,786 to 265 is severe and the paper should be more upfront about this in the abstract and results, not just in the threats section."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. All three major comments are well-taken. Comment 1 identifies a genuine selection-on-outcome concern in our filtering pipeline; we will add an analysis including zero-energy methods and discuss the bias explicitly. Comment 2 correctly notes that we lack fold-level variance reporting; we will add standard deviations and confidence intervals. Comment 3 asks for a univariate execution-time baseline and per-model ablation; we will add both. We propose no standing objections.","responses":[{"response":"The referee correctly identifies a genuine methodological concern. The selection-on-outcome problem is real: methods surviving our filtering are systematically those where execution time and energy are large enough to register at JoularJX's 1ms resolution, and execution time is simultaneously our strongest predictor. We agree that this could inflate the R² attributed to execution time. In the revision, we will address this in three ways. First, we will re-run our pipeline including the zero-energy methods (labeling them with a small epsilon value or treating them as a separate classification task) to assess how the predictive performance changes. Second, we will add an explicit discussion of this selection bias as a threat to construct validity, going beyond the current acknowledgment in §5.1. Third, we will report the distribution of execution times for excluded vs. included methods to quantify the severity of the selection effect. We acknowledge that we cannot fully eliminate this bias given JoularJX's measurement resolution constraints—it is a fundamental limitation of software-based energy profiling at this granularity. However, we believe the core finding (static features alone are uninformative) is robust to this bias, since the selection effect would, if anything, make static features appear more predictive by restricting the range to methods where code structure matters more, yet static features still yield R²≈0.","revision_made":"yes","referee_comment":"§3.3, Data Pre-processing: The filtering pipeline creates a selection-on-outcome problem because execution time both determines survival and is the strongest predictor, potentially inflating R²=0.46."},{"response":"This is a fair and accurate criticism. The differences between configurations (0.454 vs. 0.463 vs. 0.462) are indeed within the range of fold-level variance we already report (R² ranges from 0.224 to 0.567 across folds). With n=265 and per-fold validation sets of ~53 methods, these R² estimates are noisy, and we should not be presenting these small differences as meaningful without uncertainty quantification. In the revision, we will report fold-level standard deviations and 95% confidence intervals for all R² values in Tables 3 and 4. We will also add a Wilcoxon signed-rank test or paired t-test across folds for the top configurations to formally assess whether the feature selection and tuning improvements are distinguishable from noise. We expect the referee's assessment is correct—that these differences are not statistically significant—and we will adjust the language accordingly, reframing the feature selection and tuning results as 'no statistically significant improvement' rather than 'slight improvements.' The paper's core claims do not depend on these small differences; they depend on the large gap between static-only (R²≈0) and static+time (R²≈0.46), which is far outside fold-level variance.","revision_made":"yes","referee_comment":"§4.1/§4.2: R² differences across configurations (0.454 vs. 0.463 vs. 0.462) are within fold-level variance; no confidence intervals, standard deviations, or statistical tests are reported."},{"response":"We agree completely. This is the most important analytical gap the referee identifies. Given that energy and execution time are physically related (energy = power × time), isolating the univariate predictive power of execution time is essential for interpreting our results. In the revision, we will add: (1) a univariate regression using only log(execution_time) as predictor, reporting R² for all eleven models (or at minimum for RF, the best performer); (2) the full ablation table showing R² without execution time for all models, not just RF; and (3) a direct comparison of execution-time-only R² vs. execution-time-plus-static-features R², so readers can see whether static features add any explanatory power beyond the physical time-energy correlation. We anticipate that execution time alone will explain a large fraction of the R²=0.46, and that static features add marginal incremental value. This actually strengthens our paper's central claim: if execution time alone captures most of the predictive power and static features add little on top, it reinforces the conclusion that static metrics are insufficient. We will revise the framing to make this explicit, noting that the R²=0.46 is largely driven by the physical energy-time relationship rather than by code structure. We will also discuss the implication that the remaining unexplained variance (~0.54) likely reflects factors not captured by either static features or execution time—such as varying power states, cache behavior, and JVM runtime mechanisms.","revision_made":"yes","referee_comment":"§4.2, Figure 5: The ablation does not report R² for other models without execution time, nor R² for execution time alone. A univariate log(execution_time) model would clarify how much of R²=0.46 is simply the time-energy correlation."}],"tokens_in":19261,"tokens_out":1182,"duration_ms":163470,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Two things matter here. First, the negative result is solid: static source code metrics alone yield R²≈0 for predicting method-level energy in Java. Second, the positive finding — adding execution time raises R² to 0.46 — is defensible but fragile, and the stress-test concern about selection bias is the real issue to watch.","headline":"Honest negative result on static metrics for Java method-level energy, but the positive finding (R²=0.46 with execution time) is vulnerable to selection bias from filtering zero-energy methods.","tokens_in":20321,"tokens_out":139,"would_cite":true,"duration_ms":84897,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Static code metrics can't predict Java energy; runtime can, partially","keywords":[],"falsifier":"If a subsequent study using a different energy attribution method (e.g., hardware performance counters with finer granularity, or differential measurement comparing method-present vs. method-absent executions) found that static features alone yielded substantially higher R², the central null result would be undermined. Alternatively, if the 265 surviving methods are shown to be a non-representative subset (e.g., only long-running, CPU-bound methods), the finding that static features are uninformative might not generalize.","tokens_in":19259,"feed_emoji":"⚡","tokens_out":1209,"duration_ms":115131,"temperature":0.7,"pith_summary":"This paper asks a practical question: can you predict how much energy a Java method will consume just by looking at its source code? The authors profile 2,786 Java methods from benchmark repositories, extract 33 static features (loop counts, cyclomatic complexity, library usage, etc.), and train eleven regression models to predict energy consumption measured in joules. The answer is essentially no. Static features alone produce R² values near zero across all models, meaning source code structure explains almost none of the variance in method-level energy. The paper then adds a single dynamic input — execution time in milliseconds — and R² rises to 0.46, still modest but a qualitative jump. The central mechanism the paper identifies is that method-level energy in Java is dominated by runtime behavior (how long a method runs, how often it is called, what the JVM does with it) rather than by the textual structure of the code. Feature selection and hyperparameter tuning yield marginal gains over defaults, suggesting the bottleneck is input information, not model sophistication. The authors position this as a boundary on what static analysis can achieve for energy reasoning: you cannot look at code and know its energy cost without some runtime measurement.","feed_headline":"Static code metrics fail to predict Java method energy","feed_subtitle":"33 source-level features yield R²≈0; adding execution time raises it to 0.46, showing runtime behavior dominates energy at method level.","key_machinery":"The experimental pipeline pairs JoularJX (a JVM agent that attributes RAPL-based CPU/DRAM energy readings to individual methods via sampling at 1ms intervals) with async-profiler (which samples method-level execution time via CPU events). The intersection of methods covered by both profilers, after filtering zero-energy and incomplete records, yields 265 training instances from an initial pool of 2,786. Eleven regression models are evaluated across seven feature-selection configurations and subsequent hyperparameter tuning, with Random Forest consistently performing best.","core_discovery":"The paper's central finding is a null result with a positive complement. Static source code metrics — 33 features spanning control flow, complexity, API usage, and method metadata — carry essentially no predictive signal for method-level energy consumption in Java (R²≈0 across eleven regression models). When execution time is added as a single lightweight dynamic feature, R² rises to 0.46, a substantial but still incomplete improvement. This gap isolates the mechanism: energy at method granularity is determined by runtime factors — execution duration, JVM behavior, invocation context — that are invisible to static analysis. Among static features, internal method calls and cyclomatic complex.","pith_inferences":[],"forward_implications":["Energy-aware refactoring tools that rely solely on static analysis will produce unreliable guidance at the method level; they need at least lightweight runtime profiling to be useful.","The gap between static predictability and runtime-determined energy suggests that JVM-level mechanisms (JIT compilation, garbage collection, class loading) carry energy signal that no source-level feature can capture, pointing toward JVM instrumentation as a necessary complement.","The R² ceiling of 0.46 even with execution time implies that remaining variance is driven by factors not measured here — CPU utilization, cache behavior, memory allocation patterns — which could be the next features to test.","The contrast with prior work showing R²=0.75 for isolated Python code blocks suggests that execution context (full-program vs. isolated-block profiling) is a major determinant of how predictable energy is from code features."],"fun_headline_variants":["Static metrics alone fail to predict Java method energy usage","Predicting Java method energy requires runtime execution data","Static source features yield zero predictive power for Java energy","Execution time outperforms static metrics in Java energy prediction","Java method energy prediction requires more than static metrics"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper's findings depend on JoularJX correctly attributing energy to individual methods. JoularJX samples at 1ms intervals using RAPL CPU/DRAM domains, and the paper itself cites evidence that short-lived or infrequently called methods produce noisy or zero readings under this configuration. After filtering, only 265 of the original 2,786 methods remain with complete data. If the profiler's attribution is systematically biased for this surviving subset, the R² values and (","fun_headline_variants_meta":{"raw":{"variants":["Static metrics alone fail to predict Java method energy usage","Predicting Java method energy requires runtime execution data","Static source features yield zero predictive power for Java energy","Execution time outperforms static metrics in Java energy prediction","Java method energy prediction requires more than static metrics","Static metrics prove insufficient for Java method energy prediction","Adding execution time rescues Java method energy prediction","Static code features lack predictive power for Java method energy"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":1024,"prompt_tokens":507,"completion_tokens":517,"prompt_tokens_details":null},"tokens_in":507,"tokens_out":517,"duration_ms":51107,"temperature":1.0,"reasoning_tokens":494,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T15:48:36.078130+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If a subsequent study using a different energy attribution method (e.g., hardware performance counters with finer granularity, or differential measurement comparing method-present vs. method-absent executions) found that static features alone yielded substantially higher R², the central null result would be undermined. Alternatively, if the 265 surviving methods are shown to be a non-representative subset (e.g., only long-running, CPU-bound methods), the finding that static features are uninformative might not generalize.","supporting_citations":[],"review_version":1}