{"id":"23954c95-712b-4d5e-87ac-7a1da8747de1","arxiv_id":"2608.13039","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A case study comparing XGBoost intrinsic feature importances, partial dependence, ALE, and SHAP for a district heating heat demand forecasting model.","lead":"This paper applies four existing explainable AI tools to a machine learning model that forecasts heat demand in a district heating network. It shows how the tools' feature importance rankings can be compared to build trust in the model's decisions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'strong evidence of quality' claim rests on a qualitative resemblance between ALE(tamb) and the DHS control curve, a resemblance any physically sound model would display; without quantitative comparison to held-out data or reported accuracy, the trustworthiness conclusion is unsupported.","rationale":"The paper is a short case study; its interpretations of gain, cover, weight, PD/ICE, ALE and SHAP are internally consistent and physically sensible. Credit is due for using several global importance methods and for noting complementarities (e.g., high gain but flat PD for deltae-23). The central claim, however, is not just that the methods produce interpretable plots but that the model is trustworthy. That claim rests on the ALE/control-curve resemblance in Section 4. I agree with the reader's assessment that this is the weakest assumption: it treats plot-shape similarity as a validation test without specifying a reference metric, a comparison distribution, or an accuracy requirement. My stress-test adds that the resemblance is expected by construction: the DHS control curve determines supply temperature from ambient temperature, so any model trained on data from that system will inherit a decreasing ALE(tamb). This makes the resemblance a weak positive sanity check, not 'strong evidence of quality.' The missing MAE and the absence of a temporal split (the text says only 80/20, not chronological) make the trustworthiness conclusion unsupported rather than false. The correct disposition remains conditional: the demonstration is suggestive but needs quantitative held-out validation before the trustworthiness claim can be accepted. Therefore no change to the reader's conditional verdict is needed.","tokens_in":7281,"tokens_out":6951,"duration_ms":75190,"concrete_test":"On the held-out set (ideally the final heating season, not a random split), compute the empirical conditional mean of hourly heat demand within 1°C bins of ambient temperature. Overlay the model's ALE(tamb) and the control curve after normalizing to comparable scales. If the ALE curve lies outside the empirical bin confidence intervals or deviates by more than, say, one bin standard error at any temperature, the 'strong evidence of quality' claim fails. Report test-set MAE alongside. This directly tests whether the visual resemblance is diagnostic of model accuracy or just an artifact of the operating policy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is in Section 4: the ALE plot for ambient temperature 'closely resembles the shape of the control curve', and this resemblance is called 'strong evidence of quality of forecasting model'. This inference is not secured. The control curve is part of the DHS operating policy: supply-water temperature is set as a decreasing function of ambient temperature, and heat demand inherits the same decreasing dependence. A model trained on data generated under that policy should show a decreasing ALE(tamb); the resemblance is a necessary sanity check rather than evidence of forecasting quality. The ALE y-axis is predicted heat demand, while the control curve y-axis is supply-water temperature, so the comparison is qualitative and unit-incompatible. No test-set MAE, no temporal hold-out, and no numerical comparison of the ALE curve with the empirical heat-demand-vs-temperature relationship are reported. Consequently, the central trustworthiness conclusion depends on an unvalidated visual analogy. Secondary issues (PD/ALE/SHAP mislabeled 'ante-hoc' in Section 2; unsupported claim that SHAP is 'most reliable and robust'; Figure 5 caption mismatch; random 80/20 split with no stated temporal ordering) reinforce the conditional status but do not by themselves overturn the demonstration.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a case study of global feature importance for an XGBoost heat demand forecasting model trained on four heating seasons of SCADA and meteorological data from a district heating substation. Four approaches are used: XGBoost's intrinsic gain/cover/frequency metrics, partial dependence and ICE plots, accumulated local effects, and SHAP. The author interprets the resulting importances in terms of district heating operation and argues that agreement between the ALE curve for ambient temperature and the DHS control curve constitutes strong evidence of model quality, thereby supporting the trustworthiness of the black-box forecasting model. The paper explicitly avoids permutation- and perturbation-based XAI methods and discusses complementarities and limitations of the selected techniques.","tokens_in":7500,"tokens_out":4074,"duration_ms":40796,"significance":"If the evidential claim were established, the paper would offer a useful demonstration template for validating forecasting models in regulated district heating operations without relying on model perturbation. The use of four complementary interpretation tools, the explicit acknowledgement of the independence assumption in PD, and the domain-grounded discussion of lagged demand and temperature features are positive features. However, the demonstration currently lacks quantitative validation, does not provide code or data, and the key conclusion rests on a qualitative visual analogy rather than a measured comparison; these gaps must be addressed before the trustworthiness claim can be accepted.","major_comments":[{"comment":"The conclusion that the resemblance between the ALE plot for tamb and the DHS control curve is 'strong evidence of quality of forecasting model' is not supported by the evidence presented. The control curve is the operating policy according to which supply-water temperature is a decreasing function of ambient temperature; heat demand measured under that policy will display the same decreasing dependence in any model that has learned the historical relation. A decreasing ALE(tamb) is therefore a necessary sanity check, not a validation of forecast quality. The comparison is also unit-incompatible, since the ALE y-axis is predicted heat demand while the control curve y-axis is supply-water temperature. Please add a quantitative check, for example comparing ALE(tamb) with the empirical conditional expectation E[heat demand | tamb] on a held-out season, and report test-set MAE or a similar accuracy metric against a baseline.","section":"Section 4"},{"comment":"Partial Dependence, Accumulated Local Effects, and SHAP are post-hoc model-agnostic interpretation methods; describing them as 'ante-hoc' is a conceptual error. The abstract and Section 2 claim as the paper's contribution an 'ante-hoc Explainable AI methodology', while the methods are applied after training to interpret a black-box model. This misclassification should be corrected throughout the manuscript, including the first paragraph of Section 2, the second paragraph of Section 2, and the conclusions, and the claimed novelty should be reframed accordingly.","section":"Section 2 and Abstract"},{"comment":"The description of the data split ('80% of available data is used for training and the remaining 20% is used for testing') does not state whether the split is chronological or random. For a forecasting model with 23-hour lagged features, a random split can place test instances temporally before or between training instances, allowing leakage through lagged target and temperature values. Please report the exact split rule and, if the split is not chronological, repeat the analysis with a temporal hold-out; this is essential for the trustworthiness claim.","section":"Section 2, data split"},{"comment":"The statement that SHAP 'is the most reliable and robust method', and the claim that this is 'proven' by the case study, are unsupported. The Shapley-value foundation gives SHAP desirable axiomatic properties, but it does not by itself establish stability or reliability on this particular model. No bootstrap or repeated-sampling analysis, no comparison with alternative attribution methods on known ground truth, and no error bars on the reported importances are provided. Please either remove the claim or support it with quantitative evidence, such as variance of SHAP values across bootstrap samples or a comparison of SHAP-based rankings with gain and ALE rankings.","section":"Sections 3.4 and 4"}],"minor_comments":[{"comment":"The caption of Figure 5 says 'ALE plots for the selected features in the dataset', but the figure shows a SHAP bar plot and a SHAP summary plot; the caption should be corrected.","section":"Figure 5"},{"comment":"The SHAP formula uses f(S) as the model prediction given a subset of features, which can be read as implying model refitting or a well-defined function on arbitrary subsets; for a trained XGBoost model SHAP values are usually computed with conditional expectations. Please clarify the definition to avoid ambiguity.","section":"Section 3.4, Eq. (2)"},{"comment":"The hyperparameter configuration of the XGBoost model is not reported beyond the number of trees; listing the learning rate, maximum depth, and regularization parameters would improve reproducibility.","section":"Section 2"},{"comment":"Reference [6] is cited as a Springer volume without volume, page, or DOI details; if the work is published, the full bibliographic information should be provided because the data preprocessing pipeline is essential to the present study.","section":"References"},{"comment":"No code or data availability statement is included; providing the preprocessing and analysis code, or at least a detailed data description, would allow the reported global importance plots to be reproduced.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be a preprint of a paper published in Thermal Science; the editor may wish to verify that the terminology and evidential issues identified here are not already resolved in the published version. The contribution is applied and methodological rather than theoretical, and the lack of code and data is a reproducibility concern for a paper whose main evidence is visual. The strongest load-bearing issue is the unsupported inference from ALE shape to forecasting quality; this needs either a quantitative validation or a substantial weakening of the claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis is a short applied paper that applies four standard XAI methods to an XGBoost heat demand forecaster. It does not introduce new methodology, data, or theory. What is useful is the domain interpretation: the discussion of why gain, cover, frequency, and SHAP disagree for features like deltae-23 is sensible and could help practitioners. The paper also acknowledges PD's independence assumption, and the ICE/ALE comparison is handled cleanly.\n\nThe soft spots are real. The central claim that the ALE(tamb) curve resembling the DHS control curve is 'strong evidence of quality of forecasting model' does not hold. Any physically sound model trained on this system will show a decreasing relationship between ambient temperature and heat demand. The control curve is a policy function relating supply water temperature to ambient temperature; the ALE curve is predicted heat demand. The units differ, the comparison is purely qualitative, and no test-set accuracy, temporal holdout, or numerical curve comparison is reported. That resemblance is a necessary sanity check, not evidence of forecasting quality.\n\nThe paper also mislabels PD, ALE, and SHAP as 'ante-hoc' in Section 2 and 4; they are post-hoc. Calling SHAP the 'most reliable and robust' method is an overclaim, and the Figure 5 caption says 'ALE plots' when it actually shows SHAP output. These are fixable, but they signal carelessness. The biggest practical problem is the absence of code or data, and no error bars on the importance measures, so the demonstration is not reproducible. The random 80/20 split without temporal ordering is also a weakness for time-series data.\n\nOn citations: the author's self-citations to prior XAI and XGBoost work in DHS are relevant and not a flaw.\n\nThis paper is for practitioners who want a worked example of these tools applied to district heating. As a case study it has some illustrative value, but the load-bearing validation argument is missing. I would treat the trustworthiness claim as a hypothesis to test, not a conclusion. I would not cite it in my own work, and I would not send it to a serious peer-review venue without major additions: open code/data, a quantitative comparison of ALE with the empirical demand-temperature relationship, and corrected terminology. As it stands, desk reject with an invitation to resubmit if the evidentiary gap is closed.","headline":"A workmanlike XAI case study for heat demand forecasting; the feature-importance discussion is plausible, but the trustworthiness claim rests on a visual resemblance that cannot carry it.","tokens_in":8036,"tokens_out":2745,"would_cite":false,"duration_ms":27984,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that four global XAI methods, read together, can validate a black-box heat-demand forecasting model, with the ALE curve's resemblance to the district-heating control curve as the central evidence.","keywords":["explainable AI","global feature importance","heat demand forecasting","district heating systems","gradient boosting","partial dependence","accumulated local effects","SHAP"],"falsifier":"Train a deliberately wrong heat-demand model on the same data, for example a model that ignores ambient temperature or one that predicts only the daily average, and run the same four explanation methods: if its ambient-temperature ALE curve still resembles the control curve, or if the trustworthy model's resemblance disappears when evaluated on a new heating season, then the resemblance test cannot separate good from bad models.","tokens_in":7069,"feed_emoji":"🔥","tokens_out":7958,"duration_ms":76348,"temperature":0.7,"pith_summary":"This paper proposes a way to validate a black-box machine-learning model that forecasts hourly district-heating demand, using global feature importance rather than accuracy metrics. It trains an XGBoost regressor on four heating seasons of substation and meteorological data, then reads four explanation views side by side: XGBoost's intrinsic gain, cover, and frequency; partial dependence and ICE curves; accumulated local effects (ALE); and SHAP. The choice of methods is deliberate, because none permutes or perturbs input features and therefore none fabricates unrealistic data values during explanation. The paper argues that the combined picture matches how district heating actually behaves, and that this match, especially the ALE curve's resemblance to the district-heating control curve, makes the model trustworthy enough for control decisions with compliance and liability stakes.","feed_headline":"Four XAI views plus a control-curve match can validate a heat-demand model","feed_subtitle":"Four explanation methods converge on the same picture: the model tracks daily seasonality and the ambient-temperature control curve.","key_machinery":"The machinery that carries the argument is the Accumulated Local Effects (ALE) plot, a global post-hoc explanation method that estimates how small local changes in a feature shift the model prediction while conditioning on the observed data distribution; unlike partial dependence, it does not assume features are independent. The ALE curve for ambient temperature is what the paper compares with the district-heating control curve, and that visual resemblance is the stated evidence of model quality. Around it, three complementary mechanisms supply the broader picture: XGBoost's intrinsic gain, cover, and frequency importances; partial dependence plots with individual conditional expectation (ICE) lines; and SHAP values aggregated over the test set.","core_discovery":"The central claim is that a heat-demand forecasting model trained with gradient boosting can be judged trustworthy by interpreting its global feature importances in the context of district-heating operation. Read together, the four views tell one coherent story: ambient temperature ($t_{amb}$) is the most frequently used split in the model, yet the largest accuracy gains come from transmitted energy at the same hour of the previous day, reflecting strong daily seasonality; SHAP ranks current transmitted energy and hour of day as the largest influences on predictions; and the partial-dependence curve for the previous-day energy is flat on average but shows in ICE plots that the feature matters mainly at the start of daily operation. The load-bearing piece is the ALE curve for ambient temperature, which falls almost linearly below about 9 °C and then more steeply with nonlinearities, closely following the shape of the DHS control curve that relates secondary supply water temperature to ambient temperature. The paper calls this resemblance strong evidence of the forecasting model's quality, and concludes from it, together with the expert-plausible readings of the other methods, that the otherwise black-box model is transparent enough to support trust in automated district-heating control.","pith_inferences":["The paper's trustworthiness conclusion could be strengthened by replacing visual resemblance with a numeric similarity score between the ALE curve and the control curve over the operating temperature range; this is an extension the paper does not perform.","The no-permutation criterion is applied at the global level, but SHAP's model-agnostic variants sample feature coalitions and can still generate unrealistic instances; checking whether exact tree-based SHAP avoids this would test how strictly the paper's motivation carries through.","The same suite could be run on a model trained without time-lagged demand features; if the ALE temperature curve still matched the control curve, it would show whether the ambient-temperature signal is learned directly or through correlated demand features.","A natural next experiment is to apply the ALE-versus-control-curve check to forecast horizons beyond one hour, where ambient temperature plays a larger relative role; the paper's data are hourly and do not address this."],"forward_implications":["If the resemblance argument holds, operating staff can use ALE-based validation to justify deploying or retaining a black-box forecaster without a separate quantitative comparison on unseen heating seasons.","Because every chosen method avoids feature permutation and perturbation, the same four-view suite can be applied in regulated settings where explanations must not depend on synthetic, out-of-distribution feature values.","The complementary reading resolves apparent contradictions such as the flat partial dependence of previous-day energy despite its high gain: the feature is important only during the opening hours of daily operation, a nuance visible in ICE plots.","The interpretive template should transfer to other district-heating substations and to other process industries that have a known control curve, allowing domain experts to validate models by checking that ALE shapes mirror engineering knowledge."],"supporting_citations":[{"why":"Supplies the district-heating control-curve context and the earlier explainable-forecasting framing that the trustworthiness argument builds on.","marker":"[1]"},{"why":"Defines the data preprocessing and training pipeline, including feature enrichment with time-lagged demand and temperature, on which the interpreted model is built.","marker":"[6]"},{"why":"Introduces XAI and LIME, which the paper contrasts with its chosen methods and excludes for relying on local linear approximation and perturbation.","marker":"[7]"},{"why":"Defines partial dependence plots, one of the four global feature-importance views used in the study.","marker":"[8]"},{"why":"Defines accumulated local effects, the method whose ambient-temperature curve is compared with the control curve as evidence of model quality.","marker":"[9]"},{"why":"Defines SHAP values, the game-theoretic global feature-importance view the paper treats as the most reliable.","marker":"[10]"},{"why":"Defines permutation importance, which the paper deliberately excludes because it creates unrealistic out-of-distribution feature values.","marker":"[11]"}],"fun_headline_variants":["Four XAI views agree: heat model matches control curve","ALE curve mirrors DHS control: heat model proven trustworthy","Control-curve match validates black-box heat forecasting model","Heat demand trust: four methods, one control-curve match"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument stands on accepting that the visual resemblance between the ALE curve of ambient temperature and the district-heating control curve is strong evidence of forecasting-model quality; the paper provides no quantitative similarity criterion, no baseline comparison with a deliberately wrong model, and no test on an unseen heating season to establish that this resemblance could not arise from a bad model.","fun_headline_variants_meta":{"raw":{"variants":["Four XAI views agree: heat model matches control curve","ALE curve mirrors DHS control: heat model proven trustworthy","Control-curve match validates black-box heat forecasting model","Heat demand trust: four methods, one control-curve match"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001091,"raw_usage":{"total_tokens":4538,"prompt_tokens":904,"completion_tokens":3634,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":3567}},"tokens_in":520,"tokens_out":3634,"duration_ms":29542,"temperature":1.0,"reasoning_tokens":3567,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:01:52.596820+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a deliberately wrong heat-demand model on the same data, for example a model that ignores ambient temperature or one that predicts only the daily average, and run the same four explanation methods: if its ambient-temperature ALE curve still resembles the control curve, or if the trustworthy model's resemblance disappears when evaluated on a new heating season, then the resemblance test cannot separate good from bad models.","supporting_citations":[{"cited_title":"Explainable heat demand forecasting for the novel control strategies of district heating systems.Annual Reviews in Control, 53:405–418, 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the district-heating control-curve context and the earlier explainable-forecasting framing that the trustworthiness argument builds on."},{"cited_title":"is all you need","cited_arxiv_id":null,"evidence_quote":"Defines the data preprocessing and training pipeline, including feature enrichment with time-lagged demand and temperature, on which the interpreted model is built."},{"cited_title":"why should i trust you?","cited_arxiv_id":null,"evidence_quote":"Introduces XAI and LIME, which the paper contrasts with its chosen methods and excludes for relying on local linear approximation and perturbation."},{"cited_title":"Apley and Jingyu Zhu","cited_arxiv_id":null,"evidence_quote":"Defines accumulated local effects, the method whose ambient-temperature curve is compared with the control curve as evidence of model quality."}],"review_version":1}