{"id":"b71503d5-ba9f-4395-8177-1a10dbd2b3a2","arxiv_id":"2502.00088","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"The proposed expected accuracy interval is a simple symmetric band around current accuracy, but it lacks derivation and fails on part of the paper's own data.","lead":"This paper proposes a metric called the expected accuracy interval (EAI), which uses the model's current accuracy and the SHAP contribution of the most important feature to predict a range for accuracy after that feature is removed or permuted. The observed accuracies in the paper's own experiments frequently fall outside the predicted range, so the interval does not reliably bound what it claims to predict.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim collapses at Eq. (2): Expected Delta = initial_acc × FCP is asserted without derivation, making EAI a rescaling of SHAP, and the paper's own table captions report rows outside the resulting interval.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing point: Eq. (2) is asserted, not derived, and the linear scaling of accuracy by FCP is unsupported. My reading agrees and treats this as the single point on which the central claim stands or falls. If Eq. (2) were true, the EAI interval would follow trivially and the remaining criticisms (poor evaluation protocol, in-sample accuracy) could be fixed by better experiments. If Eq. (2) is false, the metric is vacuous regardless of how cleanly the experiments are run. The paper provides no derivation, no formal verification, no code, and no calibration test. Its own empirical tables are the only evidence, and the captions state that some observed accuracies fall outside the proposed interval, which is direct falsification under the paper's own criterion. The additional protocol problem of training and testing on the full dataset further undermines any quantitative claim, but it is secondary to the unsupported core equation. I therefore recommend REJECT, consistent with the reader's verdict, and my agreement_with_reader is 'agree' because the weakest assumption aligns. No independent support (e.g., machine-checked proof or reproducible code) offsets this failure. The synthetic test above would settle the concern definitively: if the predicted equality holds across correlation levels, the metric has content; if not, the rejection is confirmed.","tokens_in":5807,"tokens_out":8928,"duration_ms":86392,"concrete_test":"Generate 10,000 samples with correlated features X1,X2 (rho in {0,0.3,0.6,0.9}) and a binary label from a logistic model with beta1=beta2=1 plus noise. Fit a logistic regression, compute SHAP FCP and Expected Delta via Eqs. (1)-(2). Remove the top-SHAP feature, refit on the training split, and measure accuracy on a held-out test set. Repeat 100 times per rho. If the observed drop systematically differs from initial_acc × FCP, especially under high collinearity, Eq. (2) is false. Also compute how often the held-out retrained accuracy falls inside the EAI interval; a genuine interval should cover observed accuracy, not merely bracket initial_acc.","verdict_should_be":"REJECT","load_bearing_attack":"The proposed metric rests entirely on Eq. (2): Expected Delta = initial_acc × FCP, with FCP defined in Eq. (1) as the SHAP score of the most significant feature divided by the sum of all SHAP scores. No derivation or model of retraining is given. SHAP values are additive in the model output, not in accuracy; FCP need not lie in [0,1] unless extra constraints are imposed, and no such constraints are stated. Under collinearity, the paper's own motivating argument predicts that removing the top feature should cause a smaller drop than its individual contribution because correlated features remain; Eq. (2) instead assumes the drop is exactly current accuracy times that contribution. The paper also reports accuracy increasing after removal (e.g., Table 1, iteration 16: 0.5164 to 0.6716), which Eq. (2) cannot represent with a positive FCP. The EAI in Eqs. (3)-(5) is just the symmetric interval [initial_acc − Expected Delta, initial_acc + Expected Delta]; it has no coverage guarantee for the accuracy of a retrained model. The only empirical support is Tables 1–3, and the captions explicitly mark rows falling outside the interval. Thus, unless Eq. (2) is derived or controlled tests show the equality holds, EAI is not a prediction of ROAR/PI accuracy; it is a rescaling of SHAP contribution shares.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that ROAR (RemOve And Retrain) and permutation importance, two common proxy-based XAI evaluation metrics, can mislead when features are collinear or when there are multiple informative features, and it supports this with empirical examples on one diabetes classification dataset, one wine-quality regression dataset, and one simulated classification dataset. To address the limitation, the paper proposes an Expected Accuracy Interval (EAI) computed from the current model accuracy and the SHAP contribution share of the most significant feature: FCP = SMSF/SSOAF, Expected Delta = initial_acc * FCP, and EAI = [initial_acc - Expected Delta, initial_acc + Expected Delta]. The paper reports, in Tables 1-3, the observed accuracy after sequentially removing the most significant feature and compares it with the proposed interval.","tokens_in":6136,"tokens_out":5070,"duration_ms":42345,"significance":"If the EAI were valid, it would give practitioners a simple way to anticipate the effect of removing the most informative feature, especially in settings with collinear features, and the paper does usefully illustrate that ROAR and permutation importance do not always produce sharp accuracy drops. However, the central contribution is not established: Equation (2) is asserted without derivation or justification, the evaluation protocol trains and tests on the full dataset, and multiple observed accuracies in the paper's own tables fall outside the claimed intervals. The proposed metric is therefore currently a rescaling of the SHAP contribution share rather than a validated prediction, and the empirical evidence contradicts the abstract's central claim.","major_comments":[{"comment":"Equation (2), Expected Delta = initial_acc * FCP, is asserted without any derivation, theoretical justification, or empirical calibration. It assumes that the change in accuracy after removing a feature is exactly the product of the current accuracy and the SHAP contribution share of that feature. This linear and symmetric assumption is not implied by SHAP's additivity, which concerns model outputs rather than accuracy, and it is not supported by the paper's own discussion of collinear features, where the paper argues that removing a collinear feature should produce a smaller drop than its individual contribution. In addition, FCP need not lie in [0,1] when SHAP scores can be negative, yet the interval construction in Eqs. (3)-(5) implicitly assumes a positive bounded FCP. Since the EAI is defined entirely through Eq. (2), the central predictive claim is unsupported.","section":"Section 3, Eq. (2)"},{"comment":"The implementation section states that \"The whole data was used in the training and test.\" This means there is no held-out validation set, no cross-validation, and no repeated sampling protocol. The reported accuracies in Tables 1-3 are therefore optimistically biased and not reproducible as estimates of generalization performance. This makes the comparison between observed accuracies and the EAI interval not a valid test of the proposed prediction.","section":"Section 4.3, Implementation"},{"comment":"The captions of Tables 1 and 2 explicitly state that rows highlighted in red are outside the expected accuracy interval, and the observed values confirm this. For example, in Table 1, after iteration 16 the accuracy increases from 0.5164 to 0.6716, which is above the upper bound 0.6397 computed from the previous iteration, and after iteration 19 the observed accuracy 0.1513 is below the lower bound 0.2952. In Table 2, iterations 2, 5, 6, 7, and 8 show observed accuracies below the lower bound of the claimed interval. These violations directly contradict the abstract's claim that EAI predicts the upper and lower bounds of the accuracy when ROAR or permutation importance is implemented.","section":"Tables 1 and 2"},{"comment":"The paper's motivation is internally inconsistent with its proposed formula. Section 2 argues that when features are collinear, removing the most significant feature may not cause a sharp decline because other correlated features remain; yet Eq. (2) sets the expected accuracy drop exactly equal to the current accuracy times the SHAP contribution share, which assumes the drop is proportional to that feature's independent contribution. The empirical observation that accuracy can increase after removal (e.g., Table 1, iteration 16) cannot be represented by Eq. (2) at all when FCP is positive. The proposed interval therefore does not follow from the stated limitations of ROAR and permutation importance.","section":"Section 2 vs. Section 3"}],"minor_comments":[{"comment":"The abstract contains the typos \"the the\" and \"IP\" where \"PI\" (permutation importance) is intended.","section":"Abstract"},{"comment":"Equation (5) reads EAI = [LI - UI], but the intended expression is EAI = [LI, UI] or [LI, UI] with a comma; as written, the formula is not the interval described in the text.","section":"Section 3, Eq. (5)"},{"comment":"The text says \"More details about the data can be found here\" and \"More details about the dataset can be found here,\" but no working links are provided; this should be replaced with proper references or hyperlinks.","section":"Section 4.1"},{"comment":"The sentence \"The figure shows the most significant feature based on the SHAP score in the first iteration is GenHlth\" refers to Table 1, not a figure; the cross-reference should be corrected.","section":"Section 5.1"},{"comment":"There is a typo in the limitations paragraph: \"I addition\" should be \"In addition.\" Also, \"RemOve And Retrain\" should be \"Remove and Retrain,\" and Figure 1's label \"retain\" should likely be \"retrain.\"","section":"Section 6"},{"comment":"The data availability statement says the simulated dataset can be downloaded from the supplementary, but no supplementary material or link is included in the manuscript; this should be fixed for reproducibility.","section":"Section 8"}],"recommendation":"reject","confidential_remarks":"The paper's central proposal is not supported by its own evidence, and the core equation (Eq. (2)) is asserted without derivation. Even a substantial revision would need to replace the proposed metric with a derived and validated interval, which goes beyond what the current manuscript provides. The paper's empirical demonstrations of ROAR/PI limitations are useful but do not rescue the main claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's empirical reminder—that removing the top SHAP feature under collinearity does not reliably drop accuracy and can even raise it—is worth keeping. But the proposed EAI metric is a deterministic rescaling of SHAP and is contradicted by the paper's own tables.\n\nWhat is genuinely here: the paper runs ROAR-style retraining on three datasets (diabetes, wine, simulated) and shows accuracy often stays flat or rises after removing the top feature. That is a real phenomenon, and the tables make it visible. For practitioners who use ROAR/PI as their only XAI check, this is a useful caution. The author also correctly notes that ROAR/PI give no numerical expectation for the change in performance. Those observations are fine as far as they go. The citations are standard; the two self-citations are on topic.\n\nThe soft spots are large. Eq. (2) asserts Expected Delta = initial_acc × FCP without derivation or model of retraining. Since FCP is just the normalized SHAP score of the top feature, EAI is an algebraic rearrangement of the current accuracy and that SHAP share; it adds no information beyond what went in. It has no coverage guarantee for the accuracy of a retrained model. FCP need not lie in [0,1] unless extra constraints are stated, and none are. The paper's own motivation—collinearity makes feature contributions nonlinear—argues against the linear scaling in Eq. (2). And the evidence does not support the claim: the table captions mark rows outside the interval, and the observed accuracy increases after removal (e.g., Table 1, iteration 16 to 17: 0.5164 to 0.6716) cannot be represented by Eq. (2) with a positive FCP.\n\nThe evaluation protocol is also unsound: the model is trained and tested on the whole dataset in each iteration, so the reported accuracies are optimistic and not reproducible. That would matter even for the descriptive part of the paper. The two real datasets are publicly available and the code appears straightforward, so the empirical demonstration could be redone properly, but as it stands the numbers are not trustworthy.\n\nWho gets value from this: a reader who wants a concrete example of ROAR/PI misbehaving under collinearity might find the tables illustrative. Nobody should use EAI as a prediction interval. The paper does not deserve a serious referee; the central claim is a rescaling that the paper's own data refute, and the evaluation protocol would need to be redone from scratch. My recommendation is to desk reject.","headline":"The ROAR/PI caution is real, but EAI is a rescaling of SHAP and the paper's own tables contradict it.","tokens_in":6607,"tokens_out":4179,"would_cite":false,"duration_ms":35800,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes Expected Accuracy Interval (EAI), a metric that predicts the upper and lower bounds of model accuracy after removing or permuting the most informative feature.","keywords":["explainable AI","ROAR","permutation importance","SHAP","expected accuracy interval","multicollinearity","feature importance evaluation","proxy-based evaluation"],"falsifier":"A direct falsifier is to compute, on datasets with controlled degrees of collinearity, the fraction of removed-feature iterations in which the observed retrained accuracy falls outside the EAI interval; if that fraction is high across many seeds, the interval is not predictive. Even simpler, fit a regression of the observed accuracy drop on $\\mathrm{initial\\_acc} \\times FCP$; if the slope is not near 1, Equation 2 is not the right mechanism.","tokens_in":5615,"feed_emoji":"🎯","tokens_out":5011,"duration_ms":47223,"temperature":0.7,"pith_summary":"This paper argues that two standard proxy measures for evaluating explainable-AI methods, ROAR (remove and retrain) and permutation importance, do not reliably reveal the most informative feature when features are collinear, when several features are informative, or when baseline accuracy is low. It then proposes a new metric, the Expected Accuracy Interval (EAI), which converts the top feature's SHAP contribution share and the current model accuracy into a predicted interval for the accuracy after that feature is removed or permuted. The paper's empirical demonstrations on two real datasets and one simulated dataset show cases where accuracy barely drops or even rises after removing the top feature, and it reports that EAI is especially useful in those collinear settings. If the metric works as claimed, practitioners gain a concrete, score-based expectation of how much removing a feature should hurt, rather than relying on the qualitative expectation of a sharp decline.","feed_headline":"EAI metric predicts accuracy bounds after top-feature removal","feed_subtitle":"ROAR and permutation importance mislead under collinearity; the new interval gives a concrete check.","key_machinery":"The load-bearing object is the feature contribution percentage $FCP = S_{MSF}/S_{SOAF}$ (Equation 1), combined with the proportionality assumption $\\mathrm{Expected}\\,\\Delta = \\mathrm{initial\\_acc} \\times FCP$ (Equation 2). These feed the interval $EAI = [\\mathrm{initial\\_acc} - \\mathrm{Expected}\\,\\Delta,\\; \\mathrm{initial\\_acc} + \\mathrm{Expected}\\,\\Delta]$ (Equations 3-5). $FCP$ is meant to translate a model-agnostic explanation score, specifically SHAP's per-feature attribution, into a forecast of how much removing that feature changes accuracy, and the interval width is used as a signal of whether the remaining features have similar impact.","core_discovery":"The central claim is that EAI predicts the post-removal accuracy interval as $[\\mathrm{initial\\_acc} - \\mathrm{initial\\_acc} \\times FCP, \\; \\mathrm{initial\\_acc} + \\mathrm{initial\\_acc} \\times FCP]$, where $FCP$ is the SHAP score of the most significant feature divided by the sum of all SHAP scores. The paper demonstrates with tabular regression and classification experiments that ROAR and permutation importance often fail to produce a sharp performance drop when the top feature is collinear with other strong features, and that accuracy can even increase after removing the top feature. It therefore proposes EAI as a supplement that gives a numerical interval rather than a direction-only expectation, with wider intervals indicating that the next most significant feature has a larger impact relative to the removed one.","pith_inferences":["I expect the same interval construction to transfer to any per-feature score, such as LIME-style weights or integrated gradients, because Equations 1-2 only consume a score vector; this is an extension the paper does not test.","If the linear scaling in Equation 2 is calibrated or replaced by a learned mapping, EAI could become a tighter predictor on low-accuracy models, where the paper's own tables show some iterations falling outside the interval.","The width of EAI could be reinterpreted as a collinearity diagnostic: intervals that stay wide after removing top features signal redundancy in the feature set, a use that could be tested on synthetic data with controlled correlation.","A natural testable extension is to see whether EAI's hit rate remains high when the underlying model is a nonlinear tree ensemble instead of logistic or linear regression, since SHAP values behave differently there."],"forward_implications":["If EAI holds, a practitioner can remove the top SHAP feature and expect the retrained model's accuracy to fall inside the stated interval, making ROAR and permutation-importance evaluations quantitative rather than direction-only.","For collinear models, EAI provides a reason not to treat a small accuracy drop as a failure of the XAI method, since the interval encodes the replacement effect of collinear features.","A wider interval after removing the top feature indicates the newly most significant feature carries greater impact, which can guide feature-selection and risk-factor interpretation.","EAI requires only an existing score-based explanation, so it can be added to any pipeline that already computes SHAP-like per-feature contributions."],"supporting_citations":[{"why":"Defines the ROAR remove-and-retrain proxy that EAI is designed to predict the outcome of.","marker":"[3]"},{"why":"Identifies multicollinearity as a problem for XAI, motivating the paper's central limitation.","marker":"[4]"},{"why":"Provides SHAP, the per-feature score source used to compute the feature contribution percentage FCP.","marker":"[5]"},{"why":"Prior work on dependent features in XAI that supports the collinearity concerns behind EAI.","marker":"[6]"},{"why":"Defines permutation importance, the other proxy the paper revisits.","marker":"[7]"},{"why":"Supplies the CDC Diabetes Health Indicators dataset used in the binary classification experiments.","marker":"[8]"},{"why":"Supplies the Wine Quality dataset used in the linear regression experiments.","marker":"[9]"}],"fun_headline_variants":["EAI predicts accuracy bounds after collinear feature removal","ROAR and PI mislead; EAI gives accuracy interval","EAI: interval check for ROAR/permutation failures","Accuracy bounds after top-feature removal: EAI's prediction","Collinearity breaks ROAR/PI; EAI predicts the interval"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The prediction stands on the assumption in Equation 2 that a feature's SHAP share of total scores translates linearly into the same share of model accuracy; if that linear scaling is wrong, EAI is only a rescaling of SHAP, not an accuracy forecast.","fun_headline_variants_meta":{"raw":{"variants":["EAI predicts accuracy bounds after collinear feature removal","ROAR and PI mislead; EAI gives accuracy interval","EAI: interval check for ROAR/permutation failures","Accuracy bounds after top-feature removal: EAI's prediction","Collinearity breaks ROAR/PI; EAI predicts the interval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000635,"raw_usage":{"total_tokens":2901,"prompt_tokens":892,"completion_tokens":2009,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":1923}},"tokens_in":508,"tokens_out":2009,"duration_ms":13033,"temperature":1.0,"reasoning_tokens":1923,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T20:35:05.204971+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct falsifier is to compute, on datasets with controlled degrees of collinearity, the fraction of removed-feature iterations in which the observed retrained accuracy falls outside the EAI interval; if that fraction is high across many seeds, the interval is not predictive. Even simpler, fit a regression of the observed accuracy drop on $\\mathrm{initial\\_acc} \\times FCP$; if the slope is not near 1, Equation 2 is not the right mechanism.","supporting_citations":[{"cited_title":"A benchmark for interpretability methods in deep neural networks","cited_arxiv_id":null,"evidence_quote":"Defines the ROAR remove-and-retrain proxy that EAI is designed to predict the outcome of."},{"cited_title":"Multilayer Perceptron of Software Complexity Metrics for Explainable Multi- collinearity Mitigation and Defect Localization","cited_arxiv_id":null,"evidence_quote":"Identifies multicollinearity as a problem for XAI, motivating the paper's central limitation."},{"cited_title":"Characterizing the Contribution of Dependent Features in XAI Methods","cited_arxiv_id":null,"evidence_quote":"Prior work on dependent features in XAI that supports the collinearity concerns behind EAI."},{"cited_title":"Permutation importance: a corrected feature importance measure","cited_arxiv_id":null,"evidence_quote":"Defines permutation importance, the other proxy the paper revisits."},{"cited_title":"CDC National Health Report: leading causes of morbidity and mortality and associated behavioral risk and protective factors–United States, 2005-2013","cited_arxiv_id":null,"evidence_quote":"Supplies the CDC Diabetes Health Indicators dataset used in the binary classification experiments."},{"cited_title":"Modeling wine preferences by data mining from physicochemical properties","cited_arxiv_id":null,"evidence_quote":"Supplies the Wine Quality dataset used in the linear regression experiments."}],"review_version":1}