{"id":"4b8782c4-53a0-4362-ad9a-2f69ea1abe9b","arxiv_id":"2501.14136","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On synthetic AND/OR/XOR datasets with perfectly accurate models, every tested saliency method sometimes ranks a truly irrelevant input above a necessary one, so the scores cannot be trusted as relevance rankings.","lead":"This paper tests 14 popular AI explanation methods on simple logic puzzles where the correct important clues are known, and finds that none of them reliably highlight the inputs the model actually needs. The authors also introduce a new weight-based reading of explanation scores and show which methods work best for which logical situations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NIB/GIB failure metrics require every input from any minimal set (and every Rmax input) to score above baseline, conflicting with the paper's own 'at least one relevant input' Assumption B and with LeRF; the all-methods-fail claim may be an artifact of this exhaustive scoring rule.","rationale":"I focus on the metric construction rather than on model error because the paper's own framing makes the model-error issue hard to adjudicate: a 100%-accurate model can still have nondeterministic logit sensitivity to baseline inputs, and the paper explicitly wants explanations of the classification decision, not of the logit. The metric quantification issue, by contrast, is internal to the paper: NIB/GIB's universal reading contradicts the existential goal stated in Section III-A and the 'at least one relevant input' wording of Assumption B. The paper has substantial assets: 21 datasets, 14 first-order methods, 336 experiment configurations, released code, and per-scenario rankings that are practically useful. But the headline negative result depends on the strictest possible reading of what a correct explanation must contain. The proposed concrete test would show whether the all-methods-fail result disappears once a valid single minimal set is accepted. Until that is checked, the central claim is conditional. This does not change the reader's CONDITIONAL verdict, but it identifies a different and more specific condition than the reader's baseline-sensitivity concern.","tokens_in":26433,"tokens_out":9582,"duration_ms":93694,"concrete_test":"Recompute NIB and GIB using an existential criterion on the released code and the 100%-accuracy split models: for each sample, pass if there exists at least one minimal set r in Rd_min such that every input in r is scored strictly above the highest baseline input of that sample. For the redundant scenarios, also record the fraction of samples for which at least one sufficient input (one negative for AND-negative, one positive for OR-positive) is above baseline. If several of the 14 methods drop to 0% under this criterion, the universal failure claim is not supported as stated; if all remain above 0% by a large margin, the exhaustive scoring rule is not the sole driver and the conclusion is robust to the metric choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the quantification in the NIB/GIB definitions (Section IV-C, building on Section III-A). NIB counts a sample as failed when any input that belongs to any minimal set r in Rd_min is scored below the highest baseline input; GIB additionally counts every input in Rd_max that falls below the baseline. For an OR-positive sample with several positive inputs, or an AND-negative sample with several negative inputs, any single positive/negative input is a sufficient explanation; the paper's Assumption B says only 'at least one input needs to be relevant'. A saliency map that ranks one sufficient input above the baseline and the other redundant sufficient inputs below it is therefore fully consistent with Assumptions A-E and with the LeRF protocol (removing the low-scored redundant inputs first does not reduce accuracy while another sufficient input remains). The NIB/GIB definitions nevertheless mark such samples as failures, because the low-scored redundant inputs belong to some Rd_min/Rd_max. Section III-A describes the check as whether the ranking 'matches one possible reasoning r in Rd_min', which is an existential criterion, but NIB/GIB as defined and reported implement a universal criterion over all possible minimal sets and over all Rd_max inputs. The central claim that no method can consistently capture needed classification information is thus partly an artifact of demanding exhaustive enumeration of all alternative logical proofs, not of demanding one coherent saliency ranking.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the authors' ANDOR logical dataset framework to evaluate the reliability of saliency maps. It trains CNN and Transformer models on 21 ANDOR dataset configurations and applies 14 first-order and 4 second-order attribution methods, spanning 336 experiments and 404,880 trained models. The central empirical claim is that no analysed saliency method consistently ranks all logically necessary inputs above a non-informative baseline; IntegratedGradients, LRP-Rollout, and Attention are identified as the best, yet still imperfect. The paper also proposes the Global Coherence Representation (GCR) as a weight-based, global interpretation and evaluation approach, and it provides per-scenario rankings of methods across multiple metrics (NIB, GIB, logical accuracy, DCA, correlation, GCR fidelity).","tokens_in":26653,"tokens_out":9608,"duration_ms":81059,"significance":"If the central negative result holds, it is an important contribution to XAI, suggesting fundamental limits of first-order attribution maps even in fully controlled logical tasks with externally defined ground truth. The scale of the experiments and the explicit logical ground truth are strengths, as is the open-source code. The paper also offers practical per-scenario comparisons of 14 first-order and 4 second-order methods, which is valuable. However, the validity of the main claim depends on the strictness of the NIB/GIB metrics and on the assumption that a 100%-accurate model's decision behavior is fully reducible to the logical formula. The GCR proposal is interesting but requires additional validation to avoid circularity.","major_comments":[{"comment":"The NIB and GIB definitions implement a universal quantification over all minimal/maximal sets, which is stronger than the paper's own Assumption B and the Section III-A statement that the ranking should \"match one possible reasoning r in Rd_min\". For an OR-positive sample with two positive inputs, if one positive input scores above the baseline and the other below, NIB counts the sample as failed even though a sufficient minimal set is correctly ranked. Similarly, GIB counts every Rd_max input that falls below the baseline as a miss. This inflates failure rates and directly drives the central \"all methods fail\" claim. Please report an existential variant (e.g., the percentage of samples with at least one fully correct minimal set) and a redundant-aware variant, and clarify which of the paper's claims each metric supports.","section":"IV-C, III-A, III-C"},{"comment":"The per-method interpretation mode (AsIs/Cutoff/Absolute) is selected based on the NIB metric on the same test data used for the reported NIB results. This is a selection-on-target procedure: methods with more flexible post-processing benefit, and the reported NIB values are optimistically biased. While this does not rescue any method to a perfect score, it affects the cross-method ranking (Table IV) and the claim that IntegratedGradients is best at information capturing. Please select modes on a held-out validation set or pre-specify them per method, and report results for all three modes so readers can assess sensitivity.","section":"V-A, Table II"},{"comment":"The paper's own correlation analysis shows that baseline inputs alter model outputs: \"changing irrelevant inputs will likely (slightly) change the numeric output.\" This means the 100%-accurate models are not exact implementations of the ANDOR formula, so NIB/GIB failures could measure model sensitivity to baseline inputs rather than explanation quality. The Limitations section acknowledges that the model output does not have zero error, but it asserts that relevant inputs should still be scored highest because inputs are independent. This assertion needs quantitative support: for example, report the effect size of baseline inputs on output logits relative to relevant inputs, or verify that the NIB/GIB failures persist for models with near-zero loss.","section":"V-E, VI-B"},{"comment":"The text and Figure 12 report \"significant (p < 0.5) Pearson correlations\". If p<0.5 is literal, this is not a valid significance threshold and invalidates the correlation analysis, since nearly all correlations would be \"significant\". If it is a typo for p<0.05, the percentage-of-significant-cases panel changes substantially. Please correct the threshold and recompute the correlation results.","section":"V-E, Figure 12"},{"comment":"The GCR is both proposed as an interpretation framework and used as an evaluation tool via GCR Fidelity and tGCR Fidelity. The membership functions (Eq. 2 and Eq. 4) are defined by the authors without an axiomatic justification, and using them to evaluate saliency methods is partly circular. High GCR fidelity may reflect the averaging properties of the GCR rather than the quality of the saliency scores. Please validate the GCR against independent aggregation schemes (e.g., simple linear probes or other global aggregation baselines) and clearly state which conclusions depend on the GCR's specific assumptions.","section":"III-D, V-F"}],"minor_comments":[{"comment":"The assumptions are labeled (A), (B), (C), (D), (E), (F), (H), (I), skipping (G); please renumber or add a missing item.","section":"III-C"},{"comment":"Typo: \"confirmations bias\" should be \"confirmation bias\".","section":"Abstract"},{"comment":"The Full-DCA definition uses \"N rB\" which is undefined.","section":"IV-C"},{"comment":"The text refers to Assumption I when the relevant assumption on undesired correlations is Assumption H; please fix the cross-reference.","section":"V-E"},{"comment":"The dataset size for 2inQuaternary is given as 65526; it should be 65536 (=4^8).","section":"III-F"},{"comment":"The caption \"verage Full-DCA\" should be \"Average Full-DCA\".","section":"Appendix Figure 18"},{"comment":"Table III labels GCR FCAM/GTM values as \"Acc.\" while the text describes them as \"global differentiability\"; please clarify whether these are accuracies or normalized scores.","section":"V-H"},{"comment":"The related work could more explicitly summarize how this work differs from the authors' previous [4] in terms of new datasets, metrics, and second-order methods.","section":"II"}],"recommendation":"major_revision","confidential_remarks":"This manuscript is a direct extension of the authors' prior [4] and heavily relies on the authors' own GCR framework. The main risks are the NIB/GIB quantification issue and the selection of interpretation modes on the target metric; both are fixable. The scale of the experiments is a clear strength. The paper would benefit from having the code and configuration details preserved for review. The p<0.5 threshold appears to be a simple typo, but it should be corrected before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is a solid, heavily empirical extension of the authors' ANDOR benchmark, with real value as a comparative resource. But the headline result — 'all saliency methods fail to capture all needed classification information' — is weaker than it looks, because the main failure metrics are stricter than the paper's own assumptions.\n\nWhat's genuinely new: 12 additional dataset configurations, two new saliency methods, several new metrics, scenario-specific rankings, and a second-order GCR analysis. The experimental scale is real: 336 experiments, 404,880 trained models, and the code is public. The per-scenario tables (which method works for AND-like, OR-like, XOR-like data) are practically useful, and the authors are honest about limitations, including the real-valued output issue and the impossibility of zero-error models.\n\nThe soft spots, in order of severity. First, the NIB/GIB definitions (Section IV-C) implement a universal criterion: a sample fails if any input that belongs to any minimal set in Rd_min is scored below the baseline. But Section III-A states the check should determine whether the ranking 'matches one possible reasoning r in Rd_min', an existential criterion. These are not equivalent. For an OR-positive sample with several positive inputs, a saliency map that ranks one sufficient input above baseline and a redundant sufficient input below it satisfies Assumption B and would survive a LeRF removal test, yet NIB/GIB mark it as a failure. This directly inflates the 'no method works' claim. The fix is to either align the metric with the existential criterion or explicitly redefine the assumptions to require all minimal-set inputs to be above baseline; the paper does not do that. Second, the per-method interpretation mode is selected on the NIB metric itself; at least the authors disclose it, but it biases the main result. Third, 'p < 0.5' in the correlation section is presumably a typo for p < 0.05, but as written it is meaningless and should be corrected. Fourth, the GCR is the authors' own framework and serves both as the proposed interpretation method and as part of the evaluation, so the 'practical advantages' of second-order attribution are not independently established.\n\nWho this is for: XAI practitioners choosing methods, and anyone building controlled benchmarks for attribution evaluation. It deserves a serious referee — the experimental design is reproducible and the core result, once the metric definition is fixed, is still instructive even if it turns out to be 'saliency maps are ambiguous about which sufficient explanation they choose' rather than 'they capture no needed information.' My recommendation: send it to peer review, but expect a major revision on the metric definitions and the overclaim.","headline":"Useful controlled benchmark and scenario rankings, but the central 'all methods fail' claim rests on NIB/GIB metrics that are stricter than the paper's own existential-check assumption.","tokens_in":27265,"tokens_out":4080,"would_cite":true,"duration_ms":35074,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that all 14 first-order saliency methods fail to rank logically necessary inputs above irrelevant baselines across all 21 tested logical dataset configurations, making saliency maps ambiguous as relevance rankings.","keywords":["deep learning","attention","explainability","interpretability","logic data","attribution scores","evaluation","ANDOR"],"falsifier":"Retrain the same ANDOR models with a training objective that explicitly penalizes output sensitivity to baseline inputs, then re-apply NIB/GIB and the threshold-based masking. If no method still fails, the paper's conclusion that saliency maps are inherently ambiguous would be weakened; if the same failures persist in models with zero measured baseline sensitivity, the central claim would be strongly supported.","tokens_in":26115,"feed_emoji":"🔍","tokens_out":5361,"duration_ms":46228,"temperature":0.7,"pith_summary":"The paper aims to establish that attribution-based saliency methods cannot be trusted as unambiguous importance rankings, even in simple logical classification tasks where the ground-truth reasoning is fully known. It extends the ANDOR framework with five dataset configurations and 21 scenario settings, testing 14 first-order and 4 second-order attribution methods on CNN and Transformer models that reach 100% accuracy. Using a non-informative input baseline, the paper finds that no method consistently ranks all minimally and maximally relevant inputs above the baseline in every scenario. The best overall methods, IntegratedGradients, LRP-Rollout, and Attention, still fail in specific settings, and XOR-heavy scenarios are the hardest. The paper concludes that a single per-input score is an ambiguous explanation and argues for weight-based interpretations that can omit inputs, such as the Global Coherence Representation.","feed_headline":"All 14 saliency methods fail logical relevance tests","feed_subtitle":"None consistently ranks needed inputs above irrelevant baselines, so the paper calls for weight-based explanations.","key_machinery":"The central machinery is the ANDOR dataset framework, which generates fully controlled logical classification tasks with AND, OR, and XOR gates plus an irrelevant baseline block, so the complete set of minimally and maximally sufficient feature sets (R_min and R_max) is known for every sample. The paper measures each saliency method's ranking against those ground-truth information sets with the NIB and GIB metrics, and it applies the Global Coherence Representation (GCR), a symbolic global weight-aggregation model that turns local attribution scores into per-symbol, per-position class weights and can classify by weighted membership. The GCR's first-order GTM and second-order FCAM variants show whether saliency scores can function as class-relevance weights when inputs are fully omitted.","core_discovery":"Under the paper's Assumptions A-I, every one of the 14 first-order saliency methods violates at least one expected property in at least one of the 21 ANDOR dataset configurations: for some sample, a logically necessary input from the minimal information set R_min, or a globally relevant input from R_max, is scored below the highest baseline (irrelevant) input. The paper shows that failures concentrate in complementary-information cases and XOR gates, that information is sometimes leaked into masked inputs (DCA metrics), and that second-order aggregation (FCAM) recovers some global class differentiability that first-order scores lose. The authors conclude that saliency scores are ambiguous as local relevance rankings and that a weight-based interpretation, where the scores act as class-relevance weights and inputs can actually be omitted, is a more desirable verification direction.","pith_inferences":["If the reported failures partly reflect real-valued model outputs rather than explanation-method flaws, then the NIB/GIB metrics may conflate explanation quality with model complexity; a cleaner test would measure NIB/GIB on models with explicitly zero sensitivity to baseline inputs.","The scenario ranking suggests a practical selector: for AND-OR-like tasks, perturbation-based methods may be defensible, while XOR-heavy tasks should probably receive higher-order explanations by default.","A direct extension would replace the fixed 'highest baseline score' threshold with per-method calibrated thresholds; if ranking conclusions change, part of the observed ambiguity is a threshold artifact rather than an inherent property of saliency scores."],"forward_implications":["No analysed first-order saliency map can be read as a standalone ranking of feature importance; every method violates at least one of the paper's stated assumptions in at least one tested configuration.","XOR-like or complementary-information settings are the hardest for all methods, supporting the paper's case that second-order attribution scores are needed to capture joint input relevance.","Methods with good local information capture, such as IntegratedGradients, do not necessarily produce globally comparable scores, so local and global interpretations of the same method can disagree.","A weight-based interpretation such as the GCR can preserve class differentiability even when first-order rankings fail, suggesting that input-omitting interpretations are more promising than raw heatmap rankings."],"supporting_citations":[{"why":"Supplies the ANDOR logical dataset framework and the earlier finding that all tested methods failed to capture full classification information.","marker":"[4]"},{"why":"Provides the remove-and-retrain masking methodology that the paper adapts for its LeRF-based evaluation.","marker":"[27]"},{"why":"Earlier logical-dataset evaluation that the paper extends by adding a non-informative baseline and exhaustive input coverage.","marker":"[41]"},{"why":"Documents logic traps and confirmation bias in attribution evaluation, motivating the paper's assumption-based benchmark.","marker":"[2]"},{"why":"SHAP-IQ supplies the second-order interaction scores used as one of the four second-order attribution methods.","marker":"[13]"},{"why":"Provides the Transformer LRP implementation and CLS-token aggregation used by several first- and second-order methods.","marker":"[55]"},{"why":"Introduces the Global Coherence Representation that the paper applies for weight-based saliency interpretation and fidelity evaluation.","marker":"[5]"}],"fun_headline_variants":["All 14 saliency methods flunk logical relevance tests","Saliency maps ambiguous: every method fails logic checks","Logical tests expose all 14 saliency methods as ambiguous","14/14 saliency methods fail logical relevance, proving ambiguity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the ANDOR truth tables completely specify which inputs a correct explanation must mark as relevant, and that every relevant input must outscore every baseline input in the same sample; if the perfectly accurate model's real-valued output is genuinely sensitive to those baseline inputs, the reported failures may measure model complexity rather than explanation quality.","fun_headline_variants_meta":{"raw":{"variants":["All 14 saliency methods flunk logical relevance tests","Saliency maps ambiguous: every method fails logic checks","Logical tests expose all 14 saliency methods as ambiguous","14/14 saliency methods fail logical relevance, proving ambiguity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0005,"raw_usage":{"total_tokens":2402,"prompt_tokens":854,"completion_tokens":1548,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":470,"completion_tokens_details":{"reasoning_tokens":1481}},"tokens_in":470,"tokens_out":1548,"duration_ms":13282,"temperature":1.0,"reasoning_tokens":1481,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:21:28.091627+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same ANDOR models with a training objective that explicitly penalizes output sensitivity to baseline inputs, then re-apply NIB/GIB and the threshold-based masking. If no method still fails, the paper's conclusion that saliency maps are inherently ambiguous would be weakened; if the same failures persist in models with zero measured baseline sensitivity, the central claim would be strongly supported.","supporting_citations":[{"cited_title":"Saliency methods are encoders: Analysing logical relations towards interpreta- tion,","cited_arxiv_id":null,"evidence_quote":"Supplies the ANDOR logical dataset framework and the earlier finding that all tested methods failed to capture full classification information."},{"cited_title":"A benchmark for interpretability methods in deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides the remove-and-retrain masking methodology that the paper adapts for its LeRF-based evaluation."},{"cited_title":"Evaluation of post-hoc xai approaches through synthetic tabular data,","cited_arxiv_id":null,"evidence_quote":"Earlier logical-dataset evaluation that the paper extends by adding a non-informative baseline and exhaustive input coverage."},{"cited_title":"Logic Traps in Evaluating Attribution Scores","cited_arxiv_id":"2109.05463","evidence_quote":"Documents logic traps and confirmation bias in attribution evaluation, motivating the paper's assumption-based benchmark."},{"cited_title":"Shap-iq: Unified approximation of any-order shapley interactions,","cited_arxiv_id":null,"evidence_quote":"SHAP-IQ supplies the second-order interaction scores used as one of the four second-order attribution methods."},{"cited_title":"Transformer inter- pretability beyond attention visualization,","cited_arxiv_id":null,"evidence_quote":"Provides the Transformer LRP implementation and CLS-token aggregation used by several first- and second-order methods."},{"cited_title":"Constructing global coherence representations: Identifying interpretability and coherences of transformer attention in time series data,","cited_arxiv_id":null,"evidence_quote":"Introduces the Global Coherence Representation that the paper applies for weight-based saliency interpretation and fidelity evaluation."}],"review_version":1}