{"id":"250675b4-bed3-4ce0-ada0-d66cdfb79a90","arxiv_id":"2412.09964","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Hi-Fi decomposes the LOCO feature importance into unique, redundant, and synergistic components by finding feature subsets that minimize and maximize prediction-error reduction.","lead":"The authors propose Hi-Fi, a method that splits a feature's importance score in regression models into a direct, unique part and two cooperative parts, called redundant and synergistic. The method is demonstrated on wine quality and on simulated proton/pion detector data, and could help explainable AI distinguish when features act alone versus together.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core claim that min/max LOCO equals redundancy/synergy is an unproven analogy: Eq. (2) is a tautology, and the labels need a benchmark against an established PID before Figs. 3-6 can be interpreted.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the mapping from min/max LOCO to redundancy and synergy is asserted but not derived. My stress-test agrees. The identity in Eq. (2) is tautological; the only substantive content is the semantic labeling, and that labeling is not supported by any independent derivation or external benchmark. The paper is honest about this limitation (the variance-based substitute 'sacrifices the formal rigor of information theory'), and the code is available, so the additional check I propose is cheap and decisive. The toy example is not a sufficient validation because it was designed to satisfy the heuristic. Given this unresolved semantic gap, the CONDITIONAL verdict is appropriate: the method may be internally consistent and the definitions may be useful, but the central interpretative claim, that the three components correspond to unique, redundant, and synergistic information in any established sense, remains unverified. I would not reject the paper, because the definitions are explicit and reproducible and the authors do not claim more than an analogy, but I also would not accept it without the benchmark or an explicit reframing of the components as definitional rather than information-theoretic.","tokens_in":7027,"tokens_out":8321,"duration_ms":108513,"concrete_test":"Generate N=1e5 samples from the linear Gaussian model Y = aX + bZ + N with X and Z standardized Gaussian with correlation rho, and compute Hi-Fi's R, S, U for X using exhaustive subsets of {Z}. Compare these against Barrett's closed-form Gaussian PID redundancy and synergy for the triplet (X, Z, Y) across a grid of parameters, e.g., a,b in {0.2,0.5,1.0} and rho in {-0.8,-0.4,0.0,0.4,0.8}. If for any parameter point the sign or zero/nonzero pattern of R or S disagrees with the PID values, then min/max LOCO does not implement PID redundancy/synergy, and the semantic labels in Figs. 3-6 are not supported. If they agree across the grid, the analogy is empirically validated for the Gaussian case.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decomposition is algebraically trivial: for any subsets zmin and zmax, Lzmax = Lzmin + (Lempty - Lzmin) + (Lzmax - Lempty). The entire semantic content is in the assertions that the subset minimizing Lz 'captures the amount of redundancy R' and that the subset maximizing Lz 'leads to the amount of synergy S'. These assertions are made by analogy to the transfer-entropy decomposition of [13] and are not derived. The paper's own note that replacing entropy with variance 'sacrifices the formal rigor of information theory' (citing [11]) admits that the variance-based object is not the PID quantity. In PID, redundancy and synergy are defined axiomatically and are properties of the full set of sources; they need not coincide with the min and max of a single-variable LOCO over subsets. The toy example is constructed so that the heuristic gives the intended labels, so it does not independently validate the mapping. Without a benchmark against a recognized PID measure, statements such as 'X1 and X2 are synergistic, while X3 and X4 are redundant' (Fig. 1) and the analogous interpretations in Figs. 3-6 are not supported as factual findings about information-theoretic cooperation; they are consequences of the chosen definitions. This concern is load-bearing because the paper's novelty is precisely the semantic decomposition, not the LOCO values themselves.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Hi-Fi, an adaptive version of Leave One Covariate Out (LOCO) in which, for a given driver variable X, one searches for the subset of covariates that minimizes and the subset that maximizes the LOCO value. Defining U = L_{zmin}, R = L_empty - L_{zmin}, and S = L_{zmax} - L_empty leads to the identity L_{zmax} = S + R + U (Eq. 2). The authors interpret U, R, and S as unique, redundant, and synergistic predictive contributions, respectively, and illustrate the method on a constructed Gaussian toy model, a wine-quality dataset, and a GEANT-simulated proton/pion discrimination problem. A greedy search with permutation-based stopping is used for scalability, and code is provided.","tokens_in":7332,"tokens_out":4376,"duration_ms":50700,"significance":"If the min/max LOCO decomposition can be rigorously connected to information-theoretic redundancy and synergy, Hi-Fi would be a simple and computationally efficient extension of one of the most widely used feature-importance measures. The algebraic identity is correct, the code is open, and the particle-physics interpretation involving the Bethe-Bloch formula is physically plausible. However, the central conceptual contribution currently rests on an analogy that is asserted rather than derived, and the empirical claims lack any form of uncertainty quantification. The paper is therefore potentially useful but needs substantial additional support before the labels 'unique', 'redundant', and 'synergistic' can be taken as factual descriptions of feature behavior.","major_comments":[{"comment":"The decomposition in Eq. (2) is an algebraic identity: for any subsets zmin and zmax, L_{zmax} = L_{zmin} + (L_empty - L_{zmin}) + (L_{zmax} - L_empty). The substantive claim is that the subset minimizing Lz 'captures the amount of redundancy R' and the subset maximizing Lz 'leads to the amount of synergy S,' but this mapping is asserted rather than derived. The paper's own concession that replacing entropy with variance 'sacrifices the formal rigor of information theory' (citing [11]) means that the labels unique, redundant, and synergistic are not automatically the PID quantities. I request either a formal derivation showing under which conditions min/max LOCO equals the corresponding PID components, or a benchmark against a recognized PID measure on small synthetic systems, or a clear reframing of the method as a heuristic whose names are only analogical. As written, the interpretations in Figs. 1, 3, and 5 are not supported as factual statements about information-theoretic redundancy and synergy.","section":"Eq. (2) and surrounding definitions"},{"comment":"The toy example does not independently validate the mapping. Because U, R, and S are defined directly from Lzmin and Lzmax, a correctly implemented computation will always recover the labels used to construct the example. The example shows internal consistency, but it does not show that min/max LOCO tracks redundancy and synergy in the information-theoretic sense. A benchmark in which the true PID redundancy and synergy are known, such as a comparison against a recognized PID measure on the same Gaussian or categorical system, would be needed.","section":"Toy example and Figs. 1-2"},{"comment":"The manuscript states that exhaustive evaluation gave the same results as the greedy search on the analyzed datasets, but it does not report the exhaustive-search procedure, the stopping thresholds, or any comparison. Because the greedy stopping rule involves a permutation-test significance threshold and multiple-comparison correction, the reader cannot assess whether the reported multiplets are stable or artifact-driven. Please provide the implementation details and either the exhaustive results or a sensitivity analysis across thresholds.","section":"Greedy search and final paragraph"},{"comment":"None of the empirical claims in Figs. 3-6 are accompanied by uncertainty quantification. The predictions are evaluated on the training data (per footnote 14), and the differences between, for example, the synergistic role assigned to residual sugar and the beta-p synergy are presented as point estimates only. The absence of confidence intervals or cross-validated errors makes it impossible to judge whether the reported differences are meaningful. This is load-bearing for the paper's concrete claims about wine quality and particle discrimination.","section":"Empirical results and Figs. 3-6"}],"minor_comments":[{"comment":"The caption of Fig. 3 refers to the 'particle discrimination problem,' but the figure shows the Wine Quality dataset; the caption should be corrected.","section":"Fig. 3 caption"},{"comment":"The text says 'UCL Machine Learning Repository'; the correct name is the UCI Machine Learning Repository.","section":"Text near reference [19]"},{"comment":"The polynomial kernel degree is stated, but regularization or kernel scaling parameters, if any, are not specified, which makes exact reproduction difficult despite the provided code.","section":"Reproducibility details"},{"comment":"The notation for the full set Z and the subset z is visually similar; please distinguish them clearly, for example by using bold or calligraphic symbols, to avoid ambiguity in Eqs. (1) and (2).","section":"Notation"}],"recommendation":"major_revision","confidential_remarks":"The paper comes from a group well known in the information-decomposition community, and the applications are plausible. The fundamental issue is that the decomposition is tautological and the semantic labels are not justified by derivation or by comparison with an established PID measure. This is fixable in principle by adding formal conditions or benchmarks and by softening the claims. If the authors do not address the semantic-mapping concern, I would not recommend publication. The figure caption error and the lack of uncertainty quantification should also be corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take: the paper is a useful heuristic wrapped in an overclaim. The new move is replacing conditional mutual information with LOCO in the transfer entropy decomposition from [13], yielding a variance-based decomposition of feature importance into unique, redundant, and synergistic parts. That's a real and potentially practical extension for XAI, and the authors are transparent about sacrificing information-theoretic rigor. The toy example is well-constructed: linear suppressor effects for synergy via correlation, a nonlinear product term for structural synergy, and positive correlations for redundancy. Code is available, and the real-data applications (wine, GEANT particle discrimination) give plausible, interpretable results. The explicit comparison with DIP [22] is also a plus; the complementarity claim seems fair.\n\nThe soft spot is load-bearing, and it's the one the stress test identifies. The definitions U = Lzmin, R = L∅ - Lzmin, S = Lzmax - L∅ make Eq. (2) an algebraic identity. The semantic content rests entirely on the assertion that the subset minimizing Lz captures redundancy and the subset maximizing Lz captures synergy. That assertion is by analogy with [13], not derived. In PID, redundancy and synergy are axiomatic properties of how information from multiple sources combines; there is no reason the min and max of a single-variable LOCO over subsets must coincide with those quantities. The toy example is set up so the mapping works, so it doesn't independently validate the mapping. Given that the paper's novelty is precisely the semantic decomposition, the interpretations in Figures 3-6 go beyond what is established.\n\nThat said, I don't think this is fatal if framed differently. The method defines a well-specified, computable decomposition of LOCO: a baseline two-body component, a component obtained by removing the most redundant multiplet, and a component obtained by adding the most synergistic multiplet. That is a coherent exploratory tool even without the PID pedigree. What's missing is a benchmark: compare the Hi-Fi labels against an established PID measure on data where PID is computable (small n, discrete or Gaussian), to show that min/max LOCO actually tracks redundancy and synergy in the PID sense. The paper also lacks error bars and cross-validation. With n up to millions the in-sample estimates are probably fine, but for wine (n=6497) bootstrapping would have been easy. The greedy/exhaustive equivalence is stated without details; they say they verified it on these datasets, but no results are shown, and the greedy search procedure is only briefly described.\n\nWho is this for? XAI practitioners who want a quick, variance-based way to see cooperative effects. They should treat U/R/S as descriptive of the regression model, not as information-theoretic quantities. The paper deserves peer review — the idea is worth airing, but it needs revision to either soften the semantic claims or provide external validation. I'd recommend sending it out, with a request for a PID benchmark and uncertainty quantification.","headline":"Useful heuristic for decomposing LOCO, but the synergy/redundancy labels ride on an unproven analogy and need external benchmarking.","tokens_in":7857,"tokens_out":1866,"would_cite":false,"duration_ms":21625,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a single Leave One Covariate Out importance score can be decomposed into unique, redundant, and synergistic components by comparing the smallest and largest LOCO values over feature subsets.","keywords":["feature importance","Leave One Covariate Out","predictability decomposition","synergy","redundancy","high-order interactions","explainable artificial intelligence","suppressor variables"],"falsifier":"For a small Gaussian system where a full information-theoretic partial information decomposition can be computed exactly, compare its redundancy and synergy values with Hi-Fi's $R$ and $S$; a qualitative disagreement across correlation strengths would show that the min/max LOCO labels are not tracking the quantities they are named after.","tokens_in":6809,"feed_emoji":"🧩","tokens_out":7970,"duration_ms":79281,"temperature":0.7,"pith_summary":"This paper claims that the usual single-number feature importance score can hide how features cooperate. It introduces Hi-Fi, a version of Leave One Covariate Out that searches for the subset of other features which most lowers the score and the subset which most raises it. The gap between those extremes is read as redundancy and synergy, and the leftover baseline is the feature's unique contribution. If right, a model's explanation can distinguish a feature that acts alone from one that matters only together with others, and can name which partners matter. The method is tested on a toy model, wine quality data, and proton/pion discrimination from simulated detector signals.","feed_headline":"Feature importance becomes three parts: unique, redundant, synergistic","feed_subtitle":"An adaptive leave-one-out score finds the feature subsets that raise and lower predictive power, revealing when features cooperate.","key_machinery":"The carrying identity is Eq. (2), $L_{z_{\\max}}(X \\to Y) = S + R + U$, with $U = L_{z_{\\min}}$, $R = L_{\\emptyset} - L_{z_{\\min}}$, and $S = L_{z_{\\max}} - L_{\\emptyset}$, where $L_z$ is LOCO over subset $z$ and $L_{\\emptyset}$ is the pairwise explained variance. The identity converts the definitions of redundancy as information retrievable from multiple sources and synergy as information available only from the whole into a search over feature subsets. The subsets are found exhaustively for small $n$ and, for larger $n$, by a greedy one-variable-at-a-time search with a permutation-based stopping rule.","core_discovery":"On the paper's own terms, LOCO computed with the best subset $z_{\\max}$ decomposes exactly as $L_{z_{\\max}} = S + R + U$. Here $U$ is the LOCO value at the subset that minimizes it, the unique two-body contribution; $R$ is the gap between the pairwise explained variance and that minimum, the redundancy shared with other features; and $S$ is the gap from pairwise variance to the maximizing LOCO, the synergy supplied by partners. The authors therefore claim that the maximal predictive power of $X$ to $Y$ is a sum of a pure two-body effect and two cooperative effects, and that the optimizing subsets name which other features participate.","pith_inferences":["Beyond the paper's demonstrations, a direct check of the nomenclature would compute a true partial information decomposition on a Gaussian system and compare its $R$ and $S$ with Hi-Fi's components; the paper does not supply this validation.","The greedy search's stopping rule has only been compared with exhaustive search on small-$n$ datasets; in high-dimensional feature sets the global min or max could be unreachable, so the labels may depend on the search path.","Because the decomposition is built on prediction-error variance, the same recipe could be applied to other losses, making Hi-Fi a general way to expose cooperation in classification or survival models, not just regression.","The synergy term effectively detects suppressor variables, so Hi-Fi could double as a screening tool for suppression in regression diagnostics."],"forward_implications":["Features that are redundant with others look less important under standard LOCO; Hi-Fi restores them by assigning the lost predictive power to $R$.","Features that matter only in combination, such as suppressors, are missed by pairwise explained variance; Hi-Fi assigns their effect to $S$ and names the partners.","In the particle discrimination example, $\\beta$ and $p$ are synergistic because together they encode particle mass, while the detector responses are redundant with $\\beta$ and $p$ because of Bethe-Bloch energy loss.","On the datasets analyzed, the greedy search reaches the same decomposition as exhaustive search, so the method can scale beyond the toy case.","The components depend on the chosen regression hypothesis space, so robustness can be checked by repeating the analysis with several models."],"supporting_citations":[{"why":"Defines Leave One Covariate Out as the reduction in prediction error that Hi-Fi decomposes.","marker":"[3]"},{"why":"Proposes the unique/redundant/synergistic decomposition of transfer entropy that Hi-Fi adapts by substituting LOCO for conditional mutual information.","marker":"[13]"},{"why":"Supplies the argument that variance-based predictability substitutes sacrifice formal information-theoretic rigor, the trade-off the paper accepts.","marker":"[11]"},{"why":"Introduces transfer entropy, the information measure whose unique/redundant/synergistic split is being ported to regression.","marker":"[12]"},{"why":"Recent LOCO decomposition into interactions and dependencies that the paper positions as complementary to its explicit redundancy-synergy split.","marker":"[22]"},{"why":"Defines suppressor variables, the phenomenon named as the source of synergy.","marker":"[16]"},{"why":"Supplies the Wine Quality dataset used in the first application.","marker":"[19]"},{"why":"Supplies the simulated detector dataset used for proton/pion discrimination.","marker":"[21]"}],"fun_headline_variants":["Feature importance = two-body plus shared plus synergy","Decompose feature importance: unique, redundant, synergistic","Max LOCO = unique + redundant + synergistic parts","Feature importance reveals three components: unique, shared, synergistic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the smallest and largest LOCO values over feature subsets can be read as redundancy and synergy; the paper states this mapping rather than proves it, and the variance-based measure deliberately gives up formal information-theoretic guarantees.","fun_headline_variants_meta":{"raw":{"variants":["Feature importance = two-body plus shared plus synergy","Decompose feature importance: unique, redundant, synergistic","Max LOCO = unique + redundant + synergistic parts","Feature importance reveals three components: unique, shared, synergistic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000352,"raw_usage":{"total_tokens":1883,"prompt_tokens":873,"completion_tokens":1010,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":946}},"tokens_in":489,"tokens_out":1010,"duration_ms":9177,"temperature":1.0,"reasoning_tokens":946,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:30:54.518014+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a small Gaussian system where a full information-theoretic partial information decomposition can be computed exactly, compare its redundancy and synergy values with Hi-Fi's $R$ and $S$; a qualitative disagreement across correlation strengths would show that the min/max LOCO labels are not tracking the quantities they are named after.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Leave One Covariate Out as the reduction in prediction error that Hi-Fi decomposes."},{"cited_title":"Stramaglia, L","cited_arxiv_id":null,"evidence_quote":"Proposes the unique/redundant/synergistic decomposition of transfer entropy that Hi-Fi adapts by substituting LOCO for conditional mutual information."},{"cited_title":"Barrett, Phys","cited_arxiv_id":null,"evidence_quote":"Supplies the argument that variance-based predictability substitutes sacrifice formal information-theoretic rigor, the trade-off the paper accepts."},{"cited_title":"Conger, Educational and Psychological Measurement 34, 35 (1974)","cited_arxiv_id":null,"evidence_quote":"Defines suppressor variables, the phenomenon named as the source of synergy."},{"cited_title":"Cortez, A","cited_arxiv_id":null,"evidence_quote":"Supplies the Wine Quality dataset used in the first application."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the simulated detector dataset used for proton/pion discrimination."}],"review_version":1}