{"id":"ddcdde48-008e-4b6b-84b2-866ac51bace2","arxiv_id":"2506.01988","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"The paper proposes SIG, a graph and MILP-based summary of random forest decision rules that shows global feature interactions through pruned graphs and decision-feature interaction tables.","lead":"This paper presents a surrogate interpretable graph that extracts decision rules from a random forest, clusters them, and visualizes the most frequent feature interactions as a pruned graph with decision-feature tables. It aims to make global model explanations in healthcare more readable and claims better scaling than TreeSHAP on high-dimensional datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed SIG complexity advantage over TreeSHAP is not established: Appendix A derives no SIG complexity, and the actual pipeline's clustering and MILP steps may dominate asymptotically.","rationale":"The paper's central claim is that SIG+MILP scales gracefully and provides clear global explanations where TreeSHAP becomes infeasible. The mathematical support for this claim is Appendix A, but that appendix only proves that TreeSHAP's complexity diverges as f→∞; it never states or bounds SIG's complexity. Moreover, the components the paper itself specifies have asymptotic terms that can dominate: agglomerative clustering on the TF-IDF matrix is quadratic in the number of rules, and the MILP over ordered feature pairs is at least quadratic in f. The sentence 'effective complexity remains lower in practical high-dimensional scenarios' is therefore an assertion, not a demonstrated consequence. Even if the graph outputs were perfectly faithful to the forest, the scalability claim would remain unsupported without a formal complexity analysis. The reader's rationale already flags the invalid proof and missing runtime detail; my concern is more specific and more load-bearing than the co-occurrence-faithfulness assumption listed as the reader's weakest_assumption, hence 'partial' agreement. The proposed re-derivation is decisive: it either produces the missing bound or exposes that the claimed sublinear behavior is incompatible with the stated algorithms. No independent support—machine-checked proof, released code, or falsifiable prediction—is provided to offset this gap. The reader's REJECT verdict remains appropriate; no verdict change is needed.","tokens_in":17710,"tokens_out":5157,"duration_ms":55619,"concrete_test":"Independently derive the end-to-end asymptotic complexity of the SIG pipeline as a function of N, f, T, and d, following Algorithms 1–5 exactly: count rule extraction, TF-IDF construction, agglomerative clustering, and MILP variable/constraint sizes. Then compare the resulting bound with the stated TreeSHAP bound O(N·T·L^2 + N·f^2) as f→∞. If clustering contributes Ω(R^2) with R = O(T·L), or if the MILP formulation is Ω(f^2) or NP-hard, then the claimed 'effective complexity remains lower regardless of dataset size' is false; if a rigorous sublinear regime can be stated and proved under explicit sparsity assumptions, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central scalability claim ('effective complexity remains lower in practical high-dimensional scenarios, regardless of the dataset size') rests on Appendix A. That appendix writes Time_TreeSHAP = O(N·T·L^2 + N·f^2), takes the limit f→∞, and stops. It never defines or bounds the runtime of the SIG pipeline. The pipeline's own algorithms are not obviously sublinear: Algorithm 1 extracts R = O(T·L) rules with O(d) conditions; Algorithm 3 builds a TF-IDF matrix of size R×V, with V up to O(R·d); Algorithm 4 runs agglomerative clustering on that matrix, which is at best O(R^2 log R) and typically O(R^2·V); Algorithm 5 solves a MILP whose candidate edge set includes all ordered feature pairs, giving O(f^2) binary variables, and whose objective/constraints are only sketched (minimize total selected edges subject to 'path consistency' and 'DAG constraint', with no coverage term). Thus Table 14's 'Sublinear if rules are sparse and compressible' is an assumption, not a result. No formal comparison with TreeSHAP is actually made, so the strongest claim fails on its own terms unless SIG's end-to-end complexity is derived and shown to be lower.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Surrogate Interpretable Graph (SIG), a pipeline for global feature-interaction analysis in random forests. The pipeline extracts decision rules from trained trees (Algorithm 1), tokenizes and TF-IDF-encodes them (Algorithms 2–3), clusters rules via agglomerative clustering (Algorithm 4), and then builds a directed weighted graph whose edges are pruned by a mixed-integer linear program (Algorithm 5). The output is a sparse graph, a per-decision-feature-interaction (DFI) table, and a set of hierarchical feature-interaction chains. The authors compare SIG with TreeSHAP, claiming that SIG scales gracefully in high-dimensional healthcare datasets where TreeSHAP becomes infeasible, and they present a runtime comparison on five health datasets. The paper includes appendices with a 'mathematical proof' of TreeSHAP's quadratic-in-features complexity, additional experiments, and dataset descriptions.","tokens_in":17953,"tokens_out":5653,"duration_ms":55999,"significance":"The goal of the paper is worthy: a compact, global, rule-grounded visualization of feature interactions in random forests would be a useful interpretability tool for high-stakes healthcare applications. The proposed pipeline is concrete and the authors make an explicit attempt to compare against TreeSHAP, a standard baseline. The strength of the paper is its clear algorithmic skeleton (Algorithms 1–5) and the framing of the output as a human-readable graph and DFI table, with a stated intention to avoid the per-instance complexity of TreeSHAP. However, the central claims are not established. No end-to-end complexity of the SIG pipeline is derived; the MILP is not specified in enough detail to be reproduced or to justify the pruning choices; the reported DFI 'findings' are summary statistics of the same rules the method extracts, with no external validation; and the runtime comparison lacks necessary experimental detail. If the scalability claim and the objectivity of the MILP pruning were rigorously established, the contribution would be significant; in its current form, the paper is a description of a heuristic with hand-set parameters and unsupported performance conclusions.","major_comments":[{"comment":"The 'mathematical proof' does not establish any scalability advantage for SIG. Equation (A.1) states Time_TreeSHAP = O(N·T·L^2 + N·f^2) and then computes lim_{f→∞} Time_TreeSHAP = ∞, but it never defines or bounds the runtime of the SIG pipeline (Algorithms 1–5). The pipeline includes agglomerative clustering on an R×V TF-IDF matrix (at least O(R^2 log R) in typical implementations, and O(R^2·V) without special methods) and a MILP whose candidate edge set can be as large as O(f^2) binary variables. Table 14's entry 'Sublinear if rules are sparse and compressible' is an assumption, not a derived result. Consequently, the central claim in the Introduction and Conclusion that 'the effective complexity remains lower in practical high-dimensional scenarios, regardless of the dataset size' is unsupported by the provided analysis.","section":"Appendix A"},{"comment":"The MILP formulation is not sufficiently specified to be reproducible or to justify the claim that it 'preserves the connection with the most information.' The objective is only stated as 'minimize total selected edges' and the constraints are named ('path consistency,' 'DAG constraint') without equations. There is no coverage term that ties selected edges to the frequency of co-occurrence in the clustered rules, so minimizing the edge count alone could select an arbitrary sparse subgraph. The edge budgets are hand-chosen per dataset (15, 5, 20), and the paper itself reports that for the diabetes and Alzheimer's datasets the initial budget of 15 edges caused 'over-optimization.' The DFI counts and hierarchical interactions therefore depend directly on arbitrary thresholds, and the reported 'dominant' interactions are not robust findings.","section":"Algorithm 5 / Section 3.5"},{"comment":"The conclusions about which features are 'responsible for the decision' or are the 'most important interacting features' are derived from co-occurrence frequencies of features in the same rules extracted from the random forest. These are descriptive summaries of the method's own construction, not validated interaction effects. The statement that 'the number of unique DFIs is 10, which also shows that the number of estimators chosen for the RF classifier, i.e., 15, is optimal' is a non sequitur: the number of DFIs is an output of the pipeline and cannot by itself establish the optimality of n_estimators. Without external validation—for example, comparison with interaction importance measures on held-out data, permutation tests, or domain-expert evaluation—the DFI tables do not support the paper's interpretive claims.","section":"Section 4, Tables 3–6"},{"comment":"The runtime comparison between SIG and TreeSHAP is not sufficient to support the scalability claim. The figure has no error bars, no confidence intervals, and no reporting of the number of runs. Crucially, the experiment omits essential hyperparameters: the number of estimators and maximum depth of the random forest, the cluster count k used in Algorithm 4, the MILP solver and its time limit, and the exact TreeSHAP implementation and settings. Without these details, the reader cannot determine whether the plotted runtimes reflect algorithmic complexity or particular solver/configuration choices. A table with the actual runtime values and the hyperparameter settings should be provided before any performance comparison is claimed.","section":"Figure 7"}],"minor_comments":[{"comment":"The algorithm numbering is inconsistent: Section 3.2 refers to 'Algorithm 3' for the tokenization and standardization step, but the caption for that step is Algorithm 2, while Algorithm 3 is TF-IDF encoding in Section 3.3.","section":"Section 3.2"},{"comment":"The Shapley Interaction Index formula contains a malformed set notation: 'T⊆\\S' should be written as a set complement, e.g., 'T ⊆ S^c' or 'T ∩ S = ∅' as stated below; the current notation is confusing and likely a typesetting error.","section":"Section 2, SII formula"},{"comment":"The DFI tables appear as empty headers in the manuscript; the filled cells that indicate feature usage are not visible. Since these tables are a core result, the actual entries must be included in the printed version.","section":"Tables 3, 5, 8, 10, 12"},{"comment":"There are numerous typographical and terminology issues, including 'spatial interpologram' (Section 4), 'signed graph' and 'sign' for the graph, 'Cholestrol', 'ontribute', and inconsistent use of the abbreviation GRF for both 'Generalized Random Forests' and 'Graph Random Forest'.","section":"Throughout"},{"comment":"The heuristic for choosing the number of clusters—'the square root of f + N, rounded to the nearest perfect square'—is not justified and is ambiguously stated; a perfect-square rounding rule is not standard and should be replaced by a data-dependent criterion or a sensitivity analysis.","section":"Section 3.4"},{"comment":"The claimed complexity bound Time_TreeSHAP = O(N·T·L^2 + N·f^2) is stated without a citation or derivation; reference [25] ('Linear Tree SHAP') is not an obvious source for this interaction-time bound, and the bound should be attributed or proven.","section":"Appendix A"}],"recommendation":"reject","confidential_remarks":"The paper's central claims—scalability relative to TreeSHAP and the meaningfulness of the extracted dominant interactions—are not supported by the current manuscript. The 'mathematical proof' in Appendix A is not a comparison; the MILP is under-specified; and the runtime experiment lacks enough detail to be evaluated. These are load-bearing issues that would require a substantial rework of the paper, including a rigorous complexity analysis, a concrete MILP formulation with proper constraints and coverage terms, and a validated experimental protocol. I therefore recommend rejection, though a future submission with these gaps addressed could be considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this is a reasonable pipeline for turning random forest decision paths into a pruned graph and a per-interaction table, but the central claim about scalability over TreeSHAP is not supported. The appendix that is supposed to prove it doesn't derive SIG's complexity at all; it writes down TreeSHAP's O(N T L^2 + N f^2) and then notes the limit as f goes to infinity is infinite. That tells us nothing about SIG's end-to-end cost. If anything, the agglomerative clustering (O(R^2) or worse) and the MILP over O(f^2) candidate edges look like they could dominate.\n\nWhat's genuinely useful: the specific integration — rule extraction, TF-IDF, clustering, MILP pruning, and the DFI table — reads as a coherent recipe, even if every ingredient is in the literature. The DFI table is a nice concrete artifact for documentation and screening, and the limitations section is honest about not having user studies. The citation list covers SIRUS, Forest-ORE, feature graphs, and TreeSHAP-IQ, so the authors are at least working in the right neighborhood.\n\nSoft spots beyond the non-proof: the runtime chart has no error bars and no stated hyperparameters, MILP solver, or edge-budget rationale; the text admits budgets like 15, 5, 20 were adjusted after seeing results. The dominant DFIs are summaries of the same rules that were extracted to build the graph, so calling them findings about the data is circular — they are findings about the method's own construction. That is acceptable for a descriptive summary tool, but the paper claims more. And there is no comparison to the rule-extraction baselines it cites, which is a real gap if the pitch is \"better interpretability.\"\n\nWho is this for? A health analyst who wants a visual, global snapshot of what an RF is doing, for documentation or screening, could get value from the pipeline once it is cleaned up. A reader looking for a rigorous complexity result or validated interpretability should look elsewhere.\n\nRecommendation: I would not send this to peer review in its current form. The headline claim is load-bearing and unsupported, and the evaluation is too thin. A desk reject with an invitation to resubmit after the complexity analysis is actually derived, the runtime study is properly configured with error bars and hyperparameters, and a baseline comparison is added would be the right call. If those pieces land, the idea could deserve referee time.","headline":"A plausible visualization pipeline for summarizing RF decision rules as a graph, but the central scalability claim over TreeSHAP rests on a non-proof and the evaluation is too thin to support it.","tokens_in":18479,"tokens_out":3608,"would_cite":false,"duration_ms":35158,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a random forest can be reduced to a pruned feature-co-occurrence graph that gives global, human-readable explanations and stays practical where TreeSHAP becomes infeasible.","keywords":["random forests","feature interactions","global interpretability","explainable AI","mixed-integer linear programming","rule extraction","health informatics","Shapley values"],"falsifier":"Compute the full unpruned co-occurrence graph from a forest's extracted rules and compare it with the MILP-pruned SIG: if a feature appears in a majority of decision paths but is absent from the pruned graph, or if the dominant feature chains change materially when the edge budget is changed, then the claim that SIG captures the dominant global interactions is falsified.","tokens_in":17482,"feed_emoji":"🌳","tokens_out":7515,"duration_ms":70202,"temperature":0.7,"pith_summary":"The paper proposes the surrogate interpretability graph (SIG), a pipeline that converts a trained random forest into a small directed graph whose nodes are features and whose edges record how often features appear together in the forest's decision rules. The goal is to give clinicians and domain experts a global, readable account of which feature interactions drive predictions, summarized as a table of decision-feature interactions (DFIs). The paper claims that this graph, pruned by mixed-integer linear programming with a user-set edge budget, stays interpretable and computationally practical precisely in high-dimensional health datasets where TreeSHAP interaction values become infeasible. If the claim holds, SIG offers a path to transparent explanation of tree ensembles without per-instance Shapley computations.","feed_headline":"Graph turns random forests into readable feature maps","feed_subtitle":"Prunes a forest's decision rules into the dominant feature chains, beating TreeSHAP when features explode.","key_machinery":"The central object is the surrogate interpretable graph (SIG): a weighted directed graph whose nodes are features and whose edges record how often two features appear together along ordered decision paths. The load-bearing mechanism is the co-occurrence count over ordered feature pairs in extracted rules, combined with TF-IDF weighting of tokenized rules, agglomerative clustering to merge redundant rules, and a mixed-integer linear program that selects a small edge set while preserving path structure. The final DFI table lists the resulting dominant chains, such as $f_{10}\\rightarrow f_{13}\\rightarrow f_{12}\\rightarrow f_{1}\\rightarrow f_{14}$, as the model's hierarchical feature interactions.","core_discovery":"The central claim is that global feature interactions of a random forest can be faithfully captured by counting ordered feature co-occurrences along extracted decision paths, then compressing that co-occurrence structure into a sparse directed graph. After rules are tokenized, weighted with TF-IDF, clustered, and pruned by a MILP edge-selection objective, the resulting SIG identifies a small number of dominant decision-feature interactions, such as ten for the chronic kidney and heart datasets, six for the air-quality health dataset, five for diabetes, and four for Alzheimer's. The paper also argues, with an asymptotic comparison, that TreeSHAP's per-instance interaction computation grows quadratically in the number of features while SIG's cost remains sublinear or constant after rule extraction, making SIG the scalable option when feature counts explode.","pith_inferences":["If faithfulness holds, the MILP edge budget can be tuned as a completeness-versus-simplicity dial; one could test whether features excluded from the pruned graph ever carry prediction-critical interactions.","Replacing the TF-IDF and clustering stages with direct frequency counts of co-occurring feature pairs would reveal whether the NLP-style weighting changes the recovered graph, a testable robustness check.","Applying the pipeline to gradient-boosted trees, which the paper names as future work, would test whether the ordered chains survive boosting's sequential re-weighting of trees.","A clinician-facing user study, which the paper notes is absent, could settle whether the graph's hierarchies actually improve trust and decision-making in practice."],"forward_implications":["A forest's global behavior can be compressed to a graph whose edge count is chosen by the user, so explanation size stays bounded regardless of dataset size.","The DFI table gives ordered feature chains, revealing which features act as origins, intermediates, and endpoints in the forest's decision paths.","Runtime stays low after the one-time rule extraction, whereas TreeSHAP's interaction computation grows as the square of the feature count and becomes impractical for wide clinical or molecular datasets.","The same graph can be regenerated for different forests or datasets, making global interpretability a reusable artifact rather than a per-instance calculation.","Because the graph names the actual features and rules, it can be audited by non-experts, which helps meet transparency expectations in high-stakes health applications."],"supporting_citations":[{"why":"Supplies the random forest model class whose decision rules are the input to the method.","marker":"[1]"},{"why":"Provides the stable rule-set extraction approach that the paper builds on and contrasts with.","marker":"[15]"},{"why":"Supplies a rule-extraction baseline for turning forest logic into readable rules.","marker":"[16]"},{"why":"Defines TreeSHAP, the feature-interaction baseline the paper compares against.","marker":"[19]"},{"why":"Provides TreeSHAP-based global explanation methods that motivate the comparison.","marker":"[20]"},{"why":"Documents scalability limits of TreeSHAP interaction values that SIG is designed to avoid.","marker":"[21]"},{"why":"Supplies the linear-time TreeSHAP variant used for the runtime comparison.","marker":"[25]"},{"why":"Formulates an optimal rule ensemble via MILP, the formulation SIG adapts for graph pruning.","marker":"[45]"}],"fun_headline_variants":["Random forests get a graph that shows dominant feature chains","SIG: a sparse graph that makes random forests interpretable","Graph surrogate for random forests reveals key feature interactions","Random forest interactions, compressed into a readable graph","SIG scales beyond TreeSHAP when features explode"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the assumption that co-occurrence of features along decision paths, after clustering and pruning, faithfully reflects the most important global feature interactions.","fun_headline_variants_meta":{"raw":{"variants":["Random forests get a graph that shows dominant feature chains","SIG: a sparse graph that makes random forests interpretable","Graph surrogate for random forests reveals key feature interactions","Random forest interactions, compressed into a readable graph","SIG scales beyond TreeSHAP when features explode"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00027,"raw_usage":{"total_tokens":1570,"prompt_tokens":833,"completion_tokens":737,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":449,"completion_tokens_details":{"reasoning_tokens":661}},"tokens_in":449,"tokens_out":737,"duration_ms":7109,"temperature":1.0,"reasoning_tokens":661,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:44:29.511179+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the full unpruned co-occurrence graph from a forest's extracted rules and compare it with the MILP-pruned SIG: if a feature appears in a majority of decision paths but is absent from the pruned graph, or if the dominant feature chains change materially when the edge budget is changed, then the claim that SIG captures the dominant global interactions is falsified.","supporting_citations":[{"cited_title":"Forest-ore: Mining an optimal rule ensemble to interpret random forest models,","cited_arxiv_id":null,"evidence_quote":"Formulates an optimal rule ensemble via MILP, the formulation SIG adapts for graph pruning."},{"cited_title":"Sirus: Stable and interpretable rule set for classification,","cited_arxiv_id":null,"evidence_quote":"Provides the stable rule-set extraction approach that the paper builds on and contrasts with."},{"cited_title":"Interpretable random forests via rule extraction,","cited_arxiv_id":null,"evidence_quote":"Supplies a rule-extraction baseline for turning forest logic into readable rules."},{"cited_title":"Beyond treeshap: Efficient computation of any-order shapley interactions for tree ensembles,","cited_arxiv_id":null,"evidence_quote":"Documents scalability limits of TreeSHAP interaction values that SIG is designed to avoid."},{"cited_title":"Linear tree shap,","cited_arxiv_id":null,"evidence_quote":"Supplies the linear-time TreeSHAP variant used for the runtime comparison."}],"review_version":1}