{"id":"77e3007e-a697-4b07-8c26-3f8cb78ffded","arxiv_id":"2505.11189","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RuleSHAP, which combines global SHAP values with XGBoost and LASSO rule selection, recovers injected univariate, conjunctive, and non-convex LLM behavior triggers more faithfully than RuleFit and SHAP alone.","lead":"This paper tests whether global explainability tools can uncover hidden behavior-shaping rules in large language models by converting topics and outputs into numeric scores, and introduces RuleSHAP, a SHAP-guided rule extractor. On injected bias rules, RuleSHAP finds the true trigger rule at rank one about 82% more often than RuleFit on average.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Injected ground truth is a wrapper that reads the same u vectors used as XAI features, so the benchmark shows wrapper-rule recovery, not that self-reported beliefs causally drive LLM behaviour.","rationale":"The reader's verdict is CONDITIONAL, and my analysis supports that verdict rather than moving it. The controlled comparison is internally consistent: Table 2, the ablations, and the Wilcoxon results support RuleSHAP as an improvement for recovering wrapper-injected rules from abstracted numeric features. That is a real, if narrow, contribution. The load-bearing weakness is not the algorithm mechanics but the interpretation: the ground-truth wrapper reads the same u vectors that serve as XAI input features, so the benchmark validates rule recovery under ideal conditions where the trigger features are exactly the explanation features. The correlation certificates cannot repair this because they are in-sample and, for injected data, guaranteed by construction; in the real-data case study they are also in-sample and can be significant with negligible effect sizes. The paper's own limitations acknowledge the LLM-as-a-judge circularity, which further weakens the claim that the extracted rules describe genuine belief-driven mechanisms. However, this is an overreach in framing rather than an internal inconsistency in the core comparison, so conditional acceptance with demands for held-out evaluation, repeated runs, and a clearer separation between injected-benchmark and real-world claims remains the appropriate outcome.","tokens_in":25520,"tokens_out":8976,"duration_ms":102866,"concrete_test":"Run a controlled experiment in which the injected wrapper is triggered by an independent topic-property vector u' (e.g., human annotations or a second LLM's ratings) while RuleSHAP still receives the tested LLM's self-reported u as features. If MRR@1 collapses to near baseline, the current results only demonstrate recovery when the trigger features are identical to the explanation features, confirming that the benchmark is a wrapper-detection test rather than a test of belief-driven causation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that global XAI over 'statistically validated numeric abstractions of LLM beliefs' can recover belief-driven heuristics, and that RuleSHAP does this better. The load-bearing condition is that the 11-dimensional self-reported belief vector u is a stable, causally upstream representation of the mechanism that shapes the output abstractions v. Section 3 states this as the main assumption, but the experiments never test it. Every injected 'behaviour' is produced by a Python wrapper that reads exactly those u values and then instructs the LLM to change length, subjectivity, or readability (Section 5, Appendix D). Hence the u-to-v association is guaranteed by construction: the rules RuleSHAP recovers are rules of the wrapper, not evidence about internal belief-mediated generation. The correlation certificates (Section 6, Appendix I) are computed in-sample on the same u and v and therefore cannot break this circularity; for injected data they are tautological, and for the real-data case study (Appendix K) a rule can pass significance with a correlation near 0.05, which the author flags as a possible hallucination. The limitations section also concedes that the three LLM-as-a-judge output features may be circular; those judge scores are part of v and will track the objective proxies that the wrapper manipulates, so rules over those targets are again constructed. None of this refutes the narrow algorithm-comparison result, but it does undermine the broader claim that the pipeline reveals beliefs that cause LLM behaviour.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a pipeline for global rule extraction from LLMs. It maps topics to an 11-dimensional belief vector u (elicited from the LLM itself) and outputs to 7 proxy/judge scores v, then applies global XAI methods on (u, v). To obtain ground truth, the authors inject three families of bias rules (univariate, conjunctive, non-convex) via system instructions that are triggered by the same u vectors. They introduce RuleSHAP, which couples global SHAP aggregations with XGBoost feature weighting and SHAP-aware LASSO. Across five LLMs and SDG-related topics, RuleSHAP is reported to outperform RuleFit, SHAP, and decision-tree baselines on mean reciprocal rank (MRR@1 average 0.568 vs. 0.31 for RuleFit) while producing more compact rule sets. The paper also reports correlation certificates and a real-data case study on LLM explanations of SDG topics.","tokens_in":25661,"tokens_out":6309,"duration_ms":62968,"significance":"If the central claims are taken at face value, RuleSHAP would be a useful model-agnostic audit tool for LLM bias detection, and the paper would contribute a reproducible benchmark for global XAI methods in the text domain. The manuscript has clear strengths: a public replication package, an a priori power analysis, evaluation across five LLMs, a three-tier complexity design, and a careful ablation isolating the contribution of the SHAP-guided XGBoost step. The narrow algorithmic comparison (RuleSHAP vs. RuleFit on the injected benchmark) is supported by Table 2 and the reported Wilcoxon tests. However, the paper's broader title claim -- that the method reveals 'injected behaviours in LLMs' as belief-driven heuristics -- is not supported by the experimental design, because the injected wrapper reads exactly the same u features that are given to the XAI methods. The significance of the work is therefore contingent on substantially reframing or strengthening the claims.","major_comments":[{"comment":"The pipeline's main assumption (§3) is that the self-reported belief vector u is a stable, causally upstream representation of the mechanism shaping output abstractions v. The injected ground truth, however, is a Python wrapper that reads the exact same u vectors produced by ϕx and prepends system instructions based on threshold conditions on those u values (Appendix D). Consequently, the u-to-v association is guaranteed by construction, and any rule-extraction method that sees u can recover the wrapper's threshold rules. The correlation certificates in §6 are computed in-sample on the same u and v, so they provide no independent evidence that u encodes internal LLM beliefs. To support the central claim, the authors must either inject rules based on variables not present in u (e.g., topic metadata or labels independent of the LLM's self-reports) or provide independent evidence (e.g., activation-level probes or controlled manipulation of internal representations) that u is causally upstream of v. Without this, the paper demonstrates wrapper-rule recovery, not belief-driven behavior discovery.","section":"§3, §5, Appendix D"},{"comment":"No held-out split is described anywhere in the evaluation. The MRR metrics appear to be computed on the same topic set that was used to fit the rule-extraction surrogates (XGBoost and LASSO). Given that the candidate rule sets contain thousands of rules and the injected rules are simple threshold functions over an 11-dimensional u, in-sample evaluation can substantially overestimate MRR through overfitting. The authors should report out-of-sample MRR via topic-level cross-validation or repeated resampling, and should provide confidence intervals over multiple seeds or repeated trials. Without such an evaluation, the claimed +82% MRR@1 advantage over RuleFit, while plausible, is not yet robustly established.","section":"§6, Table 2"},{"comment":"The 'correlation certificates' are presented as validating the abstractions, but for the injected data they are tautological: the wrapper reads u, so high distance correlations between u and v are structurally guaranteed. For the real-data case study, the certificates are unthresholded: with N≈600–10,000 per LLM, distance correlations as low as dCorr=0.058 (§I.3) achieve p<0.001. Appendix K itself flags a rule with correlation near 0.05 as a possible hallucination. The paper should define a minimum effect size that qualifies as a 'correlation certificate' and apply it consistently to accept or reject rules. As written, the phrase 'statistically validated abstractions' overstates what the significance tests establish.","section":"§6, Appendix I, Appendix K"},{"comment":"Three of the seven output features are LLM-as-a-judge scores for framing, information overload, and oversimplification. The authors acknowledge in the Limitations that these 'may be judge-biased and introduce invalid circularity.' This circularity is material to the main comparison: the injected wrapper changes length, subjectivity, and readability, and the judge is explicitly asked to rate those biases, so rules over the judge-based targets can be induced by the wrapper even if the LLM's internal beliefs play no causal role. The authors argue that the judge scores correlate with independent proxies, but those proxies are also changed by the wrapper, so this correlation does not break the circularity. The analysis should either be repeated with only the four LLM-independent output features, or the results for judge-based targets should be reported separately and clearly qualified.","section":"§3, Appendix B.2, §7 Limitations"}],"minor_comments":[{"comment":"The abstract reports an average MRR@1 improvement of +82% over RuleFit, while the introduction (p. 2) reports +94% (MRR@1) on average. Please reconcile these numbers with Table 2 and Appendix J and specify the exact aggregation used for each figure.","section":"Abstract vs. Introduction"},{"comment":"The text rendering of Tables 2 and 6 contains formatting artifacts (e.g., '29230.14' and '9430.35' on row breaks). Please ensure the final PDF renders rule counts and MRR values as separate, unambiguous columns.","section":"Tables 2 and 6"},{"comment":"The description of SHAP background values and the nearest-neighbour matching procedure (finding j with ∥buk−uj∥2 minimal) is non-standard and should be clarified: in particular, how the minimum feature values from T are used as background, and how the matching procedure interacts with SHAP's perturbation distribution.","section":"§4 Step 1"},{"comment":"The term 'correlation certificate' is used informally. I suggest giving a formal definition of what constitutes a valid certificate (test, effect-size threshold, and whether it is used for rule filtering or only for post-hoc validation) and stating how certificates are applied to accept or red-flag rules.","section":"§6, Appendix I"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's narrow algorithmic contribution is reasonable and the ablation is informative, but the framing in the title and abstract goes beyond what the experimental design can support. The injected benchmark cannot distinguish wrapper-rule recovery from discovery of belief-driven internal heuristics, and the in-sample evaluation further weakens the quantitative claims. I recommend a major revision that either (a) narrows the claims to wrapper-rule recovery with an explicit statement of the circularity, or (b) adds a non-circular ground-truth injection (e.g., rules based on topic metadata not in u) and a held-out evaluation. The discrepancy between the +82% and +94% improvement figures should also be resolved. The authors have been transparent about limitations, which is commendable and should be preserved in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this paper delivers a believable incremental algorithm contribution, and a benchmark that is too circular to support the broader 'belief-driven heuristics in LLMs' framing. RuleSHAP couples global SHAP aggregates with XGBoost feature weighting and SHAP-aware LASSO, and the ablation shows Step 2 matters. On five LLMs, it beats RuleFit on MRR@1 by a large margin, with Wilcoxon p-values low enough to survive correction. That part is solid.\n\nWhat's genuinely new is the integration itself and the abstraction pipeline that maps LLM self-reported topic beliefs (the u vectors) and output proxies into numeric spaces so off-the-shelf global XAI can be applied to text. That's useful for people building rule-extraction tools for LLM auditing. The paper is also honest about several limitations, which earns credit.\n\nThe soft spots are real and they are exactly where the stress-test points. The injected ground truth is a Python wrapper that reads the same u vectors that are fed to the XAI. So the rules RuleSHAP recovers are rules of the wrapper, not evidence that self-reported beliefs causally drive LLM generation. The correlation certificates are in-sample and, for injected data, guaranteed by construction. That doesn't refute the narrow algorithm-comparison claim—the wrapper is a deterministic data-generating process, and recovering its rules is a legitimate test of rule-induction—but it does mean the paper's title question, 'reveal injected behaviours,' is more accurate than 'reveal LLM beliefs.' The real-data case study has one rule with correlation near 0.05 that the author flags as a possible hallucination; that's minor but should be disclosed more prominently.\n\nOther issues are minor: results come from a single run per condition, no confidence intervals, and the SHAP aggregation formula was chosen on the evaluation data. Also, the abstract and Section 1 disagree on the improvement (+94% vs +82%), which should be reconciled. The rule sets are still huge (hundreds to thousands), so 'compact' is relative to RuleFit, not to anything human-auditable.\n\nWho is this for? Researchers working on rule extraction or LLM auditing might want a baseline to compare against. It deserves a serious referee, with requests for held-out evaluation, repeated runs, and a clearer separation between injected-benchmark results and real-world claims.\n\nMy recommendation: send it to review, but push for the evaluation fixes.","headline":"RuleSHAP is a plausible incremental improvement over RuleFit for rule extraction, but the injected benchmark guarantees the very association it claims to discover; treat the narrow algorithm comparison as the contribution.","tokens_in":26312,"tokens_out":2329,"would_cite":true,"duration_ms":21841,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Global XAI on numerical scores of an LLM's self-reported beliefs can recover injected bias-triggering heuristics as symbolic rules, and RuleSHAP finds them more often than RuleFit or SHAP.","keywords":["explainable AI","rule extraction","SHAP","RuleSHAP","LLM beliefs","misinformation","global explanations","large language models"],"falsifier":"Run RuleSHAP on a model whose injected trigger uses a feature that is deliberately absent from the elicited belief vectors—for example, make response length depend on the number of words in the topic phrase while all 11 rated properties are held approximately constant. If RuleSHAP still surfaces the injected rule, the method recovers triggers independent of the abstraction; if it does not, the recovered rules are confined to the pre-chosen belief dimensions. A second check: hold out part of the topic set, fit the abstraction correlation certificates only on the training half, and verify whether the extracted top rules still rank injected triggers on the held-out half at the same MRR.","tokens_in":25176,"feed_emoji":"🧠","tokens_out":8305,"duration_ms":71403,"temperature":0.7,"pith_summary":"The paper tries to establish that the belief-driven shortcuts behind some LLM misbehavior can be pulled out as readable if-then rules, even though the models only take text in and out. It does this by first turning topics into numbers: the LLM rates each topic on scales like commonality, positivity, and controversy, and the explanations it generates are scored for length, readability, sentiment, and subjectivity. On these numeric abstractions, ordinary global explanation tools can run; the paper shows that SHAP flags the right features but gives no rules, RuleFit gives rules but misses many, and a new hybrid called RuleSHAP recovers injected univariate, conjunctive, and non-convex trigger rules more reliably. Across five GPT and Llama models, RuleSHAP improves top-1 rule recovery over RuleFit by about 82 percent on average, with p-values below 0.003. If this holds, auditors could surface an LLM's behavioural triggers without touching its weights.","feed_headline":"RuleSHAP surfaces hidden LLM bias rules 82% better than RuleFit","feed_subtitle":"Global XAI on LLM belief scores recovers injected heuristics as symbolic rules across GPT and Llama.","key_machinery":"The load-bearing machinery is a two-part abstraction-plus-rule-induction pipeline. First, each topic is mapped to an 11-dimensional vector $u$ of the LLM's self-reported belief scores (1–5) on properties like commonality, positivity, controversy, conceptual density, interdisciplinarity, and temporal or geographic variability, while each generated explanation is mapped to a 7-dimensional vector $v$ of length, Gunning fog readability, sentiment, subjectivity, and three LLM-as-judge bias ratings. Second, RuleSHAP couples global SHAP aggregates with rule induction: it computes per-feature Shapley values over all points, aggregates them as mean absolute value plus standard deviation, normalizes to $(0,1]$, uses these as feature weights in XGBoost (with `colsample_bylevel` set so roughly one feature is sampled per split), and then runs a LASSO whose penalty for each candidate rule is scaled inversely by the average SHAP weight of the features in that rule. The design makes global SHAP steer both the tree construction and the sparse rule selection, while dropping the support-based weighting that RuleFit uses for rule importance.","core_discovery":"The paper's central claim is that a statistically validated mapping from text to numbers—the LLM's own 1-to-5 belief ratings of topic properties such as commonality, positivity, controversy, and interdisciplinarity, paired with seven numeric output metrics (length, readability, sentiment, subjectivity, and three LLM-as-judge bias estimates)—makes LLM behaviour accessible to global XAI, and that the belief-driven heuristics behind misinformation-related biases can be recovered as explicit rules. To get ground truth, the authors inject 14 bias rules of three complexity levels (univariate, conjunctive, non-convex) through system-instruction wrappers, and measure how often each XAI method ranks the injected rule at the top (MRR@1, MRR@3, MRR@10). They report that RuleFit under-detects non-univariate triggers, that global SHAP ranks them better but cannot express them symbolically, and that RuleSHAP—global SHAP aggregates feeding both XGBoost split sampling and a SHAP-aware LASSO—detects more of them with more compact rule sets. Correlation certificates (distance correlation for the non-linear injections) support that the numeric abstractions capture real input-output regularities, and an uninjected case study on 30,000-plus SDG-related topics finds systematic rules such as common topics being simplified and positive topics being answered more positively.","pith_inferences":["I infer the method's portability hinges on the stability of the belief elicitation: if rewording the rating prompts changes the extracted rules, then the rules describe the model's answers to a particular questionnaire rather than a stable behavioural trigger. This is testable by re-running the pipeline with two differently worded belief-elicitation templates and comparing rule sets.","The reliance on in-sample correlation certificates means the reported effect sizes are partly built into the injected data, since the behavioural wrapper reads the same $u$ scores used for explanation; an out-of-sample or shuffled-$u$ control would be needed to separate genuine trigger recovery from guaranteed correlation.","The SHAP-weighting trick generalizes in principle to any tree-based surrogate and any sparse linear selector, so RuleSHAP's design could be lifted into other rule-extraction toolchains, not just XGBoost-plus-LASSO.","The non-convex failures suggest that the next bottleneck is representational: parity- or set-membership conditions may need engineered interaction features or modulo-aware abstractions before global rules can be recovered."],"forward_implications":["An LLM auditor can run the whole pipeline without access to weights or hidden states: only topic prompts, generated texts, and the model's own ratings are needed.","Because RuleSHAP cuts rule overproduction by about 10.9 percent on average while raising faithfulness, rule-set size and detection quality do not trade off in the tested settings.","Conjunctive triggers (e.g., low commonality and moderate positivity) are the clearest win: RuleSHAP's MRR@1 on conjunctive rules is roughly 0.72 versus 0.32 for RuleFit, so multi-feature heuristics become findable rather than missed.","Non-convex triggers such as the interdisciplinarity-in-{1,3,5} modulo condition remain the hardest case for every tested method, implying that rule extraction still has a complexity ceiling even for simple injected heuristics.","On uninjected SDG-topic data, the same pipeline surfaces recurring belief-driven patterns—simplified language for common topics, sentiment echoing a topic's perceived valence, longer texts for technical topics—so the method is not only validated on synthetic injections."],"supporting_citations":[{"why":"Supplies RuleFit, the main rule-extraction baseline that RuleSHAP is compared against and improves upon.","marker":"[14]"},{"why":"Supplies SHAP's game-theoretic feature attributions, which RuleSHAP aggregates into feature weights for boosting and LASSO.","marker":"[12]"},{"why":"Motivates the 'bag of heuristics' view of LLM behaviour, the premise that belief-driven heuristics are discoverable in LLMs.","marker":"[10]"},{"why":"Supports the premise that LLMs encode global beliefs, which the abstraction pipeline elicits as numeric scores.","marker":"[9]"},{"why":"Provides distance correlation, used for correlation certificates validating non-linear input-output abstractions.","marker":"[29]"},{"why":"Supplies XGBoost, the gradient-boosting engine whose split sampling RuleSHAP steers with SHAP weights.","marker":"[40]"},{"why":"Supplies LASSO regression, whose penalty RuleSHAP modifies with SHAP-based rule weights in Step 3.","marker":"[39]"},{"why":"Provides the LLM-as-a-judge protocol used to generate three of the seven output-abstraction scores.","marker":"[35]"}],"fun_headline_variants":["RuleSHAP surfaces complex LLM bias rules 82% better than RuleFit","Global XAI meets rule extraction: RuleSHAP lifts MRR@1 by 82%","RuleSHAP recovers non-linear heuristics from LLMs, +82% MRR vs RuleFit","RuleSHAP exposes hidden LLM heuristics as rules, 82% better ranking","New RuleSHAP method reveals LLM bias triggers with 82% gain over RuleFit"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that an LLM's self-reported 1-to-5 belief scores for a topic are stable, causally upstream features that determine the bias-related properties of its outputs; if those ratings do not mirror the internal representations that actually shape generation, the extracted rules describe correlations between the model's own questionnaire answers and output proxies, not the mechanisms behind the behaviour.","fun_headline_variants_meta":{"raw":{"variants":["RuleSHAP surfaces complex LLM bias rules 82% better than RuleFit","Global XAI meets rule extraction: RuleSHAP lifts MRR@1 by 82%","RuleSHAP recovers non-linear heuristics from LLMs, +82% MRR vs RuleFit","RuleSHAP exposes hidden LLM heuristics as rules, 82% better ranking","New RuleSHAP method reveals LLM bias triggers with 82% gain over RuleFit"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000283,"raw_usage":{"total_tokens":1751,"prompt_tokens":1106,"completion_tokens":645,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":722,"completion_tokens_details":{"reasoning_tokens":528}},"tokens_in":722,"tokens_out":645,"duration_ms":6047,"temperature":1.0,"reasoning_tokens":528,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:56:25.490573+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RuleSHAP on a model whose injected trigger uses a feature that is deliberately absent from the elicited belief vectors—for example, make response length depend on the number of words in the topic phrase while all 11 rated properties are held approximately constant. If RuleSHAP still surfaces the injected rule, the method recovers triggers independent of the abstraction; if it does not, the recovered rules are confined to the pre-chosen belief dimensions. A second check: hold out part of the topic set, fit the abstraction correlation certificates only on the training half, and verify whether the extracted top rules still rank injected triggers on the held-out half at the same MRR.","supporting_citations":[{"cited_title":"Predictive learning via rule ensembles.The Annals of Applied Statistics, pages 916–954, 2008","cited_arxiv_id":null,"evidence_quote":"Supplies RuleFit, the main rule-extraction baseline that RuleSHAP is compared against and improves upon."},{"cited_title":"Brownian distance covariance.The Annals of Applied Statistics, pages 1236–1265, 2009","cited_arxiv_id":null,"evidence_quote":"Provides distance correlation, used for correlation certificates validating non-linear input-output abstractions."},{"cited_title":"Lasso regression.Journal of British Surgery, 105(10): 1348–1348, 2018","cited_arxiv_id":null,"evidence_quote":"Supplies LASSO regression, whose penalty RuleSHAP modifies with SHAP-based rule weights in Step 3."}],"review_version":1}