{"id":"1e33a3c1-f66d-46b0-a4df-031c26f1690f","arxiv_id":"2608.10766","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ROT explains individual AI predictions by fitting a single additive model with feature dropout to observed input-output pairs, yielding feature importances based on predictiveness rather than perturbation.","lead":"This paper introduces 'Rule of Thumb' (ROT), an explainable AI method that identifies which input features best predict an AI system's outputs by fitting one simple additive model to data the system has already produced. It targets black-box LLMs and proprietary systems where model weights are unavailable and where querying the model is expensive or impossible.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (1)'s additive surrogate-fidelity assumption is unverified in the paper's headline settings, and feature interactions make it false; ROT's local importances may describe the surrogate, not the target AI.","rationale":"The reader's weakest assumption is the right target: the central promise is that ROT explains the actual AI behavior from observed pairs, and Eq. (1) is the only link between the fitted additive functions and C(x). I checked whether the paper anywhere validates that link. Appendix D reports ROT's training/test accuracy only for the GPT-4o-mini image experiment (98.5%/97.6%), and even that is for the full feature set, not per-subset. No such numbers appear for judicial, movie, resume, or Amazon. The Conclusion asserts no additional assumptions are made, which conflicts with the acknowledged 'simplifying assumptions' in §2.1. The failure mode is not merely theoretical: any interaction (AND, XOR, or context-dependence) violates additive approximability, so the per-datapoint importance ranking can be wrong even when overall accuracy is decent. The synthetic-subset check I propose would settle whether the additive surrogate is faithful in controlled settings and would give a template for reporting fidelity in real black-box audits. I also considered the B.1.2 uniqueness claim, which is suspect under correlated features, but fidelity is the first-order issue: even a unique additive fit need not describe C. Because the method may still be useful when the additive approximation is accurate, and because the missing evidence is obtainable, I keep the reader's conditional verdict rather than moving to reject.","tokens_in":49131,"tokens_out":5796,"duration_ms":67889,"concrete_test":"Using the released code, generate synthetic targets with known feature interactions (e.g., C = x1 XOR x2, C = x1 AND x2, plus a linear target), draw the same number of observations as the judicial experiment, fit ROT via Eq. (2), and report (i) held-out subset prediction accuracy of F(Σ_{j∈J} f_{θ_j}(x_j) + G) for every singleton and pair J, and (ii) the rank of the true relevant features among ROT's |f_{θ_j}(x_j)| per datapoint. If singleton accuracy is near chance or the relevant feature is not ranked first when interactions are present, Eq. (1)'s additive assumption is doing the explanatory work, and the paper's claims must be scoped to approximately additive target systems.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (1) asserts that for every feature subset J, C(x) ≈ F(Σ_{j∈J} f_{θ_j}(x_j) + G). This additive, main-effects model is the entire basis for interpreting |f_{θ_j}(x_j)| as a local importance score. The paper calls this a 'simplifying assumption' in §2.1, but the Conclusion claims ROT 'does not make additional assumptions,' and no experiment quantifies the approximation error of Eq. (1) against the target system. In the judicial, movie, resume, and Amazon settings, the paper never reports how well the ROT surrogate predicts C(x) on held-out inputs or on masked subsets; only the image experiment reports ROT accuracy, and only for the full-feature subset. If C(x) contains feature interactions, no additive model can satisfy Eq. (1) for all J: for example, with C(x) = x1 XOR x2, every singleton subset has conditional expectation 1/2 under uniform inputs, so fitted additive contributions will not indicate that both features are jointly decisive for a particular prediction. The local explanation is then an artifact of the surrogate, not a description of the AI's behavior, and the core claim that ROT 'identifies the most relevant features' is unverified in the paper's headline settings.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Rule-of-Thumb (ROT), a post-hoc, model-agnostic local feature-importance method. ROT fits a single additive model—linear or logistic in sums of per-feature learned functions—to observed input/output pairs of a target AI system, using dropout over feature subsets as the training distribution (Eq. (2)). The learned per-feature functions are then presented as local importances. The authors demonstrate ROT on zero-shot LLM classification (judicial, movie, resume, image), on an audit of Amazon's recommendation system without a mimic model, on scientific-discovery scenarios including adversarially manipulated models, and on runtime benchmarks. They claim that ROT is substantially faster than SHAP/LIME and well suited to regulatory auditing.","tokens_in":49279,"tokens_out":4827,"duration_ms":54440,"significance":"ROT addresses a real gap: explaining API-only or historically logged AI systems without additional queries. If surrogate fidelity is adequate, the efficiency gains (a single fit reused for many explanations) and the ability to include non-input features are valuable contributions. Strengths include the released code, the breadth of experimental domains, the comparison of judicial and movie explanations against human annotations, the concrete adversarial-attack and mimic-model experiments, and the reported runtime measurements. The central conceptual claim, however, depends entirely on the additive approximation in Eq. (1), and this dependence is not tested in most of the headline settings. The paper would be a meaningful contribution if the authors can quantify when that approximation holds; in the present form, the central claim is not yet empirically supported.","major_comments":[{"comment":"The method's validity rests on the additive approximation C(x) ≈ F(Σ_{j∈J} f_{θ_j}(x_j) + G) for every feature subset J. Because ROT importances are exactly the fitted f_{θ_j}, any gap between this surrogate and the target system means the explanation describes the surrogate, not the AI. The manuscript never reports surrogate fidelity in the judicial, movie, resume, or Amazon settings; only the image experiment reports ROT test accuracy (Appendix D.1, 97.6%), and that is for the full-feature input only, not for masked subsets. Please report held-out agreement (or loss) between ROT's predictions and C(x) for all used settings, ideally broken down by mask size. Without this, the central claim that ROT \"identifies the most relevant features\" is unverified.","section":"§2.1, Eq. (1)"},{"comment":"If the target system has feature interactions, no additive model can satisfy Eq. (1) for all J. A simple example is C(x) = x1 XOR x2 over uniform binary inputs: every singleton-mask prediction is 1/2, so any fitted additive surrogate assigns equal, low importance to both features and fails to indicate that they are jointly decisive. The constructed scenarios in §3.3 are main-effects-like and do not test this failure mode. Please add interaction diagnostics (e.g., fidelity of the surrogate on singleton masks, or comparison with a model that includes pairwise terms) and state more carefully the conditions under which ROT importances are valid.","section":"§2.1, Eq. (1); §3.3"},{"comment":"The conclusion states that ROT \"does so without making additional assumptions,\" but §2.1 explicitly says \"We make some simplifying assumptions to answer this efficiently,\" and Eq. (1) is exactly such an assumption. This is internally inconsistent. The paper should either weaken the conclusion or justify why the additive assumption is not an additional assumption relative to existing sensitivity-based methods. The distinction matters because the practical value of ROT depends on whether the assumption holds for the target system.","section":"§5 vs §2.1"}],"minor_comments":[{"comment":"The sentence \"without access to its weights, ors without making additional API calls\" contains a typo: \"ors\" should be \"or\".","section":"§3.1.2"},{"comment":"The word \"adversial\" appears several times and should be \"adversarial\".","section":"§3.3.2 and Appendix D.8"},{"comment":"The row labeled \"Final Average\" reports a negative number of explanations (-6786.2), which appears to be an arithmetic artifact and should be corrected or removed.","section":"Table S4"},{"comment":"The reported 4.9% statistic is stated as a lower bound, but the same paragraph suggests the true value could be as high as 49%; please clarify which number is used in the main text and how the LLM filter's error rate was assessed.","section":"Appendix D.6"},{"comment":"The notation in Eq. (2) is dense and difficult to parse; rewriting the objective as an explicit expectation over randomly masked feature subsets would improve readability.","section":"Eq. (2)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is broad and would benefit from a sharper scope: the legal and regulatory discussion goes beyond the empirical evidence presented, and the central technical risk—the unverified additive-assumption fidelity—should be addressed with quantitative surrogate-fidelity measurements. If those measurements are favorable, the paper could be suitable for publication; without them, the headline claims are not yet supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my read on the ROT paper. The core trick is simple and the paper is right that it fills a real gap: if you can only observe an AI system's input-output pairs, you can fit a single additive model with feature dropout and use its per-feature functions as local importances. That gives you fast, model-agnostic explanations for API-only or proprietary systems. The applications to zero-shot LLM classification, non-input features, and auditing without mimic models are genuinely new and practically useful. The speed advantage is real, and the human-alignment numbers (e.g., wAUROC 0.77 vs SHAP's 0.74 on the judicial task) plus the adversarial-robustness results on simple if-then models show the approach has legs.\n\nWhat gives me pause is the gap between the claim and the evidence. The entire explanation rests on Eq. (1), which assumes the target system is well-approximated by an additive function of per-feature contributions for every feature subset. That is a strong assumption, and the paper never reports how well the ROT surrogate predicts C(x) in the judicial, movie, resume, or Amazon experiments. Only the image experiment gives a test accuracy, and even that is on the full-feature model, not on the masked subsets. If the target has interactions, an additive surrogate can assign zero importance to features that are jointly decisive—the XOR example in the stress test is real. So the importances may be describing the surrogate, not the AI. The paper also contradicts itself: §2.1 says \"we make some simplifying assumptions,\" while the Conclusion claims ROT \"does not make additional assumptions.\"\n\nThe \"no mimic models\" framing is inflated. ROT is itself a surrogate model—a global additive one rather than a local per-datapoint one, but still a mimic. The Rashomon problem reappears if the additive model class is wrong for the target. On evaluation, there are no confidence intervals, the judicial experiment drops cases and post-hoc flips signs, and the adversarial robustness is only demonstrated on simple if-statement models. These are addressable, but they matter.\n\nWho should read it? XAI practitioners who need fast attributions for closed models will find the method and code immediately useful. It deserves a serious referee, but it needs a major revision: report surrogate fidelity on every experiment, discuss or bound the additive approximation error, and tone down the \"no assumptions\" language. I would send it out, not desk reject it.","headline":"ROT is a practical, fast additive surrogate for explaining API-only systems, but the paper never verifies how faithfully that surrogate tracks the target system in its headline experiments.","tokens_in":49905,"tokens_out":3281,"would_cite":true,"duration_ms":36788,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Fitting one additive model with dropout to an AI's observed input-output pairs yields local feature importance explanations that identify which features most predict its decisions, without needing weights, queries, or mimic models.","keywords":["Rule of Thumb explanations","feature attribution","model-agnostic XAI","dropout","zero-shot classification","black-box auditing","additive models","adversarial robustness"],"falsifier":"Fit ROT to a system with a known interactive decision rule — for example, output flips only when two particular features are both present — using only observed input-output pairs, then check whether the fitted additive model reproduces the system's outputs on held-out inputs and whether the interacting features receive top importance. The paper reports ROT's own held-out prediction accuracy only for the image experiment (97.6 percent) and does not report surrogate fidelity for the judicial, movie, resume, or Amazon settings, so measuring how well the fitted additive model predicts the observed decisions in those settings would settle whether the approximation holds where the claims are made.","tokens_in":48832,"feed_emoji":"🔍","tokens_out":12167,"duration_ms":112360,"temperature":0.7,"pith_summary":"The paper introduces 'Rule of Thumb' (ROT), a post-hoc, per-datapoint explanation method that defines a feature's importance by how much knowing its value should update a prediction of an AI system's behavior, rather than by how much perturbing the input changes the output. ROT fits one additive model to input-output pairs the system has already produced, using dropout over feature subsets so that the fitted per-feature contributions predict the system's output from any subset of features, and these contributions become the explanation. Because it needs only existing predictions, ROT can explain closed systems accessible solely through an API — the setting where perturbation-based explainers such as SHAP and LIME need thousands of extra queries and gradient methods need model weights. The paper demonstrates ROT on zero-shot image and text classification with commercial LLMs, a direct audit of Amazon's proprietary recommender, and scientific-discovery scenarios where features were deliberately made uninformative, reporting that ROT aligns with human annotations at least as well as SHAP, resists adversarial 'fairwashing' attacks that fool SHAP and LIME, and makes each additional explanation nearly free.","feed_headline":"Fit one additive rule to get per-decision AI explanations","feed_subtitle":"Works from past predictions alone, even through an API, where perturbation-based explainers are infeasible.","key_machinery":"The load-bearing machinery is the all-subsets additive objective, Equation (2):\n$$L = \\sum_{J\\subseteq \\mathcal{J}}\\sum_{x\\in X} $p^{{|J|}}$(1-p)^{|\\mathcal{J}|-|J|}\\,\\ell\\Bigl[C(x),\\, F\\Bigl(\\sum_{j\\in J} f_{\\theta_j}(x_j)+G\\Bigr)\\Bigr],$$\nwhich demands that the system's output $C(x)$ be well estimated from any subset $J$ of revealed features. Minimizing it is equivalent to training the additive model with dropout at probability $p$ on the input features, so the sum over all $2^{|\\mathcal{J}|}$ subsets collapses into one stochastic fit. The per-feature functions $f_{\\theta_j}$ take a linear form, a linear-plus-mixture-of-Gaussians form, or, for text, a shared linear form over token embeddings; each fitted $f_{\\theta_j}(x_j)$ doubles as the signed importance of that feature value. The single global fit, rather than a per-datapoint fit, is what makes the first explanation expensive and every later explanation nearly instantaneous.","core_discovery":"In the paper's own framing, important inputs are those most predictive of the outputs of an AI system. Given a model $C(\\cdot)$ over the full feature set $\\mathcal{J}$, ROT seeks additive functions $f_{\\theta_j}$ and a global bias $G$ such that $C(x) \\approx F\\bigl(\\sum_{j\\in J} f_{\\theta_j}(x_j) + G\\bigr)$ holds for every subset of features $J$ and every datapoint $x$, and it finds them by minimizing the expected loss over all feature subsets and all datapoints. Dropout makes this exponential-looking objective tractable, and the number of calls to $C$ equals the number of datapoints, independent of the number of features. The fitted value $f_{\\theta_j}(x_j)$ is the signed importance of feature $j$ taking value $x_j$, and the feature with the largest $|f_{\\theta_j}(x_j)|$ is the one that should most change confidence in the prediction. The paper argues that this predictiveness reading of importance — closer to ablation studies than to sensitivity analysis — is what lets ROT explain zero-shot LLM classification without extra API calls, audit proprietary systems without mimic models, and generate scientific hypotheses without trusting out-of-distribution behavior. In experiments, ROT's token importances align with human annotations about as well as SHAP's (weighted AUROC 0.77 versus 0.74 on judicial texts, 0.72 versus 0.50 for a random baseline on movie reviews), and in the Amazon audit ROT highlights 'sold by amazon' as an important feature, an insight the mimic-based SHAP explanation did not surface.","pith_inferences":["ROT scores predictiveness, not causation: any scientific conclusion drawn from ROT should be cross-checked by measuring how well the additive surrogate predicts the phenomenon model on held-out subsets, a fidelity figure the paper reports only for the image experiment.","The non-input-feature trick suggests a log-only audit protocol: to probe whether an opaque system relies on protected attributes, append those attributes to the explanation feature set without changing the model's inputs, fit on historical decisions, and read off their importances — no new query ever required.","The worst case for ROT is interaction-heavy decision rules, so a natural stress test is a benchmark whose outputs depend on pairwise or higher-order feature combinations; such a benchmark would map where the additive assumption starts to break.","Because the per-datapoint cost is essentially zero after the first fit, ROT could be run as a continuous monitoring layer that re-explains every decision of a live API service, turning explanation from a per-request luxury into an always-on audit signal."],"forward_implications":["Zero-shot LLM classification becomes explainable at scale: ROT fits once on predictions the model has already produced, and after the first fit each extra explanation takes under 0.1 milliseconds, where the paper measures roughly 13 million additional ROT explanations per single SHAP explanation on consumer hardware.","Proprietary-model audits no longer need a mimic model: ROT explains Amazon's recommendation system directly from scraped observations, yields one consistent importance profile where mimic-based SHAP and LIME disagree with each other, and surfaces 'sold by amazon' as an important feature — a possible self-preferencing signal the mimic-based audit missed.","For scientific discovery, ROT correctly dismisses a deliberately uninformative feature: after a transformation makes age statistically uncorrelated with diabetes, ROT assigns it negligible importance while SHAP and LIME find it somewhat important, so hypothesis generation is not misled by perturbations off the data manifold.","Adversarial 'fairwashing' attacks fail against ROT: across ten attack experiments designed to hide a sensitive feature behind foil features, ROT recovers the sensitive feature as most important on over 89 percent of datapoints (100 percent in six experiments), while SHAP and LIME succeed on at most 5 percent.","ROT matches the access conditions regulators actually have: the EU AI Act's Article 74(12) lets market surveillance authorities use fixed datasets and observe outputs without querying arbitrary inputs, and ROT computes explanations from exactly that kind of access, supporting audits for self-preferencing and adversarial robustness."],"supporting_citations":[{"why":"SHAP, the main sensitivity-based baseline; ROT is compared against it for alignment with human annotations, runtimes, and the Amazon mimic-model audit.","marker":"[8]"},{"why":"LIME, the second perturbation-based baseline, compared against ROT on runtimes, uninformative-feature detection, and adversarial attacks.","marker":"[1]"},{"why":"Dropout, the training technique that makes the all-subsets objective tractable; the paper shows minimizing Equation (2) is equivalent to feature dropout.","marker":"[14]"},{"why":"PredEx judicial case data with human segment annotations; supplies the wAUROC comparison between ROT and other explainers.","marker":"[17]"},{"why":"ERASER benchmark movie review annotations; ground truth for measuring ROT's token-level alignment on API-based sentiment classification.","marker":"[19]"},{"why":"The zero-shot resume-filtering study ROT replicates, including the race, gender, and political-orientation probes; ROT verifies the original finding of negligible importance.","marker":"[10]"},{"why":"The Markup's Amazon recommendation audit, which ROT replicates and critiques by showing mimic-based SHAP explanations change with the mimic model.","marker":"[20]"},{"why":"FairPCA, used to transform the Pima diabetes data so that age is statistically uncorrelated with the target, creating the uninformative-feature test.","marker":"[55]"},{"why":"The adversarial attack methodology that fools SHAP and LIME with foil features; ROT's resistance is measured on the exact attack models and datasets.","marker":"[56]"}],"fun_headline_variants":["Explain black-box AI without model access or API calls","Rule-of-thumb XAI: fast, model-agnostic, audit-ready","Dropout-based explainer reveals key features per decision","Partial-information explainer beats SHAP speed on audits","One additive rule explains any AI, even zero-shot LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the target system's behavior is well approximated by the additive rule — its output given any subset of revealed feature values is roughly the sigmoid (or identity) function of the sum of per-feature contributions; if the system's true decision rule depends on interactions the additive family cannot express, the fitted importances describe the surrogate, not the system.","fun_headline_variants_meta":{"raw":{"variants":["Explain black-box AI without model access or API calls","Rule-of-thumb XAI: fast, model-agnostic, audit-ready","Dropout-based explainer reveals key features per decision","Partial-information explainer beats SHAP speed on audits","One additive rule explains any AI, even zero-shot LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00075,"raw_usage":{"total_tokens":3397,"prompt_tokens":1060,"completion_tokens":2337,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":676,"completion_tokens_details":{"reasoning_tokens":2252}},"tokens_in":676,"tokens_out":2337,"duration_ms":16362,"temperature":1.0,"reasoning_tokens":2252,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:46:25.230738+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fit ROT to a system with a known interactive decision rule — for example, output flips only when two particular features are both present — using only observed input-output pairs, then check whether the fitted additive model reproduces the system's outputs on held-out inputs and whether the interacting features receive top importance. The paper reports ROT's own held-out prediction accuracy only for the image experiment (97.6 percent) and does not report surrogate fidelity for the judicial, movie, resume, or Amazon settings, so measuring how well the fitted additive model predicts the observed decisions in those settings would settle whether the approximation holds where the claims are made.","supporting_citations":[],"review_version":1}