{"id":"5349d7f7-0ea4-4d9c-9218-189ef1b7e67e","arxiv_id":"2412.13158","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Stratifying marginal Shapley by feature region avoids off-manifold extrapolation and, with a causal direction and a chosen reference constant, reproduces causal Shapley values on a linear spline and an insurance example.","lead":"This paper studies how marginal Shapley values, a common way to explain machine learning predictions, can go wrong when features are correlated and force the model to extrapolate. It proposes a stratified method that computes Shapley values within feature subgroups and can match causal Shapley values, demonstrated on a synthetic spline and French auto insurance data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The toy equivalence is arithmetically correct, but the 'stratified approach' is underspecified outside two features: no general rule is given for choosing strata or redistributing the stratum baseline, so the claimed extrapolation-free route to causal Shapley values is not yet a defined method.","rationale":"I verified the algebraic claims in the toy model. The value functions in Eq. (3.2) and Appendix B are correct, and substituting xi = beta0 + beta12*gamma into Table 4 reproduces Table 6 for both causal directions. This is a legitimate, non-obvious observation and deserves credit. The reader's verdict of CONDITIONAL is therefore appropriate. My stress-test adds weight to the conditional status by identifying exactly why the toy result does not yet constitute a general method. The transition from Table 3 to Table 4 uses causal knowledge to decide how the non-constant baseline is redistributed, but no algorithm is given for multi-feature settings. The real-data section stratifies only on driver age and marginalizes all other features jointly, so it does not demonstrate that extrapolation is globally avoided. The proposed concrete test, a three-feature extension with an explicit recursive stratification rule, would settle whether the equivalence is a structural property of marginal averaging or an artifact of the two-feature deterministic threshold model. Until that test is run, the paper's broader claims should be treated as a promising conjecture rather than an established method.","tokens_in":10446,"tokens_out":9656,"duration_ms":91692,"concrete_test":"Specify the stratified algorithm as an explicit procedure for a three-feature linear spline with a known DAG, e.g. X3 -> X2 = 1(X1 > 0), X1 ~ N(0,1), X3 ~ Bernoulli(0.5), f = beta0 + beta1*X1 + beta2*X2 + beta3*X3 + beta12*X1*X2 + beta23*X2*X3. Implement the natural recursive extension of Section 3: stratify on causal parents in topological order, compute within-stratum marginal Shapley values, and redistribute the baseline according to the causal ordering; then compare the result with causal Shapley values from Heskes et al. If the procedure cannot be stated without consulting the target causal Shapley values, or if the outputs differ, the general claim fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim in the abstract is that the proposed stratified marginal averaging avoids extrapolation and, with causal information, replicates causal Shapley values. The two-feature derivation in Section 3 and Appendix B checks out: Table 4 with xi = beta0 + beta12*gamma is exactly Table 6. But that equality depends on three special facts: (i) the stratifying variable X2 is a deterministic function of X1, so the two strata are exactly the support; (ii) the model is linear within each stratum, so within-stratum marginal Shapley values are unambiguous; and (iii) the causal direction tells the analyst which feature must receive the entire baseline transfer. No general construction is given for selecting strata, for handling more than two features, or for redistributing a non-constant baseline across features in a multi-feature DAG. In the real-data example (Section 4), only DrivAge is stratified; all other features are still marginalized jointly, so correlations among them, e.g. Area/Density, which the paper itself notes, can still force off-support evaluations. Thus the paper has not shown that the stratified approach avoids model extrapolation or that it is a method for replicating causal Shapley values beyond the toy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies the extrapolation problem in marginal (interventional) Shapley values for correlated features. Using a two-feature linear spline model f(X) = β0 + β1X1 + β12X1X2 with X2 = I(X1 > 0), it shows that linear and constant extrapolation conventions lead to different and unintuitive Shapley decompositions. It then proposes a 'stratified' approach: compute Shapley values within strata of the causal parent feature using within-stratum background data, then redistribute stratum-specific baselines based on causal direction. For the toy model, the resulting Table 4 with ξ = β0 + β12γ exactly reproduces the causal Shapley values in Table 6 for both directions X1 → X2 and X2 → X1. The method is illustrated on French motor third-party liability claims data, stratifying on driver age.","tokens_in":10672,"tokens_out":11191,"duration_ms":96072,"significance":"The paper's closed-form computations are internally consistent; I verified Tables 1, 2, 4, and 6 and the Appendix B derivation. The two-feature equivalence between stratified marginal averaging and causal Shapley values is a genuinely interesting observation and is stated with an explicit caveat ('at least for a simple problem considered here'). If the construction could be generalized, it would offer a computationally cheaper route to causal Shapley values. However, the significance is currently limited to the toy setting: no general definition of strata, no multi-feature baseline redistribution rule, and no validation against causal Shapley values in the real-data example are provided.","major_comments":[{"comment":"The method is not defined in general. No rule is given for selecting the stratifying variable(s), for handling more than two features, for continuous stratifiers with many levels, or for distributing the stratum baseline among multiple features in a DAG. The real-data application in Section 4 uses a single ad hoc transfer of the entire non-constant baseline to DrivAge. Because the abstract claims a general 'approach,' this under-specification is load-bearing; the paper should either provide a general construction or explicitly limit the claim to the two-feature example.","section":"Section 3, Table 4"},{"comment":"The equivalence with causal Shapley values is demonstrated only for a two-feature model with deterministic dependence and linearity within strata. No argument is given that within-stratum marginal averaging equals causal conditional averaging for general DAGs or for M > 2 features. The conclusion is appropriately cautious, but the abstract and Section 3's statement that 'causal Shapley values can be computed without conditional averaging' are stronger than what is proven.","section":"Appendix B, Table 6, Conclusion"},{"comment":"Stratifying on DrivAge does not by itself avoid extrapolation for the remaining features: they are still marginalized jointly within each age stratum. The manuscript notes the high Area/Density correlation and drops Area, but that is a preprocessing choice, not a property of the method; other correlations among the remaining features can still force off-support evaluations. Moreover, no causal Shapley values are computed on the real data, so the real-data section provides no evidence for the replication claim.","section":"Section 4, Figures 4-7"},{"comment":"The reference constant ξ is free in the stratified approach, and the paper discusses several possible choices. The equality with causal Shapley values holds only for ξ = β0 + β12γ = E[f], which is indeed the natural causal φ0, but the paper should state this dependence explicitly and explain how ξ would be chosen in a multi-feature problem. As written, the replication of causal Shapley values appears to require knowing the causal baseline in advance.","section":"Table 4, Section 3"}],"minor_comments":[{"comment":"The phrase 'liner extrapolation' should be 'linear extrapolation'.","section":"Section 2.2"},{"comment":"The URLs for references [12] and [13] appear to be swapped: [12] points to the SHAP 'Explaining a simple OR function' page while [13] points to the scikit-learn user guide. The in-text citation for Histogram-Based Gradient Boosting should point to the scikit-learn documentation.","section":"References [12] and [13]"},{"comment":"At x1* = 0, the expression (1 + sign(x1*))/2 gives 0.5, but X2 is defined as I(X1 > 0), so the conditional expectation is discontinuous at 0; the statement should be qualified as holding almost everywhere.","section":"Appendix B, Eq. (B.2)"},{"comment":"The caption of Figure 4b should state explicitly that DrivAge is excluded from the 'rest of the features' panel.","section":"Section 4, Figure 4"},{"comment":"The description of how the age Shapley value is computed ('a non-constant portion of reference model should be transferred to the driver's age feature') is informal; please specify the formula, e.g., φ_age = f(x) - ξ - Σ_{j≠age} φ_j.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is closer to an application note than a full methodological development. The two-feature derivation is sound, but the general claims in the abstract and introduction need either a real general construction or substantial qualification. Given the journal context, I would only support acceptance after the scope is made explicit or the method is generalized."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the two-feature equivalence between stratified marginal Shapley and causal Shapley is real, and the arithmetic in Tables 4 and 6 checks out. That is the paper's contribution. It is a modest but genuine result, and the paper knows it is modest. The broader claim—that this points to a general way to compute causal Shapley values without conditional averaging—is plausible but not yet supported beyond the toy.\n\nThe paper does three things well. First, it gives a clean closed-form demonstration of how extrapolation corrupts marginal Shapley values for a linear spline, comparing linear and constant extrapolation. Second, it introduces the stratified approach: compute marginal Shapley within each stratum, then redistribute the per-stratum baseline using causal direction. Third, Appendix B shows that for the toy, with the natural choice of the baseline constant, the stratified values coincide with the causal Shapley values of Heskes et al. and Frye et al. That equality is the payoff.\n\nWhere it is soft: the method is not actually specified for more than two features. There is no general rule for picking strata, and no rule for how to distribute the baseline across features in a multi-feature DAG. The equality relies on the stratifying variable being a deterministic function of X1, on the model being linear within strata, and on the causal direction telling you which feature gets the baseline. That is a real limitation, and the stress-test note is right about it. The real-data section also lacks a causal-Shapley ground truth, error bars, or code; it is an illustration, not a benchmark. Minor: references [12] and [13] look swapped; the scikit-learn user guide URL points to SHAP docs and vice versa. Also, the choice of ξ is free, and while ξ=β0+β12γ is the natural one because it makes the stratified and causal values align, the paper does not give a principled way to choose ξ in a new problem.\n\nI think the paper is worth refereeing. The toy result is sound, the writing is clear, and the idea is novel enough to be interesting to people working on causal Shapley and extrapolation. A good referee report would ask for (a) a definition of the stratified method for more than two features, or an explicit statement of when it is applicable; (b) a real-data comparison against actual causal Shapley values using conditional averaging, with error bars; (c) code and data. Those are heavy asks, but they are the difference between a nice toy and a method.\n\nI'd take it to reading group and would cite the toy equivalence if I were writing about causal Shapley approximations.","headline":"The two-feature stratified-vs-causal equivalence is real and the math checks out; the paper is a solid, honest contribution, but the method is not yet defined beyond the toy, so treat the broader claim as a hypothesis.","tokens_in":11235,"tokens_out":2744,"would_cite":true,"duration_ms":25723,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that marginal Shapley values, which average over features in ways that force the model into sparse or impossible regions, can be made extrapolation-free by computing attributions inside strata of a causal feature and then…","keywords":["Shapley values","model extrapolation","marginal averaging","conditional averaging","causal Shapley values","stratified Shapley values","feature attribution","correlated features"],"falsifier":"Compute stratified marginal and causal conditional Shapley values for the linear spline model with an additional within-stratum correlation, for instance $X_3$ correlated with $X_1$ inside each sign region; if the two sets of attributions diverge, the claimed equivalence between stratified marginal averaging and causal conditional averaging is refuted.","tokens_in":10196,"feed_emoji":"📊","tokens_out":7228,"duration_ms":56922,"temperature":0.7,"pith_summary":"The paper claims that marginal Shapley values, which average over features in ways that force the model into sparse or impossible regions, can be made extrapolation-free by computing attributions inside strata of a causal feature and then redistributing the stratum-dependent baseline according to the causal direction. For the model $f(X)=\\beta_0+\\beta_1 X_1+\\beta_{12} X_1 X_2$ with $X_2=I(X_1>0)$, this stratified method reproduces exactly the causal Shapley values that use conditional averaging, for both causal directions $X_1\\to X_2$ and $X_2\\to X_1$. The result matters because it suggests causal attributions can be obtained from marginal averaging alone, avoiding the cost and complexity of conditional expectations. On a real motor-claims example, the stratified approach changes the attribution of Bonus Malus and exposes an age-related interaction that standard marginal Shapley values do not show.","feed_headline":"Stratified margins replicate causal Shapley values","feed_subtitle":"A marginal-averaging variant avoids extrapolation and matches conditional causal attributions on a spline toy.","key_machinery":"The central object is the stratified Shapley value: compute the usual marginal-average Shapley values separately on each stratum of a causal feature, obtaining a stratum-specific reference $\\phi_0$, then use the assumed causal direction to decide which feature receives the non-constant part of $\\phi_0$, leaving a common reference $\\xi$. The toy model carrying the argument is the linear spline $f(X)=\\beta_0+\\beta_1 X_1+\\beta_{12} X_1 X_2$ with $X_2=I(X_1>0)$, and the identity that makes the claim concrete is the match between the stratified formulas in Table 4 and the causal Shapley formulas in Table 6 when $\\xi=\\beta_0+\\beta_{12}\\gamma$ with $\\gamma=E[X_1X_2]=1/\\sqrt{2\\pi}$. This identity does the work of showing that conditional averaging can be replaced by within-stratum marginal averaging for this model class.","core_discovery":"On its own terms, the paper establishes that stratifying by a causally upstream feature, computing Shapley values separately within each stratum using marginal averaging, and then transferring the non-constant portion of the stratum reference value to the causal feature produces attributions identical to causal Shapley values for the linear spline model $f(X)=\\beta_0+\\beta_1 X_1+\\beta_{12} X_1 X_2$ with $X_2=I(X_1>0)$. The equality holds with shared baseline $\\xi=\\beta_0+\\beta_{12}\\gamma$, where $\\gamma=E[X_1X_2]=1/\\sqrt{2\\pi}$, and holds in both causal directions: under $X_1\\to X_2$ the attribution $\\phi_2=0$ with all baseline transfer going to $X_1$, and under $X_2\\to X_1$ the transfer goes to $X_2$. The paper presents this equivalence as evidence that causal Shapley values, normally defined through conditional averaging and requiring conditional expectation estimates, can be recovered from extrapolation-free marginal averaging whenever the correct stratification is known.","pith_inferences":["Editorial inference: the equivalence suggests a broader recipe—whenever a causal variable partitions the feature space so that no residual within-stratum correlation forces off-manifold evaluations, marginal averaging inside each stratum may approximate causal Shapley values; this can be tested on any model with a known causal graph.","Editorial inference: the method would likely weaken if the stratifying variable is continuous and strata are chosen by bins, because extrapolation reappears at bin boundaries; the paper's real-data demonstration sidesteps this by treating age as discrete.","Editorial inference: the comparison between stratified and conditional causal Shapley values could be used as a diagnostic—disagreement would signal a misspecified causal direction or residual within-stratum dependence.","Editorial inference: the caution about choosing $\\xi$ points to a practical heuristic not proposed in the paper: anchor $\\xi$ to a quantity that minimizes the distortion of feature attributions, such as the observed average prediction, or select $\\xi$ to preserve efficiency."],"forward_implications":["If the stratified method is correct, causal Shapley values for this model class can be computed without conditional averaging, which is usually more complicated and computationally expensive.","The method intentionally violates the Symmetry axiom of the original Shapley framework, so attributions are causal rather than purely game-theoretic.","The freedom to choose $\\xi$ lets an analyst answer different questions, such as comparing an observation to a region average, a representative observation, or the overall model mean, though extreme $\\xi$ choices can distort attributions.","On the French motor-claims data, stratifying by driver age yields Bonus Malus attributions with much stronger age dependence than standard marginal Shapley values, including a low-score signal for very young drivers that standard values hide.","The resulting attributions depend on the analyst's causal assumptions and stratification choice, so they are only as meaningful as those inputs."],"supporting_citations":[{"why":"Supplies the cooperative game setup and the Shapley value formula that every value function in the paper instantiates.","marker":"[1]"},{"why":"Frames model explanations as Shapley values with conditional or marginal value functions, the objects the paper critiques and modifies.","marker":"[2]"},{"why":"Prior work by the author arguing that conditional averaging implicitly assumes causality, motivating the paper's use of marginal averaging with explicit causal direction.","marker":"[6]"},{"why":"Documents that marginal averaging forces models to extrapolate into sparse regions, the problem the stratified approach is designed to remove.","marker":"[7]"},{"why":"Defines causal Shapley values with conditional averaging, the benchmark the stratified method matches in the toy model.","marker":"[8]"},{"why":"Provides the asymmetric Shapley formulation, the other causal benchmark whose value functions appear in Appendix B.","marker":"[9]"},{"why":"Supplies the French motor claims dataset and the Poisson regression modeling framework used in the real-data demonstration.","marker":"[11]"},{"why":"Describes using a background data subset for Shapley computation, the practical mechanism behind computing within-stratum marginal averages.","marker":"[13]"},{"why":"Used conditional Shapley values on the same motor-claims data, providing the comparison and rationale for the real-data discussion.","marker":"[14]"}],"fun_headline_variants":["Stratified margins dodge extrapolation, match causal Shapley","New marginal route to causal Shapley values without extrapolation","Avoid extrapolation: stratified marginals equal causal Shapley","Splitting by cause fixes marginal Shapley extrapolation","Causal Shapley from extrapolation-free stratified marginals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method presupposes that the analyst knows the true causal direction and the correct stratification, and that stratifying on that feature eliminates all problematic extrapolation; if either assumption fails, the stratified values will not equal a meaningful causal attribution.","fun_headline_variants_meta":{"raw":{"variants":["Stratified margins dodge extrapolation, match causal Shapley","New marginal route to causal Shapley values without extrapolation","Avoid extrapolation: stratified marginals equal causal Shapley","Splitting by cause fixes marginal Shapley extrapolation","Causal Shapley from extrapolation-free stratified marginals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1398,"prompt_tokens":928,"completion_tokens":470,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":382}},"tokens_in":544,"tokens_out":470,"duration_ms":4846,"temperature":1.0,"reasoning_tokens":382,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:22:39.728648+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute stratified marginal and causal conditional Shapley values for the linear spline model with an additional within-stratum correlation, for instance $X_3$ correlated with $X_1$ inside each sign region; if the two sets of attributions diverge, the claimed equivalence between stratified marginal averaging and causal conditional averaging is refuted.","supporting_citations":[{"cited_title":"A value for n-person games,","cited_arxiv_id":null,"evidence_quote":"Supplies the cooperative game setup and the Shapley value formula that every value function in the paper instantiates."},{"cited_title":"Causal Analysis of Shapley Values: Conditional vs. Marginal","cited_arxiv_id":"2409.06157","evidence_quote":"Prior work by the author arguing that conditional averaging implicitly assumes causality, motivating the paper's use of marginal averaging with explicit causal direction."},{"cited_title":"Causal Shapley values: Exploiting causal knowledge to explain individual predictions of complex models,","cited_arxiv_id":null,"evidence_quote":"Defines causal Shapley values with conditional averaging, the benchmark the stratified method matches in the toy model."},{"cited_title":"Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability,","cited_arxiv_id":null,"evidence_quote":"Provides the asymmetric Shapley formulation, the other causal benchmark whose value functions appear in Appendix B."},{"cited_title":"Wüthrich and M","cited_arxiv_id":null,"evidence_quote":"Supplies the French motor claims dataset and the Poisson regression modeling framework used in the real-data demonstration."},{"cited_title":"Explaining a simple OR function,","cited_arxiv_id":null,"evidence_quote":"Describes using a background data subset for Shapley computation, the practical mechanism behind computing within-stratum marginal averages."},{"cited_title":"Conditional expectation network for SHAP","cited_arxiv_id":"2307.10654","evidence_quote":"Used conditional Shapley values on the same motor-claims data, providing the comparison and rationale for the real-data discussion."}],"review_version":1}