{"id":"3d7e404f-5918-4fb7-af05-e97d692ac64c","arxiv_id":"2412.03722","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A mixed-integer optimization framework selects effort-constrained feature changes with estimated probabilities to reclassify individuals in tree ensembles, with best-case, worst-case, and tail-risk variants.","lead":"This paper proposes optimization models that choose which features a person should try to change, under a limited effort budget, to make an ensemble tree classifier most likely to relabel them into a desired class. The authors test the models on an obesity dataset and compare their feature rankings with standard random-forest importance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Max-path, Min-path, and kappa-path objectives maximize a single selected leaf path, not the probability of majority-vote reclassification; the central claimed equivalence is false as stated.","rationale":"The paper's abstract and conclusions promise methods that maximize the probability of reaching the target class. The optimization models, however, maximize the probability of one path or a robust version of one path through the forest. Because the final prediction is a majority vote, the probability of the target class is a sum over many voting configurations. The paper never shows that maximizing a single summand maximizes the sum; the three-tree instance demonstrates that the rankings can be reversed. This is not a matter of calibration or estimation noise: even with perfectly known branch probabilities, the Max-path, Min-path, and kappa-path models solve a different problem. The empirical validation in Section 4 is also weakened by using the same synthetic perturbation model to estimate probabilities and to evaluate rankings, but that is secondary to the objective gap. The models and case study are clearly described, and the robust kappa-path idea is a reasonable extension, so a revised version that either restricts the claims to 'most probable path' or adds a correct sum over voting configurations would be worth considering. Given the current formulation, the central claim as stated is not supported, so I would reject the paper in its present form.","tokens_in":17283,"tokens_out":11150,"duration_ms":114842,"concrete_test":"Enumerate all leaf configurations for a small forest (e.g., R=3 or R=5 shallow trees) with fixed per-node probabilities. For each feasible effort allocation, compute (i) the Max-path objective value defined in Section 3.1 and (ii) the true reclassification probability, obtained by summing the product of leaf probabilities over all leaf configurations whose weighted vote total favors the target class. Then compare the effort allocations chosen by (i) and (ii). The three-tree example above (all trees 0.6 under A; two trees 0.7 and one 0 under B) already predicts disagreement, so the test should also solve the full model with constraints (2)-(13) on such an instance; if Max-path chooses B while the true probability is higher under A, the central claim requires revision.","verdict_should_be":"REJECT","load_bearing_attack":"Section 3.1's Max-path objective is not the probability that the forest assigns class k*. The forest's decision is a majority vote over trees (Section 2, constraint (5)), so, for independent trees, the probability of the event 'class k* wins the vote' is the sum, over every leaf configuration with at least half the tree weights voting for k*, of the product of the corresponding per-leaf probabilities. The Max-path objective instead fixes one leaf z_l^r per tree, uses theta_l^r and gamma^r to set theta=1 for trees treated as non-essential, and maximizes product_r (sum_l z_l^r theta_l^r). This equals the probability of a single specially selected leaf configuration, not the sum over all configurations that produce the target class. Nothing in constraints (2)-(13) forces the argmax of this single-configuration probability to coincide with the argmax of the true reclassification probability; in general it does not. Min-path and kappa-path optimize the minimum or a selected quantile of per-tree leaf probabilities, so they inherit the same gap. A concrete three-tree example shows the reversal: if allocation A gives each tree probability 0.6 of voting k* and allocation B gives two trees probability 0.7 and the third 0, the Max-path objective prefers B (0.49 vs 0.36 for the selected path), while the true majority-vote probability is 0.648 for A and 0.49 for B. Thus the case-study rankings and Tables 3-4 evaluate a different criterion than the abstract and conclusions claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a mathematical-optimization approach to feature-shift (recourse) in tree ensembles. Given a trained forest, per-node branch probabilities with and without effort, and an effort budget, the authors formulate three MINLP models (Max-path, Min-path, kappa-path) that choose which features an individual should modify and how much effort to spend, and they use the resulting solutions to rank feature importance. The method is applied to an obesity dataset with a random forest of 25 trees; the rankings are evaluated by simulating perturbed futures and measuring reclassification rates.","tokens_in":17598,"tokens_out":12514,"duration_ms":130891,"significance":"The probabilistic-effort viewpoint is potentially useful and the models are nontrivial; the paper contains a clear notation and a real-data case study. However, the central claim that the models maximize the probability of reaching the target class is not supported by the objective functions, and the empirical validation is circular because the same synthetic perturbation mechanism is used both to estimate the model inputs and to score the outputs. If the objective mismatch were corrected, the contribution would be a heuristic for the probability of a single path, not the claimed optimal reclassification procedure.","major_comments":[{"comment":"The objective in (Max-path), max ∑_r log(∑_l z^r_l θ^r_l), is not the probability that the forest assigns the target class k*. For independent trees the reclassification probability is the sum, over all leaf configurations whose weighted majority is k*, of the product of the per-leaf path probabilities. The proposed objective selects one leaf per tree, sets θ^r_l=1 for non-essential trees via γ^r and φ^r_l, and maximizes the product over the remaining essential trees. This is the probability of a single sufficient leaf configuration, not of the majority-vote event. Constraints (2)-(13) do not connect the two. A concrete reversal: with three trees, one k* leaf per tree, and vote probabilities (0.6,0.6,0.6) under allocation A versus (0.7,0.7,0) under B, Max-path prefers B (0.49>0.36), while the true majority-vote probability is 0.648 for A and 0.49 for B. Min-path and kappa-path optimize selected quantiles of the same per-tree leaf probabilities and inherit this gap.","section":"Section 3.1, Eq. (Max-path)"},{"comment":"The empirical validation is circular. In Section 4.2 the branch probabilities p^r_lt and p^r_lte are estimated by perturbing each feature 1000 times with δ∼U[0,σ_j] (no effort) or δ∼U[0,1.5σ_j] (effort), and by Bernoulli switches with parameter max(1−p,0.2) for binary features. Section 4.3 then evaluates the resulting rankings by applying exactly the same perturbation mechanism to the test set and measuring reclassification rates. Thus the simulation scores the rankings under the same generative model that produced the probabilities; it cannot provide independent evidence that the method identifies features that are impactful under a realistic change process. The ad hoc choice of the perturbation scales is also a free parameter of the whole experiment.","section":"Sections 4.2-4.3, Tables 3-4"},{"comment":"The count of non-essential trees in (12) is off by one. With R equally weighted trees in a binary forest, the desired class wins if at least ceil(R/2) trees vote for it, so the number of non-essential positive trees should be (#positive votes) − ceil(R/2). The paper subtracts R/2+1, which is one too large for even R and non-integer for odd R. The R=25 case study is only consistent if R/2 is read as integer division (floor(R/2)+1); the general formulation as written does not match the definition of essential trees in the text.","section":"Section 3.1, constraint (12)"}],"minor_comments":[{"comment":"Because φ^r_l=1 for leaves not in the desired class, θ^r_l may be set to 1 for every selected leaf of a negative-voting tree; if the objective is meant to be the probability of reaching the feature vector x, this probability is overestimated. The text should state clearly that the objective is only the probability of the essential path configuration, not the probability of the feature-shift vector itself.","section":"Section 3.1, constraint (13)"},{"comment":"The tables report point estimates only, with no standard errors or confidence intervals across the 100 simulations; some entries are close (e.g., η=4: 25.22 vs 26.07 for RSR/RFR), so the reader cannot assess whether observed differences are meaningful. Please report variability across simulations.","section":"Section 4.3, Tables 3-4"},{"comment":"The κ-path model maximizes the κ-th smallest per-tree leaf probability, not the cumulative probability of the κ smallest paths; the text referring to the 'joint probability of the κ less probable paths' and the analogy to CVaR should be reconciled with constraint (18) and the stated objective.","section":"Section 3.2, Eq. (κ-path)"},{"comment":"The notation τ^r_l appears to be used both for the set of leaves of tree r and for a specific leaf; please standardize the index sets to avoid confusion between τ^r and membership in a leaf set.","section":"Throughout, notation"}],"recommendation":"reject","confidential_remarks":"I recommend rejection because the central objective mismatch in Section 3.1 is not a local presentation issue: the models optimize a different quantity from the one claimed in the abstract and conclusions. The circular validation in Sections 4.2-4.3 compounds the problem. If the authors reframe the contribution as a heuristic for the most likely single path, the paper would need new validation and substantially weaker claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper adds something genuinely new to algorithmic recourse: instead of finding the closest point to the target class, it asks which features an individual should change, under a budget, when each branch in a tree ensemble has an estimated probability conditional on effort. The MINLP formulations for best-case, worst-case, and kappa-path objectives are coherent, and the feature-importance ranking derived from effort allocation is a sensible application. The obesity case study uses real data, and the results, taken as a heuristic, suggest the method can beat random forest variable importance for intervention design.\n\nThe main flaw is the gap between the abstract and what the Max-path objective actually solves. The paper claims to maximize the probability of being reclassified into the target class, but the objective is the product of probabilities of a single selected leaf configuration among the 'essential' trees. That is not the probability that the forest's majority vote lands on the target class. For independent trees, the true probability is a sum over all leaf configurations that win the vote. A simple three-tree example shows the Max-path can prefer an allocation with lower true reclassification probability. The Min-path and kappa-path objectives inherit the same gap because they also optimize over selected paths. The authors seem aware that single-path probability is small (Section 3.2), so the method is best understood as a heuristic for selecting a high-probability path, not a maximizer of reclassification probability. The abstract, conclusions, and some of the simulation interpretation need to be revised accordingly.\n\nA second, softer issue is the validation loop. The branch probabilities in Section 4.2 are estimated from synthetic perturbations, and the same perturbation model is then used to score the rankings in Section 4.3. This makes the experiments a test of internal consistency rather than external validity. It would be stronger to validate on a separate dataset with observed feature changes over time, or at least to vary the perturbation assumption.\n\nThe probability estimates themselves are ad hoc (uniform shifts, a Bernoulli parameter floored at 0.2), but that is secondary; the method is presented as a framework, not a calibrated model.\n\nThis paper deserves peer review. The contribution is useful and the formulations are a step beyond distance-based counterfactuals. But it needs major revision: either rename the objectives as selecting a most likely path, or include a summation over voting configurations to match the claimed probability, and then rework the validation. I would bring it to a reading group because the modeling gap is instructive.\n\nRecommendation: send to referees, with a clear instruction to check the probability claim and the circularity.","headline":"New probabilistic effort-budget counterfactual optimization for tree ensembles, but the max-path objective is not the reclassification probability it claims to maximize.","tokens_in":18120,"tokens_out":2854,"would_cite":true,"duration_ms":28371,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C11","90C26"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that finding the feature changes most likely to flip a tree ensemble's verdict is an effort-allocation problem, and that solving it yields feature rankings that outperform standard tree-ensemble rankings.","keywords":["feature shifts","counterfactual explanations","tree ensembles","mixed-integer nonlinear programming","recourse","feature importance","random forest","k-sum optimization"],"falsifier":"Longitudinal panel data with actual observed feature changes and subsequent classifications would settle the claim: estimate node-level change probabilities from the observed transitions, train a tree ensemble on the first wave, run the models with an effort budget, and compare the model's chosen effort features with random and random-forest rankings on second-wave outcomes; the central claim holds only if the model-selected features reclassify more people than the baselines under real, not simulated, changes.","tokens_in":17055,"feed_emoji":"🎯","tokens_out":10920,"duration_ms":95074,"temperature":0.7,"pith_summary":"The paper argues that the right way to ask which feature changes will reclassify an observation is to ask which changes an individual is actually able and likely to make, not which shifted point is closest in feature space. Given a trained tree ensemble, per-node branch probabilities with and without effort, and an effort budget, the authors formulate three mixed-integer nonlinear programs—Max-path, Min-path, and kappa-path—that allocate effort across features to maximize the probability that the ensemble votes for the desired class. If the probability estimates are reliable, the models yield both the most probable feature shift and a feature-importance ranking that excludes immutable variables. On an obesity dataset, the rankings produced this way reclassify more test-set individuals than random-forest importances or random rankings in simulated future scenarios.","feed_headline":"Optimization finds which few features actually change a forest's call","feed_subtitle":"Optimization with change probabilities picks where to act; obesity tests show it beats standard rankings.","key_machinery":"The load-bearing object is the path-probability product at a leaf. For leaf $\\ell$ in tree $r$, the probability of reaching $\\ell$ after the effort decisions is $\\prod_{t \\in A(\\ell)} \\bigl(p^r_{\\ell t} + \\sum_{e=0}^{E} \\beta^r_{t e}(p^r_{\\ell t e} - p^r_{\\ell t})\\bigr)$, where $\\beta^r_{t e}=1$ means $e$ units of effort are put into the feature split at node $t$ and $A(\\ell)$ is the set of ancestors of $\\ell$. Constraint (13) turns this product into an upper bound on the auxiliary variable $\\theta^r_\\ell$, with correction terms for leaves that do not target the desired class and for non-essential trees; because the objective maximizes a sum of logarithms of these $\\theta$-values, the model prefers allocations that give high probability to the required number of essential trees. The $\\kappa$-path variant borrows the k-sum/CVaR idea to maximize the $\\kappa$-th smallest path probability, giving a tunable middle ground between best-case and worst-case behavior. This machinery lets effort allocation and feature-shift location be chosen together, rather than treating the shifted point as a free variable minimized by distance.","core_discovery":"At the center of the paper is a new definition of the feature-shift problem for tree ensembles. Instead of minimizing a distance from the original observation to the closest point that the forest classifies as desired, the authors maximize the probability that the observation reaches such a point after a limited amount of effort. The input data are, for every node in every tree, the probability that the observation follows the branch leading to a given leaf both without effort, $p^r_{\\ell t}$, and with $e$ units of effort, $p^r_{\\ell t e}$. The decision variables select one leaf per tree, the effort level applied to each feature (shared across all splits that use that feature), and which trees count as essential for the majority vote. The objective is the log-probability of the path through the essential trees; a robust variant maximizes the worst-case path, and a $\\kappa$-path variant maximizes the sorted $\\kappa$-th path probability subject to a minimum cumulative probability threshold. On a real obesity dataset, rankings built from the effort allocations of these models reclassify more test-set individuals in simulations than rankings from the random forest itself or from random selections, with the 50%-path version reaching 93.33% of the feasible-to-change individuals at an effort budget of three features.","pith_inferences":["If the probability estimates were learned from real longitudinal transitions rather than synthetic perturbations, the same models would likely produce recourses that are more plausible and better calibrated, because the optimization is only as good as those inputs.","The path-probability decomposition is not tied to binary classification or unweighted forests; the essential-tree threshold can be replaced by any majority rule, so the approach should carry over to weighted and multiclass ensembles, though the paper only tests the binary case.","The method could be used prospectively to design interventions: rather than asking which features matter globally, a policymaker could ask which features, for a given population and effort budget, maximize the number of individuals who cross the classification threshold."],"forward_implications":["Counterfactual explanations for tree ensembles can be built to maximize the probability of the shift, not minimize its distance, so recourses can be realistic even when no close point exists.","The effort cap $\\eta$ forces the solution to concentrate on a few features, giving a natural sparsity prior and producing feature-importance rankings that skip immutable variables such as age and gender.","The three model variants span a risk spectrum from best-case (Max-path) through intermediate ($\\kappa$-path) to worst-case (Min-path), and the 50%-path version delivered the best reclassification rates in the case study.","Applying the models to the obesity data produced rankings whose top features differ from random-forest importances, and these rankings reclassified more test-set individuals in simulations.","The approach treats per-feature change probabilities as first-class inputs, so it can be applied to any tree ensemble with univariate splits once those probabilities are available."],"supporting_citations":[{"why":"Prior optimal counterfactual explanations in tree ensembles; the main baseline approach this paper replaces with probabilistic effort allocation.","marker":"[24]"},{"why":"Earlier optimization-based counterfactual explanations in classification, one of the distance-based approaches the paper argues against.","marker":"[6]"},{"why":"Source of the obesity dataset used for the case study and simulations.","marker":"[9]"},{"why":"Robust counterfactual explanations for tree-based ensembles, which motivates the worst-case and kappa-path variants.","marker":"[10]"},{"why":"Defines k-sum optimization, the mechanism behind the kappa-path model's robust objective.","marker":"[25]"},{"why":"Surveys CVaR beyond finance and supplies the risk-measure interpretation used for the kappa-path family.","marker":"[12]"}],"fun_headline_variants":["Probability-maximizing feature shifts beat distance-minimizing in forests","Optimize effort: which features to change for a tree ensemble's new call","Forest reclassification: pick features by probability, not proximity","Effort-aware feature selection reclassifies more in tree ensembles","Maximize reclassification odds: new feature-shift method for forests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire method hinges on the input probabilities that a person will change each feature, with and without effort, being known in advance and independent across trees; in the case study those probabilities are manufactured by 1000 synthetic perturbations with distributions chosen by the authors, so if the numbers do not reflect real change behavior the recommended features and the reported gains inherit that error.","fun_headline_variants_meta":{"raw":{"variants":["Probability-maximizing feature shifts beat distance-minimizing in forests","Optimize effort: which features to change for a tree ensemble's new call","Forest reclassification: pick features by probability, not proximity","Effort-aware feature selection reclassifies more in tree ensembles","Maximize reclassification odds: new feature-shift method for forests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000861,"raw_usage":{"total_tokens":3738,"prompt_tokens":952,"completion_tokens":2786,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":2696}},"tokens_in":568,"tokens_out":2786,"duration_ms":19083,"temperature":1.0,"reasoning_tokens":2696,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:09:35.978585+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Longitudinal panel data with actual observed feature changes and subsequent classifications would settle the claim: estimate node-level change probabilities from the observed transitions, train a tree ensemble on the first wave, run the models with an effort budget, and compare the model's chosen effort features with random and random-forest rankings on second-wave outcomes; the central claim holds only if the model-selected features reclassify more people than the baselines under real, not simulated, changes.","supporting_citations":[{"cited_title":"Optimal counterfactual explanations in tree ensembles, in: International conference on machine learning, PMLR","cited_arxiv_id":null,"evidence_quote":"Prior optimal counterfactual explanations in tree ensembles; the main baseline approach this paper replaces with probabilistic effort allocation."},{"cited_title":"Generating collec- tive counterfactual explanations in score-based classification via mathematical optimization","cited_arxiv_id":null,"evidence_quote":"Earlier optimization-based counterfactual explanations in classification, one of the distance-based approaches the paper argues against."},{"cited_title":"Obesity level estimation software based on decision trees","cited_arxiv_id":null,"evidence_quote":"Source of the obesity dataset used for the case study and simulations."},{"cited_title":"Robust counter- factual explanations for tree-based ensembles, in: International conference on machine learning, PMLR","cited_arxiv_id":null,"evidence_quote":"Robust counterfactual explanations for tree-based ensembles, which motivates the worst-case and kappa-path variants."},{"cited_title":"Revisiting k-sum optimization","cited_arxiv_id":null,"evidence_quote":"Defines k-sum optimization, the mechanism behind the kappa-path model's robust objective."},{"cited_title":"Conditional value-at-risk beyond finance: a survey","cited_arxiv_id":null,"evidence_quote":"Surveys CVaR beyond finance and supplies the risk-measure interpretation used for the kappa-path family."}],"review_version":1}