{"id":"054892a9-93d1-4ee0-8129-756a3b9aa8b5","arxiv_id":"2505.07351","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GenRe trains an autoregressive transformer on pairs sampled from positive examples with probability proportional to exp(-lambda * cost), and generates recourse by forward sampling, outperforming search-based baselines.","lead":"The paper introduces GenRe, a generative model that produces algorithmic recourse by training on synthetic pairs of denied and accepted profiles, then generating recourse by direct sampling. It reports that GenRe beats eight prior recourse methods on a combined validity, plausibility, and cost score across three benchmark datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The consistency guarantee is proven for a different sampler than the one used in training: D_h^+ with γ≥0.5 and Algorithm 1's TopK truncation of Q are absent from Theorem 4.1, and neither γ nor K is reported.","rationale":"The paper's headline contributions are (i) a training-pair construction Q that is consistent with the ideal recourse distribution and (ii) an inference procedure that only forward-samples from the trained R_θ. The second contribution inherits its guarantees from the first. The weak point is therefore the mapping from Theorem 4.1's idealized sampler to the algorithm actually executed. Section 4.1 defines D_h^+ with a threshold γ≥0.5; the theorem's support argument only works at γ=0.5 with an exact h; Algorithm 1 additionally replaces Q by a TopK truncation. Neither γ nor K appears in the hyperparameter tables (C.2.4, D.2, D.3), so the reader cannot tell whether the trained model is trained on the distribution whose consistency is proved. This is a concrete, checkable gap rather than a disagreement with the empirical methodology: the experiments are broad (three datasets, eight baselines, ablations against nearest-neighbor and diffusion guidance), and the code is promised, so a conditional acceptance with a request to report γ/K and revise the theorem is the right outcome. The reader's weakest assumption names the γ support problem; I agree with it and extend it to the TopK truncation, hence 'partial' agreement. No change in verdict is needed.","tokens_in":21077,"tokens_out":10168,"duration_ms":95409,"concrete_test":"Read the released repository to recover the default γ from Algorithm 1 and the K used in Q_trunc. Then run a synthetic version of Section 4.4 with known P(y+|x) that includes a band 0.5<P(y+|x)≤γ, and compute ‖E_Q[x+|x]−E_R[x+|x]‖ for N+∈{100,1000,10000} using the reported γ and K. If γ>0.5 or K<|D_h^+| and the bias does not decay with N+, Theorem 4.1/4.2 does not apply to the actual training sampler; if γ=0.5 and K=|D_h^+|, the support condition is satisfied and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that GenRe's training produces a consistent estimate of the ideal recourse density R(x+|x) ∝ exp(-λC(x,x+)) P(x+|y+) V(x+), so forward sampling from R_θ at inference is a valid substitute for search. Theorem 4.1 is stated for the idealized sampler Q in Eq. 6, but the implementation deviates from that sampler in two concrete ways.\n\nFirst, D_h^+ is defined in Section 4.1 with h(x)>γ for γ≥0.5, while the proof sets V(x+)=δ(P(y+|x)>0.5)=δ(h(x)>0.5) and, in Eq. 9, sums the numerator over D1 but the denominator over D_h^+. If γ>0.5, positive regions with 0.5<P(y+|x)≤γ are part of the target support (V=1) but are excluded from D_h^+, so Q has zero weight where R has positive weight; the self-normalized importance-sampling argument collapses. The paper never reports the γ used in any experiment. If the intended value is γ=0.5, the support condition is restored, but the text says γ≥0.5 and no experimental γ is given.\n\nSecond, Algorithm 1 (Appendix B) does not sample from Q: line 3 defines Q_trunc(•|x)=TopK(Q(•|x,D_h^+),K) and line 6 draws training pairs from Q_trunc. Truncation changes the proposal in an uncontrolled way; no consistency or rate statement covers it, and K is never listed among the hyperparameters in Appendix C.2. A fixed K also blocks the asymptotic coverage needed for the O(1/N+) claim in Theorem 4.2.\n\nThird, even with γ=0.5 and no truncation, Theorem 4.1 assumes h agrees with P(y+|x) on the 0.5 threshold, while the experiments train an uncalibrated ANN h with 69.6–77.3% accuracy (Table 1) and give no misspecification bound. Since consistency is advertised as the reason forward sampling works, the gap between the proven sampler and the actual sampler leaves the central theoretical claim unsupported for the deployed pipeline. The empirical evidence is strong, so the right response is to report γ and K, re-prove consistency for the implemented sampler, and quantify sensitivity to h misspecification.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GenRe, an autoregressive conditional generative model for algorithmic recourse. Training pairs are synthesized from unpaired labeled data by pairing each negative instance x with positive instances x+ weighted by exp(-lambda C(x,x+)) among classifier-confident positives (D_h^+). The model is trained with a kernel-mixture cross-entropy loss, and at inference recourse is obtained by forward sampling, followed by selecting the sample with the highest classifier probability. The authors prove, under an idealized assumption that h matches P(y+|x), that the empirical pairing distribution Q consistently estimates an ideal recourse density R, and they report experiments on Adult, COMPAS, and HELOC against eight baselines, claiming that GenRe achieves the best composite score and the best cost-validity-plausibility trade-off.","tokens_in":21508,"tokens_out":7877,"duration_ms":74010,"significance":"If the central claims hold, the paper offers a useful alternative to inference-time optimization for recourse: a conditional recourse likelihood model that can be sampled from directly, which is conceptually cleaner and avoids brittle gradient-based search. The empirical study is broad (three datasets, eight baselines, a separate random-forest gold classifier, sensitivity analyses over lambda, temperature, and kernel width, plus a comparison against diffusion models with guidance), and the code is released. The main theoretical contribution, however, is currently a consistency result for the synthesized pair distribution Q rather than for the trained generator R_theta, and several implementation details (gamma, K) needed to connect the theory to the experiments are not reported. These gaps affect the strength of the headline claims but appear addressable.","major_comments":[{"comment":"Theorem 4.1 establishes only that the empirical conditional distribution Q(x+|x) is a consistent self-normalized importance-sampling estimate of the ideal R(x+|x). It does not establish that the trained autoregressive model R_theta converges to R. The training loss in Eqs. (7)-(8) is a likelihood-type objective on pairs drawn from Q_trunc, but no generalization bound, universal-consistency argument, or sample-complexity statement connects R_theta to R. The conclusion states \"We proved these pairs are consistent, and hence GenRe's method of training on them is consistent with the ideal recourse objective,\" which is a non sequitur unless R_theta is assumed to fit the sampled pairs perfectly. Because the abstract's consistency claim refers to GenRe's training, this gap is load-bearing.","section":"Section 4.4, Theorem 4.1"},{"comment":"The support condition for the importance-sampling argument is not satisfied by the implemented sampler. D_h^+ is defined with h(x)>gamma for a chosen gamma>=0.5, while the proof sets V(x+)=delta(P(y+|x)>0.5) and, in Eq. (9), sums the numerator over D1 but the denominator over D_h^+. If gamma>0.5, instances with 0.5<P(y+|x)<=gamma are in the target support (V=1) but have zero probability under Q, violating the proposal-support condition for self-normalized importance sampling. The manuscript never reports the gamma value used in any experiment. If gamma=0.5 was used, the support condition is restored, but the text and Algorithm 1 must say so; otherwise the consistency guarantee does not apply to the implemented sampler.","section":"Section 4.1 and Theorem 4.1"},{"comment":"Algorithm 1 does not sample from Q: line 3 defines Q_trunc by TopK truncation, and line 6 draws training pairs from Q_trunc. Neither K nor the truncation step appears in Theorem 4.1, and with a fixed K the asymptotic consistency and the O(1/N+) rate in Theorem 4.2 do not follow. K is not listed among the hyperparameters in Appendix C.2. The paper should either prove a statement for the truncated proposal, report K and justify its value empirically, or explicitly reposition the theory as applying to an untruncated idealized version of the training procedure.","section":"Algorithm 1 and Appendix C.2"},{"comment":"The rate claim in Theorem 4.2 is stated \"when h(x) is the actual conditional distribution P(Y|X)\", but in the experiments h is a three-layer ANN with accuracies of 77.33%, 69.60%, and 74.23% on the three datasets (Table 1), and the gold labels come from a separate calibrated Random Forest. The paper provides no misspecification analysis, so the consistency and rate results do not directly cover the experimental regime. At minimum, the theoretical section should be reframed as an idealized analysis, and the experiments should report the agreement or calibration between h and the gold classifier that is needed to connect the two.","section":"Theorem 4.2 and Section 5.1"},{"comment":"The composite metric Score=Val+LOF-Cost/d is used to support the headline claim that GenRe provides the best overall trade-off. Dividing cost by d makes the cost term negligible when d is large (d=21 for HELOC), and the equal additive combination of validity and LOF is arbitrary. Moreover, on cost alone GenRe is worse than Wachter and DICE on Adult (0.69 vs 0.31 and 0.22, respectively), so the abstract's phrase \"superior performance across multiple metrics\" is not literally accurate as stated. I recommend reporting the trade-offs with a cost-weight sweep, or using a composite metric whose functional form is justified.","section":"Table 2 and Section 5.2"}],"minor_comments":[{"comment":"The text says eight prior recourse methods are compared but then lists nine names (Wachter, GS, DICE, ROAR, PROBE, REVISE, CRUDS, CCHVAE, and TAP); Table 2 contains no REVISE row. Please clarify which methods are actually reported.","section":"Section 5.1"},{"comment":"Section 5.1 reports tau=10.00 and sigma=0.00, but Algorithm 2 samples x'_j = mu_{j,k}+sigma_{j,k}*epsilon and Appendix D.2 reports sigma values in {2e-7, ..., 2e-2} with no 0.00 setting. Please reconcile these settings.","section":"Section 5.1 and Algorithm 2"},{"comment":"The proof defines V(x+) as delta(P(y+|x)>0.5), which appears to be a typo for P(y+|x+) (or h(x+)); please correct the notation.","section":"Theorem 4.1 proof"},{"comment":"The caption states that the x-axis is on an exponential scale, but the axis labels show linear cost values; please clarify or correct the caption.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing you should know: GenRe is a real step forward for algorithmic recourse. It replaces inference-time optimization with a trained conditional sampler, and it shows convincingly that this works better than the VAE-latent-search baselines. That is new and worth taking seriously. The paper is already out at ICLR 2025, so peer review is behind it; the question is whether the theory holds up enough to rely on.\n\nWhat's actually new: the training-pair synthesis. They take unpaired data, build pairs by weighting positive examples with exp(-lambda * C(x, x+)), and train an autoregressive transformer conditioned on the negative instance. At inference they just sample and pick the sample with highest classifier probability. The method is simple, reproducible (code is public), and the experiments are thorough: three datasets, eight baselines, ablations on lambda, temperature, truncation, nearest-neighbor comparison, and a diffusion-guidance comparison. The Score metric is ad hoc—dividing cost by d heavily downweights cost—but the raw numbers (validity, LOF, cost) still favor GenRe, so the overall conclusion isn't riding on that metric alone.\n\nWhere it gets soft: the consistency theorem. Theorem 4.1 proves consistency for the idealized Q defined in Eq. 6 with D_h^+ filtered at gamma >= 0.5. But the proof assumes V(x+) = delta(P(y+|x) > 0.5) = delta(h(x) > 0.5), meaning gamma = 0.5. If gamma > 0.5, the proposal assigns zero mass to valid positive regions; the self-normalized importance-sampling argument collapses. And the actual training uses Algorithm 1, which truncates Q to the top-K pairs per input; that truncation is not covered by the theorem. Gamma and K are not reported anywhere, so you cannot tell which sampler the proof addresses. Also, the inference-time selection (10 samples, pick highest h probability) does not obviously minimize cost, despite the abstract claiming \"minimum cost recourse.\" Those are not fatal to the empirical method—forward sampling clearly works in their experiments—but they mean the consistency proof is decoration, not a guarantee. The paper would be more honest if it dropped or heavily caveated the theorem.\n\nBottom line: this is a paper for anyone working on recourse or counterfactual explanations. It deserves a serious referee (it already got one) and it deserves citing for the paired-training approach. I'd cite it, but for the method, not the theory. If I were reviewing it today I'd ask for gamma and K to be reported, and for the proof to be restated for the actual sampler or removed.","headline":"A genuinely new and empirically strong recourse generator, but the consistency proof covers a different sampler than the one actually trained; treat it as an engineering contribution, not a theoretical one.","tokens_in":22080,"tokens_out":2975,"would_cite":true,"duration_ms":27257,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that algorithmic recourse should be modeled as a conditional sampling distribution, and that training a generator on cost-weighted positive pairs produces recourse that is more valid, more plausible, and less costly than…","keywords":["algorithmic recourse","counterfactual explanations","generative model","recourse likelihood","self-normalized importance sampling","autoregressive transformer","recourse robustness","machine learning fairness"],"falsifier":"On synthetic data with a known true model P(y+|x), train the classifier to be imperfectly calibrated and set the pair filter threshold γ above 0.5; because Q assigns zero probability to genuinely valid positives whose true P(y+|x) lies between 0.5 and γ, the importance-sampling support condition fails, and one should observe the sampled recourse's validity drop below what Theorem 4.1 predicts as γ increases. Measuring that drop would settle the role of the classifier fidelity assumption.","tokens_in":20886,"feed_emoji":"🎯","tokens_out":10017,"duration_ms":93602,"temperature":0.7,"pith_summary":"The paper tackles algorithmic recourse—the problem of telling someone denied a loan, job, or parole what minimal, realistic changes would flip the decision. It claims the three competing goals of recourse (low cost, plausibility in the desired class, and validity of the outcome) should be fused into one trainable distribution over alternate profiles, rather than balanced by per-user optimization at inference. Because direct pairs of 'person X and their good recourse' are unavailable, it synthesizes training pairs by sampling high-confidence positive examples with probability proportional to exp(-λ cost). It proves this synthetic pairing is a consistent estimator of the ideal recourse distribution whenever the classifier's confident-positive region agrees with the true favorable-outcome region. If correct, the method replaces brittle gradient search with a single forward pass, yields multiple options per user, and matches or beats eight existing recourse methods on standard benchmarks.","feed_headline":"Sampling, not searching, gives better algorithmic recourse","feed_subtitle":"A trained generative model produces valid, plausible, low-cost recourse in one forward pass, beating eight baselines.","key_machinery":"The argument turns on the ideal recourse distribution R(x+|x) ∝ exp(-λ C(x,x+)) P(x+|y+) V(x+), and on the empirical proposal Q(x+|x) that replaces it during training. Q puts probability on high-confidence positive instances proportional to exp(-λ cost); the proof that Q is a consistent, self-normalized importance-sampling estimate of R is what licenses training a likelihood model on pairs and then only sampling at inference. The model itself is an autoregressive encoder-decoder transformer whose final layer outputs a kernel mixture per feature, which lets sampling proceed feature-by-feature.","core_discovery":"The central claim is that a conditional recourse likelihood model R_θ(x+|x) can be trained to output the ideal recourse distribution exp(-λ C(x,x+)) P(x+|y+) V(x+), where C is the cost of moving from x to x+, P(x+|y+) is the density of plausible positives, and V(x+) indicates that the outcome flips. The paper constructs an empirical sampling distribution Q(x+|x) over positives h-calibrated above a threshold, weighted by exp(-λ C(x,x+)), and proves via self-normalized importance sampling that training on pairs from Q is consistent—with error in the expected recourse instance decreasing at rate 1/N+—when the classifier h agrees with the true conditional probability on which points are favorable. The trained model is an autoregressive transformer with per-feature kernel-density outputs, so inference is forward sampling; empirically on Adult Income, COMPAS, and HELOC, GenRe achieves the best combined validity, plausibility, and cost among nine methods, with scores near the theoretical maximum of 2 on every dataset.","pith_inferences":["A testable extension is to apply the same pair-sampling recipe to high-dimensional data (images, text) with a continuous decoder; the consistency argument should survive, but the support condition on Q becomes harder to satisfy as the positive manifold grows sparse.","The paper never reports the exact value of γ it used; an immediate experiment is to sweep γ between 0.5 and higher values and check whether validity or the 1/N+ convergence rate degrades, which would indicate the thresholding is actively discarding valid positives.","The nearest-neighbor ablation suggests the learned model's advantage lies in interpolating between training positives; if so, GenRe should show larger gains over NNR on datasets where positive regions are sparsely populated, a prediction that could be tested by subsampling D+_h.","Amortizing recourse into a trained sampler fits a broader 'inference = one forward pass' trend; if this holds, recourse could be offered in real-time interactive settings, and the training data could be updated as the deployed model drifts, yielding a form of recourse lifelong learning."],"forward_implications":["Once GenRe is trained, a new denial gets recourse by a single forward pass, so per-user response time and cost drop dramatically compared with iterative optimization.","Sampling gives users a menu of recourse options—potentially diverse in which features change—so the user can pick the action they can actually implement.","The training procedure requires only unpaired labeled data, a cost function, and a classifier, so it transfers to any setting where these three are available without needing recourse examples.","Because the objectives are fused during training rather than at inference, GenRe's behavior degrades gracefully as the cost weight λ changes, unlike inference-time joint optimization which shows sharp swings in validity and plausibility.","The consistency theorem ties the quality of sampling-based recourse to the fidelity of the classifier, giving a concrete quantity (the agreement between h and P(y+|x)) that practitioners can audit before deploying the recourse model."],"supporting_citations":[{"why":"Supplies the ℓ1 cost function and the classic gradient-search objective that GenRe's ideal distribution extends, and is the primary cost-minimizing baseline.","marker":"Wachter et al., 2017"},{"why":"Formalizes actionable recourse with cost and validity constraints, providing the formulation underlying Equation 1.","marker":"Ustun et al., 2019"},{"why":"Introduces CRUDS, the VAE-based likelihood baseline that GenRe is contrasted against; the comparison in Section 5.3 shows training-based fusion beats inference-time fusion.","marker":"Downs et al., 2020"},{"why":"Introduces CCHVAE, a VAE baseline using latent-space search, which the paper argues suffers mode collapse and motivates the conditional likelihood approach.","marker":"Pawelczyk et al., 2020a"},{"why":"Introduces PROBE, a robustness-oriented baseline that ignores data density; provides the margin-based trade-off comparison.","marker":"Pawelczyk et al., 2023"},{"why":"Introduces ROAR, a robust-recourse baseline under model shift, used as a comparison for cost-validity trade-offs.","marker":"Upadhyay et al., 2021"},{"why":"Introduces TAP, a verifier-based validity baseline, showing the limitations of verification at inference time.","marker":"Friedbaum et al., 2024"},{"why":"Provides the self-normalized importance-sampling consistency and variance results used in Theorems 4.1 and 4.2.","marker":"Koller & Friedman, 2009"},{"why":"Supplies the transformer encoder-decoder architecture used as the recourse likelihood model R_theta, enabling autoregressive forward sampling.","marker":"Vaswani et al., 2017"}],"fun_headline_variants":["Generative recourse: sample it, don't search it","GenRe: Forward sampling beats costly search for recourse","From search to sampling: better algorithmic recourse","One forward pass yields valid, plausible, low-cost recourse","Training a generative model beats search-based recourse"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All guarantees and much of the empirical success depend on the classifier h being a faithful proxy for the true probability of a favorable outcome, so that the high-confidence positives it selects are genuinely valid and the final sampled recourse truly flips the decision.","fun_headline_variants_meta":{"raw":{"variants":["Generative recourse: sample it, don't search it","GenRe: Forward sampling beats costly search for recourse","From search to sampling: better algorithmic recourse","One forward pass yields valid, plausible, low-cost recourse","Training a generative model beats search-based recourse"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1349,"prompt_tokens":968,"completion_tokens":381,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":307}},"tokens_in":584,"tokens_out":381,"duration_ms":3852,"temperature":1.0,"reasoning_tokens":307,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:18:25.692131+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On synthetic data with a known true model P(y+|x), train the classifier to be imperfectly calibrated and set the pair filter threshold γ above 0.5; because Q assigns zero probability to genuinely valid positives whose true P(y+|x) lies between 0.5 and γ, the importance-sampling support condition fails, and one should observe the sampled recourse's validity drop below what Theorem 4.1 predicts as γ increases. Measuring that drop would settle the role of the classifier fidelity assumption.","supporting_citations":[{"cited_title":"Counterfactual explanations without opening the black box: Automated decisions and the gdpr","cited_arxiv_id":null,"evidence_quote":"Supplies the ℓ1 cost function and the classic gradient-search objective that GenRe's ideal distribution extends, and is the primary cost-minimizing baseline."},{"cited_title":"Actionable recourse in linear classification","cited_arxiv_id":null,"evidence_quote":"Formalizes actionable recourse with cost and validity constraints, providing the formulation underlying Equation 1."},{"cited_title":"Cruds: Counterfactual recourse using disentangled subspaces","cited_arxiv_id":null,"evidence_quote":"Introduces CRUDS, the VAE-based likelihood baseline that GenRe is contrasted against; the comparison in Section 5.3 shows training-based fusion beats inference-time fusion."},{"cited_title":"Probabilistically robust recourse: Navigating the trade-offs between costs and robustness in algorithmic recourse","cited_arxiv_id":null,"evidence_quote":"Introduces PROBE, a robustness-oriented baseline that ignores data density; provides the margin-based trade-off comparison."},{"cited_title":"Towards robust and reliable algorithmic recourse","cited_arxiv_id":null,"evidence_quote":"Introduces ROAR, a robust-recourse baseline under model shift, used as a comparison for cost-validity trade-offs."},{"cited_title":"Trustworthy Actionable Perturbations","cited_arxiv_id":"2405.11195","evidence_quote":"Introduces TAP, a verifier-based validity baseline, showing the limitations of verification at inference time."}],"review_version":1}