{"id":"4b3c72a2-3359-487e-a745-e1d6f66e17b5","arxiv_id":"2504.20642","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Restricting demographic parity enforcement to the decision-making region of a score distribution yields better fairness-performance trade-offs than enforcing it globally.","lead":"This paper proposes focusing fairness constraints only on the score range where resource allocation decisions are actually made, rather than across the entire score distribution. It shows on three datasets that this decision-centric approach often achieves better fairness-performance trade-offs than global fairness enforcement.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (6) never uses τ: k% is fixed from a λ=0 baseline, and the penalized top-k% is a per-group quantile set that can drift away from [τ,1] during training; the claim that fairness is induced 'specifically within the decision-making region' is therefore asserted, not measured.","rationale":"The paper's contribution is a clean formalization: evaluate and enforce demographic parity only over the decision region [τ,1] via ABPCτ, ABCCτ, and AUC-PRτ, motivated by online resource allocation in which the threshold drifts with changing constraints. The formalization is coherent, the experiments are transparent, code is released, and λ is swept over a grid so results are shown as Pareto fronts rather than cherry-picked points. The paper also honestly flags its own limitation, and the reader's weakest_assumption identifies the same issue.\n\nThe concern is load-bearing because the optimization in Eq. (6) never uses τ. k% is fixed once from a λ=0 baseline, and the penalty is a 1-Wasserstein distance between per-group top-k% conditional score distributions. The claimed mechanism, fairness induced specifically within the decision-making region, holds only if that top-k% set tracks [τ,1] as training shifts the scores. This is not guaranteed: per-group quantile boundaries differ from τ whenever group distributions differ, and the paper's own footnote shows the penalty has a strong downward pull on scores, since the direct quantile variant collapsed all scores below τ. Section 5.3's alignment argument also presupposes the proxy, because ABCCτ is a truncated 1-Wasserstein integral over [τ,1] while the penalty is the 1-Wasserstein distance of the top-k% distributions; they coincide only when the sets match.\n\nThe empirical results are real evidence in the paper's favor: ABCCτ improves and, importantly, the less-aligned ABPCτ evaluation (Figure 4) also shows an advantage, so the result is not purely an evaluation-alignment artifact. But the magnitude and the 'consistently' claim could depend on dataset-specific proxy behavior, and the headline interpretation is unverified at the mechanism level. The concrete test, measuring the overlap between the penalized top-k% set and {score ≥ τ} on the test set, settles the mechanism question in one step. This supports the reader's CONDITIONAL verdict: the authors should add the proxy-overlap analysis, and ideally repeated seeds given the acknowledged training instability, before the 'consistently' claim is accepted at full strength.","tokens_in":18337,"tokens_out":18210,"duration_ms":155294,"concrete_test":"On the held-out test set, for every decision-centric model in Figures 4–6, compute the fraction of instances in each group's top-k% set (as defined during training) with score ≥ τ, and report each group's top-k% score boundary versus τ across the λ grid. If the overlap is materially below 1 (e.g., < 0.9) for any dataset, τ, or λ, the penalty operated substantially outside the decision region and the 'specifically within the decision-making region' mechanism is unverified. As a robustness follow-up, re-run Figure 5 with k% re-estimated on the validation set each epoch to re-anchor the top-k% to the current model's τ-quantile; if the decision-centric Pareto fronts then shift by more than the decision-centric versus global gap, the reported advantage depends on proxy drift rather than on genuine decision-region enforcement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 5.1) is that decision-centric fairness 'consistently delivers superior trade-offs' because it induces fairness specifically within the decision-making region. The optimization, however, never references τ. In Eq. (6), L_unfairness = IPM(ỹ^(k%)_0, ỹ^(k%)_1), with k% fixed once from the proportion of validation scores above τ under a λ=0 baseline (Section 3.2). During training, the penalty acts on the top-k% of each group's current predicted scores, a per-group, per-batch quantile set whose boundary is model-dependent and free to drift. The paper's own Section 3.2 footnote and conclusion call this 'a proxy for the ideal case where fairness is enforced strictly within the decision-making region,' and report that the direct τ-based quantile penalty was abandoned because even tiny λ collapsed all scores below τ.\n\nThe risk is concrete. The per-group top-k% boundaries differ from each other and from τ whenever group score distributions differ, so the penalized region is not [τ,1] even at initialization; as training shifts scores, the boundary can drift further. Nothing in the loss prevents the penalized region from disconnecting from [τ,1], and the documented downward pull makes misalignment plausible. Section 5.3's explanation that ABCCτ 'aligns closely' with the decision-centric penalty presupposes the proxy: ABCCτ is the truncated 1-Wasserstein integral over [τ,1], while the penalty is the 1-Wasserstein distance of the top-k% conditional distributions; the two coincide only if the top-k% set tracks [τ,1].\n\nThis is a mechanism gap between the formalization (ABPCτ, ABCCτ over [τ,1]) and the optimization. The empirical ABCCτ and ABPCτ improvements on three datasets are encouraging, and the less-aligned ABPCτ results support the method, but the claimed 'specifically within the decision-making region' mechanism is untested, as the paper itself acknowledges.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a decision-centric view of demographic parity for binary classifiers used in resource allocation: rather than enforcing parity over the full score distribution, fairness should be evaluated and induced only over the decision-relevant threshold region [τ, 1]. The authors introduce truncated variants ABPCτ and ABCCτ of existing distribution-level fairness metrics, a decision-centric performance metric AUC-PRτ, and a training penalty (Eq. (6)) that penalizes the 1-Wasserstein distance between the top-k% score distributions of two protected groups, with k% derived from the baseline model's selection rate above τ. They compare this decision-centric approach with a global fairness penalty on three datasets (one semi-synthetic, one proprietary), across varying bias rates and thresholds, and report that decision-centric models often dominate the global approach in the AUC-PRτ versus ABCCτ/ABPCτ plane. The paper also discusses the legal and practical motivation for focusing fairness on the actionable region.","tokens_in":18657,"tokens_out":5982,"duration_ms":64176,"significance":"The problem is well motivated: online resource allocation with dynamic thresholds makes threshold-specific post-hoc fairness difficult, and enforcing fairness over irrelevant score regions can needlessly degrade predictive performance. Formalizing ABPCτ, ABCCτ, and AUC-PRτ is a useful contribution, and the paper ships public code, uses multiple datasets, and transparently discusses its own limitations. If the empirical dominance were established with uncertainty quantification and with a penalty that demonstrably acts on [τ, 1], the proposed method would be practically valuable. In its current form, the strength of the empirical claims exceeds what the evidence supports.","major_comments":[{"comment":"The training penalty never uses the decision threshold τ. The value of k% is fixed once from a λ = 0 baseline on the validation set, and the objective then compares the top-k% of each group's current predicted scores. Because the boundaries of those per-group top-k% sets are group-specific and model-dependent, the penalized set need not coincide with [τ, 1] at initialization and can drift during training: scores below τ can be penalized while scores above τ can be excluded. The statement that the penalty 'specifically focuses on actionable instances' is therefore asserted rather than measured. The paper itself calls this a proxy in the Section 3.2 footnote and in the conclusion. To support the central claim that the decision-centric approach induces fairness 'specifically within the decision-making region,' the authors should quantify the overlap between the penalized top-k% regions and [τ, 1] during and after training (for example, report the proportion of penalized instances with score ≥ τ, and the proportion of instances with score ≥ τ that are penalized), or compare against a stabilized direct τ-based penalty.","section":"Section 3.2, Eq. (6)"},{"comment":"The main fairness axis ABCCτ is the integral of the CDF gap over [τ, 1], which coincides with the 1-Wasserstein distance, while the decision-centric objective penalizes the Wasserstein distance between top-k% conditional scores. Thus the evaluation metric is aligned with the decision-centric objective in a way it is not with the global objective. The paper acknowledges in Section 5.3 that this alignment 'may (partly) explain' the larger differences between the Pareto fronts, but the Section 5.1 claim of 'consistently superior trade-offs' is stated without conditioning on that caveat. I ask the authors to report the analysis with ABPCτ as the primary decision-region fairness metric (Figure 4 shows a much smaller separation than Figure 5), and to add at least one evaluation in which the fairness metric is not aligned with the training loss, or to explicitly restrict the strength of the conclusion.","section":"Section 5.3, Figures 4 and 5"},{"comment":"All reported Pareto fronts appear to be based on a single run per configuration: a single train/validation/test split, a single initialization, and hyperparameters selected only for λ = 0. Given that the differences between global and decision-centric fronts are small on some datasets (notably Adult and the low-bias TelecomKaggle variants), the central claim of consistent superiority needs repeated seeds and error bars or bootstrap intervals. Without such uncertainty quantification, the observed separation between the fronts could reflect stochastic variation rather than a systematic advantage of the decision-centric penalty.","section":"Section 5, Figures 4-6"}],"minor_comments":[{"comment":"AUC-PRτ is described informally with a figure; please provide a formal definition of the partial PR curve, including how the curve is normalized, since this metric is one of the two evaluation axes used in the Pareto comparisons.","section":"Section 4.2"},{"comment":"The notation 'top-k%' is ambiguous: it should be made explicit that the percentile is applied per protected group, and the relationship between the fixed scalar k% and the group-specific quantile boundaries should be stated precisely.","section":"Section 3.2, Eq. (6) and Figure 2"},{"comment":"The legends use 'Global reg.' and 'Local reg.' while the text refers to the 'global fairness approach' and 'decision-centric approach'; please use consistent terminology throughout the figures and text.","section":"Figures 4-6"},{"comment":"There are several typographical issues: 'V os' appears in the reference to Peeperkorn and De Vos, Figure 2 contains the label 'top□k%', and Figures C5 and C6 captions contain 'e ffect'.","section":"References and figures"},{"comment":"The Churn dataset is proprietary, so the results on it cannot be reproduced with the released code; please state this explicitly in the reproducibility section and consider evaluating on a public proxy dataset as well.","section":"Section 4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest and well written, and the core idea is timely. My main concern is that the current evaluation may partly reward the decision-centric method for optimizing a proxy of the evaluation metric, and that the absence of repeated runs makes the 'consistently superior' claim fragile. A revision that adds alignment diagnostics, alternative evaluation metrics, and uncertainty quantification would substantially strengthen the paper. I would not reject it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is worth your time. It does two practical things: it truncates Han et al.'s ABPC/ABCC distribution-level fairness metrics to a decision region [τ,1], and it operationalizes fairness induction by penalizing the 1-Wasserstein distance between the top-k% score distributions of two groups. The truncated metrics are straightforward extensions, and the training penalty is essentially the Peeperkorn–De Vos Wasserstein IPM applied to a per-group top-k% slice. The newness is modest, but the framing is sensible and the writing is transparent.\n\nWhat the paper does well: the experiments are coherent, the Pareto-front comparisons are clearly presented, and the decision-centric method does consistently move the frontier in the favorable direction across three datasets and several bias/threshold settings. The authors also deserve credit for shipping code, for flagging the proprietary Churn dataset, and for explicitly acknowledging the proxy nature of their top-k% penalty in both Section 3.2 and the conclusion. The less-aligned ABPCτ results still favor their method, which reduces the worry that everything is driven purely by evaluation-optimization alignment.\n\nNow the soft spots. First, there are no repeated runs or error bars; every Pareto point appears to be a single seed, which makes the 'consistently superior' claim more brittle than the plots suggest. Second, the paper itself admits in Section 5.3 that ABCCτ aligns closely with the decision-centric penalty, and that this may partly explain the larger gaps. That is honest but means part of the reported advantage is circular. Third—and this is the more substantive issue—Equation (6) never references τ. The k% is fixed once from a λ=0 baseline, and then the penalty acts on each group's current top-k% scores, which are per-group quantile sets that can drift away from [τ,1] as training proceeds. So the central claim that fairness is induced 'specifically within the decision-making region' is asserted, not measured. The authors call this a proxy, and the direct τ-based quantile penalty was abandoned because it collapsed scores below τ. That is a reasonable engineering choice, but it means the mechanism gap between the formalization (ABPCτ, ABCCτ over [τ,1]) and the optimization (top-k% Wasserstein) is real and unresolved.\n\nNone of this sinks the paper. The empirical evidence is directionally consistent, the authors are candid about limitations, and the decision-centric framing is genuinely useful for practitioners who face variable thresholds. I would send this to peer review. The referee should ask for repeated-seed results, a sensitivity analysis of the k% proxy (e.g., does the top-k% set actually stay close to [τ,1] during training?), and a softening of the claimed mechanism where the paper currently says 'specifically within.'\n\nFor you: it's a solid paper for anyone working on fairness in resource allocation or on distribution-level fairness metrics. I would cite the truncated metric definitions if I needed a decision-region variant. Worth a serious referee; not a desk reject.\n\nBest.","headline":"A useful, honest paper that defines decision-region fairness metrics and a top-k% training penalty; the empirical trade-offs are plausible, but the claimed mechanism of enforcing fairness 'specifically within' [τ,1] is exaggerated because the penalty never uses τ and the proxy can drift.","tokens_in":19304,"tokens_out":1524,"would_cite":true,"duration_ms":17477,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Enforce fairness only where scores actually trigger resource decisions.","keywords":["decision-centric fairness","demographic parity","resource allocation","fairness-accuracy trade-off","Wasserstein distance","top-k score distributions","churn prediction","AUC-PR"],"falsifier":"Train the decision-centric model and monitor, at each epoch, the actual decision-region disparity ABPC$\\tau$ on a held-out set; if the top-$k$ penalty can be reduced while ABPC$\\tau$ stays unchanged or worsens, the proxy is doing the wrong work. Conversely, if any model that reduces the top-$k$ Wasserstein distance reliably reduces ABPC$\\tau$ across bias levels and thresholds, the central claim is supported.","tokens_in":18115,"feed_emoji":"⚖️","tokens_out":8650,"duration_ms":75551,"temperature":0.7,"pith_summary":"This paper argues that when a predictive score is used to allocate resources—loans, retention offers, fraud investigations—fairness should be measured and induced only in the score range that can actually trigger an action, not across the full distribution of scores. It formalizes this idea by truncating distribution-level demographic parity metrics to the decision region $[\\tau,1]$, yielding ABPC$\\tau$ and ABCC$\\tau$, and by training classifiers with a loss that penalizes differences between groups only among the top-$k\\%$ of scores. Across a public telecom churn dataset, a proprietary churn dataset, and the Adult dataset, the authors claim this decision-centric approach consistently achieves better predictive performance at equal decision-region fairness than a global approach, with the gap widening as historical bias grows and as the decision region shrinks. The intended payoff is that online resource allocation with changing thresholds can be kept fair without sacrificing as much score quality.","feed_headline":"Fairness should be enforced only where scores trigger decisions","feed_subtitle":"Restricting demographic-parity constraints to actionable scores keeps resource allocation fair without sacrificing predictive quality.","key_machinery":"The machinery has three parts. First, decision-centric fairness metrics: ABPC$\\tau = \\int_{\\tau}^{1} |f_0(x)-f_1(x)|\\,dx$ and ABCC$\\tau = \\int_{\\tau}^{1} |F_0(x)-F_1(x)|\\,dx$, which restrict the area-between-curves demographic parity measures to the actionable score range $[\\tau,1]$. Second, the induction mechanism: a composite loss $L = (1-\\lambda)\\cdot L_{\\mathrm{BCE}} + \\lambda\\cdot L_{\\mathrm{unfairness}}$, where $L_{\\mathrm{unfairness}}$ is the 1-Wasserstein distance, computed via Sinkhorn approximations, between the distributions of the top-$k\\%$ predicted scores of each protected group, with $k\\%$ chosen from a baseline model as the validation proportion above $\\tau$; this proxy keeps the penalty focused on actionable instances without using $\\tau$ directly in the loss. Third, the evaluation metric AUC-PR$\\tau$, the area under the partial precision-recall curve for thresholds above $\\tau$, measures predictive performance only where decisions occur. Together these define what it means for a model to be optimized for decision-centric demographic parity rather than global parity.","core_discovery":"The paper's central claim is that demographic parity in resource allocation should be defined and optimized relative to the decision-making region—the thresholds $\\tau$ an operator might actually use—because enforcing parity outside that region constrains the model without improving any real decision. To carry this out, the paper localizes distribution-level fairness metrics by integrating only over $[\\tau,1]$, and trains neural classifiers with a composite loss in which the unfairness penalty is the 1-Wasserstein distance between the top-$k\\%$ score distributions of the two protected groups, where $k\\%$ is fixed from the proportion of baseline validation scores above $\\tau$. The empirical comparison on three datasets is meant to show that this decision-centric penalty produces Pareto-superior trade-offs between decision-region predictive performance, measured by a truncated area under the precision-recall curve (AUC-PR$\\tau$), and decision-region fairness, relative to a global penalty applied over the whole score range, and that the benefit increases with bias in the data and with a smaller (higher-$\\tau$) decision region.","pith_inferences":["If the top-$k$ proxy were replaced by a penalty that uses $\\tau$ directly, the method would target the true decision region; the paper reports that quantile-based versions were training-unstable, so a stable implementation would be a natural test of whether the proxy is the main source of slippage.","The same localization logic should transfer to ranking-based resource allocation and uplift modeling, where only the top of the ranking is acted upon; measuring decision-centric parity there could avoid the classification-threshold mismatch entirely.","A legal reading of the argument suggests that local demographic parity may satisfy disparate-impact requirements whenever individuals outside the actionable range are never acted upon; testing this against regulatory interpretations would determine how far the equivalence holds.","On datasets with multiple or intersecting protected attributes, the top-$k$ Wasserstein penalty could be applied per group combination, though variance in small subgroups may require larger batches or smoothed estimates."],"forward_implications":["In online settings where resource constraints shift the decision threshold over time, a model trained with the decision-centric penalty can maintain demographic parity across the whole range of plausible thresholds without retraining or post-hoc group-dependent thresholds.","At the same level of decision-region demographic parity, the decision-centric method is claimed to retain higher AUC-PR$\\tau$ than global enforcement, meaning fewer wasted resources or missed positive cases for the same fairness guarantee.","The advantage over global fairness grows with stronger historical bias and with narrower decision regions (higher $\\tau$), and disappears at $\\tau=0$, where the two approaches coincide.","In some configurations the unfairness penalty acts as regularization, so both fairness and decision-region predictive performance improve relative to the unpenalized baseline.","For deployment, the authors recommend selecting models on the ABCC$\\tau$ / AUC-PR$\\tau$ trade-off, using ABPC$\\tau$ as a secondary check, because ABCC$\\tau$ reflects how far probability mass must move between groups rather than only how much mass differs."],"supporting_citations":[{"why":"Supplies the distribution-level demographic parity metrics ABPC and ABCC that the paper truncates to the decision region.","marker":"Han et al., 2023"},{"why":"Supplies the global fairness training scheme using a BCE loss plus a Wasserstein unfairness penalty, which the decision-centric loss modifies.","marker":"Peeperkorn and De Vos, 2024"},{"why":"Provides the integral probability metric and 1-Wasserstein formulation with gradient approximation used in the unfairness penalty.","marker":"Shalit et al., 2017"},{"why":"Provides the Sinkhorn distance approximation used to compute the Wasserstein penalty efficiently during training.","marker":"Cuturi, 2013"},{"why":"Anchors the disparate-impact and legal-actionability motivation for demographic parity that the decision-centric framing relies on.","marker":"Feldman et al., 2015"},{"why":"Motivates the dynamic churn-prediction setting with changing resource constraints and thresholds that the method targets.","marker":"Ali and Arıtürk, 2014"}],"fun_headline_variants":["Fairness only where scores trigger decisions, not globally","Decision-centric fairness: localize parity to actionable scores","Enforce demographic parity only in the decision-making region","Better fairness–accuracy trade-offs by focusing on decision thresholds","Localize fairness constraints to the scores that drive allocation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the top-$k\\%$ of scores—with $k$ fixed once from a baseline model's validation scores above $\\tau$—stays a faithful stand-in for the true decision region $[\\tau,1]$ while the model is being retrained, even though the score distribution shifts and the penalty never uses $\\tau$ directly.","fun_headline_variants_meta":{"raw":{"variants":["Fairness only where scores trigger decisions, not globally","Decision-centric fairness: localize parity to actionable scores","Enforce demographic parity only in the decision-making region","Better fairness–accuracy trade-offs by focusing on decision thresholds","Localize fairness constraints to the scores that drive allocation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000196,"raw_usage":{"total_tokens":1373,"prompt_tokens":970,"completion_tokens":403,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":326}},"tokens_in":586,"tokens_out":403,"duration_ms":4172,"temperature":1.0,"reasoning_tokens":326,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:23:28.884378+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the decision-centric model and monitor, at each epoch, the actual decision-region disparity ABPC$\\tau$ on a held-out set; if the top-$k$ penalty can be reduced while ABPC$\\tau$ stays unchanged or worsens, the proxy is doing the wrong work. Conversely, if any model that reduces the top-$k$ Wasserstein distance reliably reduces ABPC$\\tau$ across bias levels and thresholds, the central claim is supported.","supporting_citations":[],"review_version":1}