Inherent Trade-Offs in the Fair Determination of Risk Scores
read the original abstract
Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.
This paper has not been read by Pith yet.
Forward citations
Cited by 19 Pith papers
-
Toward Calibrated Mixture-of-Experts Under Distribution Shift
Expert calibration suffices for MoE calibration under distribution shifts in hard-routed models but not soft-routed ones; adversarial reweighting improves the accuracy-calibration tradeoff across models and shifts.
-
Toward Calibrated, Fair, and accurate Deepfake Detection
Face-Feature Tuning is a label-free logit remapping method that reduces FPR/TPR gaps across groups in deepfake detection while preserving overall accuracy.
-
Long-term Fairness with Selective Labels
A framework and RL algorithm for long-term fairness under selective labels that decomposes the true fairness measure into observed fairness plus prediction bias and provides sufficient conditions based on predictor co...
-
Multi-User Dueling Bandits: A Fair Approach using Nash Social Welfare
The work establishes a regret lower bound of Ω(T^{2/3} min(K,D)^{1/3}) for fair multi-user dueling bandits with heterogeneous Condorcet winners and gives algorithms achieving matching upper bounds up to logs.
-
AgentFairBench: Do LLM Agents Discriminate When They Act?
AgentFairBench is a multi-domain benchmark for demographic disparity in LLM agent actions, with a pilot showing no significant effect for Claude Haiku 4.5 after arity-matched noise correction.
-
Learning Fair Demand Models
Compares enforcing parity-wise and Rawlsian fairness in demand estimation versus price optimization stages in a linear demand pricing model, characterizing conditions for higher social welfare and showing coincidence ...
-
Demystifying the Optimal Fair Classifier in Multi-Class Classification
Derives tractable optimal fair multi-class classifier and supplies in-processing and post-processing algorithms that converge to the accuracy-fairness Pareto frontier.
-
Inside Baseball: The Automated Ball-Strike System as an Object Lesson in Technological Rule Enforcement
An STS case study of MLB's Automated Ball-Strike System reveals that clear rules still require complex sociotechnical translation and calls for practice-based evaluation of automated enforcement systems.
-
Revisiting Fairness Impossibility with Endogenous Behavior
Error-rate balance and predictive parity become compatible under endogenous behavior by adjusting stakes differently across groups, introducing a new form of unequal treatment in consequences.
-
Responsible Evaluation of AI for Mental Health
Proposes an interdisciplinary framework and taxonomy for responsible evaluation of AI mental health tools based on analysis of 135 publications identifying gaps in metrics, expert involvement, safety, and equity.
-
Aligning AI With Shared Human Values
Introduces ETHICS benchmark showing current language models have promising but incomplete ability to predict basic human ethical judgments on text scenarios.
-
Pareto-Guided Teacher Alignment for Fair Personalized Text Generation
Fairness mitigation in personalized text generation is objective-dependent with methods occupying different regions of the fairness-personalization Pareto frontier rather than any single strategy dominating all objectives.
-
What Medicine Taught Us About Fairness and What It Missed: Lessons from Reconsidering Race-Specific Lung Function Reference Algorithms
Analysis of lung function algorithms shows GLI-Global implicitly treats roughly 62% of the Black-White FEV1 gap as exposure-related and that clinical studies applied sufficiency-style fairness criteria before formal A...
-
Inside Baseball: The Automated Ball-Strike System as an Object Lesson in Technological Rule Enforcement
A qualitative case study of MLB's ABS shows that technological enforcement of rules involves bridging historically contested ground truths and balancing stakeholder ecosystems rather than simple measurement of distanc...
-
Secondary Bounded Rationality: A Theory of How Algorithms Reproduce Structural Inequality in AI Hiring
Secondary bounded rationality describes how AI recruitment algorithms reproduce structural inequality by optimizing for biased proxies of competence drawn from cultural and social capital disparities.
-
A Gaia-linked High-purity QSO Candidate Catalog in Selected Fields with Extinction-binned Calibration and Spectrum-informed Training
The P3 selector achieves 0.9809 purity and 0.8869 completeness for QSO candidates in selected fields, outperforming Gaia's official probabilities.
-
Contextual Multi-Objective Optimization: Rethinking Objectives in Frontier AI Systems
Frontier AI needs contextual multi-objective optimization to select and balance multiple context-dependent objectives rather than relying on single stable goals.
-
Statistical and Structural Approaches to Algorithmic Fairness
Thesis claims to identify and address reliance on deterministic point estimates and isolated-individual modeling in algorithmic fairness.
-
Fairness Definitions and Metrics in Deep Reinforcement Learning for Drug Discovery in Healthcare: A Rapid Evidence Review
A synthesis of fairness definitions and metrics for DRL molecule generation, focusing on dataset composition, reward functions, and parity across cancer versus non-cancer targets.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.