Pith. sign in

REVIEW 4 major objections 5 minor 19 references

Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in ML

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that most bias mitigation algorithms can achieve comparable fairness–accuracy tradeoffs once hyperparameters are optimized, so perceived superiority may reflect evaluation setup rather than the algorithm.

desk verdict Useful empirical counterpoint to single-setting fairness leaderboards, but the parity claim rests on visual Pareto overlap rather than a quantitative comparability criterion. read the letter →

arxiv 2411.11101 v2 pith:ZAZFIEUY submitted 2024-11-17 cs.LG cs.AIcs.CY

Prakhar Ganesh , Usman Gohar , Lu Cheng , Golnoosh Farnadi This is my paper · ORCID
classification cs.LGcs.AIcs.CY
keywords fairnessinmachinelearningbiasmitigationalgorithmshyperparametersensitivityfairness–utilitytradeoffParetofrontmodelmultiplicitybenchmarkingdemographicparity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Bias mitigation algorithms in machine learning are usually benchmarked under one uniform setup, with the assumption that identical training conditions make comparisons fair. This paper argues that those conditions are anything but neutral: fairness scores swing with random seeds, batch size, learning rate, model architecture, and the algorithm's own control parameter. Running seven mitigation algorithms on seven tabular datasets across multiple hyperparameter settings, the authors find no consistently winning method; different algorithms lead under different settings, and some fail to converge while others excel. Filtering each algorithm to the models on the fairness–accuracy Pareto front after a hyperparameter search, they conclude that most mitigation techniques can reach comparable tradeoffs. If true, algorithm rankings in fairness benchmarks can be an artifact of the chosen evaluation setup, and choosing an algorithm should hinge on context—runtime, guarantees, robustness, and arbitrariness—rather than a single tradeoff curve.

What carries the argument

The load-bearing object is the Pareto front of the fairness–utility tradeoff: the set of models from each algorithm that are not dominated on both demographic parity (or equalized odds or equal opportunity) and accuracy. The authors sweep seven hyperparameter settings per dataset, three control parameter values per algorithm, and multiple random seeds, then keep only the non-dominated models for each algorithm. Comparing these fronts is what turns 'which algorithm is best' into 'which algorithms can reach a given tradeoff at all,' and it is what reveals that multiple methods share nearly the same frontier on most datasets. The same object also carries the negative result: under a single fixed setting, the front collapses to one point per algorithm, and rankings that look stable are actually slices through a much larger and overlapping cloud of achievable outcomes.

What would settle it

Run the same seven algorithms on the same datasets with a substantially expanded hyperparameter search—for instance, a Bayesian optimization budget with dozens of trials per algorithm—and recompute each algorithm's Pareto front. If some algorithms that look competitive under the paper's grid are consistently dominated, or if algorithms that look weak are pulled onto the front, the claim that most techniques are competitive would fail; a quantitative coverage analysis showing the fronts still move materially after additional search would also falsify it.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the fairness–utility ordering of bias mitigation techniques is unstable across the learning pipeline, and that this instability undermines one-dimensional benchmarking. Using the benchmark's seven datasets and seven in-processing algorithms, the authors vary random seeds, control parameters, batch size, learning rate, and architecture, and plot demographic parity and accuracy tradeoffs. They observe that no algorithm dominates: HSIC, which the benchmark singled out as offering the best tradeoff, is matched or beaten by DiffDP and PRemover under other settings, and algorithms like AdvDebias and LAFTR only stand out under large-batch, high-learning-rate regimes where others fail to converge. Across datasets, HSIC's advantage fades on small datasets like COMPAS and German, and LAFTR's weakness appears tied to one-hot feature explosion rather than to the method alone. When the authors form the Pareto front of each algorithm over the searched hyperparameters, several algorithms occupy similar fronts on almost every dataset, which they take as evidence that most mitigation techniques are competitive given freedom to tune. They then show that runtime and prediction multiplicity distinguish algorithms even when tradeoffs do not.

Load-bearing premise

The load-bearing assumption is that the hyperparameter and seed grid searched here is wide enough to reveal each algorithm's true range of fairness–accuracy outcomes, even though the paper does not check whether a broader or different search would change the rankings.

Editorial extensions

If this is right

  • No single bias mitigation algorithm should be declared the best from a single hyperparameter configuration; results should report the spread of fairness scores across settings.
  • In ranking or regulator-facing evaluations, hyperparameter optimization should be part of the protocol, because restricted comparisons can unfairly favor or penalize algorithms.
  • Given comparable tradeoffs, selection criteria like training runtime, theoretical fairness guarantees, and prediction multiplicity become decisive; for example, DiffDP is attractive for fast training and PRemover for low ambiguity on some datasets.
  • Dataset aggregation in benchmarks can hide dataset-specific failures, such as HSIC's dependence on batch size for estimating statistical dependence.
  • Deployment context—whether retraining is cheap or expensive, whether stability or peak performance matters—should determine how fairness evaluations are conducted.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the comparability result generalizes, fairness leaderboards should shift from point rankings to reporting Pareto fronts and variance bands; otherwise they will keep rewarding whichever algorithm happens to fit the chosen defaults.
  • A natural testable extension is to check whether the same equalizing effect appears under automated hyperparameter optimization with larger budgets and across pre- and post-processing methods, which the paper does not cover.
  • The multiplicity results suggest a possible tradeoff beyond fairness–utility: algorithms that enforce fairness through representation learning may increase individual-level arbitrariness, which matters for contestability and recourse even when aggregate fairness looks identical.
  • Regulatory reporting of fairness metrics could be gamed through cherry-picked hyperparameters; the paper's variance results imply that reports should include a range or distribution of fairness scores over training configurations, not a single number.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies the sensitivity of in-processing fairness mitigation algorithms to hyperparameter choices, random seeds, and pipeline settings. Using the FFB benchmark's seven tabular datasets and seven mitigation algorithms, the authors vary batch size, learning rate, architecture, control parameter, and seeds, and plot fairness-accuracy tradeoffs under demographic parity, equalized odds, and equal opportunity. They argue that no algorithm dominates across settings, that FFB's single-setting and cross-dataset aggregation hides dataset-specific trends, and that after hyperparameter search most algorithms achieve comparable Pareto-front performance. They then propose selecting algorithms by additional criteria such as runtime, theoretical guarantees, and predictive multiplicity, and support this with runtime and ambiguity tables.

Significance. If substantiated, the central claim that 'most bias mitigation techniques can achieve comparable performance, given the freedom to perform hyperparameter optimization' would challenge the validity of single-setting fairness benchmarks such as FFB and would redirect algorithm selection toward deployment-related criteria. The paper's strengths are its breadth (seven datasets, seven algorithms, three fairness metrics), its use of an external benchmark rather than self-constructed settings, and its secondary empirical contributions: runtime measurements and multiplicity/ambiguity tables that offer independent grounds for algorithm choice. The paper is also transparent in Section 5 about its scope (in-processing methods, training-time hyperparameters only). However, the headline comparability claim currently rests on visual inspection of Pareto fronts with no quantitative definition of 'comparable', no accuracy floor, no statistical support, and a relatively small and under-specified hyperparameter grid. These gaps are load-bearing for the main conclusion.

major comments (4)
  1. [Section 4.1, Figures 4, 15, 16] The central claim that 'most mitigation techniques can achieve comparable performance' is not attached to any quantitative definition of 'comparable.' The evidence is visual overlap of per-algorithm Pareto fronts, but any point on a Pareto front, including a degenerate high-fairness/low-accuracy point that would not be deployed, makes its algorithm appear 'competitive.' The text in Section 4.1 says these are 'the models that would be deployed,' yet without an accuracy floor (e.g., within a stated margin of ERM or of the best algorithm's accuracy) the displayed fronts do not establish that. Please define a comparability criterion, such as maximum allowed accuracy loss at a target fairness level, Pareto dominance gap, or hypervolume under a utility floor, and apply it to the reported results.
  2. [Appendix A, Tables 3 and 4, Section 4.1] The hyperparameter search underlying the Pareto-front claim consists of seven hyperparameter settings and three control values, with the number of random seeds left unspecified. No coverage analysis, convergence check, or comparison across seeds is reported. With as few as 21 runs per algorithm per dataset, the Pareto front is an envelope of a small grid rather than a characterization of what the algorithms can achieve under hyperparameter optimization; a different or denser grid could change which algorithms appear competitive. Please report the number of seeds, use identical seeds across algorithms and datasets, and provide a sensitivity analysis of the Pareto fronts to the grid, or use a larger random search.
  3. [Figures 2-4, Tables 1-2] No error bars, confidence intervals, or statistical tests are provided, so the claimed 'significant variance' and 'comparable performance' are not distinguished from run-to-run noise. For instance, Table 2 reports large multiplicity differences (e.g., AdvDebias 0.87 vs. ERM 0.55 on German) without any measure of variability across seeds, and Figure 3 presents dataset-level tradeoffs as point clouds with no summary statistics. Please report distributions over seeds and control values (mean, standard deviation, or min/max) and apply a statistical test or effect-size measure to the pairwise algorithm comparisons that underlie the headline conclusion.
  4. [Section 3.3] The dataset-specific explanations, such as HSIC's weakness on small-batch datasets and LAFTR's sensitivity to one-hot-encoded features, are post hoc and not tested. These explanations are used to argue that FFB-style cross-dataset aggregation obscures trends, so they need at least a controlled comparison, e.g., varying batch size while holding other hyperparameters fixed, or varying the feature encoding while holding the algorithm fixed. Without such tests, these claims remain speculative and should be labeled as hypotheses rather than findings.
minor comments (5)
  1. [Table 3] The table lists 'DiffEOdd' twice and omits DiffEOpp; the second row should presumably be DiffEOpp.
  2. [Appendix A.3] The number of random seeds used for each setting is never stated; please include this information in the table or text.
  3. [Figure 4 and Appendix D] The captions use lowercase 'pareto front'; please use 'Pareto front' consistently and define the term at first use in Section 4.1.
  4. [References] The 'Baharlouei et al.' reference lacks a year and publication venue; please complete it.
  5. [Reproducibility] The paper borrows the FFB codebase but does not release its own modified code or configuration files; a reproducibility statement listing exact changes (e.g., which hyperparameters were added, how seeds were handled) would strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claim is an empirical, externally benchmarked finding rather than a derivation from its own assumptions.

full rationale

The paper's main claim, that most bias mitigation techniques achieve comparable fairness-utility tradeoffs when hyperparameter optimization is allowed, is an empirical claim supported by experiments on the external FFB benchmark and seven public datasets. The Pareto-front analysis in Section 4.1 is a post-hoc selection of best-performing runs per algorithm, not a quantity that is defined in terms of the conclusion: the conclusion ('several algorithms can provide competitive tradeoffs') is inferred from the plotted fronts, not encoded in their construction. The variance results in Section 3 similarly come from training runs under varied hyperparameters and are not derived from the thesis. Self-citations (Ganesh et al., 2023; Gohar et al., 2024; Ganesh, 2024) are used to motivate the importance of variance and instability, but none of these citations supplies a theorem or result that logically forces the paper's comparability conclusion; the conclusion rests on the paper's own empirical plots. Criticisms about the grid being small, 'comparable' lacking a quantitative threshold, and Pareto fronts being assessed visually are legitimate concerns about evidential strength and falsifiability, but they do not make the claim true by definition or reduce any prediction to a fitted input. No circular step can be exhibited with a quote showing that an output equals an input by construction, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted to derive a result. The central claim is an empirical observation, but it relies on assumptions about the benchmark implementation, the adequacy of the hyperparameter grid, and the choice of metrics. No new theoretical entities are introduced.

assumptions (4)
  • domain assumption The FFB benchmark's dataset splits, preprocessing, and algorithm implementations are correct and representative.
    All experiments build on Han et al.'s open-source FFB code; any implementation or preprocessing error propagates into the comparisons. Invoked in Section 3.1.
  • ad hoc to paper The grid of seven hyperparameter settings and three control values is an adequate proxy for hyperparameter optimization.
    The central comparability claim depends on this hand-chosen grid. Table 4 lists settings selected by the authors; no coverage or convergence analysis is given, so results may not reflect true best achievable performance.
  • domain assumption Accuracy and group fairness metrics (DP, EO, EOdds) at the final epoch are the relevant dimensions for comparing mitigation algorithms.
    The conclusion of comparable performance is about these metrics only; other criteria such as calibration or individual fairness could separate the algorithms. Stated in Section 3.1.
  • ad hoc to paper Pareto-front filtering is a valid method for comparing algorithms after hyperparameter optimization.
    Section 4.1 selects the best performing models per algorithm; this assumes best-case performance is the right comparison and ignores the cost or likelihood of finding those settings.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in ML." pith.science (2026). https://pith.science/paper/ZAZFIEUY

@misc{pith2026241111101,
  author       = {Pith},
  title        = {Pith review of: Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in ML},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZAZFIEUY}},
  note         = {Machine review of arXiv:2411.11101}
}
read the original abstract

With fairness concerns gaining significant attention in Machine Learning (ML), several bias mitigation techniques have been proposed, often compared against each other to find the best method. These benchmarking efforts tend to use a common setup for evaluation under the assumption that providing a uniform environment ensures a fair comparison. However, bias mitigation techniques are sensitive to hyperparameter choices, random seeds, feature selection, etc., meaning that comparison on just one setting can unfairly favour certain algorithms. In this work, we show significant variance in fairness achieved by several algorithms and the influence of the learning pipeline on fairness scores. We highlight that most bias mitigation techniques can achieve comparable performance, given the freedom to perform hyperparameter optimization, suggesting that the choice of the evaluation parameters-rather than the mitigation technique itself-can sometimes create the perceived superiority of one method over another. We hope our work encourages future research on how various choices in the lifecycle of developing an algorithm impact fairness, and trends that guide the selection of appropriate algorithms.

Figures

Figures reproduced from arXiv: 2411.11101 by the authors.

Figure 1
Figure 1. We postulate that after accounting for the variance in fairness due to the hyperparameter [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Fairness-utility (demographic parity-accuracy) tradeoff across various settings for the Adult [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results for other fairness metrics are present in Appendix C. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (14 more)
Figure 3
Figure 3. Figure 3: Fairness-utility (demographic parity-accuracy) tradeoff across various datasets, under their [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 4
Figure 4. Figure 4: Pareto front of the fairness-utility (demographic parity-accuracy) tradeoff across various [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Fairness-utility (equalized odds-accuracy) tradeoff across various settings for the Adult [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Fairness-utility (equal opportunity-accuracy) tradeoff across various settings for the Adult [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Fairness-utility (demographic parity-accuracy) tradeoff across various settings for the Bank [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Fairness-utility (demographic parity-accuracy) tradeoff across various settings for the [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Fairness-utility (demographic parity-accuracy) tradeoff across various settings for the [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Fairness-utility (demographic parity-accuracy) tradeoff across various settings for the [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Fairness-utility (demographic parity-accuracy) tradeoff across various settings for the [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Fairness-utility (demographic parity-accuracy) tradeoff across various settings for the [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Fairness-utility (equalized odds-accuracy) tradeoff across various datasets, under their [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Fairness-utility (equal opportunity-accuracy) tradeoff across various datasets, under their [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: Pareto front of the fairness-utility (equalized odds-accuracy) tradeoff across various [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: Pareto front of the fairness-utility (equal opportunity-accuracy) tradeoff across various [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 11 canonical work pages

  1. [4]

    Beutel, J

    A. Beutel, J. Chen, Z. Zhao, and E. H. Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075,

    arXiv

  2. [10]

    Gohar and L

    U. Gohar and L. Cheng. A survey on intersectional fairness in machine learning: Notions, mitigation, and challenges. arXiv preprint arXiv:2305.06969,

    arXiv

  3. [11]

    Gohar, S

    9 U. Gohar, S. Biswas, and H. Rajan. Towards understanding fairness and its composition in ensemble machine learning. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1533–1545. IEEE,

    work page 2023

  4. [12]

    Gohar, Z

    U. Gohar, Z. Tang, J. Wang, K. Zhang, P. L. Spirtes, Y . Liu, and L. Cheng. Long-term fairness inquiries and pursuits in machine learning: A survey of notions, methods, and challenges. arXiv preprint arXiv:2406.06736,

    arXiv

  5. [14]

    URL https://www.justice.gov/crt/fcs/T6Manual7. T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2012, Bristol, UK, September 24-28,

    work page 2012

  6. [16]

    Perrone, M

    V . Perrone, M. Donini, M. B. Zafar, R. Schmucker, K. Kenthapadi, and C. Archambeau. Fair bayesian optimization. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 854–863,

    work page 2021

  7. [18]

    B. H. Zhang, B. Lemoine, and M. Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340,

    work page 2018

  8. [1994]

    DOI: https://doi.org/10.24432/C5NC77. U. S. D. O. Justice. Title vi legal manual, section vii: Proving discrimination – disparate impact., Oct

    doi:10.24432/c5nc77

Show all 19 references
  1. [1996]

    DOI: https://doi.org/10.24432/C5XW20. R. K. Bellamy, K. Dey, M. Hind, S. C. Hoffman, S. Houde, K. Kannan, P. Lohia, J. Martino, S. Mehta, A. Mojsilovi´c, et al. Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and De...

    doi

  2. [2000]

    DOI: https://doi.org/10.24432/C5N30T. T. Adel, I. Valera, Z. Ghahramani, and A. Weller. One-network adversarial fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 2412–2420,

    doi

  3. [2014]

    DOI: https://doi.org/10.24432/C5K306. H. Noh, T. You, J. Mun, and B. Han. Regularizing deep neural networks by noise: Its interpretation and optimization. Advances in neural information processing systems, 30,

    doi

  4. [2017]

    S. Bird, M. Dudík, R. Edgar, B. Horn, R. Lutz, V . Milan, M. Sameki, H. Wallach, and K. Walker. Fairlearn: A toolkit for assessing and improving fairness in ai. Microsoft, Tech. Rep. MSR-TR- 2020-32,

    2020

  5. [2018]

    [2023], we redirect the reader to their work and the FFB benchmark code 3 for details on the underlying setup

    11 A Additional Details on Experiment Setup As we directly borrow the experiment setup from Han et al. [2023], we redirect the reader to their work and the FFB benchmark code 3 for details on the underlying setup. In this section, we briefly mention the datasets and algorithms...

    2023

  6. [2019]

    Baharlouei, M

    S. Baharlouei, M. Nouiehed, A. Beirami, and M. Razaviyayn. Rényi fair inference. In International Conference on Learning Representations. 8 I. Baldini, D. Wei, K. N. Ramamurthy, M. Yurochkin, and M. Singh. Your fairness may vary: Pretrained language model fairness in toxic tex...

    arXiv

  7. [2020]

    Black and M

    E. Black and M. Fredrikson. Leave-one-out unfairness. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 285–295,

    2021

  8. [2021]

    Black, M

    E. Black, M. Raghavan, and S. Barocas. Model multiplicity: Opportunities, concerns, and solutions. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 850–863,

    2022

  9. [2022]

    Simson, F

    10 J. Simson, F. Pfisterer, and C. Kern. One model many scores: Using multiverse analysis to prevent fairness hacking and evaluate the influence of model design decisions. InThe 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1305–1320,

    2024

  10. [2023]

    Black, T

    E. Black, T. Gillis, and Z. Y . Hall. D-hacking. InThe 2024 ACM Conference on Fairness, Account- ability, and Transparency, pages 602–615,

    2024

  11. [2024]

    Ganesh, H

    P. Ganesh, H. Chang, M. Strobel, and R. Shokri. On the impact of machine learning randomness on group fairness. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1789–1800,

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.