Pith. sign in

REVIEW 4 major objections 5 minor 120 references

Exploring the Landscape of Fairness Interventions in Software Engineering

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey maps fairness interventions in software engineering into a three-part taxonomy and identifies the field's open problems.

desk verdict A useful but uneven orientation survey of fairness interventions; the repository-maintenance count needs supporting detail, but the literature summaries are mostly faithful and the misattribution flag in the reader's report is wrong. read the letter →

arxiv 2507.18726 v1 pith:BGVLFVNW submitted 2025-07-24 cs.SE

Sadia Afrin Mim This is my paper · ORCID
classification cs.SE
keywords fairnessAIethicsbiasmitigationtoolkitsopensourcesoftwareengineeringliteraturesurveyhuman-centricevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is a survey of fairness interventions in software engineering that tries to establish that this research area can be organized into three coherent fronts: ethical concerns in general software development, fairness of AI-assisted software systems, and human-centric studies of fairness tools. It argues that such a map is needed because current interventions are fragmented across metrics, mitigation algorithms, toolkits, and qualitative studies, and because practitioners lack reliable guidance about which tools to adopt. The survey also claims that the field's next problems are definitional, adoption, and sustainability problems: fairness lacks a standardized definition, most available toolkits are hard to learn and poorly integrated into workflows, and only a minority of open-source fairness repositories show recent activity. A sympathetic reader would care because these claims, if accurate, tell researchers and practitioners where investment in fairness tooling will pay off.

What carries the argument

The central mechanism is the three-part taxonomy together with the repository-curation method. The taxonomy categorizes the surveyed literature into ethical software development, fairness in AI-driven software engineering, and human-centric fairness studies; it is what turns a list of papers into a landscape with identifiable gaps. The repository method builds a corpus of 62 fairness repositories by collecting ethics and fairness keywords from repository descriptions, starting from an earlier list of 14 toolkits, and labels each repository as active or inactive based on last activity in issues, commits, or pull requests, following a prior maintenance-classification approach. This machinery supports the survey's quantitative maintenance claim and its open-problem list.

What would settle it

An independent replication would settle the central claim: run a systematic search with explicit inclusion and exclusion criteria across a broader set of venues and check whether all identified fairness-intervention studies fit the three-part taxonomy; any substantial body of work that does not fit would show the taxonomy is incomplete. A separate check would re-measure the same 62 repositories with a different activity proxy, such as release frequency or issue-response time; if most of them count as active under that proxy, the 'only 20 active' maintenance claim would fail.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes a taxonomy of fairness-intervention research: (i) ethical and fairness concerns for generic software development, including diversity and inclusion practices, ignorance and prejudice in software fairness, and test-generation interventions; (ii) fairness concerns for AI-assisted software, including fairness-practice surveys, comparisons of existing toolkits, and interventions for machine learning fairness; and (iii) human-centric evaluations that interview practitioners about how fairness tools are actually used. It contributes a curated list of 62 open-source fairness repositories, gathered by collecting ethics-related keywords from repository descriptions and expanding an earlier taxonomy of 14 toolkits, and it reports that only 20 of those repositories are actively maintained as measured by recent issue resolution, commits, or pull requests. The survey then reads the literature through this taxonomy and derives open problems: no consistent fairness definition, a disconnect between theoretical ethics and practical implementation, fairness tools biased toward traditional binary classification, steep learning curves and poor workflow integration, weak open-source maintenance, and overly narrow human-centric study designs.

Load-bearing premise

The survey assumes that a literature search using broad fairness keywords, mostly restricted to 2018–2025 and without explicit inclusion or exclusion criteria, yields a representative sample of fairness-intervention research; if that sample is skewed, the taxonomy and its open problems will be incomplete or misleading.

Editorial extensions

If this is right

  • A researcher positioning a new fairness study can locate it in one of the three taxonomy clusters and immediately see the neighboring work it must engage with.
  • A practitioner choosing a fairness toolkit should treat maintenance as a first-class selection criterion, because the survey counts only 20 of 62 open-source fairness repositories as actively maintained.
  • Fairness support for large language models and multiclass problems is an open frontier, since most surveyed interventions center on traditional binary classification.
  • Human-centric findings imply that better documentation, onboarding, and workflow integration may improve fairness practice more than additional fairness metrics do.
  • Closing the theory-practice gap requires turning ethical principles into actionable guidelines embedded in the software lifecycle, not just academic discussion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the maintenance statistic generalizes, the sustainability of fairness infrastructure deserves as much research attention as bias-mitigation algorithms; future work could track whether the 20 active repositories persist or churn.
  • Because the survey's repository sample comes from public open-source signals, the actual landscape of fairness interventions is probably larger and less visible once internal corporate tooling and commercial products are included.
  • A concrete replication: repeat the same keyword-and-activity method at a later date to measure entry and abandonment rates of fairness repositories, which would turn the snapshot into a churn map.
  • The taxonomy's human-centric cluster suggests adoption failures are the binding constraint; interventions tested through user studies on less-known tools could identify fixes the current, mostly industry-tool studies miss.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript is a survey of fairness interventions in software engineering. It first provides background on protected attributes, sources of bias, fairness metrics, bias mitigation algorithms, and open-source fairness tools, and it reports an empirical observation that only 20 of 62 fairness repositories are actively maintained. It then proposes a three-part taxonomy—ethical and fairness concerns for generic software development, fairness concerns for AI-assisted software systems, and human-centric evaluation of fairness—and organizes a review of roughly two dozen papers under these headings in Table I. The paper closes with open research problems across the same three areas. The central claims are that the paper catalogs existing research on Ethical AI tools and provides insights to guide future work.

Significance. The paper addresses a timely and important topic, and if the reported coverage were representative, the taxonomy and open-problem list would be a useful map for researchers and practitioners. The organization by workflow and the inclusion of human-centric studies alongside tool-based interventions are genuine strengths. The paper also is transparent about the subjectivity of its taxonomy ('my understanding'), and it points to a concrete, potentially valuable data point (the maintenance status of fairness repositories). However, the contribution is currently limited by the absence of a reproducible literature-selection protocol, a documented classification procedure, and verifiable repository-activity data. These are all fixable within a revision, but without them the central 'landscape' claim cannot be independently checked.

major comments (4)
  1. [Section I (Introduction), third paragraph] The literature search is described only as using Google Scholar, ACM Digital Library, and IEEE Xplore with keywords such as 'Fairness, AI Fairness, AI Bias, and AI Ethics' and literature 'primarily 2018 to 2025.' No Boolean query, database-specific result counts, inclusion/exclusion criteria, screening protocol, or snowballing step is reported. Because the taxonomy in Section III and the open problems in Section V are derived from this corpus, the representativeness of the survey cannot be assessed. Please add a complete search protocol (exact queries per database, date of search, inclusion/exclusion criteria, screening flow, and the full list of included studies) so that the coverage claim is reproducible.
  2. [Section II-B (Open Source Fairness Tools)] The statement 'Only 20 out of 62 fairness repositories are actively maintained from our list' is not verifiable as written. The manuscript does not provide the list of 62 repositories, the date of data collection, the exact definition of 'last activity' (issues, commits, or pull requests), the threshold separating active from inactive, or the way the protocol of Coelho and Valente [35] was operationalized; the cited GitHub spreadsheets [58], [59] are not described with column definitions or access dates. Since this count is one of the few quantitative findings and is used to motivate the open problem in Section V-B3, it should either be made fully reproducible (with a data appendix or a stable external artifact) or be explicitly labeled as preliminary and non-authoritative. The text cites [35] for the activity-status methodology, but [36] appears to be the more directly relevant reference for maintenance measurement; please clarify which source and which thresholds were used.
  3. [Section IV-A (Literature on Ethical Software Engineering), Themis paragraph] The sentence 'Previously, the authors created a methodology known as causal discrimination that evaluates the connection between program behavior and sensitive properties [49]' misattributes the causal-discrimination methodology. Reference [49] is Galhotra et al., 'Fairness testing: testing software for discrimination,' and Angell et al. [13] build on that methodology; they did not create it. Please rephrase, e.g., 'building on the causal discrimination methodology of Galhotra et al. [49], the authors ...' so that the survey accurately represents the cited work.
  4. [Section III (Area Taxonomy)] The taxonomy is introduced with the sentence 'My understanding of literature study in this are can be classified as follows,' but no coding procedure, codebook, or validation step is described. The reader cannot tell how the three categories were derived, how individual papers were assigned to them, or whether the categories are exhaustive and disjoint. Because the taxonomy is a central contribution, please describe the classification method, the unit of coding, and any inter-rater or stability check; if the taxonomy is intentionally the author's subjective synthesis, say so explicitly and justify why it is still a reliable map of the field.
minor comments (5)
  1. [Section II-A3] 'Fainress through Awareness' should read 'Fairness through Awareness' [30], and 'metrices' should be 'metrics' throughout the paper (e.g., 'fairness metrices' in the same subsection).
  2. [Section IV-A and Section IV-B1] Several citations refer to authors by first name in a way that is inconsistent with the reference list: 'Daniela et al.' for [39] should be 'Damian et al.,' and 'Maria et al.' for [48] should be 'Franse et al.' (or the full author list). Please use standard surname-based citations.
  3. [Section V-B] The heading 'Developing Comprehensive Fairnes Tools' contains a typo ('Fairnes' for 'Fairness'), and the heading 'Maintenance & Open Source Engagement Analysis::' has a doubled colon.
  4. [Section I] The phrase 'As AI moving forward' is ungrammatical; also 'the section II' should be 'Section II' without the article.
  5. [Section II-B] For consistency, use 'in-processing' rather than 'inprocess' (the latter appears in the bullet list of bias mitigation algorithms).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey makes no predictive or derivational claims, and its one self-citation is not load-bearing.

full rationale

This paper is a literature survey and taxonomy, not a derivation or prediction pipeline. There are no equations that define an output in terms of an input, no fitted parameters renamed as predictions, and no claimed uniqueness theorem imported from the authors' prior work. The central contribution, cataloging existing research and organizing it into a taxonomy, is presented transparently as the author's interpretation and synthesis of the reviewed literature. The only self-citation is to the author's earlier taxonomy of 14 fairness toolkits [84], which is used as a starting point for expanding the repository list to 62 and for the active-maintenance count. This is an incremental extension of prior work, not a circular justification of the survey's conclusions. The maintenance statistic is an empirical observation reported with its methodological source [35], not a quantity derived from the thing it purports to describe. Concerns about the representativeness and reproducibility of the literature search, while legitimate methodological risks for a survey, are not circularity because the survey's claims are descriptive rather than derived from a fitted model or a self-referential argument. Accordingly, no circular step meets the standard of being quotable and reducible to its own inputs, and the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper's main contribution is a literature taxonomy, which relies on the representativeness of the selected corpus. The single empirical count relies on an activity proxy and an unlisted repository set. Neither is independently verified in the preprint.

assumptions (2)
  • domain assumption The selected literature from Google Scholar, ACM Digital Library, and IEEE Xplore is representative of fairness intervention research in software engineering from 2018-2025.
    Section I describes the search but gives no inclusion/exclusion criteria or screening process. The survey's taxonomy and open problems depend on this corpus being representative.
  • domain assumption 'Last activity' in issues, commits, or pull requests is a valid proxy for a repository being actively maintained.
    Section II-B says the analysis follows methodology from prior work [35], but no validation or cutoff date is provided. The claim that 20 of 62 repositories are active rests on this proxy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Landscape of Fairness Interventions in Software Engineering." pith.science (2026). https://pith.science/paper/BGVLFVNW

@misc{pith2026250718726,
  author       = {Pith},
  title        = {Pith review of: Exploring the Landscape of Fairness Interventions in Software Engineering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BGVLFVNW}},
  note         = {Machine review of arXiv:2507.18726}
}
read the original abstract

Current developments in AI made it broadly significant for reducing human labor and expenses across several essential domains, including healthcare and finance. However, the application of AI in the actual world poses multiple risks and disadvantages due to potential risk factors in data (e.g., biased dataset). Practitioners developed a number of fairness interventions for addressing these kinds of problems. The paper acts as a survey, summarizing the various studies and approaches that have been developed to address fairness issues

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

120 extracted references · 75 canonical work pages

  1. [49]

    Fairness testing: testing software for discrimination

    Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. Fairness testing: testing software for discrimination. In Proceedings of the 2017 11th Joint meeting on foundations of software engineering , pages 498–510, 2017

    work page 2017

  2. [35]

    Why modern open source projects fail

    Jailton Coelho and Marco Tulio Valente. Why modern open source projects fail. In Proceedings of the 2017 11th Joint meeting on foundations of software engineering , pages 186–196, 2017

    2017

  3. [58]

    Fairness-repository-mining-

    INSPIRED-GMU. Fairness-repository-mining-. Available at GitHub: Applicationsfoundagainstethicalkeywords.xlsx, 2023. https://github. com/INSPIRED-GMU/Fairness-repository-mining-

    work page 2023

  4. [59]

    Fairness-repository-mining-

    INSPIRED-GMU. Fairness-repository-mining-. Available at GitHub: Applicationsfoundagainsttool-specifickey-words.xlsx, 2023. https:// github.com/INSPIRED-GMU/Fairness-repository-mining-

    work page 2023

  5. [36]

    Is this github project maintained? measuring the level of maintenance activity of open-source projects

    Jailton Coelho, Marco Tulio Valente, Luciano Milen, and Luciana L Silva. Is this github project maintained? measuring the level of maintenance activity of open-source projects. Information and Software Technology, 122:106274, 2020

    2020

  6. [13]

    Themis: Automatically testing software for discrimination

    Rico Angell, Brittany Johnson, Yuriy Brun, and Alexandra Meliou. Themis: Automatically testing software for discrimination. In Pro- ceedings of the 2018 26th ACM Joint meeting on european software engineering conference and symposium on the foundations of software engineering, pages 871–875, 2018

    2018

  7. [1]

    https://deon.drivendata.org/

    An ethics checklist data scientists. https://deon.drivendata.org/

  8. [2]

    https://ai.facebook.com/blog/ how-were-using-fairness-flow-to-help-build-ai-that-works-better-for-everyone/

    Fairness Flow. https://ai.facebook.com/blog/ how-were-using-fairness-flow-to-help-build-ai-that-works-better-for-everyone/

Show all 120 references
  1. [3]

    https://github.com/pymetrics/audit-ai./

    pymetric/audit-ai. https://github.com/pymetrics/audit-ai./

  2. [4]

    https://github.com/linkedin/LiFT/, 2021

    2021

  3. [5]

    https://github.com/Tizpaz/Parfait-ML/, 2021

    2021

  4. [6]

    https://github.com/dchen236/FairFace/, 2021

    2021

  5. [7]

    {TensorFlow}: a system for {Large-Scale} machine learning

    Mart ´ın Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. {TensorFlow}: a system for {Large-Scale} machine learning. In 12th USENIX symposium on operating systems design and implem...

    2016

  6. [8]

    Civil rights act of 1964

    Civil Rights Act. Civil rights act of 1964. Title VII, Equal Employment Opportunities, 1964

    1964

  7. [9]

    An empirical study on the survival rate of github projects

    Adem Ait, Javier Luis C ´anovas Izquierdo, and Jordi Cabot. An empirical study on the survival rate of github projects. In Proceedings of the 19th International Conference on Mining Software Repositories , pages 365–375, 2022

    2022

  8. [10]

    A taxonomy and mapping of computer-based critiquing tools

    Norhayati Mohd Ali, John Hosking, and John Grundy. A taxonomy and mapping of computer-based critiquing tools. IEEE Transactions on Software Engineering , 39(11):1494–1520, 2013

    2013

  9. [11]

    Uncovering and mitigating algorithmic bias through learned latent structure

    Alexander Amini, Ava P Soleimany, Wilko Schwarting, Sangeeta N Bhatia, and Daniela Rus. Uncovering and mitigating algorithmic bias through learned latent structure. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society , pages 289–295, 2019

    2019

  10. [12]

    Capturing the relationship between sentence triplets for llm and human-generated texts to enhance sentence embeddings

    Na Min An, Sania Waheed, and James Thorne. Capturing the relationship between sentence triplets for llm and human-generated texts to enhance sentence embeddings. In Findings of the Association for Computational Linguistics: EACL 2024 , pages 624–638, 2024

    2024

  11. [14]

    Fairness tool evaluation submis- sion 0b7e

    Anonymous. Fairness tool evaluation submis- sion 0b7e. https://anonymous.4open.science/r/ Fairness-tool-evaluation-submission-0B7E/, 2024

    2024

  12. [15]

    Data bias, intelligent systems and criminal justice outcomes

    Jacob O Arowosegbe. Data bias, intelligent systems and criminal justice outcomes. International Journal of Law and Information Technology, 31(1):22–45, 2023

    2023

  13. [16]

    Putting ai ethics to work: are the tools fit for purpose? AI and Ethics , 2(3):405–429, 2022

    Jacqui Ayling and Adriane Chapman. Putting ai ethics to work: are the tools fit for purpose? AI and Ethics , 2(3):405–429, 2022

    2022

  14. [17]

    Gpt-4: A review on advancements and opportunities in natural language processing

    Jawid Ahmad Baktash and Mursal Dawodi. Gpt-4: A review on advancements and opportunities in natural language processing. arXiv preprint arXiv:2305.03195, 2023

    2023 arXiv

  15. [18]

    Themis-ml: A fairness-aware machine learning inter- face for end-to-end discrimination discovery and mitigation

    Niels Bantilan. Themis-ml: A fairness-aware machine learning inter- face for end-to-end discrimination discovery and mitigation. Journal of Technology in Human Services , 36(1):15–30, 2018

    2018

  16. [19]

    Who will leave the company?: a large-scale industry study of developer turnover by mining monthly work report

    Lingfeng Bao, Zhenchang Xing, Xin Xia, David Lo, and Shanping Li. Who will leave the company?: a large-scale industry study of developer turnover by mining monthly work report. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), pages 170–181....

    2017

  17. [20]

    Social network- ing meets software development: Perspectives from github, msdn, stack exchange, and topcoder

    Andrew Begel, Jan Bosch, and Margaret-Anne Storey. Social network- ing meets software development: Perspectives from github, msdn, stack exchange, and topcoder. IEEE software, 30(1):52–66, 2013

    2013

  18. [21]

    Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias

    Rachel KE Bellamy, Kuntal Dey, Michael Hind, Samuel C Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovi ´c, et al. Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of...

    2019

  19. [22]

    Marcus Vinicius Bertoncello, Gustavo Pinto, Igor Scaliante Wiese, and Igor Steinmacher. Pull requests or commits? which method should we use to study contributors’ behavior? In 2020 IEEE 27th Interna- tional Conference on Software Analysis, Evolution and Reengineering (SANER),...

    2020

  20. [23]

    Stefano A Bini. Artificial intelligence, machine learning, deep learning, and cognitive computing: what do these terms mean and how will they impact health care? The Journal of arthroplasty , 33(8):2358–2361, 2018

    2018

  21. [24]

    Fairlearn: A toolkit for assessing and improving fairness in ai

    Sarah Bird, Miro Dud ´ık, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. Fairlearn: A toolkit for assessing and improving fairness in ai. Microsoft, Tech. Rep. MSR-TR-2020-32 , 2020

    2020

  22. [25]

    Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness

    Sumon Biswas and Hridesh Rajan. Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness. In Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software ...

    2020

  23. [26]

    What’s in a github star? understanding repository starring practices in a social coding platform

    Hudson Borges and Marco Tulio Valente. What’s in a github star? understanding repository starring practices in a social coding platform. Journal of Systems and Software , 146:112–129, 2018

    2018

  24. [27]

    Software fairness

    Yuriy Brun and Alexandra Meliou. Software fairness. In Proceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering , pages 754–759, 2018

    2018

  25. [28]

    GraphQL in action

    Samer Buna. GraphQL in action . Simon and Schuster, 2021

    2021

  26. [29]

    Reuse and maintenance practices among divergent forks in three software ecosystems

    John Businge, Moses Openja, Sarah Nadi, and Thorsten Berger. Reuse and maintenance practices among divergent forks in three software ecosystems. Empirical Software Engineering , 27(2):54, 2022

    2022

  27. [30]

    A clarification of the nuances in the fairness metrics landscape

    Alessandro Castelnovo, Riccardo Crupi, Greta Greco, Daniele Regoli, Ilaria Giuseppina Penco, and Andrea Claudio Cosentini. A clarification of the nuances in the fairness metrics landscape. Scientific Reports , 12(1):4209, 2022

    2022

  28. [31]

    Fairness in machine learning: A survey

    Simon Caton and Christian Haas. Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053 , 2020

    2010 arXiv

  29. [32]

    Fairness in machine learning: A survey

    Simon Caton and Christian Haas. Fairness in machine learning: A survey. ACM Comput. Surv., 56(7), April 2024

    2024

  30. [33]

    A comprehensive empirical study of bias mitigation methods for machine learning classifiers

    Zhenpeng Chen, Jie M Zhang, Federica Sarro, and Mark Harman. A comprehensive empirical study of bias mitigation methods for machine learning classifiers. ACM transactions on software engineering and methodology, 32(4):1–30, 2023. 8

    2023

  31. [34]

    Fairness improvement with multiple protected attributes: How far are we? In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , pages 1–13, 2024

    Zhenpeng Chen, Jie M Zhang, Federica Sarro, and Mark Harman. Fairness improvement with multiple protected attributes: How far are we? In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , pages 1–13, 2024

    2024

  32. [37]

    Social coding in github: transparency and collaboration in an open software repository

    Laura Dabbish, Colleen Stuart, Jason Tsay, and Jim Herbsleb. Social coding in github: transparency and collaboration in an open software repository. In Proceedings of the ACM 2012 conference on computer supported cooperative work , pages 1277–1286, 2012

    2012

  33. [38]

    Sampling projects in github for msr studies

    Ozren Dabic, Emad Aghajani, and Gabriele Bavota. Sampling projects in github for msr studies. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) , pages 560–564. IEEE, 2021

    2021

  34. [39]

    Equity, Diversity, and Inclusion in Software Engineering: Best Practices and Insights

    Daniela Damian, Kelly Blincoe, Denae Ford, Alexander Serebrenik, and Zainab Masood. Equity, Diversity, and Inclusion in Software Engineering: Best Practices and Insights . Springer Nature, 2024

    2024

  35. [40]

    Identifying and characterizing unmaintained projects in github

    Jailton Junior de Sousa Coelho et al. Identifying and characterizing unmaintained projects in github. 2019

    2019

  36. [41]

    A taxonomy and catalog of runtime software-fault monitoring tools

    Nelly Delgado, Ann Q Gates, and Steve Roach. A taxonomy and catalog of runtime software-fault monitoring tools. IEEE Transactions on software Engineering , 30(12):859–872, 2004

    2004

  37. [42]

    Exploring how machine learning practitioners (try to) use fairness toolkits

    Wesley Hanwen Deng, Manish Nagireddy, Michelle Seng Ah Lee, Jatinder Singh, Zhiwei Steven Wu, Kenneth Holstein, and Haiyi Zhu. Exploring how machine learning practitioners (try to) use fairness toolkits. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, an...

    2022

  38. [43]

    The eu ai act: a summary of its significance and scope

    Lilian Edwards. The eu ai act: a summary of its significance and scope. Artificial Intelligence (the EU AI Act) , 1:25, 2021

    2021

  39. [44]

    Predicting long-time contributors for github projects using machine learning

    Vijaya Kumar Eluri, Thomas A Mazzuchi, and Shahram Sarkani. Predicting long-time contributors for github projects using machine learning. Information and Software Technology , 138:106616, 2021

    2021

  40. [45]

    Certifying and removing dis- parate impact

    Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheideg- ger, and Suresh Venkatasubramanian. Certifying and removing dis- parate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , pages 259–268, 2015

    2015

  41. [46]

    Gender bias in translation using google translate: Problems and solution

    Tira Nur Fitria. Gender bias in translation using google translate: Problems and solution. Language Circle: Journal of Language and Literature, 15(2), 2021

    2021

  42. [47]

    2020 survey of artificial general intelligence projects for ethics, risk, and policy

    McKenna Fitzgerald, Aaron Boddy, and Seth D Baum. 2020 survey of artificial general intelligence projects for ethics, risk, and policy. Global Catastrophic Risk Institute Technical Report , pages 20–1, 2020

    2020

  43. [48]

    Practical and open source best practices for ethical machine learning

    Jeroen Franse, Violeta Misheva, and Daniel S Vale. Practical and open source best practices for ethical machine learning. In Towards Trustworthy Artificial Intelligent Systems, pages 77–84. Springer, 2022

    2022

  44. [50]

    What is fair? defining fairness in machine learning for health

    Jianhui Gao, Benson Chou, Zachary R McCaw, Hilary Thurston, Paul Varghese, Chuan Hong, and Jessica Gronsbell. What is fair? defining fairness in machine learning for health. arXiv preprint arXiv:2406.09307, 2024

    2024 arXiv

  45. [51]

    Fairness metrics: A comparative analysis

    Pratyush Garg, John Villasenor, and Virginia Foggo. Fairness metrics: A comparative analysis. In 2020 IEEE international conference on big data (Big Data) , pages 3662–3666. IEEE, 2020

    2020

  46. [52]

    Justicia: A stochastic sat approach to formally verify fairness

    Bishwamittra Ghosh, Debabrota Basu, and Kuldeep S Meel. Justicia: A stochastic sat approach to formally verify fairness. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 7554–7563, 2021

    2021

  47. [53]

    The quest for open source projects that use uml

    Mining GitHub. The quest for open source projects that use uml

  48. [54]

    Investigating labeler bias in face annotation for machine learning

    Luke Haliburton, Sinksar Ghebremedhin, Robin Welsch, Albrecht Schmidt, and Sven Mayer. Investigating labeler bias in face annotation for machine learning. In HHAI 2024: Hybrid Human AI Systems for the Social Good , pages 145–161. IOS Press, 2024

    2024

  49. [55]

    Olf-ml: An offensive language framework for detection, categorization, and of- fense target identification using text processing and machine learning algorithms

    MD Nahid Hasan, Kazi Shadman Sakib, Taghrid Tahani Preeti, Jeza Allohibi, Abdulmajeed Atiah Alharbi, and Jia Uddin. Olf-ml: An offensive language framework for detection, categorization, and of- fense target identification using text processing and machine learning algorithms....

    2024

  50. [56]

    Same file, different changes: the potential of meta-maintenance on github

    Hideaki Hata, Raula Gaikovina Kula, Takashi Ishio, and Christoph Treude. Same file, different changes: the potential of meta-maintenance on github. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) , pages 773–784. IEEE, 2021

    2021

  51. [57]

    Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods

    Max Hort, Jie M Zhang, Federica Sarro, and Mark Harman. Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software En...

    2021

  52. [60]

    Assurance of machine learning/tinyml in safety-critical domains

    Zain Iqbal. Assurance of machine learning/tinyml in safety-critical domains. In 2022 IEEE Symposium on Visual Languages and Human- Centric Computing (VL/HCC) , pages 1–2. IEEE, 2022

    2022

  53. [61]

    Github projects

    Oskar Jarczyk, Bła ˙zej Gruszka, Szymon Jaroszewicz, Leszek Bukowski, and Adam Wierzbicki. Github projects. quality analysis of open-source software. In Social Informatics: 6th International Conference, SocInfo 2014, Barcelona, Spain, November 11-13, 2014. Proceedings 6, pages...

    2014

  54. [62]

    Availability and usage of platform- specific apis: A first empirical study

    Ricardo Job and Andre Hora. Availability and usage of platform- specific apis: A first empirical study. In 2024 IEEE/ACM 21st In- ternational Conference on Mining Software Repositories (MSR) , pages 27–31. IEEE, 2024

    2024

  55. [63]

    Fairkit, fairkit, on the wall, who’s the fairest of them all? supporting data scientists in training fair models

    Brittany Johnson, Jesse Bartola, Rico Angell, Katherine Keith, Sam Witty, Stephen J Giguere, and Yuriy Brun. Fairkit, fairkit, on the wall, who’s the fairest of them all? supporting data scientists in training fair models. arXiv preprint arXiv:2012.09951 , 2020

    2012 arXiv

  56. [64]

    Make your tools sparkle with trust: The picse framework for trust in software tools

    Brittany Johnson, Christian Bird, Denae Ford, Nicole Forsgren, and Thomas Zimmermann. Make your tools sparkle with trust: The picse framework for trust in software tools. In 2023 IEEE/ACM 45th Inter- national Conference on Software Engineering: Software Engineering in Practice...

    2023

  57. [65]

    Fairkit-learn: a fairness evaluation and comparison toolkit

    Brittany Johnson and Yuriy Brun. Fairkit-learn: a fairness evaluation and comparison toolkit. In Proceedings of the ACM/IEEE 44th Interna- tional Conference on Software Engineering: Companion Proceedings , pages 70–74, 2022

    2022

  58. [66]

    Towards ethical data-driven software: filling the gaps in ethics research & practice

    Brittany Johnson and Justin Smith. Towards ethical data-driven software: filling the gaps in ethics research & practice. In 2021 IEEE/ACM 2nd International Workshop on Ethics in Software Engi- neering Research and Practice (SEthics) , pages 18–25. IEEE, 2021

    2021

  59. [67]

    Decision theory for discrimination-aware classification

    Faisal Kamiran, Asim Karim, and Xiangliang Zhang. Decision theory for discrimination-aware classification. In 2012 IEEE 12th interna- tional conference on data mining , pages 924–929. IEEE, 2012

    2012

  60. [68]

    Fairness-aware classifier with prejudice remover regularizer

    Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Ma- chine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2012, Bristol, UK, September 24-28, 2012. Proceedings, Part...

    2012

  61. [69]

    An overview of ethical issues in using ai systems in hiring with a case study of amazon’s ai based hiring tool

    Akhil Alfons Kodiyan. An overview of ethical issues in using ai systems in hiring with a case study of amazon’s ai based hiring tool. Researchgate Preprint, pages 1–19, 2019

    2019

  62. [70]

    A survey on datasets for fairness-aware machine learning

    Tai Le Quy, Arjun Roy, Vasileios Iosifidis, Wenbin Zhang, and Eirini Ntoutsi. A survey on datasets for fairness-aware machine learning. Wi- ley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , 12(3):e1452, 2022

    2022

  63. [71]

    The landscape and gaps in open source fairness toolkits

    Michelle Seng Ah Lee and Jat Singh. The landscape and gaps in open source fairness toolkits. In Proceedings of the 2021 CHI conference on human factors in computing systems , pages 1–13, 2021

    2021

  64. [72]

    The impact of gdpr on global technology development, 2019

    He Li, Lu Yu, and Wu He. The impact of gdpr on global technology development, 2019

    2019

  65. [73]

    David J. Lilja. Measuring computer performance : A practitioner’s guide. 2000

    2000

  66. [74]

    The possessive investment in whiteness: How white people profit from identity politics

    George Lipsitz. The possessive investment in whiteness: How white people profit from identity politics . Temple University Press, 2006

    2006

  67. [75]

    Bias mitigation post-processing for individual and group fairness

    Pranay K Lohia, Karthikeyan Natesan Ramamurthy, Manish Bhide, Diptikalyan Saha, Kush R Varshney, and Ruchir Puri. Bias mitigation post-processing for individual and group fairness. In Icassp 2019- 9 2019 ieee international conference on acoustics, speech and signal processing ...

    2019

  68. [76]

    Assessing the fairness of ai systems: Ai practitioners’ processes, challenges, and needs for support

    Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wort- man Vaughan, and Hanna Wallach. Assessing the fairness of ai systems: Ai practitioners’ processes, challenges, and needs for support. Proceed- ings of the ACM on Human-Computer Interaction , 6(CSCW1):1–26, 2022

    2022

  69. [77]

    Survey on causal-based machine learning fairness notions

    Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. Survey on causal-based machine learning fairness notions. arXiv preprint arXiv:2010.09553, 2020

    2010 arXiv

  70. [78]

    On the applicability of machine learning fairness notions

    Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. On the applicability of machine learning fairness notions. ACM SIGKDD Explorations Newsletter, 23(1):14–23, 2021

    2021

  71. [79]

    A tax- onomy of tools and approaches for fairification

    Dario Mangione, Leonardo Candela, and Donatella Castelli. A tax- onomy of tools and approaches for fairification. In CEUR Workshop Proceedings, Padova, Italy, pages 24–25, 2022

    2022

  72. [80]

    Ethical issues in focus by the autonomous vehicles industry

    Andreia Martinho, Nils Herber, Maarten Kroesen, and Caspar Chorus. Ethical issues in focus by the autonomous vehicles industry. Transport reviews, 41(5):556–577, 2021

    2021

  73. [81]

    Mining co-change information to understand when build changes are necessary

    Shane McIntosh, Bram Adams, Meiyappan Nagappan, and Ahmed E Hassan. Mining co-change information to understand when build changes are necessary. In 2014 IEEE International Conference on Software Maintenance and Evolution , pages 241–250. IEEE, 2014

    2014

  74. [82]

    Statistical methods for reliability data

    William Q Meeker, Luis A Escobar, and Francis G Pascual. Statistical methods for reliability data . John Wiley & Sons, 2022

    2022

  75. [83]

    A survey on bias and fairness in machine learning

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) , 54(6):1–35, 2021

    2021

  76. [84]

    A taxonomy of machine learning fairness tool specifications, features and workflows

    Sadia Afrin Mim, Justin Smith, and Brittany Johnson. A taxonomy of machine learning fairness tool specifications, features and workflows. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pages 222–225, 2023

    2023

  77. [85]

    Peer interaction effectively, yet infrequently, enables programmers to discover new tools

    Emerson Murphy-Hill and Gail C Murphy. Peer interaction effectively, yet infrequently, enables programmers to discover new tools. In Proceedings of the ACM 2011 conference on Computer supported cooperative work, pages 405–414, 2011

    2011

  78. [86]

    An automated approach to assess the similarity of github repositories

    Phuong T Nguyen, Juri Di Rocco, Riccardo Rubei, and Davide Di Ruscio. An automated approach to assess the similarity of github repositories. Software Quality Journal , 28:595–631, 2020

    2020

  79. [87]

    From literature to practice: Exploring fairness testing tools for the software industry adoption

    Thanh Nguyen, Maria Teresa Baldassarre, Luiz Fernando de Lima, and Ronnie de Souza Santos. From literature to practice: Exploring fairness testing tools for the software industry adoption. In Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engine...

    2024

  80. [88]

    Assessing and mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ecg analysis

    Peter A Noseworthy, Zachi I Attia, LaPrincess C Brewer, Sharonne N Hayes, Xiaoxi Yao, Suraj Kapa, Paul A Friedman, and Francisco Lopez-Jimenez. Assessing and mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ecg ...

    2020

  81. [89]

    Bias and unfairness in machine learning models: a systematic review on datasets, tools, fairness metrics, and identification and mitigation methods

    Tiago P Pagano, Rafael B Loureiro, Fernanda VN Lisboa, Rodrigo M Peixoto, Guilherme AS Guimar˜aes, Gustavo OR Cruz, Maira M Araujo, Lucas L Santos, Marco AS Cruz, Ewerton LS Oliveira, et al. Bias and unfairness in machine learning models: a systematic review on datasets, tools...

    2023

  82. [90]

    Scikit-learn: Machine learning in python

    Fabian Pedregosa, Ga ¨el Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Pret- tenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research , 12:2825–2830, 2011

    2011

  83. [91]

    A review on fairness in machine learning

    Dana Pessach and Erez Shmueli. A review on fairness in machine learning. ACM Computing Surveys (CSUR) , 55(3):1–44, 2022

    2022

  84. [92]

    U. Peters. Algorithmic political bias in artificial intelligence systems. Philosophy& Technology, 35(2):25, March 30 2022

    2022

  85. [93]

    Towards fairness in practice: A practitioner-oriented rubric for evaluating fair ml toolkits

    Brianna Richardson, Jean Garcia-Gathright, Samuel F Way, Jennifer Thom, and Henriette Cramer. Towards fairness in practice: A practitioner-oriented rubric for evaluating fair ml toolkits. In Proceed- ings of the 2021 CHI Conference on Human Factors in Computing Systems, pages ...

    2021

  86. [94]

    A framework for fairness: A systematic review of existing fair ai solutions

    Brianna Richardson and Juan E Gilbert. A framework for fairness: A systematic review of existing fair ai solutions. arXiv preprint arXiv:2112.05700, 2021

    2021 arXiv

  87. [95]

    Estimating development effort in free/open source software projects by mining software repos- itories: a case study of openstack

    Gregorio Robles, Jes ´us M Gonz ´alez-Barahona, Carlos Cervig ´on, An- drea Capiluppi, and Daniel Izquierdo-Cort´azar. Estimating development effort in free/open source software projects by mining software repos- itories: a case study of openstack. In Proceedings of the 11th W...

    2014

  88. [96]

    {SourceFinder}: Finding malware {Source-Code} from publicly available repositories in {GitHub}

    Md Omar Faruk Rokon, Risul Islam, Ahmad Darki, Evangelos E Papalexakis, and Michalis Faloutsos. {SourceFinder}: Finding malware {Source-Code} from publicly available repositories in {GitHub}. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID...

    2020

  89. [97]

    Aequitas: A bias and fairness audit toolkit

    Pedro Saleiro, Benedict Kuester, Loren Hinkson, Jesse London, Abby Stevens, Ari Anisfeld, Kit T Rodolfa, and Rayid Ghani. Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577, 2018

    2018 arXiv

  90. [98]

    Towards mining norms in open source software repositories

    Bastin Tony Roy Savarimuthu and Hoa Khanh Dam. Towards mining norms in open source software repositories. In Agents and Data Mining Interaction: 9th International Workshop, ADMI 2013, Saint Paul, MN, USA, May 6-7, 2013, Revised Selected Papers 9, pages 26–39. Springer, 2014

    2013

  91. [99]

    Discriminatory effect and the fair housing act

    Robert G Schwemm. Discriminatory effect and the fair housing act. Notre Dame Law., 54:199, 1978

    1978

  92. [100]

    Towards efficient software engineering in the era of ai and ml: Best practices and challenges

    Varun Shah. Towards efficient software engineering in the era of ai and ml: Best practices and challenges. International Journal of Computer Science and Technology, 3(3):63–78, 2019

    2019

  93. [101]

    Taxonomy of automated software testing tools

    Kamran Shaukat, Usman Shaukat, Faran Feroz, Shahraiz Kayani, and Ali Akbar. Taxonomy of automated software testing tools. Interna- tional Journal of Computer Science and Innovation , 1:7–18, 2015

    2015

  94. [102]

    Fat forensics: a python toolbox for algorithmic fairness, accountability and trans- parency

    Kacper Sokol, Raul Santos-Rodriguez, and Peter Flach. Fat forensics: a python toolbox for algorithmic fairness, accountability and trans- parency. arXiv preprint arXiv:1909.05167 , 2019

    1909 arXiv

  95. [103]

    Fat forensics: A python toolbox for algorithmic fairness, accountability and trans- parency

    Kacper Sokol, Raul Santos-Rodriguez, and Peter Flach. Fat forensics: A python toolbox for algorithmic fairness, accountability and trans- parency. Software Impacts, 14:100406, 2022

    2022

  96. [104]

    The language interpretability tool: Extensible, interactive visualizations and analysis for nlp models

    Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, et al. The language interpretability tool: Extensible, interactive visualizations and analysis for nlp models. arXiv prepri...

    2008 arXiv

  97. [105]

    Fairness-aware configuration of machine learning libraries

    Saeid Tizpaz-Niari, Ashish Kumar, Gang Tan, and Ashutosh Trivedi. Fairness-aware configuration of machine learning libraries. In Proceed- ings of the 44th International Conference on Software Engineering , pages 909–920, 2022

    2022

  98. [106]

    Sveva fair: A frame- work for evaluating fairness in speaker verification

    Wiebke Toussaint and Aaron Yi Ding. Sveva fair: A frame- work for evaluating fairness in speaker verification. arXiv preprint arXiv:2107.12049, 2021

    2021 arXiv

  99. [107]

    An analysis of bias in facial image processing: A review of datasets

    Amarachi M Udefi, Segun Aina, Aderonke R Lawal, and Adeniran I Oluwarantie. An analysis of bias in facial image processing: A review of datasets. International Journal of Advanced Computer Science and Applications, 14(5), 2023

    2023

  100. [108]

    Bias in ai amplifies our own biases, December 2024

    University College London. Bias in ai amplifies our own biases, December 2024

    2024

  101. [109]

    In-processing modeling techniques for machine learning fairness: A survey

    Mingyang Wan, Daochen Zha, Ninghao Liu, and Na Zou. In-processing modeling techniques for machine learning fairness: A survey. ACM Transactions on Knowledge Discovery from Data , 17(3):1–27, 2023

    2023

  102. [110]

    Mitigating bias in face recognition using skewness-aware reinforcement learning

    Mei Wang and Weihong Deng. Mitigating bias in face recognition using skewness-aware reinforcement learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9322–9331, 2020

    2020

  103. [111]

    The what-if tool: Interactive probing of machine learning models

    James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wat- tenberg, Fernanda Vi ´egas, and Jimbo Wilson. The what-if tool: Interactive probing of machine learning models. IEEE transactions on visualization and computer graphics , 26(1):56–65, 2019

    2019

  104. [112]

    ethical ai

    David Gray Widder, Dawn Nafus, Laura Dabbish, and James Herbsleb. Limits and possibilities for “ethical ai” in open source: A study of deepfakes. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 2035–2046, 2022

    2022

  105. [113]

    Seeing like a toolkit: How toolkits envision the work of ai ethics

    Richmond Y Wong, Michael A Madaio, and Nick Merrill. Seeing like a toolkit: How toolkits envision the work of ai ethics. Proceedings of the ACM on Human-Computer Interaction , 7(CSCW1):1–27, 2023

    2023

  106. [114]

    The rise and potential of large language model based agents: A survey

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864 , 2023

    2023 arXiv

  107. [115]

    What do users ask in 10 open-source ai repositories? an empirical study of github issues

    Zhou Yang, Chenyu Wang, Jieke Shi, Thong Hoang, Pavneet Kochhar, Qinghua Lu, Zhenchang Xing, and David Lo. What do users ask in 10 open-source ai repositories? an empirical study of github issues. In 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (...

    2023

  108. [116]

    Investigating how practitioners use human-ai guidelines: A case study on the people+ ai guidebook

    Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fernanda Vi ´egas. Investigating how practitioners use human-ai guidelines: A case study on the people+ ai guidebook. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , pages 1–13, 2023

    2023

  109. [117]

    Measuring the maintain- ability of open-source software

    Liguo Yu, Stephen R Schach, and Kai Chen. Measuring the maintain- ability of open-source software. In 2005 International Symposium on Empirical Software Engineering, 2005. , pages 7–pp. IEEE, 2005

    2005

  110. [118]

    Conservative ai and social inequality: conceptualizing alternatives to bias through social theory

    Mike Zajko. Conservative ai and social inequality: conceptualizing alternatives to bias through social theory. Ai & Society , 36(3):1047– 1056, 2021

    2021

  111. [119]

    ” ignorance and prejudice” in software fairness

    Jie M Zhang and Mark Harman. ” ignorance and prejudice” in software fairness. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) , pages 1436–1447. IEEE, 2021

    2021

  112. [120]

    What the fork: a study of inefficient and efficient forking practices in social coding

    Shurui Zhou, Bogdan Vasilescu, and Christian K ¨astner. What the fork: a study of inefficient and efficient forking practices in social coding. In Proceedings of the 2019 27th ACM joint meeting on european software engineering conference and symposium on the foundations of sof...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.