REVIEW 9 cited by
Bias and Fairness in Large Language Models: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this paper, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly-available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.
Forward citations
Cited by 9 Pith papers
-
WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipedia
WETBench shows that existing machine-generated text detectors, particularly zero-shot methods, underperform on task-specific Wikipedia editing scenarios, with supervised detectors averaging 78% accuracy and zero-shot ...
-
Exploring Gender Bias Beyond Occupational Titles
The paper presents GenderLexicon and a ClozeGender score, reporting that action verbs and object nouns carry gender bias beyond occupational stereotypes in English and Japanese language models.
-
Explicit Preference Optimization: No Need for an Implicit Reward Model
EXPO is a pair of explicit preference-optimization losses that provably avoid DPO's uniform-regularization and poor-interpolation failure modes and outperform DPO on Anthropic HH and IMDb.
-
Adultification Bias in LLMs and Text-to-Image Models
Large language and text-to-image models show measurable adultification bias, portraying Black girls as more mature, culpable, and sexualized than White girls in several tested models.
-
A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
A query-only audit framework using dynamically generated dialect prompts finds that Amazon Rufus gives lower-quality and more incorrect responses to minoritized English dialects, with typos making the gap worse.
-
A Close Reading Approach to Gender Narrative Biases in AI-Generated Stories
A close reading of 15 AI-generated stories finds that even when character counts are balanced, narrative roles, descriptions, and plot dynamics remain gender-stereotyped (e.g., every villain is male).
-
Towards High Supervised Learning Utility Training Data Generation: Data Pruning and Column Reordering
PRRO combines signal-based data pruning and column reordering to improve the supervised learning utility of synthetic tabular data, but its evaluation is undermined by data manipulation and an ill-defined correlation measure.
-
A quantum semantic framework for natural language processing
The paper reports CHSH inequality violations from LLM interpretations of ambiguous sentences and uses them to claim that linguistic meaning is non-classical and observer-dependent.
-
The Hitchhiker's Guide to Agentic AI: From Foundations to Systems
A survey-style reference book mapping the full agentic-AI stack from transformer internals to production deployment, with no new research result.
Discussion (0). Sign in to comment.