Pith. sign in

REVIEW 4 major objections 6 minor 68 references

Breaking Down Bias: On The Limits of Generalizable Pruning Strategies

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Racial bias in LLMs is not a single general concept that pruning can remove across contexts.

desk verdict Careful empirical study of pruning for bias mitigation, but the headline generalization claim is confounded by pruned-set size; the low cross-context overlap is real, the SMD comparison is not apples-to-apples. read the letter →

arxiv 2502.07771 v1 pith:GIUJLYDH submitted 2025-02-11 cs.CL cs.AIcs.CYcs.LG

Sibo Ma , Alejandro Salinas , Peter Henderson , Julian Nyarko This is my paper · ORCID
classification cs.CLcs.AIcs.CYcs.LG
keywords largelanguagemodelsracialbiasmodelpruninglocalizationgeneralizationmitigationlegalliabilityLlama-3-8B-Instruct
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether racial bias in large language models can be understood and removed as a single general concept, the way a 'bias neuron' or a small set of biased components would allow. The authors prune components of Llama-3-8B-Instruct that are disproportionately influential for prompts with Black-associated names, then test whether the same pruned set reduces bias in other contexts. They find that context-matched pruning nearly eliminates measured bias, while cross-context pruning leaves a substantial gap. They conclude that racial bias is only partly encoded generally; much of it is tied to the specific decision context, so one-size-fits-all developer-side mitigation will be limited. If correct, this makes a concrete policy case for placing legal responsibility on deployers who adapt models to particular use cases.

What carries the argument

The argument runs on a localization-and-prune pipeline. Each neuron is scored by the product of its activation and its output weight (WandA-style), and each attention head is scored by the maximum attention weight that following tokens place on the race-revealing name tokens. The minority-specific set D is formed by taking the top-scoring components for Black-associated prompts and subtracting those that also rank highly for white-associated prompts; pruning zeroes those neurons or value matrices. Generalizability is then measured by comparing three constructions: prompt-specific sets, a leave-one-out within-context intersection, and a cross-context intersection over nine variations spanning Services, Activities, and Finance. The shrinking size of the pruned set as it is required to be shared across contexts is what carries the conclusion that bias is context-specific.

What would settle it

Take the cross-context intersection $D_{\text{ctx}}$ and enlarge it to the same number of components pruned in the prompt-specific condition (for example, by adding the next-best-scoring components), then measure SMD on the Purchase prompts; if bias reduction matches the prompt-specific near-zero result, the conclusion that bias is context-specific rather than a budget artifact would collapse.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that there is no universal 'bias neuron' that drives disparate outcomes across contexts. Using pruning as a probe, the paper shows that racial bias separates into a context-general part and a much larger context-specific part: when the components chosen for pruning come from the same prompt or within the same scenario, bias can be reduced to near zero without hurting the model's ability to produce plausible numeric answers; when the components are instead required to be shared across nine variations in three different scenarios, the reduction is only partial. Neuron-level pruning consistently beats attention-head pruning, and the overlap of pruned neurons across scenarios is low, supporting the conclusion that the biased circuitry is not a small universal set.

Load-bearing premise

The paper assumes the weaker effect of generalized pruning is due to context-specific bias, not just to the fact that the generalized approach prunes fewer parts of the model.

Editorial extensions

If this is right

  • Context-specific neuron pruning can bring measured racial bias close to zero while keeping the inlier ratio at or above 0.98, so pruning is a viable narrow-context mitigation.
  • Generalized pruning is only partially effective: cross-context pruning leaves mean SMD around -0.37 for neurons and -0.57 for attention heads, and financial-to-commercial transfer cuts bias by only about 40 percent.
  • Neuron-level pruning outperforms attention-head pruning in every generalization regime tested, so finer-grained interventions are the better target for future work.
  • Because the biased components overlap little across contexts, effective mitigation is likely to require access to the specific deployment context, supporting legal and regulatory designs that assign responsibility to deployers.
  • Distributional robustness checks with Earth Mover's Distance confirm the same pattern, suggesting the finding is not an artifact of the standardized mean difference metric.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit: if bias is mostly context-specific, then general-purpose 'debiasing' benchmarks may overstate the safety of a foundation model, and evaluations should be built from the actual distribution of downstream deployments.
  • The localization results suggest that bias may be encoded in distributed, context-conditioned circuits rather than a single direction in activation space; a testable extension would be to check whether the same pattern holds in larger or instruction-tuned models with more layers.
  • A natural next experiment would compare cross-context pruning at a matched component budget, separating context specificity from the mere size of the pruned set.
  • The framework could be extended beyond binary Black/white name comparisons to other demographic groups and other output modalities, which would clarify how widely the context-specificity conclusion holds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies whether racial bias in LLMs is encoded as a general, context-independent concept or as a context-specific phenomenon, using pruning as an intervention. On LLaMA-3-8B-Instruct, the authors localize neurons and attention heads that are disproportionately important for Black-associated names in an advice-seeking prompt framework, then prune them under three regimes: prompt-specific (each variation pruned and evaluated on itself), within-context (leave-one-out intersection over nine Purchase-scenario variations), and cross-context (intersection over nine variations from Services, Activities, and Finance, evaluated on the Purchase scenario). They report that prompt-specific pruning nearly eliminates the standardized mean difference (SMD) in outputs, while within-context and cross-context pruning are progressively less effective, and that neuron pruning outperforms attention-head pruning. These results are interpreted as evidence that racial bias is only partially represented as a general concept, with implications for allocating legal responsibility to deployers rather than only developers.

Significance. If the central claim is established, the paper would make a valuable empirical contribution to the interpretability and AI-governance literature: it would provide a concrete, falsifiable test of the 'universal bias neuron' hypothesis and inform the developer-vs-deployer liability debate. The work has notable strengths: a clearly specified localization method (WandA-based neuron scoring and attention-weight-based head scoring), a multi-scenario design that goes beyond a single prompt template, and robustness checks using Earth Mover's Distance and non-winsorized SMD that corroborate the qualitative ordering of the pruning regimes. However, the main comparison across regimes is confounded by the number of pruned components, and one of the headline examples in the abstract is not directly tested. These issues undermine the strength of the generalization claim as currently presented, but they are addressable within the manuscript's scope.

major comments (4)
  1. [§4.3, Eqns (11)–(13); §5.1, Fig. 2] The comparison across prompt-specific, within-context, and cross-context pruning is confounded by the number of pruned components. D_LOO_k and D_ctx are intersections of nine sets, so their cardinalities are likely much smaller than the prompt-specific D_k. The reported SMD reductions (e.g., mean ≈ +0.07, ≈ −0.20, ≈ −0.37 for neuron pruning) could therefore reflect a dose effect rather than context-specific degradation. The manuscript does not report |D_k|, |D_LOO_k|, or |D_ctx|, and it does not control for pruning budget. Please report these cardinalities and add an equal-budget analysis (for example, prune the same number of top-ranked components under each regime, or prune random subsets of D_k with size equal to |D_ctx|) to separate the effect of set size from the effect of context distance.
  2. [Abstract and §5.1] The abstract states that 'a model that is trained on removing racial biases in the context of financial decision-making poorly generalizes to biases in commercial transactions' and the introduction cites 'about 40% reduction' for this finance-to-commerce transfer. However, the cross-context condition pools Services, Activities, and Finance (M = 9 variations, Eq. 13); no finance-only transfer experiment is reported. The 40% figure appears to be the mean reduction for the pooled cross-context set. Either add a direct finance-only transfer test (construct D from Finance variations only and evaluate on Purchase) or revise the abstract and text to accurately describe the pooled cross-context result.
  3. [§4.4 and Appendix A.2] The thresholds τ_min and τ_maj are optimized on the Purchase data, which is the same domain used for the prompt-specific and within-context evaluations. This in-sample tuning likely inflates the prompt-specific results and may also affect the cross-context comparison, since the same thresholds are applied to D_ctx even though they were not selected for that regime. The paper should report whether the qualitative ordering of the three regimes is robust to (i) thresholds chosen on a held-out validation set and (ii) per-regime threshold optimization, to ensure the central generalization claim is not an artifact of threshold selection.
  4. [§5.1 and Fig. 2] The paper reports only mean SMD values across the ten Purchase variations and provides no uncertainty quantification (e.g., standard errors, bootstrap confidence intervals, or a paired test across variations). Given that the cross-context vs within-context difference is a key load-bearing comparison, it is important to know whether the differences (e.g., ≈ −0.20 vs ≈ −0.37 for neuron pruning) are statistically distinguishable. Please add per-variation measures of dispersion and an appropriate paired significance test.
minor comments (6)
  1. [Appendix A.2] The text says the grid search covers '66 combinations,' but the ranges in Eqns (14)–(17) yield 55 combinations for each component type (sum_{k=1}^{10} k). Please correct this number or clarify how 66 is obtained.
  2. [§3.1.1, Eqns (1)–(2)] The notation S^{neuron}_{t,n} = Act_{t,n} W_n is ambiguous: W_n appears to be a vector in R^o, but the summation over o is not written. Please specify the dimensions and whether the sum in Eq. (2) is over the output dimension as well as the token dimension.
  3. [§4.2, Eq. (10)] The pooled standard deviation formula uses '100 · |G|' as sample sizes, but this is never explicitly explained in the main text (only implied by the footnote about 100 repeated samples). Please define s_black and s_white and clarify that the 100 factor reflects repeated sampling.
  4. [Fig. 2 caption] The caption says '(b) Attention Head Pruning on Bias and Utility,' but the actual panel labels are '(a) Neuron pruning' and '(b) Attention Head Pruning'; the caption text appears truncated or misformatted. Please make the caption consistent with the figure panels.
  5. [§5.2, Fig. 3] The overlap heatmap in Fig. 3 normalizes by the size of the non-Purchase variation's pruned set (the denominator is stated to be 'the total size of pruned neurons for the corresponding scenario's variation'). It would be informative to also report overlap as a fraction of the Purchase variation's set, or to include a random-overlap baseline, so that the observed 0.12–0.16 fractions are interpretable.
  6. [§7, Limitations] The limitations section is honest about the scope of the study, but it does not mention the set-size confound between the pruning regimes. Adding a sentence acknowledging this and pointing to the equal-budget control in the revised analyses would strengthen the paper.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the central context-specificity claim is tested on external cross-context sets, with only a minor in-sample evaluation and a non-load-bearing self-citation.

full rationale

The paper's central claim—that racial bias is partially context-specific and that no universal 'bias neuron' drives outcomes across contexts—has independent empirical content. The pruning targets are derived from activation/attention scores (Eqs. 1-7), not from the SMD evaluation metric, so the pruning procedure is not constructed to minimize the measured outcome by definition. Prompt-specific pruning is evaluated on the same variations used to build D_k (Sec. 4.3.1), which is an in-sample demonstration, but the paper explicitly labels this the 'best-case scenario' and acknowledges it 'may overfit,' so it is not presented as an out-of-sample prediction. The cross-context evaluation uses D_ctx (Eq. 13), computed from Services, Activities, and Finance prompts, and then measures SMD on the Purchase variations; this is a genuine transfer test. The intersection construction of D_LOO_k and D_ctx does create a plausible dose/coverage confound with pruned-set size, and the paper does not report cardinalities or an equal-budget control. That is a validity concern, but it is not circularity: SMD is not equal by construction to the number of pruned components, and Fig. 3 offers an additional, size-normalized overlap analysis. The self-citation to Haim et al. [25], which shares two authors with the present paper, supplies the prompt template and name lists, but the paper's legal and conceptual conclusions do not rest on the validity of that prior paper's claims. No circular step meeting the evidentiary bar of an explicit reduction of a result to its own inputs is present.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper's conclusions rest on an operational definition of bias as mean output differences for Black versus white names, a causal interpretation of activation and attention scores, fitted pruning thresholds, and an interpretive leap from low cross-context overlap to context-specific bias. The free parameters are the pruning thresholds tuned on the Purchase scenario. No new theoretical entities are introduced.

free parameters (3)
  • Neuron pruning thresholds tau_min and tau_maj = tau_min=0.40, tau_maj=0.35
    Grid search on Purchase scenario to balance SMD reduction and inlier ratio; no separate validation set is described (Section 4.4, Appendix A.2).
  • Attention head pruning thresholds tau_min and tau_maj = tau_min=40 heads, tau_maj=5 heads
    Same grid-search procedure on Purchase scenario (Section 4.4, Appendix A.2).
  • Winsorization percentile for utility range = not specified
    Appendix A.4 says outputs are winsorized at predefined percentiles, but the percentile values are not given, leaving an unstated tuning choice.
assumptions (5)
  • domain assumption Bias is operationalized as the difference in numeric outputs for prompts with white-associated versus Black-associated names.
    Section 4.1 and Eq. 9; this equates racial bias with mean outcome gaps in single-turn advice prompts, excluding implicit associations and distributional aspects.
  • domain assumption Components with high minority-group activation or attention scores causally drive biased outputs, so zeroing them reduces bias.
    Sections 3.2 and 3.3; no causal verification or ablation with random components is provided.
  • domain assumption Intersection of component sets across prompt variations is a valid measure of generalizability; absence from the intersection indicates context-specific representation.
    Section 4.3.3 and Figure 3; this interpretive step anchors the central conclusion.
  • domain assumption Llama-3-8B-Instruct is representative enough to support conclusions about how LLMs encode racial bias in general.
    Section 1 and Limitations; only one model is tested.
  • ad hoc to paper The max attention-weight scoring for heads (Eq. 4) is an appropriate importance measure.
    Footnote 1 states that maximizing over both dimensions yielded the best results; this is a hand-chosen heuristic rather than a derived or benchmarked criterion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Breaking Down Bias: On The Limits of Generalizable Pruning Strategies." pith.science (2026). https://pith.science/paper/GIUJLYDH

@misc{pith2026250207771,
  author       = {Pith},
  title        = {Pith review of: Breaking Down Bias: On The Limits of Generalizable Pruning Strategies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GIUJLYDH}},
  note         = {Machine review of arXiv:2502.07771}
}
read the original abstract

We employ model pruning to examine how LLMs conceptualize racial biases, and whether a generalizable mitigation strategy for such biases appears feasible. Our analysis yields several novel insights. We find that pruning can be an effective method to reduce bias without significantly increasing anomalous model behavior. Neuron-based pruning strategies generally yield better results than approaches pruning entire attention heads. However, our results also show that the effectiveness of either approach quickly deteriorates as pruning strategies become more generalized. For instance, a model that is trained on removing racial biases in the context of financial decision-making poorly generalizes to biases in commercial transactions. Overall, our analysis suggests that racial biases are only partially represented as a general concept within language models. The other part of these biases is highly context-specific, suggesting that generalizable mitigation strategies may be of limited effectiveness. Our findings have important implications for legal frameworks surrounding AI. In particular, they suggest that an effective mitigation strategy should include the allocation of legal responsibility on those that deploy models in a specific use case.

Figures

Figures reproduced from arXiv: 2502.07771 by the authors.

Figure 1
Figure 1. Illustration of our pruning-based bias mitigation method. Initially, the unpruned [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Impact of Neuron and Attention Head Pruning on Bias and Utility. The top [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Overlap in biased neurons between Purchase variations and variations from other scenarios. Heat is defined as a fraction, with the numerator being the intersection of pruned neurons between every scenario’s variation and each Purchase variations. The denominator is the total size of pruned neurons for the corresponding scenario’s variation. Higher values indicate stronger overlap. 5.3 Location of Pruned Neurons [PI… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Neuron Pruning Distribution across Layers and Subcomponents. This heatmap [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Grid search results for neuron pruning. Plot on the left is SMD and the plot on [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]
Figure 6
Figure 6. Figure 6: Grid search results for neuron pruning. Plot on the left is SMD and the plot on [PITH_FULL_IMAGE:figures/full_fig_p026_6.png]
Figure 7
Figure 7. Figure 7: Overlap of pruned neurons and attention heads across [PITH_FULL_IMAGE:figures/full_fig_p027_7.png]
Figure 8
Figure 8. Figure 8: Impact of Neuron and Attention Head Pruning on Bias as Measured by SMD [PITH_FULL_IMAGE:figures/full_fig_p028_8.png]
Figure 9
Figure 9. Figure 9: Impact of Neuron and Attention Head Pruning on Bias as Measured by EMD. The [PITH_FULL_IMAGE:figures/full_fig_p028_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 45 canonical work pages

  1. [1]

    Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models

    Rishabh Adiga, Besmira Nushi, and Varun Chandrasekaran. Attention Speaks Vol- umes: Localizing and Mitigating Bias in Language Models . 2024. arXiv: 2410.22517 [cs.CL]. url: https://arxiv.org/abs/2410.22517. 17

    work page Pith review arXiv 2024

  2. [2]

    Mean Difference, Standardized Mean Difference (SMD), and Their Use in Meta-Analysis: As Simple as It Gets

    Chittaranjan Andrade. “Mean Difference, Standardized Mean Difference (SMD), and Their Use in Meta-Analysis: As Simple as It Gets”. In: Primary Care Companion for CNS Disorders 81.5 (2020). doi: 10.4088/JCP.20f13681

    doi:10.4088/jcp.20f13681 2020

  3. [3]

    Measuring Implicit Bias in Explicitly Unbiased Large Language Models

    Xuechunzi Bai et al. Measuring Implicit Bias in Explicitly Unbiased Large Language Models. 2024. arXiv: 2402.04105 [cs.CY]. url: https://arxiv.org/abs/2402. 04105

    arXiv 2024

  4. [4]

    Taking the next step with generative artificial intelligence: The transformative role of multimodal large language models in science education

    Arne Bewersdorff et al. “Taking the next step with generative artificial intelligence: The transformative role of multimodal large language models in science education”. In: Learning and Individual Differences 118 (2025), p. 102601

    work page 2025

  5. [5]

    On the opportunities and risks of foundation models

    Rishi Bommasani et al. “On the opportunities and risks of foundation models”. In: arXiv preprint arXiv:2108.07258 (2021)

    arXiv 2021

  6. [6]

    Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions

    Angana Borah and Rada Mihalcea. “Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions”. In: arXiv preprint arXiv:2410.02584 (2024)

    arXiv 2024

  7. [7]

    General purpose technologies

    Timothy Bresnahan. “General purpose technologies”. In: Handbook of the Economics of Innovation 2 (2010), pp. 761–791

    work page 2010

  8. [8]

    Prompting change: exploring prompt engineering in large language model AI and its potential to transform education

    William Cain. “Prompting change: exploring prompt engineering in large language model AI and its potential to transform education”. In: TechTrends 68.1 (2024), pp. 47–57

    work page 2024

Show all 68 references
  1. [9]

    Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks

    Ting-Yun Chang, Jesse Thomason, and Robin Jia. Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks . 2024. arXiv: 2311. 09060 [cs.CL]. url: https://arxiv.org/abs/2311.09060

    2024 arXiv

  2. [10]

    Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted

    Ruchika Chavhan et al. Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted . 2024. arXiv: 2406 . 18566 [cs.CV]. url: https : //arxiv.org/abs/2406.18566

    2024 arXiv

  3. [11]

    An overview of domain-specific foundation model: key technolo- gies, applications and challenges

    Haolong Chen et al. “An overview of domain-specific foundation model: key technolo- gies, applications and challenges”. In: arXiv preprint arXiv:2409.04267 (2024)

    2024

  4. [12]

    Crime News and Racialized Beliefs: Understanding the Relationship Between Local News Viewing and Perceptions of African Americans and Crime

    Travis L. Dixon. “Crime News and Racialized Beliefs: Understanding the Relationship Between Local News Viewing and Perceptions of African Americans and Crime”. In: Journal of Communication 58.1 (2008), pp. 106–125. doi: https://doi.org/10. 1111/j.1460- 2466.2007.00376.x . epri...

    2008

  5. [13]

    Evaluating Feature Steering: A Case Study in Mitigating Social Bi- ases

    Esin Durmus et al. Evaluating Feature Steering: A Case Study in Mitigating Social Bi- ases. Oct. 25, 2024. url: https://anthropic.com/research/evaluating-feature- steering

    2024

  6. [14]

    First-Person Fairness in Chatbots

    Tyna Eloundou et al. First-Person Fairness in Chatbots . 2024. arXiv: 2410.19803 [cs.CY]. url: https://arxiv.org/abs/2410.19803

    2024 arXiv

  7. [15]

    The democratization of global AI governance and the role of tech companies

    Eva Erman and Markus Furendal. “The democratization of global AI governance and the role of tech companies”. In: Nature Machine Intelligence 6.3 (2024), pp. 246–248. 18

    2024

  8. [16]

    Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial In- telligence Act)

    European Commission. Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial In- telligence Act). Brussels, 21.4.2021 COM(2021) 206 final. https://eur-lex.europa. eu/legal-content/EN/TXT/?uri=C...

    2021

  9. [17]

    Fairness and bias in artificial intelligence: A brief survey of sources, impacts, and mitigation strategies

    Emilio Ferrara. “Fairness and bias in artificial intelligence: A brief survey of sources, impacts, and mitigation strategies”. In: Sci 6.1 (2023), p. 3

    2023

  10. [18]

    How black are Lakisha and Jamal? Racial perceptions from names used in correspondence audit studies

    S Michael Gaddis. “How black are Lakisha and Jamal? Racial perceptions from names used in correspondence audit studies”. In: Sociological Science 4 (2017), pp. 469–489

    2017

  11. [19]

    The capacity for moral self-correction in large language models

    Deep Ganguli et al. “The capacity for moral self-correction in large language models”. In: arXiv preprint arXiv:2302.07459 (2023)

    2023 arXiv

  12. [20]

    From melting pots to misrepresentations: Exploring harms in generative ai

    Sanjana Gautam, Pranav Narayanan Venkit, and Sourojit Ghosh. “From melting pots to misrepresentations: Exploring harms in generative ai”. In: arXiv preprint arXiv:2403.10776 (2024)

    2024 arXiv

  13. [21]

    Prime Suspects: The Influence of Local Television News on the Viewing Public

    Franklin D. Gilliam and Shanto Iyengar. “Prime Suspects: The Influence of Local Television News on the Viewing Public”. In: American Journal of Political Science 44.3 (2000), pp. 560–573. issn: 00925853, 15405907. url: http://www.jstor.org/ stable/2669264 (visited on 01/22/2025)

    2000

  14. [22]

    Where You Live and What You Watch: The Impact of Racial Proximity and Local Television News on Attitudes about Race and Crime

    Franklin D. Gilliam, Nicholas A. Valentino, and Matthew N. Beckmann. “Where You Live and What You Watch: The Impact of Racial Proximity and Local Television News on Attitudes about Race and Crime”. In: Political Research Quarterly 55.4 (2002), pp. 755–780. issn: 10659129. url:...

    2002

  15. [23]

    Police agencies on Facebook overreport on Black suspects

    Ben Grunwald, Julian Nyarko, and John Rappaport. “Police agencies on Facebook overreport on Black suspects”. In: Proceedings of the National Academy of Sciences 119.45 (2022), e2203089119. doi: 10.1073/pnas.2203089119. eprint: https://www. pnas.org/doi/pdf/10.1073/pnas.2203089...

    2022 doi

  16. [24]

    Generative discrimination: What happens when generative AI exhibits bias, and what can be done about it

    Philipp Hacker et al. “Generative discrimination: What happens when generative AI exhibits bias, and what can be done about it”. In: arXiv preprint arXiv:2407.10329 (2024)

    2024 arXiv

  17. [25]

    What’s in a Name? Auditing Large Language Models for Race and Gender Bias

    Amit Haim, Alejandro Salinas, and Julian Nyarko. What’s in a Name? Auditing Large Language Models for Race and Gender Bias . 2024. arXiv: 2402.14875 [cs.CL]. url: https://arxiv.org/abs/2402.14875

    2024 arXiv

  18. [26]

    Ethical AI: A Policy Framework to Regulate Bias in Large Language Models

    Anuj Jain. “Ethical AI: A Policy Framework to Regulate Bias in Large Language Models”. PhD thesis. 2024

    2024

  19. [27]

    Financial statement analysis with large language models

    Alex Kim, Maximilian Muhn, and Valeri Nikolaev. “Financial statement analysis with large language models”. In: arXiv preprint arXiv:2407.17866 (2024)

    2024 arXiv

  20. [28]

    “We’d love to hire them, but

    Joleen Kirshenman and Kathryn M Neckerman. ““We’d love to hire them, but. . . ”: The meaning of race for employers”. In: Race and ethnic conflict . Routledge, 2019, pp. 152–161. 19

    2019

  21. [29]

    Acceptable Use Policies for Foundation Models

    Kevin Klyman. “Acceptable Use Policies for Foundation Models”. In: Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society . Vol. 7. 2024, pp. 752–767

    2024

  22. [30]

    Gender bias and stereotypes in large language models

    Hadas Kotek, Rikker Dockum, and David Sun. “Gender bias and stereotypes in large language models”. In: Proceedings of the ACM collective intelligence conference. 2023, pp. 12–24

    2023

  23. [31]

    Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models

    Benjamin Laufer, Jon Kleinberg, and Hoda Heidari. “Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models”. In: Proceedings of the ACM on Web Conference 2024. 2024, pp. 66–76

    2024

  24. [32]

    Evaluating the accuracy and reliability of large language models in assisting with pediatric differential diagnoses: A multicenter diagnostic study

    Masab A Mansoor et al. “Evaluating the accuracy and reliability of large language models in assisting with pediatric differential diagnoses: A multicenter diagnostic study”. In: medRxiv (2024), pp. 2024–08

    2024

  25. [33]

    Large Language Models are Geographically Biased

    Rohin Manvi et al. Large Language Models are Geographically Biased . 2024. arXiv: 2402.02680 [cs.CL]. url: https://arxiv.org/abs/2402.02680

    2024 arXiv

  26. [34]

    The imperative for regulatory oversight of large language models (or generative AI) in healthcare

    Bertalan Mesk´ o and Eric J Topol. “The imperative for regulatory oversight of large language models (or generative AI) in healthcare”. In:NPJ digital medicine 6.1 (2023), p. 120

    2023

  27. [35]

    How AI is Shaking Up the Mental Health Community: ”Rather Than Pay for Another Session, I’d Go on ChatGPT”

    Le Monde. How AI is Shaking Up the Mental Health Community: ”Rather Than Pay for Another Session, I’d Go on ChatGPT” . Accessed: 2025-01-22. 2024. url: https: //www.lemonde.fr/en/pixels/article/2024/08/18/how- ai- is- shaking- up- the-mental-health-community-rather-than-pay-fo...

    2025

  28. [36]

    Automatically Interpreting Millions of Features in Large Lan- guage Models

    Gon¸ calo Paulo et al. Automatically Interpreting Millions of Features in Large Lan- guage Models . 2024. arXiv: 2410.13928 [cs.LG]. url: https://arxiv.org/abs/ 2410.13928

    2024 arXiv

  29. [37]

    Race and Networks in the Job Search Pro- cess

    David S. Pedulla and Devah Pager. “Race and Networks in the Job Search Pro- cess”. In: American Sociological Review 84.6 (2019), pp. 983–1012. doi: 10.1177/ 0003122419883255. eprint: https://doi.org/10.1177/0003122419883255 . url: https://doi.org/10.1177/0003122419883255

    2019 doi

  30. [38]

    About a Quarter of U.S

    Pew Research Center. About a Quarter of U.S. Teens Have Used ChatGPT for School- work, Double the Share in 2023 . Accessed: 2025-01-22. 2025. url: https : / / www . pewresearch.org/short- reads/2025/01/15/about- a- quarter- of- us- teens- have-used-chatgpt-for-schoolwork-doubl...

    2023

  31. [39]

    A large-scale analysis of racial disparities in police stops across the United States

    Emma Pierson et al. “A large-scale analysis of racial disparities in police stops across the United States”. In: Nature human behaviour 4.7 (2020), pp. 736–745

    2020

  32. [40]

    Comparative perspectives on the regulation of large language models

    Cristina Poncib` o and Martin Ebers. “Comparative perspectives on the regulation of large language models”. In: Cambridge Forum on AI: Law and Governance . Vol. 1. Cambridge University Press. 2025, e10

    2025

  33. [41]

    Racial disparities in school-based disciplinary actions are associated with county-level rates of racial bias

    Travis Riddle and Stacey Sinclair. “Racial disparities in school-based disciplinary actions are associated with county-level rates of racial bias”. In: Proceedings of the National Academy of Sciences 116.17 (2019), pp. 8255–8260. doi: 10.1073/pnas. 1808307116. eprint: https://...

    2019 doi

  34. [42]

    Racist Cops, Vested “Blue

    Vincent J. Roscigno and Kayla Preito-Hodge. “Racist Cops, Vested “Blue” Inter- ests, or Both? Evidence from Four Decades of the General Social Survey”. In: Socius 7 (2021), p. 2378023120980913. doi: 10 . 1177 / 2378023120980913. eprint: https : / / doi . org / 10 . 1177 / 2378...

    2021

  35. [43]

    The unequal opportunities of large language models: Examining demographic biases in job recommendations by chatgpt and llama

    Abel Salinas et al. “The unequal opportunities of large language models: Examining demographic biases in job recommendations by chatgpt and llama”. In: Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 2023, pp. 1–15

    2023

  36. [44]

    How Implicit Bias Contributes to Racial Disparities in Maternal Morbidity and Mortality in the United States

    Bani Saluja and Zenobia Bryant. “How Implicit Bias Contributes to Racial Disparities in Maternal Morbidity and Mortality in the United States”. In: Journal of Women ’s Health 30.2 (2021). PMID: 33237843, pp. 270–273. doi: 10.1089/jwh.2020.8874 . eprint: https://doi.org/10.1089...

    2021

  37. [45]

    Who Does the Giant Number Pile Like Best: Analyzing Fairness in Hiring Contexts

    Preethi Seshadri and Seraphina Goldfarb-Tarrant. Who Does the Giant Number Pile Like Best: Analyzing Fairness in Hiring Contexts . 2025. arXiv: 2501.04316 [cs.CL]. url: https://arxiv.org/abs/2501.04316

    2025 arXiv

  38. [46]

    Toward expert-level medical question answering with large lan- guage models

    Karan Singhal et al. “Toward expert-level medical question answering with large lan- guage models”. In: Nature Medicine (2025), pp. 1–8

    2025

  39. [47]

    A Simple and Effective Pruning Approach for Large Language Models

    Mingjie Sun et al. A Simple and Effective Pruning Approach for Large Language Models. 2024. arXiv: 2306.11695 [cs.CL]. url: https://arxiv.org/abs/2306. 11695

    2024 arXiv

  40. [48]

    Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

    Adly Templeton et al. “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet”. In: Transformer Circuits Thread (2024). url: https : / / transformer-circuits.pub/2024/scaling-monosemanticity/index.html

    2024

  41. [49]

    Executive Order on Safe, Secure, and Trustworthy Artificial Intel- ligence

    The White House. Executive Order on Safe, Secure, and Trustworthy Artificial Intel- ligence. https://www.whitehouse.gov/ . Issued October 30, 2023, Accessed: Month Day, Year. 2023

    2023

  42. [50]

    Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned

    Elena Voita et al. “Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned”. In: arXiv preprint arXiv:1905.09418 (2019)

    2019 arXiv

  43. [51]

    SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning

    Hanrui Wang, Zhekai Zhang, and Song Han. “SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning”. In: 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . IEEE, Feb. 2021. doi: 10 . 1109 / hpca51647 . 2021 . 00018....

    2021

  44. [52]

    Assessing the Brittleness of Safety Alignment via Pruning and Low- Rank Modifications

    Boyi Wei et al. Assessing the Brittleness of Safety Alignment via Pruning and Low- Rank Modifications. 2024. arXiv: 2402.05162 [cs.LG]. url: https://arxiv.org/ abs/2402.05162

    2024 arXiv

  45. [53]

    Fairness & Privacy in an Age of Generative AI

    Alice Xiang. “Fairness & Privacy in an Age of Generative AI”. In: Science and Technology Law Review 25.2 (June 2024). doi: 10.52214/stlr.v25i2.12765 . url: https : / / journals . library . columbia . edu / index . php / stlr / article / view / 12765. 21

    2024 doi

  46. [54]

    The Economics of AI Foundation Models: Openness, Competition, and Governance

    Fasheng Xu et al. “The Economics of AI Foundation Models: Openness, Competition, and Governance”. In: Competition, and Governance (August 11, 2024) (2024)

    2024

  47. [55]

    Mitigating Biases for Instruction-following Language Models via Bias Neurons Elimination

    Nakyeong Yang et al. Mitigating Biases for Instruction-following Language Models via Bias Neurons Elimination . 2024. arXiv: 2311 . 09627 [cs.AI]. url: https : / / arxiv.org/abs/2311.09627

    2024 arXiv

  48. [56]

    Fairness-Aware Structured Pruning in Transformers

    Abdelrahman Zayed et al. Fairness-Aware Structured Pruning in Transformers. 2023. arXiv: 2312.15398 [cs.CL]. url: https://arxiv.org/abs/2312.15398

    2023 arXiv

  49. [57]

    AI Risk Categorization Decoded (AIR 2024): From Government Reg- ulations to Corporate Policies

    Yi Zeng et al. AI Risk Categorization Decoded (AIR 2024): From Government Reg- ulations to Corporate Policies . 2024. arXiv: 2406 . 17864 [cs.CY]. url: https : / / arxiv.org/abs/2406.17864

    2024 arXiv

  50. [58]

    Air-bench 2024: A safety benchmark based on risk categories from regulations and policies

    Yi Zeng et al. “Air-bench 2024: A safety benchmark based on risk categories from regulations and policies”. In: arXiv preprint arXiv:2407.17436 (2024)

    2024 arXiv

  51. [59]

    Know what you don’t need: Single-Shot Meta-Pruning for attention heads

    Zhengyan Zhang et al. “Know what you don’t need: Single-Shot Meta-Pruning for attention heads”. In: AI Open 2 (2021), pp. 36–42

    2021

  52. [60]

    Revolutionizing finance with llms: An overview of applications and insights

    Huaqin Zhao et al. “Revolutionizing finance with llms: An overview of applications and insights”. In: arXiv preprint arXiv:2401.11641 (2024). 22 A Appendix A.1 Prompt Selection for Pruning Evaluation To systematically evaluate the impact of pruning on model disparities and uti...

    2024

  53. [61]

    We generate 30 prompt variations (e.g., different products to purchase) using a com- bination of LLM-generated suggestions and manual inspection to ensure diversity and relevance

  54. [62]

    Each variation is processed through the LLaMA 3-Instruct 8B model to measure the initial disparity in outcomes

  55. [63]

    These selected variations serve as the foundation for subsequent pruning experiments, allowing us to focus on cases where bias is most evident

    Based on these evaluations, we select theN = 10 variations with the highest disparities for further analysis. These selected variations serve as the foundation for subsequent pruning experiments, allowing us to focus on cases where bias is most evident. A.2 Threshold Optimizat...

  56. [64]

    For each prompt variation, we explore different values of τmin and τmaj while moni- toring the size of the bias-influential set D defined in Equation 7

  57. [65]

    Empirical observations suggest τmin ≈ τmaj, leading to the choice of the following ranges

    The thresholds are selected based on the elbow point principle, where the set size stabilizes, indicating diminishing returns. Empirical observations suggest τmin ≈ τmaj, leading to the choice of the following ranges. For neuron pruning, the selected ranges were: We define the...

  58. [66]

    The resulting SMD for different parameter combinations is shown in Figure 5 (neuron pruning) and Figure 6 (attention head pruning)

    The model is pruned using the selected τmin and τmaj values across the 66 combina- tions, and the disparities and utility of each setting are analyzed. The resulting SMD for different parameter combinations is shown in Figure 5 (neuron pruning) and Figure 6 (attention head pru...

  59. [67]

    Quantitative Filtering: Since model responses may contain both numerical and non-numerical elements, we apply regular expression (regex) rules to extract quanti- tative parts of the output. If a response cannot be successfully parsed into a numerical value (e.g., free-text res...

  60. [68]

    Any response that falls outside this range is marked as a utility violation

    Range-Based Filtering: We establish a reference range for acceptable numerical values based on the minimum and maximum outputs generated by the unpruned model across all prompt variations. Any response that falls outside this range is marked as a utility violation. This range-...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.