Pith. sign in

REVIEW 4 major objections 6 minor 36 references

A Mechanistic Analysis of Gender Sensitivity in Dense Retrieval Models

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gender bias in dense retrievers traces to input embeddings and a few late-layer attention heads that also encode relevance.

desk verdict A credible first mechanistic map of gender bias in dense retrievers, with a real embedding-origin finding — but the five-head circuit claim lacks a null baseline and error bars, so treat the precision as provisional. read the letter →

arxiv 2608.05467 v1 pith:TR7SP7J6 submitted 2026-08-05 cs.IR

Catherine Chen , Maarten de Rijke , Carsten Eickhoff This is my paper · ORCID
classification cs.IR
keywords genderbiasdenseretrievalmechanisticinterpretabilityactivationpatchingpathattentionheadssteeringbi-encodermodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks where gender bias lives inside dense retrieval models and whether it can be steered. Using activation patching and path patching on five bi-encoder models with identical male, female, and neutral document variants, it finds that the asymmetric scores trace back to the input embeddings and then to a small set of late-layer attention heads. Those same heads also carry term-matching information, so gender sensitivity is not a separate circuit but is entangled with the model's ordinary relevance computation. The paper then shows that steering at the two locations produces different outcomes: embedding-level steering neutralizes score gaps in both directions, while attention-level steering shifts preference toward one gender or the other. The point of the exercise is that bias in retrieval can be studied and adjusted at the level of internal components, not only through retraining or post-hoc reranking.

What carries the argument

The machinery is a contrastive circuit-analysis setup. For each query, the model scores two document variants that are identical except for gendered tokens, and activation patching and path patching are used to swap internal activations between the low-scoring and high-scoring variants and measure which components shift the relevance score. The load-bearing object is the small set of late-layer attention heads, which act as general aggregation sites in the final residual stream and are identified by patching effects at least two standard deviations from the mean. A second piece of machinery is additive activation steering, which builds a vector from the mean activation difference between gendered variants and adds it, scaled, either to the input embedding of the gendered token or to the outputs of the identified heads.

What would settle it

A reader could re-run the same path-patching analysis with a different threshold, a different patching metric, or randomly selected late-layer heads as a control; if random heads reduce the score gap as much as the identified heads when ablated, or if the identified head set changes drastically, the claimed causal localization would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that in bi-encoder dense retrievers the gender signal enters through the input embeddings, is routed through attention, and is aggregated by a small set of late-layer attention heads that jointly encode gender and term-matching information. Path patching identifies these heads as the components that mediate score differences between male, female, and neutral document variants, and mean ablation of the identified heads reduces the score gaps across comparisons. The neutralization is incomplete on documents where the neutral term also appears in the query, which is taken as evidence that the heads are doing double duty: they match query terms and they carry gender. The paper concludes that gender sensitivity is entangled with relevance computation, which explains why ablating the heads only partially removes bias and why future debiasing must disentangle the two signals.

Load-bearing premise

The entire localization rests on the assumption that the patching procedure with its threshold truly finds the components causing the score differences, rather than components that merely appear salient under one metric and one dataset (the measured patch effects are small, and no alternative head sets or noise baselines are reported).

Editorial extensions

If this is right

  • If the identified heads are genuine aggregation sites for both gender and relevance, then any intervention that removes gender from these heads will also touch term matching, making the fairness–performance trade-off a mechanistic constraint rather than an accident of training.
  • Embedding-level steering is a symmetry-breaking knob: steering in either direction compresses the score gap between male and female variants, which makes it useful for neutralization but not for fine-grained control.
  • Attention-level steering is directional: positive coefficients favor female variants and negative coefficients favor male variants, offering an inference-time control that avoids retraining or fine-tuning.
  • Because the same heads are implicated across multiple bi-encoders and both pooling architectures, the mechanism is likely a general property of this model family, not a quirk of one checkpoint.
  • Ablation failing specifically on documents whose neutral term coincides with a query term is a direct behavioral signature of the joint encoding claim.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • It follows, though the paper does not say so, that gender steering vectors could spill over into retrieval quality: if the heads also match terms, a steering vector built from gender-contrastive pairs may shift ranking behavior on queries that share nothing with the steering data, which can be tested by measuring nDCG or mean reciprocal rank on a held-out query set before and after steering.
  • The same path-patching recipe could be applied to other protected attributes, such as age, race, or disability, by constructing document variants that differ only on those terms; the prediction would be that analogous late-layer aggregation heads appear, since the architecture has no reason to treat gender specially.
  • Embedding-level steering's symmetric neutralization may be explained by steering acting as noise injection rather than as a targeted cancellation of a gendered subspace; a test would compare it against adding random noise of matched magnitude to the same embedding.
  • If gender and term-matching directions are non-orthogonal in the head activation space, then a decomposition of the steering vector into a gender component and a term-matching component would predict which steering coefficients preserve retrieval performance; the paper leaves this as an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents a mechanistic interpretability study of gender sensitivity in five bi-encoder dense retrieval models. Using activation patching, path patching, mean ablation, and activation steering on the Grep-BiasIR dataset, the authors argue that gender-related score differences originate in the input embeddings and propagate through a small set of late-layer attention heads that jointly encode gender and term-matching signals. They further report that embedding-level steering non-specifically neutralizes score differences, while steering at the identified attention heads produces directional shifts favoring one gender or the other.

Significance. If the central claim holds, the paper would be a valuable step toward moving the study of gender bias in IR from behavioral measurement to mechanistic understanding. The use of causal patching methods, the inclusion of five bi-encoder models with different pooling architectures, and the public release of code and preprocessed data are concrete strengths. The finding that gender and term-matching signals are entangled in shared attention heads, if robust, would have direct implications for targeted debiasing. However, the empirical support for the specific five-head circuit is currently underpowered: the reported patching effects are small, no confidence intervals or significance tests are provided, and the head-selection rule is not validated against a null baseline. The steering experiments also appear to evaluate on data used to construct the steering vectors. These issues are fixable and do not undermine the value of the research question, but they need to be addressed before the mechanistic claims can be accepted.

major comments (4)
  1. [Section 4.1, Appendix D.2, Table 3] The identification of the 'small set of late-layer attention heads' (4.1, 4.5, 4.10, 5.3, 5.11) relies on a 2SD inclusion rule applied to path-patching effects, but no null baseline is reported. For a model with roughly 72 attention heads, a Gaussian null would be expected to produce a few heads above 2SD by chance. Table 3 reports raw effects of 0.02 to 0.08 with many blank entries, no confidence intervals, and no multiple-comparison correction. The claim that gender sensitivity is mediated by this specific small circuit is therefore not yet distinguished from a threshold artifact. A permutation baseline, a random-head control, or at least per-head confidence intervals is needed before the central mechanistic claim can be accepted.
  2. [Section 4.1, Appendix E, Figure 13] Mean ablation is applied to exactly the heads selected by path patching and evaluated on the same contrast pairs used for selection; this does not test selectivity. A random same-size set of heads could produce a comparable reduction, and no such control is reported. In addition, after observing that ablation fails for the largest MvN and FvN differences, the authors filter out documents where the neutral term appears in the query and then show a more uniform ablation effect (Figure 13). This post hoc filtering should be validated on a held-out split or with a pre-specified criterion, otherwise the causal role of the identified heads is not convincingly established.
  3. [Section 5, Appendix F.1] The steering vectors are computed as the mean activation difference over document pairs from Grep-BiasIR, and the evaluation appears to use the same dataset; the 'random subset (n=100)' is not stated to be disjoint from the evaluation set. This creates a circularity risk for the reported steering effects. Moreover, no significance tests or retrieval-quality metrics (e.g., nDCG on a labeled set) are provided, so the claim that attention-level steering is a 'promising inference-time intervention' is not quantitatively supported. The paper's own caveat that embedding-level steering may act as noise injection underscores the need for a more careful evaluation design.
  4. [Appendix A.1, Table 2] The preprocessing pipeline rewrites benchmark documents by substituting synonyms to equalize tokenization, inserting neutral tokens, padding with filler tokens, and filtering gendered proper names. The paper does not validate that these rewritten variants preserve the original gender-sensitivity signal or the original ranking behavior. Since all circuit analyses are performed on the modified dataset, any distortion introduced by preprocessing would propagate through every conclusion. A validation experiment comparing score differences and patching effects on original versus preprocessed variants should be reported.
minor comments (6)
  1. [Section 1] There is a missing space in 'consideredsocially problematic' in the Introduction.
  2. [Table 1] In the example document, 'a well as a high average salary' should read 'as well as a high average salary'.
  3. [Figure 4] The figure does not label its axes or units, so the reader cannot determine what quantity is plotted (score difference, preference rate, or something else).
  4. [Appendix B.2, Equation (1)] The normalization factor sqrt(1 + (s(Xj) - s(Xi))^2) is introduced without explanation; please define the motivation and cite the specific source.
  5. [Appendix D.2] The 2SD threshold description should state whether the mean and standard deviation are computed across heads, across comparisons, or pooled over both, and whether the threshold is applied per comparison or globally.
  6. [Section 5] The sentence 'if the two signals are not orthogonal' should be clarified: earlier results suggest the signals are jointly encoded, so the paper should state whether orthogonality was measured or assumed.

Circularity Check

1 steps flagged · score 4.0 of 10

Attention-level steering's directional shift is the sign of a vector fit to the same contrastive pairs; the central causal localization remains independent.

  1. fitted input called prediction [Section 5 (Results) and Appendix F.1]
    "For each comparison, the steering vector is the mean difference in activations between the two variants, computed over the relevant document pairs. ... attention-level steering directs model behavior: α= 5 favors Df (points move toward the lower right), and α=−5 favors Dm. Steering at this level produces the expected directional shift."

    The steering vector v is fit as the mean female-minus-male activation difference over the same contrastive pairs whose score differences are the target. Adding α·v (or −α·v) to the identified heads injects the exact signed direction fitted to those pairs, so observing that α=5 favors Df and α=−5 favors Dm is the sign of the fitted vector rather than an independent prediction. The heads were themselves selected by path-patching effect on the same gender comparisons, so the attention-level steering 'result' is a consistency check on the fitting data, not a held-out test.

full rationale

The central mechanistic localization is not circular: activation and path patching use contrastive pairs from the external Grep-BiasIR benchmark and identify heads via an explicit causal metric whose formula is given in Appendix B.2, rather than imported by self-citation. The self-citations (MechIR, the Polyakov et al. metric, and Lu et al.'s 'relevance scoring heads') are implementation details or analogies, not load-bearing uniqueness claims. The main circularity concern is the steering intervention: the steering vector is the mean activation difference on the same contrastive pairs, and the attention-level directional shift reported as a finding is the sign of that fitted vector, so RQ3's headline steering effect is partly forced by construction. The 2SD head-selection rule and mean-ablation confirmation also share the same pairs and lack a null baseline; this is a statistical validity risk rather than a definitional reduction, so it is not scored as a separate circular step. Overall, one auxiliary 'prediction' reduces to its fitted input while the central claim remains independently grounded, giving a partial-circularity score of 4.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The analysis depends on the circuit-analysis premise that interchange interventions localize causal components, on the benchmark premise that variants are otherwise identical, and on the normative premise that gender-irrelevant queries should not rank variants differently. Two hand-chosen parameters enter: the 2SD head-inclusion threshold and the steering coefficient alpha. The paper introduces no new entities, mediators, or forces.

free parameters (2)
  • Path patching inclusion threshold = 2 standard deviations above mean patching effect
    Heads are considered part of the computational path if their patching effect is at least 2 SD from the mean (Appendix D.2). This hand-chosen threshold determines the reported head set; with small effects (0.02 to 0.08), the selection is sensitive to this choice.
  • Steering coefficient alpha = Swept over [-5,5] in 0.25 increments; example values 5 and -5
    ActAdd steering magnitude is a user-chosen hyperparameter, not fit to an independent objective. The qualitative conclusion (directional shift at attention level) depends on the coefficient range and sign convention.
assumptions (4)
  • domain assumption Contrastive pairs that differ by a single gendered token isolate gender as the sole cause of score differences; query-relevant content is otherwise identical.
    Central to all comparisons and patching. Stated in Section 3 and based on Grep-BiasIR construction; preprocessing (Appendix A) is needed to maintain this.
  • domain assumption Activation patching and path patching with MLPs recomputed provide faithful causal attribution of the final relevance score to internal components.
    Circuit analysis methods assume interchange interventions approximate model computation; the paper adopts them from prior work (Meng et al. 2022; Wang et al. 2023; Polyakov et al. 2025).
  • domain assumption The identified late-layer attention heads are the causal carriers of gender sensitivity, not merely correlates selected by a threshold.
    Head selection uses a 2SD threshold on patching effects; ablation confirms on the same data but does not validate against alternative head sets or noise controls.
  • ad hoc to paper For gender-neutral queries, changing gendered wording should not change ranking; deviations are bias rather than legitimate relevance.
    Normative premise stated in Section 1; it defines what counts as bias, with exceptions for sex-specific health queries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Mechanistic Analysis of Gender Sensitivity in Dense Retrieval Models." pith.science (2026). https://pith.science/paper/TR7SP7J6

@misc{pith2026260805467,
  author       = {Pith},
  title        = {Pith review of: A Mechanistic Analysis of Gender Sensitivity in Dense Retrieval Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TR7SP7J6}},
  note         = {Machine review of arXiv:2608.05467}
}
read the original abstract

While gender bias in dense retrieval models is well documented, with prior work showing that models often score male-gendered documents higher than female or neutral variants, the internal mechanisms producing these disparities are poorly understood. In this paper, we mechanistically analyze bi-encoder models to localize gender sensitivity, finding that the signal originates in input embeddings and propagates through a small set of late-layer attention heads that carry both gender and term-matching signals. Guided by these findings, we test steering interventions at both identified points and find distinct effects: embedding-level steering non-specifically neutralizes score differences, while attention-level steering produces directional shifts. Our findings provide a mechanistic basis for targeted debiasing and highlight the challenge of disentangling gender from relevance signals in shared model components.

Figures

Figures reproduced from arXiv: 2608.05467 by the authors.

Figure 1
Figure 1. Activation patching effects averaged across [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Attention from the CLS (DistilBERTmulti-qa, top) and gendered tokens (DistilBERTms-marco, bottom) to other tokens at heads identified via path patching. To further characterize this behavior, we exam￾ine attention from the token positions identified by activation patching as the most important: the CLS token (CLS pooled) and gendered tokens (mean pooled). We examine attention to five token cate￾gories: CLS, SEP, gen… view at source ↗
Figure 4
Figure 4. DistilBERTms-marco steered along the MvF direction at the embedding (left) and late-layer attention (right) levels, compared to the baseline (center). Together, these patterns suggest that the iden￾tified heads carry both gender and relevance sig￾nals, functioning not as a dedicated gender circuit, but more as general aggregation sites where mul￾tiple signals converge. Lu et al. (2025) observe a similar pattern in a… view at source ↗
Figures from the paper (16 more)
Figure 5
Figure 5. Figure 5: Per-document score differences ∆f−n (x-axis) and ∆m−n (y-axis) for all models. Points above the red y = x diagonal indicate that Dm scores higher than Df relative to Dn. E Full Ablation Results To confirm the causal role of the heads identified by path patching, we mea…
Figure 6
Figure 6. Figure 6: Activation patching results for TAS-Bms-marco [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Activation patching results for DistilBERT [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Activation patching results for MiniLMmulti-qa [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Activation patching results for DistilBERT [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Activation patching results for BERTms-marco [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Path patching to the final residual stream for all models. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Mean ablating identified heads from path patching; top 30 percent of score differences. [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Mean ablation results filtering out documents where the neutral variant’s gendered term appears in the [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 14
Figure 14. Figure 14: Embedding steering results for all five models for all comparison directions, with [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Head steering results for all five models for all comparison directions, with [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Steering effects on the embeddings (top) compared to attention (bottom) for TAS-B [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]
Figure 17
Figure 17. Figure 17: Steering effects on the embeddings (top) compared to attention (bottom) for DistilBERT [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: Steering effects on the embeddings (top) compared to attention (bottom) for Mini-LM [PITH_FULL_IMAGE:figures/full_fig_p016_18.png]
Figure 19
Figure 19. Figure 19: Steering effects on the embeddings (top) compared to attention (bottom) for DistilBERT [PITH_FULL_IMAGE:figures/full_fig_p017_19.png]
Figure 20
Figure 20. Figure 20: Steering effects on the embeddings (top) compared to attention (bottom) for BERT [PITH_FULL_IMAGE:figures/full_fig_p017_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 26 canonical work pages

  1. [1]

    Sentence- BERT : Sentence Embeddings using Siamese BERT -Networks

    Reimers, Nils and Gurevych, Iryna. Sentence- BERT : Sentence Embeddings using Siamese BERT -Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. 2019

    2019

  2. [2]

    Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Axiomatic causal interventions for reverse engineering relevance computation in neural retrieval models , author=. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    work page

  3. [3]

    Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of

    Lu, Meng and Chen, Catherine and Eickhoff, Carsten , booktitle=. Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of

    work page

  4. [4]

    arXiv preprint arXiv:2308.10248 , year=

    Steering language models with activation engineering , author=. arXiv preprint arXiv:2308.10248 , year=

    arXiv

  5. [5]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Multi-attribute steering of language models via targeted intervention , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    work page

  6. [6]

    2026 , url=

    Jiachen Zhao and Jing Huang and Zhengxuan Wu and David Bau and Weiyan Shi , booktitle=. 2026 , url=

    work page 2026

  7. [7]

    Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Societal biases in retrieved contents: Measurement framework and adversarial mitigation of bert rankers , author=. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    work page

  8. [8]

    , author=

    Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases. , author=. EDBT , pages=

    work page

Show all 36 references
  1. [9]

    Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Mitigating bias in search results through contextual document reranking and neutrality regularization , author=. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  2. [10]

    European Conference on Information Retrieval , pages=

    A light-weight strategy for restraining gender biases in neural rankers , author=. European Conference on Information Retrieval , pages=. 2022 , organization=

    2022

  3. [11]

    European Conference on Information Retrieval , pages=

    Exploring gender biases in information retrieval relevance judgement datasets , author=. European Conference on Information Retrieval , pages=. 2021 , organization=

    2021

  4. [12]

    International Workshop on Algorithmic Bias in Search and Recommendation , pages=

    Do Perceived Gender Biases in Retrieval Results Affect Relevance Judgements? , author=. International Workshop on Algorithmic Bias in Search and Recommendation , pages=. 2022 , organization=

    2022

  5. [13]

    Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Do neural ranking models intensify gender bias? , author=. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  6. [14]

    Proceedings of the 14th ACM Web Science Conference 2022 , pages=

    On the characteristics of ranking-based gender bias measures , author=. Proceedings of the 14th ACM Web Science Conference 2022 , pages=

    2022

  7. [15]

    arXiv preprint arXiv:2602.03422 , year=

    RankSteer: Activation Steering for Pointwise LLM Ranking , author=. arXiv preprint arXiv:2602.03422 , year=

  8. [16]

    and Saligrama, Venkatesh and Kalai, Adam T

    Bolukbasi, Tolga and Chang, Kai-Wei and Zou, James Y. and Saligrama, Venkatesh and Kalai, Adam T. , booktitle=. Man is to computer programmer as woman is to homemaker?

  9. [17]

    arXiv preprint arXiv:2501.14457 , year=

    Understanding and mitigating gender bias in llms via interpretable neuron editing , author=. arXiv preprint arXiv:2501.14457 , year=

    arXiv

  10. [18]

    Advances in Neural Information Processing Systems , volume=

    Investigating gender bias in language models using causal mediation analysis , author=. Advances in Neural Information Processing Systems , volume=

  11. [19]

    Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 ...

    2019

  12. [20]

    Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval , pages=

    An analysis of BERT in document ranking , author=. Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval , pages=

  13. [21]

    Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    On the orthogonality of bias and utility in ad hoc retrieval , author=. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  14. [22]

    Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Towards Best Practices of Axiomatic Activation Patching in Information Retrieval , author=. Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  15. [23]

    Krieg, Klara and Parada-Cabaleiro, Emilia and Medicus, Gertraud and Lesota, Oleg and Schedl, Markus and Rekabsaz, Navid , booktitle=

  16. [24]

    Gender Bias in Contextualized Word Embeddings

    Zhao, Jieyu and Wang, Tianlu and Yatskar, Mark and Cotterell, Ryan and Ordonez, Vicente and Chang, Kai-Wei. Gender Bias in Contextualized Word Embeddings. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human ...

    2019

  17. [25]

    Siu, Vincent and Crispino, Nicholas and Park, David and Henry, Nathan W and Wang, Zhun and Liu, Yang and Song, Dawn and Wang, Chenguang , journal=

  18. [26]

    Reverse-Engineering the Retrieval Process in

    Reusch, Anja and Belinkov, Yonatan , booktitle=. Reverse-Engineering the Retrieval Process in

  19. [27]

    How do large language models understand relevance?

    Liu, Qi and Duan, Haozhe and Mao, Jiaxin and Wen, Ji-Rong , journal=. How do large language models understand relevance?. 2025 , publisher=

    2025

  20. [28]

    Machine Learning , volume=

    Gender disentangled representation learning in neural rankers , author=. Machine Learning , volume=. 2025 , publisher=

    2025

  21. [29]

    Interpretability in the Wild: a Circuit for Indirect Object Identification in

    Kevin Ro Wang and Alexandre Variengien and Arthur Conmy and Buck Shlegeris and Jacob Steinhardt , booktitle=. Interpretability in the Wild: a Circuit for Indirect Object Identification in

  22. [30]

    arXiv preprint arXiv:2304.05969 , year=

    Localizing model behavior with path patching , author=. arXiv preprint arXiv:2304.05969 , year=

    arXiv

  23. [31]

    Locating and editing factual associations in

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , booktitle=. Locating and editing factual associations in

  24. [32]

    European Conference on Information Retrieval , pages=

    Measuring Bias in Search Results Through Retrieval List Comparison , author=. European Conference on Information Retrieval , pages=. 2024 , organization=

    2024

  25. [33]

    male nurse

    Kopeinik, Simone and Mara, Martina and Ratz, Linda and Krieg, Klara and Schedl, Markus and Rekabsaz, Navid , booktitle=. Show me a "male nurse"!

  26. [34]

    2025 , organization=

    Parry, Andrew and Chen, Catherine and Eickhoff, Carsten and MacAvaney, Sean , booktitle=. 2025 , organization=

    2025

  27. [35]

    2024 , journal=

    Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet , author=. 2024 , journal=

    2024

  28. [36]

    Advances in Neural Information Processing Systems , volume=

    Refusal in language models is mediated by a single direction , author=. Advances in Neural Information Processing Systems , volume=

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.