Pith. sign in

REVIEW 4 major objections 5 minor 70 references

Who Gets the Callback? Generative AI and Gender Bias

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Open-source LLMs show large, model-dependent gender bias in hiring callbacks.

desk verdict A large and serious audit whose headline cross-model comparison is not robust: prompt-order effects flip female callback rates by huge margins for most models, and only Llama-3.1 is stable enough for the paper's own deeper analyses. read the letter →

arxiv 2504.21400 v1 pith:3FQFDOHC submitted 2025-04-30 econ.GN cs.CLq-fin.EC

Sugat Chaturvedi , Rochana Chaturvedi This is my paper · ORCID
classification econ.GNcs.CLq-fin.EC
keywords largelanguagemodelsgenerativeAIalgorithmicbiasgenderdiscriminationoccupationalsegregationresumescreeningBigFivepersonalitytraitsjobpostings
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that open-source large language models, when used to shortlist candidates, reproduce and amplify gender bias even when the two candidates are explicitly equally qualified. Across 332,044 real job postings, six models were asked to choose between "Mr. X" and "Ms. X" for an interview callback; the female callback rate runs from 1.4% (Ministral) to 87.3% (Gemma), showing that the bias is model-dependent rather than a fixed property of the task. Most models steer women toward lower-wage, stereotypically female occupations and men toward higher-wage ones, producing occupational segregation and a female wage penalty in recommended callbacks. The bias is tied to wording: job ads mentioning empathy, writing, or flexibility raise the chance of a female recommendation, while coding, hardware, finance, supervision, and night shifts lower it. If correct, the result matters because the same models are already entering real screening pipelines, and a fairness fix that works for one model or prompt does not transfer to others.

What carries the argument

The machinery is a head-to-head audit design. Each job posting is fed through a fixed prompt that asks the model to choose one of two equally qualified applicants, "Mr. X" or "Ms. X"; responses are parsed by string matching, and the female callback rate is $\text{FCR}=N_{\text{Ms.}}/(N_{\text{Ms.}}+N_{\text{Mr.}})$. To measure segregation, postings are embedded with a sentence transformer and matched to the nearest 2018 Standard Occupational Classification occupation by cosine similarity, then aggregated into the dissimilarity index $D = \frac{1}{2}\sum_o \left|N_o^f/N^f - N_o^m/N^m\right|$. To compare models on equal footing, the female-callback probability from the model's tokens is thresholded at values $\rho\in[0,1]$ so that callback parity (50% female) is imposed before segregation and wage gaps are recomputed. The same thresholding is used to relate Big Five trait infusion—ten persona prompts, high and low on each of openness, conscientiousness, extraversion, agreeableness, and emotional stability—and 99 historical-figure personas to segregation and wage disparity.

What would settle it

Take a random sample of roughly 1,000 postings, have trained coders assign 2018 SOC codes by hand, and compare those labels to the embedding-based mapping; if the agreement rate is low or the mapping errors correlate with the words that drive female callbacks, the segregation and wage-gap estimates would not survive.

Watch

Extended reading notes

Core claim

The central discovery is that the gender of an equally qualified applicant changes whether an LLM grants a callback, and the direction and size of that effect depend on which model is asked. On the same set of job postings, female callback rates are 1.39% for Ministral, 17.30% for Qwen, 41.02% for Llama-3.1, 61.33% for Granite, 73.24% for Llama-3, and 87.33% for Gemma; the most balanced model also abstains most often, refusing a gendered choice on 5.88% of postings. Models sort applicants by occupation even after the aggregate imbalance is removed: the dissimilarity index across six-digit SOC occupations is 8.25% for Llama-3.1 but reaches 49.58% for Ministral, and at callback parity the index ranges from roughly 21% to 38%. Most models post a female wage penalty, with women recommended for jobs paying 4.1 to 22.7 log points less; two models show a female wage premium, but only because their female callback rates are extremely low or skewed. The paper also claims that the behavior is partly an agreeableness and compliance effect: models agree with explicit gender requests in 55–92% of cases (Cohen's $\kappa$), and a deliberately less agreeable persona refuses on ethical grounds and reduces occupational segregation, while high openness and high conscientiousness increase it.

Load-bearing premise

The load-bearing premise is that embedding-based matching of each job posting to a U.S. occupation code is accurate enough that the measured occupation-level callback patterns actually reflect model behavior rather than misclassification.

Editorial extensions

If this is right

  • If LLMs are used to shortlist real applicants, equally qualified men and women will receive systematically different callback rates, with the direction determined by the model choice.
  • Deployment without intervention would reproduce occupational segregation: women would be steered toward care, writing, and administrative roles while men are steered toward technical, financial, and supervisory roles.
  • Models comply with explicit gender preferences in job ads; since such requests are rare (about 2% of postings) yet followed at high rates, even small amounts of gendered ad text can bias the pipeline.
  • Calibrating a decision threshold to equalize aggregate callback rates is not enough: segregation remains at 21–38% at callback parity, so fairness requires occupation-level monitoring.
  • Persona steering changes callback rates by tens of percentage points; any audit result is therefore tied to the prompt, and "debiased" prompting must be validated on the target corpus.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit: the same lexical attribution scores could be used pre-deployment to flag job advertisements whose wording predicts biased model callbacks, turning the audit into a text-screening tool.
  • Because the occupation mapping has no human validation, a natural extension is to benchmark embedding-based SOC assignment against manual coding; the paper's segregation and wage conclusions would be strengthened or weakened by that check.
  • If the agreeableness-bias explanation holds, model updates that change RLHF reward tuning could move callback rates without changing the prompt; a testable prediction is that "helpful and harmless" fine-tuned variants will show different segregation indices than base models.
  • The design compares one male and one female name only; extending the audit to multiple names, caste or religion cues, and intersectional identities would tell whether the gender gap is additive or interacts with other protected attributes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper audits six mid-sized open-source LLMs for gender bias in hiring recommendations. For each of 332,044 real job postings from India's National Career Services portal, the authors prompt each model to choose between two equally qualified candidates, Mr. X and Ms. X, and measure the female callback rate (FCR). They report large cross-model variation (FCR from 1.4% to 87.3%), compute a dissimilarity index of occupational segregation, estimate posted-wage gaps between jobs recommended to women versus men, analyze linguistic correlates of recommendations, and probe how Big Five personality infusions and historical-figure personas change model behavior. The central claim is that most models favor men, especially for higher-wage roles, and that recommendations reproduce occupational gender segregation.

Significance. The scale of the audit (over 40 million LLM queries on real job advertisements) and the direct elicitation of gendered recommendations are notable strengths, moving beyond small benchmark studies. If the headline results were robust, this would be an important contribution to the algorithmic-bias and labor-economics literatures. However, the lack of validation of the response parser and the SOC mapping, and especially the extreme sensitivity of the headline numbers to prompt order, mean that the cross-model claims are not currently supported.

major comments (4)
  1. [Section 4.1 and Table 1] The order-reversal robustness check reported in the text shows that reversing the order of 'Mr. X' and 'Ms. X' in Prompt A.1 changes the female callback rate by 18 to 98 percentage points for four of the six models (Ministral: 1.39% to 99.86%; Llama-3: 73.24% to 99.46%; Gemma: 87.33% to 99.17%; Granite: 61.33% to 79.64%). This demonstrates that for most models the outcome variable is dominated by the position of the gendered label rather than by a stable preference between equally qualified candidates, so the headline range of 1.4% to 87.3% in Table 1 and the abstract's claim that 'most models tend to favor men' are not supported. The abstract is also internally inconsistent with Table 1, where three of the six models recommend women in 61% to 87% of cases. The cross-model comparisons of the dissimilarity index and wage gaps in Table 1 inherit this non-robustness because they are computed from the same order-sensitive callback labels.
  2. [Section 3.2] The mapping of job postings to 2018 SOC occupations is performed by taking the nearest O*NET occupation summary under cosine similarity of all-mpnet-base-v2 sentence embeddings, but no validation of mapping accuracy is reported. Because the dissimilarity index (Section 3.2) and the wage-gap regressions with occupation fixed effects (Section 3.3) depend entirely on this assignment, a misassignment rate of even a few percent could materially distort the occupational-segregation results and occupation-level wage comparisons. Please provide a human-annotated validation sample, report precision/recall of the mapping, and show robustness to alternative mapping methods (e.g., keyword matching or LLM-based classification).
  3. [Section 3.1] The parsing rule that classifies a response as male or female based on the presence of 'Mr.' or 'Ms.' is not validated. Models may answer with phrases such as 'the male candidate', 'candidate X', or 'Mr. Smith', which would be misclassified or counted as abstentions, and the abstention rate under the reversed-order prompt is not reported. Please validate the parser on a random sample of outputs (e.g., 200 responses per model) and report the classification accuracy and abstention disagreement rates.
  4. [Section 5 and Table 1] The paper itself cautions in Section 5 that 'simply looking at the output tokens across models without considering the token probabilities might be misleading,' yet Table 1 and the abstract present raw token-based female callback rates as the headline result. The thresholding exercise shows that the rank ordering of models by dissimilarity index changes materially when token probabilities are used (e.g., Granite and Qwen become the least segregated), so the raw-output presentation in Table 1 is not a reliable summary of model behavior. Please either present the probability-thresholded results as the main analysis or clearly state that Table 1 reflects a biased prompt-and-parsing protocol.
minor comments (5)
  1. [Abstract] The sentence 'most models tend to favor men' is contradicted by Table 1, where Gemma, Llama-3, and Granite favor women; please revise the abstract and introduction to be consistent with the results.
  2. [Equation 3.1] The dependent variable is written as ln(wage_ijst) and the treatment as Fcallback_ijst, but notation elsewhere uses Fcallback_{p,ijst}; please standardize the subscripts.
  3. [Section 4.3] Figure 2 reports a correlation of 83.95% between female callback rate and explicit requests for women at the 2-digit level, but the figure itself is not shown in the text; please add a pointer to the figure and describe the direction of the correlation.
  4. [References] The manuscript cites several working papers and unpublished manuscripts (e.g., Bafna et al. 2025, Chaturvedi et al. 2024b); please include DOIs or stable URLs where available, and clarify which parts of the analysis rely on data or code from the authors' prior work.
  5. [Section 7.2] The text states that Margaret Sanger appears both as a women's rights advocate with high female callback rates and as a controversial figure with high refusal rates (47.68%); please reconcile these two descriptions to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

Derivation is self-contained; callback rates are measured directly from model outputs, and the self-cited skill/word taxonomies serve only as predictors or external validation, not as inputs to the outcome.

full rationale

The paper's central outcome—the female callback rate—is defined in Section 3.1 directly from the model's own output tokens (the presence of "Mr." or "Ms." in the response), not derived from any fitted parameter or from the authors' prior results. The occupational segregation dissimilarity index and wage gaps are descriptive reweightings of these measured callbacks across SOC occupations and posted wages, so they inherit the properties of the direct measurement rather than being constructed from the paper's inputs. The self-citations that appear are not load-bearing in a circular sense: Chaturvedi et al. (2024a) supplies data-driven skill categories used only as predictors in Equation 3.2, and Chaturvedi et al. (2024b) supplies gendered-word lists from a separate job portal used as an external benchmark for the language analysis; neither defines the callback outcome. The occupation-mapping method is attributed to Bafna et al. (2025), but the cosine-similarity procedure is described in the paper itself and is not a result imported by citation. The paper even reports its own order-reversal robustness check (Section 4.1), showing large positional sensitivity for several models; that is a serious validity limitation for the cross-model headline comparison, but it is not circularity because the outcome is still measured, not constructed. Similarly, the unmapped SOC assignment and the lack of validation noted in Section 3.2 are accuracy risks, not circular reductions. No equation in the paper is equivalent to its own input, and no fitted parameter is renamed as a prediction. Accordingly, the derivation chain is self-contained, and the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

This paper is an empirical audit; it does not derive a model or introduce fitted parameters or invented entities. The mapping and sampling assumptions above are the main unstated premises. The skill categories and gendered-word lists from the authors' own prior work are inputs, not fitted outputs of this paper.

assumptions (4)
  • domain assumption LLM inference is effectively deterministic, or single-run outputs are representative of the model's behavior
    Sections 3.1-3.6 issue each prompt once; temperature, top-p, and number of runs are not reported, so the reported rates assume negligible sampling variance.
  • domain assumption Cosine similarity of all-mpnet-base-v2 embeddings accurately maps job postings to the correct O*NET/SOC occupation
    Section 3.2 assigns each posting to the nearest occupation summary; no validation or accuracy statistics are provided.
  • domain assumption The NCS portal job postings are a valid sample of real-world vacancies for measuring LLM hiring recommendations
    Section 2: the portal skews toward service-sector, higher-wage roles (mean annual wage INR 278,041); external validity of the bias magnitudes depends on this sample.
  • domain assumption The Big Five trait descriptions (P2 method of Jiang et al. 2023) genuinely induce the stated personality in the model
    Section 3.5; the paper relies on prior work that these prompts are effective and interprets behavior changes as trait effects.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Who Gets the Callback? Generative AI and Gender Bias." pith.science (2026). https://pith.science/paper/3FQFDOHC

@misc{pith2026250421400,
  author       = {Pith},
  title        = {Pith review of: Who Gets the Callback? Generative AI and Gender Bias},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3FQFDOHC}},
  note         = {Machine review of arXiv:2504.21400}
}
read the original abstract

Generative artificial intelligence (AI), particularly large language models (LLMs), is being rapidly deployed in recruitment and for candidate shortlisting. We audit several mid-sized open-source LLMs for gender bias using a dataset of 332,044 real-world online job postings. For each posting, we prompt the model to recommend whether an equally qualified male or female candidate should receive an interview callback. We find that most models tend to favor men, especially for higher-wage roles. Mapping job descriptions to the Standard Occupational Classification system, we find lower callback rates for women in male-dominated occupations and higher rates in female-associated ones, indicating occupational segregation. A comprehensive analysis of linguistic features in job ads reveals strong alignment of model recommendations with traditional gender stereotypes. To examine the role of recruiter identity, we steer model behavior by infusing Big Five personality traits and simulating the perspectives of historical figures. We find that less agreeable personas reduce stereotyping, consistent with an agreeableness bias in LLMs. Our findings highlight how AI-driven hiring may perpetuate biases in the labor market and have implications for fairness and diversity within firms.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 64 canonical work pages

  1. [1]

    Hallermeier, and A

    Abraham, L., J. Hallermeier, and A. Stein (2024): Words matter: Experimental evidence from job applications, Journal of Economic Behavior & Organization, 225, 348--391

    work page 2024

  2. [2]

    Adida, C. L., D. D. Laitin, and M.-A. Valfort (2010): Identifying barriers to Muslim integration in France, Proceedings of the National Academy of Sciences, 107, 22384--22390

    work page 2010

  3. [3]

    Aher, G. V., R. I. Arriaga, and A. T. Kalai (2023): Using large language models to simulate multiple humans and replicate human subject studies, in International Conference on Machine Learning, PMLR, 337--371

    work page 2023

  4. [4]

    Larson, S

    Angwin, J., J. Larson, S. Mattu, and L. Kirchner (2022): Machine bias, in Ethics of data and analytics, Auerbach Publications, 254--264

    work page 2022

  5. [5]

    Argyle, L. P., E. C. Busby, N. Fulda, J. R. Gubler, C. Rytting, and D. Wingate (2023): Out of one, many: Using language models to simulate human samples, Political Analysis, 31, 337--351

    work page 2023

  6. [6]

    Armstrong, L., A. Liu, S. MacNeil, and D. Metaxa (2024): The Silicon Ceiling: Auditing GPT’s Race and Gender Biases in Hiring, in Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, 1--18

    work page 2024

  7. [7]

    Chaturvedi, K

    Bafna, T., S. Chaturvedi, K. Mahajan, and S. Tomar (2025): The Evolving Nature of Work: Improving Occupation Mapping with Large Language Models, Unpublished manuscript

    work page 2025

  8. [8]

    Bertrand, M. and S. Mullainathan (2004): Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination, American economic review, 94, 991--1013

    work page 2004

Show all 70 references
  1. [9]

    Blandin, and D

    Bick, A., A. Blandin, and D. J. Deming (2024): The rapid adoption of generative ai, Tech. rep., National Bureau of Economic Research

    2024

  2. [10]

    Chang, J

    Bolukbasi, T., K.-W. Chang, J. Y. Zou, V. Saligrama, and A. T. Kalai (2016): Man is to computer programmer as woman is to homemaker? debiasing word embeddings, Advances in neural information processing systems, 29

    2016

  3. [11]

    Booth, A. and A. Leigh (2010): Do employers discriminate by gender? A field experiment in female-dominated occupations, Economics Letters, 107, 236--238

    2010

  4. [12]

    Caliskan, A., J. J. Bryson, and A. Narayanan (2017): Semantics derived automatically from language corpora contain human-like biases, Science, 356, 183--186

    2017

  5. [13]

    Cao, X. and M. Kosinski (2024): Large language models and humans converge in judging public figures’ personalities, PNAS nexus, 3, pgae418

    2024

  6. [14]

    Chaturvedi, R. and S. Chaturvedi (2024): It’s all in the name: A character-based approach to infer religion, Political Analysis, 32, 34--49

    2024

  7. [15]

    Mahajan, and Z

    Chaturvedi, S., K. Mahajan, and Z. Siddique (2024 a ): Using Domain-Specific Word Embeddings to Examine the Demand for Skills, in Big Data Applications in Labor Economics, Part B, Emerald Publishing Limited, 171--223

    2024

  8. [16]

    --- -.1pt --- -.1pt --- (2024 b ): Words matter: Gender, jobs and applicant behavior, Jobs and Applicant Behavior (February 18, 2024)

    2024

  9. [17]

    Chen, L., R. Ma, A. Hann \'a k, and C. Wilson (2018): Investigating the impact of gender on rank in resume search engines, in Proceedings of the 2018 chi conference on human factors in computing systems, 1--14

    2018

  10. [18]

    Costa, P. T. and R. R. McCrae (1999): A five-factor theory of personality, Handbook of personality: Theory and research, 2, 1999

    1999

  11. [19]

    (2022): Amazon scraps secret AI recruiting tool that showed bias against women, in Ethics of data and analytics, Auerbach Publications, 296--299

    Dastin, J. (2022): Amazon scraps secret AI recruiting tool that showed bias against women, in Ethics of data and analytics, Auerbach Publications, 296--299

    2022

  12. [20]

    Deming, D. and L. B. Kahn (2018): Skill requirements across firms and labor markets: Evidence from job postings for professionals, Journal of Labor Economics, 36, S337--S369

    2018

  13. [21]

    Murahari, T

    Deshpande, A., V. Murahari, T. Rajpurohit, A. Kalyan, and K. Narasimhan (2023): Toxicity in chatgpt: Analyzing persona-assigned language models, in Findings of the Association for Computational Linguistics: EMNLP 2023, ed. by H. Bouamor, J. Pino, and K. Bali, Singapore: Associ...

    2023

  14. [22]

    Ekehammar, B. and N. Akrami (2007): Personality and prejudice: From Big Five personality factors to facets, Journal of personality, 75, 899--926

    2007

  15. [23]

    Manning, P

    Eloundou, T., S. Manning, P. Mishkin, and D. Rock (2024): GPTs are GPTs: Labor market impact potential of LLMs, Science, 384, 1306--1308

    2024

  16. [24]

    Flory, J. A., A. Leibbrandt, and J. A. List (2015): Do competitive workplaces deter female workers? A large-scale natural field experiment on job entry decisions, The Review of Economic Studies, 82, 122--155

    2015

  17. [25]

    Goldsmith-Pinkham, T

    Fuster, A., P. Goldsmith-Pinkham, T. Ramadorai, and A. Walther (2022): Predictably unequal? The effects of machine learning on credit markets, The Journal of Finance, 77, 5--47

    2022

  18. [26]

    Gaebler, J. D., S. Goel, A. Huq, and P. Tambe (2024): Auditing the Use of Language Models to Guide Hiring Decisions, arXiv preprint arXiv:2404.03086

    2024 arXiv

  19. [27]

    Askell, N

    Ganguli, D., A. Askell, N. Schiefer, T. I. Liao, K. Lukošiūtė, A. Chen, A. Goldie, A. Mirhoseini, C. Olsson, D. Hernandez, D. Drain, D. Li, E. Tran-Johnson, E. Perez, J. Kernion, J. Kerr, J. Mueller, J. Landau, K. Ndousse, K. Nguyen, L. Lovitt, M. Sellitto, N. Elhage, N. Merca...

    2023

  20. [28]

    Schiebinger, D

    Garg, N., L. Schiebinger, D. Jurafsky, and J. Zou (2018): Word embeddings quantify 100 years of gender and ethnic stereotypes, Proceedings of the National Academy of Sciences, 115, E3635--E3644

    2018

  21. [29]

    Friesen, and A

    Gaucher, D., J. Friesen, and A. C. Kay (2011): Evidence that gendered wording in job advertisements exists and sustains gender inequality. Journal of personality and social psychology, 101, 109

    2011

  22. [30]

    Gee, L. K. (2019): The more you know: Information effects on job application rates in a large field experiment, Management Science, 65, 2077--2094

    2019

  23. [31]

    Gonen, H. and Y. Goldberg (2019): Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langu...

    2019

  24. [32]

    Gosling, S. D., P. J. Rentfrow, and W. B. Swann Jr (2003): A very brief measure of the Big-Five personality domains, Journal of Research in personality, 37, 504--528

    2003

  25. [33]

    Greenwald, D. L., S. T. Howell, C. Li, and E. Yimfor (2024): Regulatory arbitrage or random errors? implications of race prediction algorithms in fair lending analysis, Journal of Financial Economics, 157, 103857

    2024

  26. [34]

    Shrivastava, A

    Gupta, S., V. Shrivastava, A. Deshpande, A. Kalyan, P. Clark, A. Sabharwal, and T. Khot (2024): Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs, in The Twelfth International Conference on Learning Representations

    2024

  27. [35]

    Tamkin, M

    Handa, K., A. Tamkin, M. McCain, S. Huang, E. Durmus, S. Heck, J. Mueller, J. Hong, S. Ritchie, T. Belonax, et al. (2025): Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations, arXiv preprint arXiv:2503.04761

    2025 arXiv

  28. [36]

    Neumark, and Q

    He, H., D. Neumark, and Q. Weng (2021): Do workers value flexible jobs? A field experiment, Journal of Labor Economics, 39, 709--738

    2021

  29. [37]

    Hofmann, V., P. R. Kalluri, D. Jurafsky, and S. King (2024): AI generates covertly racist decisions about people based on their dialect, Nature, 633, 147--154

    2024

  30. [38]

    Horton, J. J. (2023): Large language models as simulated economic agents: What can we learn from homo silicus? Tech. rep., National Bureau of Economic Research

    2023

  31. [39]

    Brahman, V

    Huang, T., F. Brahman, V. Shwartz, and S. Chaturvedi (2021): Uncovering Implicit Gender Bias in Narratives through Commonsense Inference, in Findings of the Association for Computational Linguistics: EMNLP 2021, 3866--3873

    2021

  32. [40]

    Xu, S.-C

    Jiang, G., M. Xu, S.-C. Zhu, W. Han, C. Zhang, and Y. Zhu (2023): Evaluating and inducing personality in pre-trained language models, Advances in Neural Information Processing Systems, 36, 10622--10643

    2023

  33. [41]

    Zhang, X

    Jiang, H., X. Zhang, X. Cao, C. Breazeal, D. Roy, and J. Kabbara (2024): PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits, in Findings of the Association for Computational Linguistics: NAACL 2024, 3605--3627

    2024

  34. [42]

    John, O. P. and S. Srivastava (1999): The Big Five Trait Taxonomy: History, Measurement, and Theoretical Perspective, Hand Book of Personality: Theory and Research

    1999

  35. [43]

    Ju, H. and S. Aral (2025): Collaborating with AI Agents: Field Experiments on Teamwork, Productivity, and Performance, arXiv preprint arXiv:2503.18238

    2025

  36. [44]

    Kirk, H. R., Y. Jun, F. Volpin, H. Iqbal, E. Benussi, F. Dreyer, A. Shtedritski, and Y. Asano (2021): Bias out-of-the-box: An empirical analysis of intersectional occupational biases in popular generative language models, Advances in neural information processing systems, 34, ...

    2021

  37. [45]

    Kline, P., E. K. Rose, and C. R. Walters (2022): Systemic discrimination among large US employers, The Quarterly Journal of Economics, 137, 1963--2036

    2022

  38. [46]

    Dockum, and D

    Kotek, H., R. Dockum, and D. Sun (2023): Gender bias and stereotypes in large language models, in Proceedings of the ACM collective intelligence conference, 12--24

    2023

  39. [47]

    Shen, and S

    Kuhn, P., K. Shen, and S. Zhang (2020): Gender-targeted job ads in the recruitment process: Facts from a Chinese job board, Journal of Development Economics, 102531

    2020

  40. [48]

    Lambrecht, A. and C. Tucker (2019): Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads, Management science, 65, 2966--2981

    2019

  41. [49]

    Rathelot, and A

    Le Barbanchon, T., R. Rathelot, and A. Roulet (2021): Gender differences in job search: Trading off commute against wage, The Quarterly Journal of Economics, 136, 381--426

    2021

  42. [50]

    Leibbrandt, A. and J. A. List (2015): Do women avoid salary negotiations? Evidence from a large-scale natural field experiment, Management Science, 61, 2016--2024

    2015

  43. [51]

    Li, D., L. R. Raymond, and P. Bergman (2020): Hiring as exploration, Tech. rep., National Bureau of Economic Research

    2020

  44. [52]

    Manning, B. S., K. Zhu, and J. J. Horton (2024): Automated social science: Language models as scientist and subjects, Tech. rep., National Bureau of Economic Research

    2024

  45. [53]

    Bethke, and S

    Nadeem, M., A. Bethke, and S. Reddy (2021): StereoSet: Measuring stereotypical bias in pretrained language models, in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Process...

    2021

  46. [54]

    Powers, C

    Obermeyer, Z., B. Powers, C. Vogeli, and S. Mullainathan (2019): Dissecting racial bias in an algorithm used to manage the health of populations, Science, 366, 447--453

    2019

  47. [55]

    (2011): Why do skilled immigrants struggle in the labor market? A field experiment with thirteen thousand resumes, American Economic Journal: Economic Policy, 3, 148--171

    Oreopoulos, P. (2011): Why do skilled immigrants struggle in the labor market? A field experiment with thirteen thousand resumes, American Economic Journal: Economic Policy, 3, 148--171

    2011

  48. [56]

    Pennebaker, J., R. Boyd, R. Booth, A. Ashokkumar, and M. Francis (2022): Linguistic inquiry and word count: Liwc-22. pennebaker conglomerates,

    2022

  49. [57]

    Ringer, K

    Perez, E., S. Ringer, K. Lukosiute, K. Nguyen, E. Chen, S. Heiner, C. Pettit, C. Olsson, S. Kundu, S. Kadavath, et al. (2023): Discovering Language Model Behaviors with Model-Written Evaluations, in Findings of the Association for Computational Linguistics: ACL 2023, 13387--13434

    2023

  50. [58]

    Reimers, N. and I. Gurevych (2019): Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics

    2019

  51. [59]

    (2023): The role of the ask gap in gender pay inequality, The Quarterly Journal of Economics

    Roussille, N. (2023): The role of the ask gap in gender pay inequality, The Quarterly Journal of Economics

    2023

  52. [60]

    Rudman, L. A. and P. Glick (2021): The social psychology of gender: How power and intimacy shape gender relations, Guilford Publications

    2021

  53. [61]

    Salecha, A., M. E. Ireland, S. Subrahmanya, J. Sedoc, L. H. Ungar, and J. C. Eichstaedt (2024): Large language models display human-like social desirability biases in Big Five personality surveys, PNAS nexus, 3, pgae533

    2024

  54. [62]

    Durmus, F

    Santurkar, S., E. Durmus, F. Ladhak, C. Lee, P. Liang, and T. Hashimoto (2023): Whose opinions do language models reflect? in International Conference on Machine Learning, PMLR, 29971--30004

    2023

  55. [63]

    Si, C., Z. Gan, Z. Yang, S. Wang, J. Wang, J. L. Boyd-Graber, and L. Wang (2023): Prompting GPT-3 To Be Reliable, in The Eleventh International Conference on Learning Representations

    2023

  56. [64]

    Song, K., X. Tan, T. Qin, J. Lu, and T.-Y. Liu (2020): Mpnet: Masked and permuted pre-training for language understanding, Advances in Neural Information Processing Systems, 33, 16857--16867

    2020

  57. [65]

    Brenninkmeijer, A

    Tranchero, M., C.-F. Brenninkmeijer, A. Murugan, and A. Nagaraj (2024): Theorizing with large language models, Tech. rep., National Bureau of Economic Research

    2024

  58. [66]

    Veldanda, A. K., F. Grob, S. Thakur, H. Pearce, B. Tan, R. Karri, and S. Garg (2023): Are Emily and Greg still more employable than Lakisha and Jamal? Investigating algorithmic hiring bias in the era of ChatGPT, arXiv preprint arXiv:2310.05135

    2023 arXiv

  59. [67]

    Wiles, E. and J. J. Horton (2025): Generative ai and labor market matching efficiency, Available at SSRN 5187344

    2025

  60. [68]

    Munyikwa, and J

    Wiles, E., Z. Munyikwa, and J. Horton (2025): Algorithmic writing assistance on jobseekers’ resumes increases hires, Management Science

    2025

  61. [69]

    Zhang, S. and P. J. Kuhn (2024): Measuring Bias in Job Recommender Systems: Auditing the Algorithms, Tech. rep., National Bureau of Economic Research

    2024

  62. [70]

    Zhao, J., T. Wang, M. Yatskar, R. Cotterell, V. Ordonez, and K.-W. Chang (2019): Gender Bias in Contextualized Word Embeddings, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo...

    2019

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.