Pith. sign in

REVIEW 5 major objections 6 minor 31 references

A Cross-Cultural Comparison of LLM-based Public Opinion Simulation: Evaluating Chinese and U.S. Models on Diverse Societies

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read All evaluated LLMs, regardless of origin, default to overgeneralized responses and none shows a home-field advantage, the paper argues.

desk verdict A useful measurement study with one genuinely interesting finding (Qwen2.5's degenerate uniform responses) and a broader overgeneralization claim that is overstated given the evaluation design. read the letter →

arxiv 2506.21587 v2 pith:7QUTY2AL submitted 2025-06-17 cs.CL

Weihong Qi , Fan Huang , Jisun An , Haewoon Kwak This is my paper · ORCID
classification cs.CL
keywords largelanguagemodelspublicopinionsimulationDeepSeekculturalbiasdemographicovergeneralizationANES2020Zuobiaodataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a Chinese-built open-source model, DeepSeek, can simulate public opinion in China and the U.S. better than American-built models. It compares DeepSeek-V3 and DeepSeek-R1 with GPT-4o, Qwen2.5, and Llama-3.3 on thousands of survey responses from ANES and the Zuobiao dataset. The central finding is that no model has a home-field advantage: DeepSeek does not consistently beat U.S. models on Chinese opinion, and every model systematically overgeneralizes within demographic groups. The most striking evidence is Qwen2.5's apparent success on Chinese views about capitalism, which turns out to be one uniform answer returned for 99.89% of low-income prompts. The paper concludes that LLM-based opinion simulation cannot be trusted as a faithful demographic mirror without subgroup-level checks.

What carries the argument

The experimental setup uses persona prompts: each survey respondent's demographic attributes (race, gender, age, education, income, religion, ideology, and partisanship for the U.S.; gender, age, education, and income for China) are inserted into a fixed prompt, and the model must choose one of the survey's original answer options. Accuracy is then computed as the fraction of simulated answers exactly matching the human survey response, both overall and within demographic subgroups. The subgroup breakdown is the load-bearing tool because it exposes where aggregate accuracy hides bias. Confusion matrices comparing predicted versus true labels across subgroups are what reveal that Qwen2.5's high accuracy is produced by monotony rather than understanding.

What would settle it

Repeat the same demographic prompts with paraphrased versions of the survey questions that the models are unlikely to have memorized; if accuracy falls sharply, much of the measured skill is recall of training text rather than social understanding. Separately, measure answer diversity within each demographic cell: if models produce one dominant answer for nearly all members, the overgeneralization claim is directly confirmed.

Watch

Extended reading notes

Core claim

The paper's central claim is that all evaluated LLMs, regardless of cultural origin, fail to capture within-group diversity and default to overgeneralized responses. No consistent home-field advantage exists: DeepSeek's performance is comparable to or worse than U.S. models on Chinese topics, despite being developed in China. DeepSeek-V3's best U.S. result (abortion, accuracy 0.53 vs. max 0.46 for the other models) comes from accurately simulating Democrats and liberals while missing Republicans and conservatives. On Chinese capitalism opinion, DeepSeek-V3 scores 0.36 versus Qwen2.5's 0.54, but Qwen2.5's lead is an artifact: it emits the same response for 99.89% of low-income prompts. DeepSeek-R1 is largely unusable in this setup, with invalid responses averaging 57% and reaching 99.6% on one U.S. topic. The paper thus reframes the problem from a China-versus-West divide to a systemic failure of demographic sensitivity.

Load-bearing premise

The evaluation treats a model matching exactly one human survey answer as "correct" and never tests the alternative that models might have memorized the survey text, so the accuracy numbers could partly measure retrieval rather than demographic understanding.

Editorial extensions

If this is right

  • If all current LLMs overgeneralize within groups, aggregate accuracy scores for opinion simulation are misleading: a model can look accurate on average while erasing minority views.
  • DeepSeek's lack of home-field advantage means a model's cultural origin and developer location are not reliable predictors of how well it represents that culture's public opinion.
  • Reasoning-oriented models like DeepSeek-R1 cannot currently be used for opinion simulation without output-format adaptation, because refusal and invalid-response rates reach 99.6%.
  • Prompt language alone is not a fix: switching the Zuobiao prompts to Chinese improved some topics but did not consistently improve accuracy across models or issues.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: A natural next test is to ask each model for repeated draws per persona and compare the full response distribution to the survey distribution; this would explicitly measure whether models can produce within-group spread rather than a single default answer.
  • Editorial inference: Because the exact-match accuracy metric is never checked against a memorization test, the possibility remains that high scores partly reflect retrieval of survey data seen in training, not social reasoning, and the paper does not settle that.
  • Editorial inference: The overgeneralization finding extends to other synthetic-respondent uses, such as pre-testing survey questions or simulating deliberative polls, where flattened within-group opinion could silently bias the design.
  • Editorial inference: A practical extension is a diversity audit metric, such as the entropy of a model's answer distribution within each demographic cell, flagging low-entropy stereotyped responses even when overall accuracy is high.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper evaluates five LLMs (DeepSeek-R1, DeepSeek-V3, Qwen2.5, GPT-4o, Llama-3.3) on their ability to simulate individual survey responses from the 2020 ANES (U.S.) and the Zuobiao (China) datasets. The authors construct demographic prompts from each respondent's attributes, compute per-topic and per-subgroup accuracy against the original human responses, and analyze differences between models. They report that DeepSeek-V3 outperforms other models on the U.S. abortion item, driven by better simulation of Democratic and liberal respondents, but underperforms on the Chinese capitalism item, especially for low-income and non-college-educated groups. They further claim that all evaluated LLMs overgeneralize within demographic groups, with Qwen2.5's apparent success on capitalism being an artifact of returning a single uniform answer for 99.89% of low-income prompts and 96.15% of no-college prompts. The paper also tests whether Chinese prompts improve simulation accuracy for Chinese samples, finding no consistent improvement.

Significance. If the central overgeneralization claim were properly supported, the paper would make a valuable contribution to the growing literature on LLM-based public opinion simulation and demographic bias. The study is one of few to compare Chinese- and U.S.-developed models on both an American and a Chinese survey, and the confusion-matrix analysis of Qwen2.5 is a concrete and instructive artifact. However, the manuscript currently overstates its findings: the headline conclusion that 'all LLMs overgeneralize' is confounded with the evaluation protocol, and the quantitative claims of 'significant' differences are not backed by statistical inference. With the distributional and statistical revisions described below, the paper could provide a useful cautionary benchmark for this line of research.

major comments (5)
  1. [Section IV-A and Figure 2] The evaluation protocol forces within-cell uniformity for any deterministic model, which confounds the paper's central overgeneralization claim. Because each prompt contains only coarse demographics (gender, age, education, income; for ANES also race, religion, ideology, party) and the model is greedily decoded, every respondent in the same demographic cell receives the same predicted label. The reported finding that models 'default to consistent responses within groups' (Section IV-B, Section V) is therefore partly a consequence of the experimental design, not necessarily a property of the models' conditional distributions. To support the conclusion that LLMs overgeneralize relative to human populations, the paper must compare the model's conditional label distribution within a demographic cell to the human conditional distribution, for example by sampling multiple responses per prompt (with temperature or multiple seeds) and reporting output entropy or a divergence measure. As written, the quantitative evidence of uniformity (99.89% and 96.15% for Qwen2.5) shows only that one model ignores even the coarse features for those cells; the generalization to 'all evaluated LLMs' is not established.
  2. [Abstract and Section IV-A] The claim that DeepSeek-V3 'achieves a significantly higher accuracy of 0.53' on the abortion item is unsupported because no confidence intervals or significance tests are reported anywhere in the paper. With samples of N=2,457 (ANES) and N=2,000 (Zuobiao), the observed differences could be assessed with bootstrap confidence intervals or McNemar tests for paired comparisons. This applies equally to the subgroup comparisons in Tables I–IV and to the prompt-language comparisons in Section IV-C, which also use the word 'significant' without any inferential statistics.
  3. [Section III-A] The abstract states that 'DeepSeek-V3 performs best in simulating U.S. opinions on the abortion issue compared to other topics such as climate change, gun control, immigration, and services for same-sex couples.' This is contradicted by Figure 2, where for DeepSeek-V3 the abortion accuracy (0.53) is the lowest among the five U.S. topics (climate 0.65, gun control 0.71, immigration 0.59, same-sex couple service 0.73). The intended claim is presumably that DeepSeek-V3 outperforms the other models on the abortion item, not that abortion is its best topic. The abstract and any related summary statements must be corrected to avoid a misleading representation of the results.
  4. [Section IV-B] The paper never addresses the possibility that the ANES and Zuobiao survey questions, or even the full survey response distributions, overlap with the models' pretraining data. If any of the five models memorized these publicly available survey instruments, the reported accuracy would reflect retrieval rather than social understanding, and the comparison between models would be partially a memorization test. The authors should at least discuss this risk and, if feasible, provide a control, such as paraphrasing the question stems or checking model release dates against the survey data vintage.
  5. [Section IV-B] The statement that DeepSeek-V3's abortion accuracy is 'primarily because it more accurately simulates responses when provided with Democratic or liberal personas' is a post hoc subgroup observation, not an established causal mechanism. The paper does not test whether the Democratic/liberal advantage is statistically distinguishable from other correlates, nor does it attempt to separate persona effects from other demographic interactions. The causal language should be softened to a descriptive claim, or the authors should add a mediation-style analysis that actually tests the proposed explanation.
minor comments (6)
  1. [Figure 2] The label 'Indivisualism' is a typo; it should read 'Individualism.'
  2. [Section III-B] The phrase 'the designedprompt' is missing a space and should be 'the designed prompt.'
  3. [Section III-B] The prompt template is presented in a compact, run-on format; a structured display with placeholders on separate lines would improve readability.
  4. [Section IV-A] The sentence 'The model’s long-form reasoning style, which often results in responses exceeding the output window.' is a sentence fragment and should be merged with the following sentence or rewritten.
  5. [Section IV-A] The claim that 'the potential in-built censorship mechanism of LLMs ... appears to be the main cause' of DeepSeek-R1's invalid responses is speculative and not supported by any analysis in the paper; it should be labeled as a hypothesis or supported with evidence.
  6. [Section IV-C] The conclusion that 'Chinese prompts do not lead to consistent and significant improvements' uses 'significant' without reporting any significance tests, consistent with the broader statistical issue noted above.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: external survey benchmarks, no fitted inputs, and only a non-load-bearing self-citation.

full rationale

The paper is an empirical benchmark against external ground-truth datasets (ANES and Zuobiao) and does not fit any parameter to the target predictions. Accuracy in Eq. (1) and subgroup accuracy in Eq. (2) are standard pointwise comparison metrics evaluated on held-out survey responses; model outputs are not constructed from the labels, so no fitted input is being renamed as a prediction. The only self-citation, reference [22] by Qi et al., appears in a related-work sentence about political affiliations and institutions and is not load-bearing for the method, datasets, or conclusions. The central claim that LLMs overgeneralize within demographic groups is an interpretive reading of confusion matrices and subgroup accuracies rather than a mathematical identity. A validity concern exists: because prompts contain only demographic attributes and decoding is deterministic, models necessarily produce the same answer for identical prompts, which constrains observed within-cell diversity; however, this is a measurement-interpretation issue, not circularity. No equation in the paper reduces to its own input, and no uniqueness theorem or fitted parameter is invoked to force the conclusions. The paper is therefore self-contained against external benchmarks, and no circular step is identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new fitted parameters or invented entities. Its central claims depend on four domain assumptions about survey ground truth, prompt sufficiency, data contamination, and sample representativeness. The most fragile is the untested no-contamination assumption, because high simulation accuracy could be inflated by memorized survey content.

assumptions (4)
  • domain assumption Survey ground truth assumption: ANES and Zuobiao answers represent true public opinion, and exact match is a valid accuracy measure.
    Used in Eq. (1) and Section III-A; if responses contain noise or the model should sample from a distribution rather than match an individual, the metric mischaracterizes performance.
  • domain assumption Persona sufficiency: the demographic attributes included in prompts are enough to condition a realistic opinion.
    Section III-B prompt design; demographic features differ between countries and may omit important determinants such as region, urban or rural status, or religion for China, which could bias subgroup results.
  • domain assumption No training-data contamination: LLMs' pretraining corpora do not include the specific ANES and Zuobiao survey responses used for evaluation.
    Section III-A datasets; if models memorized survey content, accuracy reflects retrieval rather than opinion simulation. This is not tested in the paper.
  • domain assumption Representativeness of the sampled Zuobiao subset: a random sample of 2,000 from 470,000 individuals preserves population distributions.
    Section III-A; random sampling is reasonable, but no repeated sampling or replicate analysis is shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Cross-Cultural Comparison of LLM-based Public Opinion Simulation: Evaluating Chinese and U.S. Models on Diverse Societies." pith.science (2026). https://pith.science/paper/7QUTY2AL

@misc{pith2026250621587,
  author       = {Pith},
  title        = {Pith review of: A Cross-Cultural Comparison of LLM-based Public Opinion Simulation: Evaluating Chinese and U.S. Models on Diverse Societies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7QUTY2AL}},
  note         = {Machine review of arXiv:2506.21587}
}
read the original abstract

This study evaluates the ability of DeepSeek, an open-source large language model (LLM), to simulate public opinions in comparison to LLMs developed by major tech companies. By comparing DeepSeek-R1 and DeepSeek-V3 with Qwen2.5, GPT-4o, and Llama-3.3 and utilizing survey data from the American National Election Studies (ANES) and the Zuobiao dataset of China, we assess these models' capacity to predict public opinions on social issues in both China and the United States, highlighting their comparative capabilities between countries. Our findings indicate that DeepSeek-V3 performs best in simulating U.S. opinions on the abortion issue compared to other topics such as climate change, gun control, immigration, and services for same-sex couples, primarily because it more accurately simulates responses when provided with Democratic or liberal personas. For Chinese samples, DeepSeek-V3 performs best in simulating opinions on foreign aid and individualism but shows limitations in modeling views on capitalism, particularly failing to capture the stances of low-income and non-college-educated individuals. It does not exhibit significant differences from other models in simulating opinions on traditionalism and the free market. Further analysis reveals that all LLMs exhibit the tendency to overgeneralize a single perspective within demographic groups, often defaulting to consistent responses within groups. These findings highlight the need to mitigate cultural and demographic biases in LLM-driven public opinion modeling, calling for approaches such as more inclusive training methodologies.

Figures

Figures reproduced from arXiv: 2506.21587 by the authors.

Figure 1
Figure 1. Overview of the Evaluation Process for LLMs in Public Opinion [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Simulation results for US and Chinese samples in public opinions [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Confusion Matrices for Public Opinion Simulation Differences Using [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Simulation results for Chinese samples in public opinions with English and Chinese Prompts [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 17 canonical work pages

  1. [1]

    GPT-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., “GPT-4 technical report,” arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    ANES 2020 Time Series Study Full Release,

    American National Election Studies, “ANES 2020 Time Series Study Full Release,” Feb. 10, 2022 version. [Online]. Available: https://www.electionstudies.org, 2021

    work page 2020

  3. [3]

    Machine learning as a model for cultural learning: Teaching an algorithm what it means to be fat,

    A. Arseniev-Koehler and J. G. Foster, “Machine learning as a model for cultural learning: Teaching an algorithm what it means to be fat,” Sociological Methods & Research, vol. 51, no. 4, pp. 1484–1539, 2022

    work page 2022

  4. [4]

    Inducing Group Fairness in Prompt-Based Language Model Decisions

    J. Atwood, P. Lahoti, A. Balashankar, F. Prost, and A. Beirami, “Inducing group fairness in LLM-based decisions,” arXiv preprint arXiv:2406.16738, 2024

    work page Pith review arXiv 2024

  5. [5]

    Fairlearn: A toolkit for assessing and improving fairness in AI,

    S. Bird, M. Dud ´ık, R. Edgar, B. Horn, R. Lutz, V . Milan, M. Sameki, H. Wallach, and K. Walker, “Fairlearn: A toolkit for assessing and improving fairness in AI,” Microsoft, Tech. Rep. MSR-TR-2020-32, 2020

    work page 2020

  6. [6]

    Synthetic replacements for human survey data? The perils of large language models,

    J. Bisbee, J. D. Clinton, C. Dorff, B. Kenkel, and J. M. Larson, “Synthetic replacements for human survey data? The perils of large language models,” Political Analysis, vol. 32, no. 4, pp. 401–416, 2024

    work page 2024

  7. [7]

    As- sessing cross-cultural alignment between ChatGPT and human societies: An empirical study,

    Y . Cao, L. Zhou, S. Lee, L. Cabello, M. Chen, and D. Hershcovich, “As- sessing cross-cultural alignment between ChatGPT and human societies: An empirical study,” arXiv preprint arXiv:2303.17466, 2023

    arXiv 2023

  8. [8]

    Fairness in machine learning: A survey,

    S. Caton and C. Haas, “Fairness in machine learning: A survey,” ACM Computing Surveys, vol. 56, no. 7, pp. 1–38, 2024

    2024

Show all 31 references
  1. [9]

    The LLaMA 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al., “The LLaMA 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  2. [10]

    R. S. Erikson and K. L. Tedin, American Public Opinion: Its Origins, Content, and Impact. Routledge, 2019

    2019

  3. [11]

    From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models,

    S. Feng, C. Y . Park, Y . Liu, and Y . Tsvetkov, “From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models,” in Proc. 61st Annu. Meeting Assoc. Comput. Linguistics (ACL), 2023

    2023

  4. [12]

    Bias and fairness in large language models: A survey,

    I. O. Gallegos, R. A. Rossi, J. Barrow, M. M. Tanjim, S. Kim, F. Dernoncourt, T. Yu, R. Zhang, and N. K. Ahmed, “Bias and fairness in large language models: A survey,” Computational Linguistics, vol. 50, no. 3, pp. 1097–1179, 2024

    2024

  5. [13]

    What’s in a name? Audit- ing large language models for race and gender bias,

    A. Haim, A. Salinas, and J. Nyarko, “What’s in a name? Audit- ing large language models for race and gender bias,” arXiv preprint arXiv:2402.14875, 2024. Capitalism Foreign Aid Traditionalism Indivisualism Free Market 0.0 0.2 0.4 0.6 0.8 1.0Accuracy 0.36 0.42 0.52 0.29 0.29 0....

    2024 arXiv

  6. [14]

    How well do LLMs represent values across cultures? Empirical analysis of LLM responses based on Hofstede cultural dimensions,

    J. Kharchenko, T. Roosta, A. Chadha, and C. Shah, “How well do LLMs represent values across cultures? Empirical analysis of LLM responses based on Hofstede cultural dimensions,” arXiv preprint arXiv:2406.14805, 2024

    2024 arXiv

  7. [15]

    DeepSeek-V3 technical report,

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al., “DeepSeek-V3 technical report,” arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  8. [16]

    Assessing the fairness of AI systems: AI practitioners’ processes, challenges, and needs for support,

    M. Madaio, L. Egede, H. Subramonyam, J. Wortman Vaughan, and H. Wallach, “Assessing the fairness of AI systems: AI practitioners’ processes, challenges, and needs for support,” Proc. ACM Human- Computer Interaction, vol. 6, no. CSCW1, pp. 1–26, 2022

    2022

  9. [17]

    McCombs and S

    M. McCombs and S. Valenzuela, Setting the Agenda: Mass Media and Public Opinion. John Wiley & Sons, 2020

    2020

  10. [18]

    Having beer after prayer? Measuring cultural bias in large language models,

    T. Naous, M. J. Ryan, A. Ritter, and W. Xu, “Having beer after prayer? Measuring cultural bias in large language models,” arXiv preprint arXiv:2305.14456, 2023

    2023 arXiv

  11. [19]

    Bulletin of the Seventh National Population Census (No. 2),

    National Bureau of Statistics of China, “Bulletin of the Seventh National Population Census (No. 2),” 2021. [Online]. Available: https://www. stats.gov.cn/sj/tjgb/rkpcgb/qgrkpcgb/202302/t20230206 1902002.html. Accessed: Feb. 15, 2025

    2021

  12. [20]

    China’s ideological spectrum,

    J. Pan and Y . Xu, “China’s ideological spectrum,” The Journal of Politics, vol. 80, no. 1, pp. 254–273, 2018

    2018

  13. [21]

    Measuring Religion in China,

    Pew Research Center, “Measuring Religion in China,” 2023. [Online]. Available: https://www.pewresearch.org/religion/2023/08/30/measuring- religion-in-china/. Accessed: Feb. 15, 2025

    2023

  14. [22]

    Representation bias in political sample sim- ulations with large language models,

    W. Qi, H. Lyu, and J. Luo, “Representation bias in political sample sim- ulations with large language models,” arXiv preprint arXiv:2407.11409, 2024

    2024 arXiv

  15. [23]

    Fairness in serving large language models,

    Y . Sheng, S. Cao, D. Li, B. Zhu, Z. Li, D. Zhuo, J. E. Gonzalez, and I. Stoica, “Fairness in serving large language models,” in Proc. 18th USENIX Symp. Operating Systems Design and Implementation (OSDI), pp. 965–988, 2024

    2024

  16. [24]

    A framework for understanding unintended consequences of machine learning,

    H. Suresh and J. V . Guttag, “A framework for understanding unintended consequences of machine learning,” arXiv preprint arXiv:1901.10002, vol. 2, no. 8, p. 73, 2019

    1901 arXiv

  17. [25]

    Cultural bias and cultural alignment of large language models,

    Y . Tao, O. Viberg, R. S. Baker, and R. F. Kizilcec, “Cultural bias and cultural alignment of large language models,” PNAS Nexus, vol. 3, no. 9, p. pgae346, 2024

    2024

  18. [26]

    Systematic biases in LLM simulations of debates,

    A. Taubenfeld, Y . Dover, R. Reichart, and A. Goldstein, “Systematic biases in LLM simulations of debates,” arXiv preprint arXiv:2402.04049, 2024

    2024 arXiv

  19. [27]

    LLaMA: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, et al., “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  20. [28]

    Kelly is a warm person, Joseph is a role model: Gender biases in LLM- generated reference letters,

    Y . Wan, G. Pu, J. Sun, A. Garimella, K.-W. Chang, and N. Peng, “Kelly is a warm person, Joseph is a role model: Gender biases in LLM- generated reference letters,” arXiv preprint arXiv:2310.09219, 2023

    2023 arXiv

  21. [29]

    Qwen2.5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al., “Qwen2.5 technical report,” arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  22. [30]

    Justice or prejudice? Quantifying biases in LLM-as-a-judge,

    J. Ye, Y . Wang, Y . Huang, D. Chen, Q. Zhang, N. Moniz, T. Gao, W. Geyer, C. Huang, P.-Y . Chen, et al., “Justice or prejudice? Quantifying biases in LLM-as-a-judge,” arXiv preprint arXiv:2410.02736, 2024

    2024 arXiv

  23. [31]

    Cultural value differences of LLMs: Prompt, language, and model size,

    Q. Zhong, Y . Yun, and A. Sun, “Cultural value differences of LLMs: Prompt, language, and model size,” arXiv preprint arXiv:2407.16891, 2024. APPENDIX A. Survey Questions on Public Opinions Public Opinion Questions for the United States Sample. The following questions from the...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.