Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereotypes and Safeguards

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Persona assignment is a general, model-dependent lever for amplifying toxic generation in Chinese LLMs, raising toxicity by over 40-fold in specific persona–social group combinations and shifting refusal behavior systematically.

desk verdict Useful refusal analysis and a large new dataset, but the headline toxicity amplification ratios are partly a measurement artifact and should not be taken at face value. read the letter →

arxiv 2506.04975 v2 pith:37DIG5OO submitted 2025-06-05 cs.CY

Geng Liu , Li Feng , Carlo Alberto Bono , Songbo Yang , Mengxiao Zhu , Francesco Pierri This is my paper
classification cs.CY
keywords ChineseLLMspersonaassignmenttoxicityamplificationrefusalbehaviorstereotypessafetymechanismsLLMsafeguardsmitigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that telling a Chinese large language model to speak in a particular persona—'Speak exactly like P'—is a reliable way to push it past its safety guardrails, and that the resulting refusal and toxicity behaviors are structured rather than random. Across four Chinese LLMs and over 1.4 million generated texts, it finds that persona assignment amplifies toxic output relative to each model's own default baseline, with the largest effects (over 40-fold) emerging from specific persona–social group combinations such as negative personas aimed at identity- or appearance-related groups. It also finds that refusal behavior is unevenly distributed: female personas trigger more refusals than male ones, restrictive prompt templates ('Toxic', 'Harmful', 'Bad') raise refusal rates, and some social groups (sexual orientation, disability, disease) are protected far more than others (age, socioeconomic status). A complementary case study on one model shows that an external LLM evaluator giving iterative feedback can cut the highest toxicity scores substantially without retraining the model. These are among the first large-scale measurements of persona-driven risk in Chinese-language LLMs, and the results imply that safety evaluations for these widely deployed models must be culturally contextualized.

What carries the argument

The device that carries the argument is the persona-conditioned system prompt ('完全模仿𝑃的说话方式…', i.e., 'Speak exactly like P…'), combined with six prompt templates (Generic, Good, Bad, Negative, Harmful, Toxic) and 240 Chinese social groups in 13 categories that serve as the target S. The central measure is the toxicity ratio $r$, defined per matched (template, social group) pair as $r = \mathrm{toxicity}_{\mathrm{Persona}} / \mathrm{toxicity}_{\mathrm{Default}}$, with the median ratio within each (template, persona) cell encoding 'amplification' relative to a model's own baseline. Refusal is measured by a fine-tuned BERT classifier trained on Qwen outputs and applied uniformly across models for comparability, and default-baseline-anchored logistic and ordinary least squares regressions separate the contributions of persona category, social group category, and prompt template to refusal probability and toxicity score.

What would settle it

Take a stratified sample of generated responses spanning all four models, the six templates, several personas, and the 13 social categories, and have native Chinese-speaking annotators rate toxicity directly; then recompute the amplification ratio $r$ using human ratings instead of the API scores. If the ratios collapse—for instance, if human raters find no 40-fold gap for Ernie or no 9-fold gap for Qwen under hateful personas—the paper's central amplification claim fails, and the comparison would also reveal whether the API over-scores some conditions the way it is documented to over-score German.

Watch

Extended reading notes

Core claim

The paper's central claim is that persona conditioning is a general mechanism for amplifying toxic generation in Chinese LLMs: comparing each persona-conditioned output against the same model's default baseline for matched (template, social group) pairs, it finds toxicity ratios $r = \mathrm{toxicity}_{\mathrm{Persona}} / \mathrm{toxicity}_{\mathrm{Default}}$ above 1 across all four models, with strongly negative personas such as 'a hateful person' or 'a nasty person' reaching median ratios around 9–10 in Qwen and DeepSeek and exceeding 40 in Ernie, where the near-zero default baseline makes the amplification numerically extreme. The paper claims this amplification is not uniform but is driven by interactions among persona category, prompt polarity, target social group, and model-specific safety mechanisms: models with high refusal rates (Hunyuan) show muted amplification, while models with lower withholding (DeepSeek) show sharper toxicity shifts under restrictive templates. Refusal behavior is claimed to be equally structured—it attenuates over ten identical prompt repetitions, shifts with persona gender, and concentrates on sensitive social categories such as sexual orientation, disease, and disability. As a complementary result, the paper shows that an iterative evaluator-guided feedback loop using an external LLM can reduce the worst Qwen outputs from a median toxicity of 0.6–0.8 down to 0.1–0.3 without retraining.

Load-bearing premise

The whole measurement rests on one premise: that the Perspective API's toxicity scores are valid and comparable for Simplified Chinese across every condition compared in the study—if the API's known language bias distorts Chinese scores differently for some templates, personas, or social groups, the reported amplification ratios and group rankings could be artifacts of the measurement rather than behavior of the models.

Editorial extensions

If this is right

  • Persona conditioning is a general bypass mechanism: users of Chinese LLM APIs can systematically raise toxic output by pairing a negative persona with an identity- or appearance-related target group, so system-role customization should be treated as a safety-relevant control.
  • Refusal safeguards are unevenly distributed: they protect some social groups (sexual orientation, disability) far more than others (age, socioeconomic status), and ten identical repetitions erode refusal rates in every model tested, so a single refusal measurement underestimates long-run exposure.
  • The gender asymmetry in refusal—female personas refusing more often when differences are significant—implies that safety behavior itself encodes gendered expectations, and persona gender should be a stratification variable in any safety audit.
  • Models that refuse more (Hunyuan, Qwen) show flatter toxicity distributions, meaning refusal rate and toxicity are partially substitutable risk indicators and must be reported together rather than separately.
  • Iterative LLM-as-evaluator feedback can reduce high toxicity scores without retraining, offering a mitigation path for closed API models where weights and gradients are inaccessible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the 40-fold figure for Ernie is inflated by a near-zero denominator, which the paper itself acknowledges, so cross-model comparisons of amplification are better read from absolute toxicity levels (e.g., DeepSeek's 0.92 for 'Younger Sister' under a nasty-persona prompt) than from ratios alone.
  • A testable extension the paper leaves implicit: run the identical persona, template, and social-group protocol on multilingual or English-only models under the same decoding settings to see whether the amplification magnitudes are a Chinese-context artifact or a general property of persona conditioning.
  • If the documented language bias of the toxicity API (shown for German in the cited literature) also distorts Simplified Chinese scores differently across templates or groups, the reported rankings of which groups are most targeted could be partly a measurement effect; a native-speaker human rating study on a stratified sample would settle whether the amplification structure survives a change of in
  • The refusal attenuation curve over repeated prompts suggests a concrete new metric—'safeguard persistence,' the number of identical repetitions needed to reach floor refusal—which could be used to compare safety robustness across models and across future model versions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports a large-scale evaluation of four Chinese LLMs (Qwen-Turbo, Ernie-4.5-Turbo-128k, DeepSeek-V3, Hunyuan-Standard) under default and persona-assigned system prompts, covering 240 social groups, six prompt templates, and three repetitions per cell (over 1.4M generated texts). It measures refusal with a fine-tuned BERT classifier and toxicity with the Perspective API, reporting refusal-rate trends over repeated prompting, gender differences in refusals, template- and social-group-dependent toxicity, and persona-vs-default toxicity ratios. It then fits separate logistic and linear regressions per model for persona category, social group, and template, and includes a Qwen-only mitigation case study using iterative evaluator feedback. The headline quantitative claim is that persona assignment can amplify toxicity by over 40-fold relative to model-specific defaults, and the interpretive claim is that persona, prompt template, and target social group 'jointly shape' refusal and toxicity.

Significance. If the quantitative results withstand scrutiny, this would be a valuable contribution: it is one of the first large-scale, culturally contextualized safety evaluations of Chinese LLMs, and it ships a large released dataset, a fine-tuned refusal classifier, and analysis/mitigation code. The descriptive findings—refusal attenuation over repetitions, gender asymmetries, social-category-specific refusal and toxicity patterns, and cross-model heterogeneity—are clearly presented and plausible. The mitigation case study, though preliminary, is a useful illustration. The main reservations concern the toxicity-ratio estimator, the mismatch between the 'jointly shape' language and the univariate regressions, and the unvalidated use of Perspective API on Chinese as ground-truth toxicity; these are load-bearing for the headline claims but addressable with additional analysis.

major comments (4)
  1. [Section 3.3 and Section 4.2.2, Eq. (1)] The toxicity ratio is constructed from cell-level maximum toxicity over non-refusal responses, divided by a default baseline that sits at the API floor for several models (e.g., Ernie Default 'Good' is 0.02 in Figure 6). Because persona assignment reduces refusal rates (Section 4.3.1, Figure 8), persona-conditioned cells typically contain more non-refusal draws than the matched default cells, and the expected maximum of k samples increases with k. The reported 'amplification' therefore conflates genuine per-response toxicity increases with (a) a selection effect whereby refusals are removed from the denominator population and (b) a max-over-more-draws effect. The paper only discloses the near-floor component, not the survivor-count/max effect. I ask for a reanalysis that either uses all generated responses with refusal as a separate outcome, or conditions on a fixed number of non-refusal draws, and that reports per-response toxicity distributions rather than cell maxima.
  2. [Abstract, RQ3, and contribution list] The paper states that persona, prompt template, and target social group 'independently and jointly shape' refusal and toxicity, but Section 3.4 specifies three separate logistic regressions and three separate linear regressions per model, each containing one determinant at a time. No interaction terms or joint model are estimated anywhere in Section 4.3. Consequently, the abstract's claim of 'interactions across several factors' is not supported by the reported analyses. Either fit joint models with interaction terms (or at least a single model containing all three main effects) and report the interaction estimates, or revise the claims to say that the three factors each have model-dependent main effects.
  3. [Section 3.3 and Section 5] The Perspective API is the sole toxicity measurement for Simplified Chinese, and all amplification ratios and regression coefficients inherit its scores. The paper cites [44] showing the API misreads German as more toxic and acknowledges that it 'may not capture all cultural nuances and specificities related to toxicity' for Chinese, but it does not provide any validation of the API against human judgments or a Chinese-specific detector in this setup. Because the comparisons are across personas, templates, and social groups, a language-dependent measurement bias that varies with those conditions would directly change the rankings and magnitudes. I request validation on a stratified sample of the generated Chinese responses (e.g., human annotation or comparison with a Chinese-specific detector such as COLDETECTOR [12]), reported per condition, before the quantitative toxicity claims can be taken at face value.
  4. [Section 4.2.2, Figure 6] The 'Default' column in the heatmap is described as 'the average toxicity of the model Default setting,' whereas Eq. (1) defines r as a ratio of cell-level toxicity values that Section 3.3 states are maxima over non-refusal responses. These are different aggregations, and it is unclear whether the denominator in Eq. (1) is the max-toxicity default cell, the average default toxicity, or something else. Please clarify the exact construction; if the denominator is an average, explain why a ratio of a maximum to an average is the quantity of interest, since otherwise the headline 40-fold figure is not reproducible from the text as written.
minor comments (5)
  1. [Section 2.1] There is a duplicated and grammatically broken sentence: 'Persona-assigned LLMs are large language models conditioned These "persona-assigned LLMs" not only...' and later 'these personas are instantiated through system-level role instructions were translated into Chinese.' Please rewrite both passages.
  2. [Section 3.1] The paragraph beginning 'By adjusting this parameter...' is repeated verbatim in the preceding paragraph; remove the duplicate.
  3. [Appendix E] The sentence 'The parameters of the training process are shown in Table??.' contains a missing table reference; either insert the table or delete the sentence.
  4. [Appendix C] The name 'Eirne' appears in the caption text and should be 'Ernie'.
  5. [Section 4.2.2] The phrase 'an extremely lowDefault baseline' is missing a space before 'Default'; please correct the typographical error.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the persona-toxicity and refusal results are direct measurements, not reductions to fitted inputs or self-cited premises.

full rationale

The paper's central claims are empirical measurements. The toxicity amplification ratio r = toxicity_Persona / toxicity_Default (Eq. 1, Section 4.2.2) is computed directly from Perspective API scores on generated responses; there is no fitted parameter that is later renamed as a prediction. The regression analyses in Sections 4.3.1 and 4.3.2 report coefficient estimates from logistic and OLS models and do not use those estimates to 'predict' the data they were fit to. The refusal classifier is trained on a labeled subset and applied uniformly, with its Qwen-specific training explicitly acknowledged as a limitation in Section 5; this is a validity concern, not a circular reduction. The social-group taxonomy from the authors' prior work [37] and the Perspective API bias citation [44] are stimulus and tool choices used to frame the experiments; neither is invoked as a theorem or premise that forces the reported outcomes. The paper's own limitations section candidly flags that the API may miss Chinese cultural nuances and that the refusal detector may not generalize across models. These are measurement-threat disclosures, not evidence that the conclusions are equivalent to their inputs. No equation in the paper reduces to another by construction, and no fitted value is presented as an independent prediction. Therefore, no circularity is present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No hand-fitted constants are used; the study is an empirical measurement, not a derivation. The regression coefficients are outputs, not assumed inputs. The load-bearing content sits in the measurement assumptions listed above, plus standard regression and decoding choices (temperature=1, top_p=0.90, presence_penalty=0.02) that are held fixed across conditions. No new theoretical entities are introduced.

assumptions (3)
  • domain assumption Perspective API provides valid toxicity scores for Simplified Chinese content in this setup.
    Section 3.3 adopts Perspective API for all toxicity measurements; Section 5 acknowledges the API may not capture all cultural nuances and cites prior evidence of cross-language bias [7,44]. All toxicity comparisons depend on this assumption.
  • domain assumption The BERT refusal classifier trained on Qwen-generated responses generalizes to DeepSeek, Ernie, and Hunyuan.
    Section 3.2 trains on Qwen data and applies the fixed detector across models; Section 5 states refusal phrasing may vary across models and the detector may not capture all model-specific expressions. Refusal-rate comparisons depend on this transfer assumption.
  • domain assumption Translated personas and the 240 social groups retain their intended meaning in Chinese.
    Section 3.1 relies on py-googletrans plus manual review for personas and adopts social groups from [37]; Section 5 notes LLMs may distort personas. The interpretation of persona effects depends on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereotypes and Safeguards." pith.science (2026). https://pith.science/paper/37DIG5OO

@misc{pith2026250604975,
  author       = {Pith},
  title        = {Pith review of: Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereotypes and Safeguards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37DIG5OO}},
  note         = {Machine review of arXiv:2506.04975}
}
read the original abstract

Recent research has highlighted that assigning specific personas to large language models (LLMs) can significantly increase harmful content generation. However, limited attention has been given to persona-driven toxicity in non-Western contexts, particularly in Chinese-based LLMs. In this paper, we perform a large-scale, cross-model analysis of refusal behavior and persona-driven toxicity amplification across four Chinese LLMs, leveraging a comprehensive dataset of over 1,400,000 generated texts. We identify significant disparities in persona-driven refusal behavior, including systematic gender differences in refusal triggering across the evaluated Chinese LLMs. Furthermore, we provide quantitative evidence of persona-driven toxicity amplification with respect to model default baselines. We show that this amplification--whose magnitude varies substantially across models--is driven by interactions across several factors, involving persona conditioning, prompting strategy, target social group, and model-specific safety mechanisms. Leveraging model-specific regression analyses, we systematically characterize how persona categories, target social groups, and prompt templates independently and jointly shape both refusal behavior and output toxicity. As a complementary case study, we further explore an iterative, evaluator-guided mitigation strategy based on model feedback with an external LLM evaluator, demonstrating that highly toxic outputs can be substantially reduced without costly model retraining. Overall, our findings highlight the importance of culturally contextualized safety evaluations for Chinese-language LLMs and provide a structured framework for assessing persona-induced risks and exploratory mitigation strategies in LLM-generated content.

Figures

Figures reproduced from arXiv: 2506.04975 by the authors.

Figure 1
Figure 1. Diagram showing the workflow of our analysis and mitigation strategy. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Template-specific refusal rates as a function of repeated prompting across four large language models. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Gender-specific refusal proportions across prompt templates for four large language models. Each [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Refusal rates across social categories under different prompt templates. Each panel corresponds to [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Template-specific toxicity distributions and refusal rates across the four studied Chinese LLMs. Each [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Template- and persona-specific toxicity amplification relative to each model’s default baseline. The [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Template-specific toxicity patterns across social attribute categories for four large language models. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Coefficient estimates from logistic regression models predicting refusal behavior across four large [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Coefficient estimates from linear regression models predicting toxicity scores across four large language [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Estimated effects of Basic Persona polarity on toxicity and refusal across four large language models. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Top ten social groups with the highest observed toxicity values under the “a nasty person” persona for four large language models. Each panel corresponds to one model and reports the social groups associated with the largest toxicity scores in this setting. satisfies …
Figure 12
Figure 12. Figure 12: Distribution of toxicity ratios comparing dictator personas (0–1000AD) with all other personas across [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]
Figure 13
Figure 13. Figure 13: Comparison of Toxicity Scores Across Different Mitigation Strategies. A: The toxicity scores of responses under different mitigation strategies: No Mitigation, Qwen Mitigation, and Ernie Mitigation. Lower scores indicate more effective toxicity reduction. B: The numbe…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups

    cs.CL 2025-10 conditional novelty 5.0 of 10

    Chinese LLMs generate more positive continuations after 'we' prompts and more negative after 'they' prompts; the feminine 'they' intensifies negativity in several pretrained models.

Reference graph

Works this paper leans on

70 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [44]

    Gianluca Nogara, Francesco Pierri, Stefano Cresci, Luca Luceri, Petter Törnberg, and Silvia Giordano. 2023. Toxic Bias: Perspective API misreads German as more toxic.arXiv preprint arXiv:2312.12651(2023)

    work page Pith review arXiv 2023

  2. [12]

    Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng, and Minlie Huang. 2022. COLD: A Benchmark for Chinese Offensive Language Detection. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, Abu Dhabi, Uni...

    doi:10.18653/v1/2022.emnlp-main.796 2022

  3. [1]

    Mahyar Abbasian, Iman Azimi, Amir M Rahmani, and Ramesh Jain. 2023. Conversational health agents: A personalized llm-powered agent framework.arXiv preprint arXiv:2310.02374(2023)

    arXiv 2023

  4. [2]

    Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large Language Models for Mathematical Reasoning: Progresses and Challenges. InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, Neele Falk, Sara Papi, and Mike Zhang (Eds.). Association for Co...

    work page 2024

  5. [3]

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in language models is mediated by a single direction.URL https://arxiv. org/abs/2406.11717(2024)

    arXiv 2024

  6. [4]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency(Virtual Event, Canada)(FAccT ’21). Association for Computing Machinery, New York, NY, USA, 610–623. doi:10.114...

    arXiv 2021

  7. [5]

    Daniel Berrar et al. 2019. Cross-validation

    work page 2019

  8. [6]

    Andrew Blair-Stanek, Nils Holzenberger, and Benjamin Van Durme. 2023. Can GPT-3 perform statutory reasoning?. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law. 22–31

    work page 2023

Show all 70 references
  1. [7]

    Rishi Bommasani, Percy Liang, and Tony Lee. 2023. Holistic evaluation of language models.Annals of the New York Academy of Sciences1525, 1 (2023), 140–146

    2023

  2. [8]

    Angana Borah and Rada Mihalcea. 2024. Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions. InFindings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computationa...

    2024 doi

  3. [9]

    Myra Cheng, Tiziano Piccardi, and Diyi Yang. 2023. CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for ...

    2023 doi

  4. [10]

    Yujun Cheng, Weiting Zhang, Zhewei Zhang, Chuan Zhang, Shengjin Wang, and Shiwen Mao. 2024. Towards Federated Large Language Models: Motivations, Methods, and Future Directions.IEEE Communications Surveys & Tutorials (2024), 1–1. doi:10.1109/COMST.2024.3503680

    2024

  5. [11]

    Jonathan H Choi, Kristin E Hickman, Amy B Monahan, and Daniel Schwarcz. 2021. ChatGPT goes to law school.J. Legal Educ.71 (2021), 387. , Vol. 1, No. 1, Article . Publication date: May 2025. Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereo...

    2021

  6. [13]

    Erik Derner, Kristina Batistič, Jan Zahálka, and Robert Babuška. 2024. A security risk taxonomy for prompt-based interaction with large language models.IEEE Access(2024)

    2024

  7. [14]

    Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models. InFindings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bal...

    2023 doi

  8. [15]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human ...

    2019 doi

  9. [16]

    Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong, Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, et al. 2024. Agent ai: Surveying the horizons of multimodal interaction.arXiv preprint arXiv:2401.03568(2024)

    2024 arXiv

  10. [17]

    Farshid Faal, Ketra Schmitt, and Jia Yuan Yu. 2023. Reward modeling for mitigating toxicity in transformer-based language models.Applied Intelligence53, 7 (2023), 8421–8435

    2023

  11. [18]

    Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models. InProceedings of the 61st Annual Meeting of the Association for Computationa...

    2023 doi

  12. [19]

    Horton, and Benjamin S

    Apostolos Filippas, John J. Horton, and Benjamin S. Manning. 2024. Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?. InProceedings of the 25th ACM Conference on Economics and Computation(New Haven, CT, USA)(EC ’24). Association for Compu...

    2024

  13. [20]

    Nicoló Fontana, Francesco Pierri, and Luca Maria Aiello. 2024. Nicer Than Humans: How do Large Language Models Behave in the Prisoner’s Dilemma?arXiv preprint arXiv:2406.13605(2024)

    2024 arXiv

  14. [21]

    Jonas Freiknecht and Wolfgang Effelsberg. 2020. Procedural Generation of Interactive Stories using Language Models. InProceedings of the 15th International Conference on the Foundations of Digital Games(Bugibba, Malta)(FDG ’20). Association for Computing Machinery, New York, N...

    2020

  15. [22]

    Gallegos, Ryan A

    Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. Bias and Fairness in Large Language Models: A Survey.Computational Linguistics 50, 3 (Sept. 2024), 1097–1179. doi:10.1162/coli_a_00524

    2024 doi

  16. [23]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2020, Trevor Cohn, Yulan He, and Yang Liu (Eds.)...

    2020 doi

  17. [24]

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang

  18. [25]

    Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, and Tushar Khot. 2024. Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs. InThe Twelfth International Conference on Learning Representations

    2024

  19. [26]

    Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith

  20. [27]

    Vojtěch Hudeček and Ondrej Dusek. 2023. Are Large Language Models All You Need for Task-Oriented Dialogue?. InProceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Svetlana Stoy- anchev, Shafiq Joty, David Schlangen, Ondrej Dusek, Case...

    2023 doi

  21. [28]

    Hadas Kotek, Rikker Dockum, and David Sun. 2023. Gender bias and stereotypes in Large Language Models. In Proceedings of The ACM Collective Intelligence Conference(Delft, Netherlands)(CI ’23). Association for Computing , Vol. 1, No. 1, Article . Publication date: May 2025. 24 ...

    2023

  22. [29]

    Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. GeDi: Generative Discriminator Guided Sequence Generation. InFindings of the Association for Computational Linguistics: EMNLP 2021, Marie-Fra...

    2021 doi

  23. [30]

    Valerio La Gatta, Chiyu Wei, Luca Luceri, Francesco Pierri, and Emilio Ferrara. 2023. Retrieving false claims on Twitter during the Russia-Ukraine conflict. InCompanion proceedings of the ACM web conference 2023. 1317–1323

    2023

  24. [31]

    Peter A Lachenbruch. 2014. McNemar test.Wiley StatsRef: Statistics Reference Online(2014)

    2014

  25. [32]

    Alina Leidinger and Richard Rogers. 2024. How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studies. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7. 839–854

    2024

  26. [33]

    Rui Li, Deji Fu, Chunyu Shi, Zhilan Huang, and Gang Lu. 2024. Efficient LLMs Training and Inference: An Introduction. IEEE Access(2024)

    2024

  27. [34]

    Yaqiong Li, Peng Zhang, Hansu Gu, Tun Lu, Siyuan Qiao, Yubo Shu, Yiyang Shao, and Ning Gu. 2024. DeMod: A Holistic Tool with Explainable Detection and Personalized Modification for Toxicity Censorship.arXiv preprint arXiv:2411.01844(2024)

    2024 arXiv

  28. [35]

    Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. InInternational Conference on Machine Learning. PMLR, 6565–6576

    2021

  29. [36]

    Lizhi Lin, Honglin Mu, Zenan Zhai, Minghan Wang, Yuxia Wang, Renxi Wang, Junjie Gao, Yixuan Zhang, Wanxiang Che, Timothy Baldwin, et al. 2025. Against The Achilles’ Heel: A Survey on Red Teaming for Generative Models. Journal of Artificial Intelligence Research82 (2025), 687–775

    2025

  30. [37]

    Geng Liu, Carlo Alberto Bono, and Francesco Pierri. 2025. Comparing diversity, negativity, and stereotypes in Chinese-language AI technologies: an investigation of Baidu, Ernie and Qwen.PeerJ Computer Science11 (2025), e2694

    2025

  31. [38]

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. 2023. Agentbench: Evaluating llms as agents.arXiv preprint arXiv:2308.03688(2023)

    2023 arXiv

  32. [39]

    Yi Liu, Junzhe Yu, Huijia Sun, Ling Shi, Gelei Deng, Yuqi Chen, and Yang Liu. 2024. Efficient Detection of Toxic Prompts in Large Language Models. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering(Sacramento, CA, USA)(ASE ’24). Assoc...

    2024

  33. [40]

    Tinh Luong, Thanh-Thien Le, Linh Ngo, and Thien Nguyen. 2024. Realistic Evaluation of Toxicity in Large Language Models. InFindings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational L...

    2024 doi

  34. [41]

    Hanjia Lyu, Jiebo Luo, Jian Kang, and Allison Koenecke. 2025. Characterizing Bias: Benchmarking Large Language Models in Simplified versus Traditional Chinese. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. 2815–2846

    2025

  35. [42]

    Manuj Malik, Jing Jiang, and Kian Ming Chai. 2024. An Empirical Analysis of the Writing Styles of Persona-Assigned LLMs. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 19369–19388

    2024

  36. [43]

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2023. A comprehensive overview of large language models.arXiv preprint arXiv:2307.06435 (2023)

    2023 arXiv

  37. [45]

    Deonna M Owens, Ryan A Rossi, Sungchul Kim, Tong Yu, Franck Dernoncourt, Xiang Chen, Ruiyi Zhang, Jiuxiang Gu, Hanieh Deilamsalehy, and Nedim Lipka. 2024. A multi-llm debiasing framework.arXiv preprint arXiv:2409.13884 (2024)

    2024 arXiv

  38. [46]

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology. 1–22

    2023

  39. [47]

    Bernstein

    Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. Social Simulacra: Creating Populated Prototypes for Social Computing Systems. InProceedings of the 35th Annual ACM Symposium on User Interface Software and Techno...

    2022

  40. [48]

    Damian Pascual, Beni Egressy, Clara Meister, Ryan Cotterell, and Roger Wattenhofer. 2021. A Plug-and-Play Method for Controlled Text Generation. InFindings of the Association for Computational Linguistics: EMNLP 2021, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Sco...

    2021

  41. [49]

    Siddhesh Pawar, Junyeong Park, Jiho Jin, Arnav Arora, Junho Myung, Srishti Yadav, Faiz Ghifari Haznitrama, Inhwa Song, Alice Oh, and Isabelle Augenstein. 2024. Survey of cultural awareness in language models: Text and beyond. arXiv preprint arXiv:2411.00860(2024)

    2024 arXiv

  42. [50]

    Parijat Rai, Saumil Sood, Vijay K Madisetti, and Arshdeep Bahga. 2024. Guardian: A multi-tiered defense architecture for thwarting prompt injection attacks on llms.Journal of Software Engineering and Applications17, 1 (2024), 43–68

    2024

  43. [51]

    Paul Röttger, Fabio Pernisi, Bertie Vidgen, and Dirk Hovy. 2024. Safetyprompts: a systematic review of open datasets for evaluating and improving large language model safety.arXiv preprint arXiv:2404.05399(2024)

    2024 arXiv

  44. [52]

    Joni Salminen, Chang Liu, Wenjing Pian, Jianxing Chi, Essi Häyhänen, and Bernard J Jansen. 2024. Deus Ex Machina and Personas from Large Language Models: Investigating the Composition of AI-Generated Persona Descriptions. InProceedings of the 2024 CHI Conference on Human Facto...

    2024

  45. [53]

    Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiangyong Zhu, and Bo Zheng. 2024. Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models. arXiv:2412.15265 [cs.CL] https://arxiv.or...

    2024 arXiv

  46. [54]

    Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Yu-Ching Hsu, Jia-Yin Foo, Chao-Wei Huang, and Yun-Nung Chen

  47. [55]

    Lei Wang and Ee-Peng Lim. 2023. Zero-Shot Next-Item Recommendation using Large Pretrained Language Models. arXiv preprint arXiv:2304.03153(2023)

    2023 arXiv

  48. [56]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail?Advances in Neural Information Processing Systems36 (2023), 80079–80110

    2023

  49. [57]

    Two tales of persona in llms: A survey of role-playing and personalization.arXiv preprint arXiv:2406.01171 (2024)

    2024 arXiv

  50. [58]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.High-Confidence Computing(2024), 100211

    2024

  51. [59]

    Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, and Kai-Wei Chang. 2022. Geomlama: Geo-diverse commonsense probing on multilingual pre-trained language models.arXiv preprint arXiv:2205.12247(2022)

    2022 arXiv

  52. [60]

    As an AI language model, I cannot

    Joel Wester, Tim Schrills, Henning Pohl, and Niels van Berkel. 2024. “As an AI language model, I cannot”: Investigating LLM Denials of User Requests. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA)(CHI ’24). Association for Co...

    2024

  53. [61]

    Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, and Ning Zhang. 2024. Don’t Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models. In33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, Philadelphia, PA

    2024

  54. [62]

    Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. 2023. Don’t trust ChatGPT when your question is not in English: a study of multilingual abilities and types of LLMs.arXiv preprint arXiv:2305.16339(2023)

    2023 arXiv

  55. [63]

    Chao Yu and Drew Margolin. 2024. Topic territoriality and the cost of civility: examining the impact of IP address disclosure on Weibo.Journal of Computer-Mediated Communication29, 6 (2024), zmae018

    2024

  56. [64]

    Di Zhou and Yinxian Zhang. 2024. Political biases and inconsistencies in bilingual GPT models—the cases of the US and China.Scientific Reports14, 1 (2024), 25048

    2024

  57. [65]

    Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang

  58. [66]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models.arXiv preprint arXiv:2303.18223(2023)

    2023 arXiv

  59. [70]

    a nasty person

    Noah Ziems, Wenhao Yu, Zhihan Zhang, and Meng Jiang. 2023. Large Language Models are Built-in Autoregressive Search Engines. InFindings of the Association for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki (Eds.). Association for Comp...

    2023

  60. [2020]

    InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.)

    Don‘t Stop Pretraining: Adapt Language Models to Domains and Tasks. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, Onli...

    doi

  61. [2023]

    InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.)

    Solving Math Word Problems via Cooperative Reasoning induced Language Models. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computat...

    doi

  62. [2024]

    Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680 (2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.