Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Closed-source AI moderation classifiers are unfair and can be bypassed by minimal paraphrasing, this paper claims.

desk verdict Useful audit with a solid robustness finding, but the headline fairness ranking is threshold-dependent and needs a caveat. read the letter →

arxiv 2501.13302 v1 pith:XS2AMAZC submitted 2025-01-23 cs.CL cs.AI

Akshit Achara , Anshuman Chhabra This is my paper · ORCID
classification cs.CLcs.AI
keywords AIsafetymoderationfairnessdemographicparityconditionalstatisticalrobustnessLLM-basedperturbationcontentOpenAPI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the closed-source AI safety moderation classifiers now used to police social media and to filter LLM fine-tuning data are neither neutral nor stable. It audits four APIs—OpenAI, Perspective, Google Cloud Natural Language, and Clarifai—and reports that, at a common 0.5 threshold, the OpenAI Moderation API shows the largest group-fairness gaps on metrics such as demographic parity and conditional statistical parity. It also reports that minimal semantic-preserving rewrites, especially GPT-3.5-based paraphrases, can flip the verdicts of all four systems, turning initially unsafe comments into safe ones. The stakes are that biased or fragile moderation can suppress minority voices and can allow harmful content into the training pipelines of future models.

What carries the argument

The central machinery is a black-box evaluation protocol: treat each closed moderation API as a classifier C that maps input text to a binary safe/unsafe verdict, then measure fairness as the absolute difference in unsafe rates between majority and minority groups (demographic parity) and within negatively regarded comments (conditional statistical parity), and robustness as f_robust = |E_X[C(X)] - E_X*[C(X*)]| over semantically similar perturbed inputs. The perturbations are produced by backtranslation and by GPT-3.5-Turbo paraphrase, and the legitimate factor for conditional statistical parity comes from a BERT-based regard classifier.

What would settle it

Recompute demographic parity and conditional statistical parity for each API at the threshold its vendor recommends, and check whether the OpenAI model still has the largest fairness gap; likewise, rerun the GPT-3.5 paraphrase on a fresh hold-out of unsafe comments and see whether the unsafe-to-safe flip rate stays far above zero.

Watch

Extended reading notes

Core claim

The paper's central discovery, stated on its own terms, is that fairness and robustness failures are measurable in all four closed-source AI safety moderation classifiers it audited. Using demographic parity and conditional statistical parity on identity-labeled Jigsaw toxicity data and a new annotated Reddit ideology dataset, it reports that the OpenAI Moderation API has the largest parity errors, while the Google Cloud Natural Language API tracks the uniformly random fairness baseline most closely. On robustness, backtranslation changes only a few predictions, but LLM-based paraphrasing with GPT-3.5 Turbo flips a substantial share of initially unsafe comments to safe for every model, with the largest flips again for OpenAI. The authors therefore conclude that current ASM guardrails can be bypassed by minimal semantic-preserving perturbations and that fairness varies by protected attribute, with sexual orientation showing the largest gaps across models.

Load-bearing premise

The comparison assumes that converting every score-based API's output to safe/unsafe with one fixed 0.5 threshold produces the same kind of decision as OpenAI's native flag, so the fairness ranking may change if per-API thresholds are used.

Editorial extensions

If this is right

  • If closed-source moderation APIs are used to filter fine-tuning data, paraphrased harmful content can enter training mixtures, since all four models can be flipped from unsafe to safe by minimal LLM-based rewrites.
  • The fairness gaps, especially around sexual orientation, mean minority-group authors may have their content removed at disproportionate rates under current default settings.
  • The perturbed samples produced by the paper can serve as a fixed benchmark for monitoring updates to closed-source moderation models.
  • The threshold experiment shows that fairness comparisons across black-box moderation systems are sensitive to the score threshold used to binarize their outputs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit: if each vendor's recommended threshold, such as 0.7 for Perspective, is used instead of 0.5, the fairness ranking among the four APIs could change, because the paper's own appendix shows Perspective's fairness improves and GCNL's worsens at 0.7.
  • The same paraphrase mechanism that flips unsafe to safe could likely be automated by an adversarial search, turning the demonstrated vulnerability into a scalable bypass; the paper names AutoDAN and PAP as related attacks but does not run them.
  • The intersectional results reported for the OpenAI model suggest that fairness measured on single protected attributes can hide worse disparities for combinations of attributes; extending the audit to all pairs of groups is a natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper audits four closed-source AI safety moderation APIs (OpenAI Moderation, Perspective, GCNL, and Clarifai) for group fairness and robustness. Using the Jigsaw toxicity subdatasets and a newly collected Reddit-Ideology corpus, it computes demographic parity and conditional statistical parity across protected groups, and measures robustness to backtranslation and GPT-3.5-based paraphrasing. The main claims are that OpenAI is less fair than the other ASMs, that sexual orientation shows the largest disparities, and that LLM-based paraphrases flip a substantial share of unsafe predictions to safe across all APIs.

Significance. The audit is timely and practically relevant. The robustness experiments, qualitative examples of successful bypasses, and the manual verification that most perturbed inputs retain their harmful character are valuable contributions, and the released code supports reproducibility. However, the headline fairness comparison is based on a single 0.5 binarization threshold, and the paper's own Appendix J shows that the ranking is threshold-sensitive. As stated, the 'OpenAI is most unfair' contribution is not yet supported; the threshold and base-rate issues need to be addressed before the fairness ranking can be taken at face value.

major comments (3)
  1. [§4, Appendix A, Appendix J] The paper's headline contribution that 'the OpenAI ASM model is more unfair as compared to the other ASMs' (contribution list and §5) rests on comparing binary labels obtained with a single threshold of 0.5 for Perspective, GCNL, and Clarifai against OpenAI's native flag (§4, Appendix A). Appendix J (Figure 7) shows that at threshold 0.7 the Perspective API becomes fairer while GCNL becomes less fair, so the relative ranking is not invariant to the binarization threshold. Since Perspective's own documentation recommends 0.7, the 0.5-based ranking cannot be assumed to reflect the operating points used in practice. The authors should report fairness metrics across a range of thresholds (or for each API's recommended threshold) and either qualify or drop the unconditional 'OpenAI is most unfair' claim.
  2. [§4, Appendix F, Figure 3] The conclusion that GCNL is the most fair ASM is hard to interpret without base rates. Appendix F and Figure 5 show that GCNL labels a much higher proportion of comments as unsafe than the other APIs; a classifier whose positive rate is near saturation will have small DP/CSP differences even if its decisions are not group-fair in any meaningful sense. The statement in §4 that GCNL is 'closely aligning to the uniformly random baseline' is therefore misleading, because GCNL's behavior is not random. Please report the overall unsafe rate per model and per group, and consider threshold-independent measures (e.g., AUC or calibration) before concluding that GCNL has no fairness issues.
  3. [§3.3, Table 2, Figure 4] The robustness measure defined in §3.3, f_robust = |E_X(C(X)) - E_{X*}(C(X*))|, is an aggregate difference in mean predictions and is not equivalent to the probability that an individual prediction changes; symmetric flips (safe→unsafe and unsafe→safe) cancel in this metric. It is therefore unclear whether the percentages in Table 2 are this aggregate metric or the label-flip rates implicit in Figure 4. The qualitative robustness claim depends on flip rates, so the paper should clarify the metric and report paired flip rates with confidence intervals or bootstrap error bars. In addition, the fairness comparisons in Figure 3 are point estimates without uncertainty quantification; the claim that OpenAI is 'more unfair' could be within sampling noise, especially for the smaller Reddit-Ideology dataset.
minor comments (5)
  1. [Throughout] There are several typos, including 'theshold' (§5), 'mailicious' (§5), 'close-sourced' (Introduction), and 'ethinity' (Appendix I); a careful proofread is needed.
  2. [§4, Reddit-Ideology] Please state where the Reddit-Ideology dataset and the manual annotations are available, and provide more detail on the annotation instructions; Cohen's kappa is reported, but the dataset is a small convenience sample and this should be acknowledged in the limitations.
  3. [Appendix A] For Perspective, GCNL, and Clarifai, the paper says a comment is unsafe if 'any of the scores are greater than or equal to 0.5'; please clarify whether this is the maximum over all label scores and how this interacts with the different label sets across APIs.
  4. [Appendix F] The choice of BERT regard 'negative' as the legitimate factor for CSP should be justified more explicitly; the CSP values depend on this choice, and the paper should discuss its sensitivity.
  5. [§5] The phrase 'no significant fairness issues' for GCNL should be replaced by a more cautious statement such as 'no large DP/CSP differences under the chosen threshold,' given that no significance tests are reported.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the fairness and robustness findings are empirical measurements with externally defined metrics and no fitted parameters folded into the claims.

full rationale

The paper's derivation chain is an empirical audit rather than a formal derivation. Fairness is measured using externally defined metrics (demographic parity and conditional statistical parity) computed from API predictions and protected-group labels; neither metric is defined in terms of the paper's conclusions. Robustness is measured by the defined quantity f_robust = |E_X(C(X)) - E_X*(C(X*))|, which directly compares observed classification rates before and after perturbation and involves no fitted parameters. The choice of a 0.5 threshold for score-based APIs is an experimental design assumption, not a fitted input or a parameter renamed as a prediction; Appendix J's demonstration that fairness rankings change with threshold is a validity/robustness caveat, not a circular step. Self-citations (e.g., Chhabra et al. for fairness background, Askari et al. for a political classifier) are used as prior-work references and tool citations, not as load-bearing justification for the central claims. No equation reduces to its own inputs, no uniqueness theorem is imported from the authors, and no ansatz is smuggled in via citation. The central empirical findings are therefore self-contained against the stated experimental setup, even if the threshold choice may limit their generalizability.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on two hand-chosen parameters (the binarization threshold and the CSP conditioning label) and on several domain assumptions about the proxies used for legitimate factors, semantic preservation under perturbation, and the representativeness of the Reddit sample. No new theoretical entities are introduced.

free parameters (2)
  • Binary threshold = 0.5
    Chosen for all score-based APIs; affects fairness ranking (Appendix J).
  • CSP conditioning value = negative
    CSP computed only on regard-negative comments; different conditioning could change results.
assumptions (4)
  • domain assumption The uniform 0.5 threshold on Perspective, GCNL, and Clarifai scores yields comparable binary labels to OpenAI's native flag.
    Section 4 models and Appendix J; the paper itself shows 0.7 changes the fairness ranking.
  • domain assumption BERT regard labels serve as legitimate factors for conditional statistical parity.
    Appendix F; CSP is computed only on comments the regard model labels as negative.
  • domain assumption Backtranslation and GPT-3.5 paraphrases preserve semantic meaning and harmfulness.
    Section 3.3; only a small manual subset (300 examples) was checked for harm retention.
  • domain assumption Comments from four political subreddits filtered by a BERT political classifier represent left/right political ideology.
    Section 4 Datasets; a convenience sample of 1147 comments annotated by three graduate students.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers." pith.science (2026). https://pith.science/paper/XS2AMAZC

@misc{pith2026250113302,
  author       = {Pith},
  title        = {Pith review of: Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XS2AMAZC}},
  note         = {Machine review of arXiv:2501.13302}
}
read the original abstract

AI Safety Moderation (ASM) classifiers are designed to moderate content on social media platforms and to serve as guardrails that prevent Large Language Models (LLMs) from being fine-tuned on unsafe inputs. Owing to their potential for disparate impact, it is crucial to ensure that these classifiers: (1) do not unfairly classify content belonging to users from minority groups as unsafe compared to those from majority groups and (2) that their behavior remains robust and consistent across similar inputs. In this work, we thus examine the fairness and robustness of four widely-used, closed-source ASM classifiers: OpenAI Moderation API, Perspective API, Google Cloud Natural Language (GCNL) API, and Clarifai API. We assess fairness using metrics such as demographic parity and conditional statistical parity, comparing their performance against ASM models and a fair-only baseline. Additionally, we analyze robustness by testing the classifiers' sensitivity to small and natural input perturbations. Our findings reveal potential fairness and robustness gaps, highlighting the need to mitigate these issues in future versions of these models.

Figures

Figures reproduced from arXiv: 2501.13302 by the authors.

Figure 1
Figure 1. The comparison highlights bias in the Ope [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A small perturbation in the input prompt may [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The demographic parity difference for the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The percentage of safe and unsafe comments predicted by all the ASM models for each of the regard labels where A represents OpenAI Moderation API, B represents Perspective API, C represents GCNL API and D represents Clarifai API. The analysis is performed on Jigsaw dat…
Figure 6
Figure 6. Figure 6: Top 3 topics for each of the datasets in consideration with examples and associated keywords. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: The demographic parity difference for the [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 20 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403

    arXiv 2023

  3. [3]

    Andrew Arsht and Daniel Etcovitch. 2018. The human cost of online content moderation. Harvard Journal of Law and Technology, 2

    work page 2018

  4. [4]

    Hadi Askari, Anshuman Chhabra, Bernhard Clemm von Hohenberg, Michael Heseltine, and Magdalena Wojcieszak. 2024. Incentivizing news consumption on social media platforms using large language models and realistic bot accounts. PNAS nexus, 3(9):pgae368

    work page 2024

  5. [5]

    Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. 2024. Measuring political bias in large language models: What is said and how it is said. arXiv preprint arXiv:2403.18932

    arXiv 2024

  6. [6]

    Daniel Borkan, Jeffrey Sorensen, Lucas Dixon, and Lucy Vasserman. 2019. https://kaggle.com/competitions/jigsaw-unintended-bias-in-toxicity-classification Jigsaw unintended bias in toxicity classification

    work page 2019

  7. [7]

    Houssem Ben Braiek and Foutse Khomh. 2024. Machine learning robustness: A primer. arXiv preprint arXiv:2404.00897

    arXiv 2024

  8. [8]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

Show all 46 references
  1. [9]

    Simon Caton and Christian Haas. 2024. Fairness in machine learning: A survey. ACM Computing Surveys, 56(7):1--38

    2024

  2. [10]

    Anshuman Chhabra, Hadi Askari, and Prasant Mohapatra. 2024 a . Revisiting zero-shot abstractive summarization in the era of large language models from the perspective of position bias. NAACL

    2024

  3. [11]

    Anshuman Chhabra, Peizhao Li, Prasant Mohapatra, and Hongfu Liu. 2023. Robust fair clustering: A novel fairness attack and defense framework. In The Eleventh International Conference on Learning Representations

    2023

  4. [12]

    what data benefits my classifier?

    Anshuman Chhabra, Peizhao Li, Prasant Mohapatra, and Hongfu Liu. 2024 b . " what data benefits my classifier?" enhancing model performance and interpretability through influence-based data selection. In The Twelfth International Conference on Learning Representations

    2024

  5. [13]

    Anshuman Chhabra, Karina Masalkovait \.e , and Prasant Mohapatra. 2021. An overview of fairness in clustering. IEEE Access, 9:130698--130720

    2021

  6. [14]

    Towards F air V ideo S ummarization

    Anshuman Chhabra, Kartik Patwari, Chandana Kuntala, Deepak Kumar Sharma, Prasant Mohapatra, et al. Towards F air V ideo S ummarization. Transactions on Machine Learning Research

  7. [15]

    https://clarifai.com/clarifai/main/models/moderation-english-text-classification

    Clarifai. https://clarifai.com/clarifai/main/models/moderation-english-text-classification

  8. [16]

    Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37--46

    1960

  9. [17]

    Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pages 797--806

    2017

  10. [18]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://api.semanticscholar.org/CorpusID:52967399 Bert: Pre-training of deep bidirectional transformers for language understanding . In North American Chapter of the Association for Computational Linguistics

    2019

  11. [19]

    Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67--73

    2018

  12. [20]

    Yi Dong, Ronghui Mu, Gaojie Jin, Yi Qi, Jinwei Hu, Xingyu Zhao, Jie Meng, Wenjie Ruan, and Xiaowei Huang. 2024. Building guardrails for large language models. arXiv preprint arXiv:2402.01822

    2024 arXiv

  13. [21]

    Esin Durmus, Karina Nguyen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXi...

    2023 arXiv

  14. [22]

    Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214--226

    2012

  15. [23]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462

    2020 arXiv

  16. [24]

    Google. a . https://perspectiveapi.com/

  17. [25]

    Google. b . https://cloud.google.com/natural-language/docs/moderating-text

  18. [26]

    Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794

    2022 arXiv

  19. [27]

    Shan Jiang, Ronald E Robertson, and Christo Wilson. 2020. Reasoning about political bias in content moderation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13669--13672

    2020

  20. [28]

    Deepak Kumar, Yousef Anees AbuHashem, and Zakir Durumeric. 2024. Watch your language: Investigating content moderation with large language models. In Proceedings of the International AAAI Conference on Web and Social Media, volume 18, pages 865--878

    2024

  21. [29]

    Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. Advances in neural information processing systems, 30

    2017

  22. [30]

    Michelle S Lam, Mitchell L Gordon, Dana \"e Metaxa, Jeffrey T Hancock, James A Landay, and Michael S Bernstein. 2022. End-user audits: A system empowering communities to lead large-scale investigations of harmful algorithmic behavior. Proceedings of the ACM on Human-Computer I...

    2022

  23. [31]

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451

    2023 arXiv

  24. [32]

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747

    2023 arXiv

  25. [33]

    Edward Ma. 2019. Nlp augmentation. https://github.com/makcedward/nlpaug

    2019

  26. [34]

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1--35

    2021

  27. [35]

    Gianluca Nogara, Francesco Pierri, Stefano Cresci, Luca Luceri, Petter T \"o rnberg, and Silvia Giordano. 2023. Toxic bias: Perspective api misreads german as more toxic. arXiv preprint arXiv:2312.12651

    2023 arXiv

  28. [36]

    https://platform.openai.com/docs/guides/moderation

    OpenAI. https://platform.openai.com/docs/guides/moderation

  29. [37]

    Esteban Ortiz-Ospina. 2019. The rise of social media. Our World in Data. Https://ourworldindata.org/rise-of-social-media

    2019

  30. [38]

    Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693

    2023 arXiv

  31. [39]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. https://doi.org/10.18653/v1/P16-1009 Improving neural machine translation models with monolingual data . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ...

    2016 doi

  32. [40]

    Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. https://doi.org/10.48550/ARXIV.1909.01326 The woman worked as a babysitter: On biases in language generation

    2019 doi arXiv

  33. [41]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: H ow D oes LLM S afety T raining F ail? In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  34. [42]

    Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, et al. 2023. A C omprehensive C apability A nalysis of GPT -3 and GPT -3.5 S eries M odels. arXiv preprint arXiv:2303.10420

    2023 arXiv

  35. [43]

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. arXiv preprint arXiv:2401.06373

    2024 arXiv

  36. [44]

    Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

  37. [45]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.