Pith. sign in

REVIEW 16 cited by

ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2203.09509 v4 pith:444SPTCJ submitted 2022-03-17 cs.CL

Thomas Hartvigsen , Saadia Gabriel , Hamid Palangi , Maarten Sap , Dipankar Ray , Ece Kamar This is my paper
classification cs.CL
keywords toxictoxigentextlanguagemachine-generateddatagroupshate
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Toxic language detection systems often falsely flag text that contains minority group mentions as toxic, as those groups are often the targets of online hate. Such over-reliance on spurious correlations also causes systems to struggle with detecting implicitly toxic language. To help mitigate these issues, we create ToxiGen, a new large-scale and machine-generated dataset of 274k toxic and benign statements about 13 minority groups. We develop a demonstration-based prompting framework and an adversarial classifier-in-the-loop decoding method to generate subtly toxic and benign text with a massive pretrained language model. Controlling machine generation in this way allows ToxiGen to cover implicitly toxic text at a larger scale, and about more demographic groups, than previous resources of human-written text. We conduct a human evaluation on a challenging subset of ToxiGen and find that annotators struggle to distinguish machine-generated text from human-written language. We also find that 94.5% of toxic examples are labeled as hate speech by human annotators. Using three publicly-available datasets, we show that finetuning a toxicity classifier on our data improves its performance on human-written data substantially. We also demonstrate that ToxiGen can be used to fight machine-generated toxicity as finetuning improves the classifier significantly on our evaluation subset. Our code and data can be found at https://github.com/microsoft/ToxiGen.

Discussion (0). Sign in to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworks

    cs.CL 2025-07 reject novelty 7.0 of 10

    The paper creates a span-level Chinese hate speech dataset and a 830-term coded hate lexicon, but its two-stage training method's reported superiority is contradicted by the paper's own COLD results.

  2. S2T-RLHF: Hierarchical Credit Assignment for Stable Preference-Based RLHF

    cs.AI 2026-05 conditional novelty 6.0 of 10

    S2T-RLHF splits each response-level RLHF reward into sentence shares and then token shares, via bargaining and Dirichlet weighting, yielding steadier training with competitive preference alignment.

  3. YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language Models

    cs.HC 2025-09 conditional novelty 6.0 of 10

    Introduces YAIR, a youth-GenAI risk benchmark, and YouthSafe, a fine-tuned classifier with AUPRC 0.94 on it, though it compares a trained model to untrained baselines.

  4. Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English

    cs.CL 2025-08 conditional novelty 6.0 of 10

    After ChatGPT's release, science and tech podcast speakers used AI-associated words like 'surpass' and 'align' more often, while control synonyms showed no average shift.

  5. Trade-offs in Image Generation: How Do Different Dimensions Interact?

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A new benchmark and VLM-as-judge metric map trade-offs among ten image-generation dimensions across 14 models, with a visualization called DTM.

  6. PL-Guard: Benchmarking Language Model Safety for Polish

    cs.CL 2025-06 reject novelty 6.0 of 10

    A small Polish BERT classifier proved more robust than larger fine-tuned LLMs at classifying safe versus unsafe Polish content, including under character-level adversarial perturbations.

  7. MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment

    cs.LG 2025-05 conditional novelty 6.0 of 10

    The paper introduces TRADE, an online-only MCP attack, and RAG-Pref, a retrieval-based preference alignment method that together with DPO improves strict refusal of falsely benign MCP exploits from 6.7% to 24.1% on average.

  8. Large Language Models Often Know When They Are Being Evaluated

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Frontier language models distinguish evaluation transcripts from deployment transcripts with AUC up to 0.83, below the authors' human baseline of 0.92.

  9. Lifelong Safety Alignment for Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.

  10. Fence: Specialized SLM Guardrails for LLM Applications

    cs.AI 2026-05 conditional novelty 5.0 of 10

    Small language models trained on GAN-style adversarial synthetic data outperform prompted large models on use-case-specific guardrails for off-topic and prompt-injection detection.

  11. Decoding the Rule Book: Extracting Hidden Moderation Criteria from Reddit Communities

    cs.CL 2025-09 conditional novelty 5.0 of 10

    Lexical scores from an interpretable classifier reveal that moderation criteria differ sharply across subreddits, though the extracted criteria only partially match the accuracy of full-text neural models.

  12. Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weights

    cs.AI 2025-06 reject novelty 5.0 of 10

    The paper claims a state-of-the-art, low-cost open-weight web agent, but its WebVoyager result is inflated by training on the evaluation benchmark.

  13. AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs

    cs.LG 2025-05 conditional novelty 5.0 of 10

    AutoMixAlign adaptively reweights or resamples task data during DPO training to match specialist-model losses, improving average performance on helpfulness, coding, and safety benchmarks compared to standard DPO and m...

  14. Beyond Explainable AI (XAI): An Overdue Paradigm Shift and Post-XAI Research Directions

    cs.CY 2026-02 unverdicted novelty 4.0 of 10

    Current XAI methods for DNNs and LLMs rest on paradoxes and false assumptions that demand a paradigm shift to verification protocols, scientific foundations, context-aware design, and faithful model analysis rather th...

  15. SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use

    cs.CL 2025-05 conditional novelty 4.0 of 10

    A new cross-lingual benchmark shows large language models comply with explicit requests to use swear words far more often in Indic languages than in English, revealing a safety alignment gap.

  16. The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs

    cs.CL 2025-06 conditional novelty 3.0 of 10

    A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.

Pith tools