Pith. sign in

REVIEW 5 cited by

RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.07940 v2 pith:XETSP6YZ submitted 2024-05-13 cs.CL

Liam Dugan , Alyssa Hwang , Filip Trhlik , Josh Magnus Ludan , Andrew Zhu , Hainiu Xu , Daphne Ippolito , Chris Callison-Burch This is my paper · ORCID
classification cs.CL
keywords adversarialdetectorsmodelsraidattacksbenchmarkmachine-generatedtext
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Many commercial and open-source models claim to detect machine-generated text with extremely high accuracy (99% or more). However, very few of these detectors are evaluated on shared benchmark datasets and even when they are, the datasets used for evaluation are insufficiently challenging-lacking variations in sampling strategy, adversarial attacks, and open-source generative models. In this work we present RAID: the largest and most challenging benchmark dataset for machine-generated text detection. RAID includes over 6 million generations spanning 11 models, 8 domains, 11 adversarial attacks and 4 decoding strategies. Using RAID, we evaluate the out-of-domain and adversarial robustness of 8 open- and 4 closed-source detectors and find that current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models. We release our data along with a leaderboard to encourage future research.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark

    cs.CL 2026-01 conditional novelty 6.0 of 10

    Adding human-alignment augmentation (roleplaying, BPO, self-refine, RLDF) to machine-generated text both fools existing detectors and improves the generalization of detectors fine-tuned on it.

  2. Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Fine-tuning LLMs with DPO to push generated news and abstracts toward human style substantially reduces the F1 scores of state-of-the-art machine-generated text detectors.

  3. Small Is Enough: Per-User Style Rewriting of AI-Edited Text via LoRA Adapters

    cs.CL 2026-07 conditional novelty 5.0 of 10

    LoRA-adapted 0.5B-7B language models all reach the same automatic rewriting score (0.69), indicating model size does not change measured quality for this single-user style-rewriting task.

  4. Hijacking Text Heritage: Hiding the Human Signature through Homoglyphic Substitution

    cs.CR 2026-04 unverdicted novelty 5.0 of 10

    Replacing characters in at least 37.5% of words with visual homoglyphs degrades authorship verification scores enough to obfuscate style, with diminishing returns past 50%.

  5. A Comprehensive Dataset for Human vs. AI Generated Text Detection

    cs.CL 2025-10 reject novelty 4.0 of 10

    A dataset of ~58k NYT articles plus AI rewrites from six LLMs, evaluated with a rewrite-distance baseline reaching 58.35% detection and 8.92% attribution accuracy.

Pith tools