Pith. sign in

REVIEW 2 cited by

GenAI Content Detection Task 3: Cross-Domain Machine-Generated Text Detection Challenge

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.08913 v1 pith:WYZEK4RZ submitted 2025-01-15 cs.CL cs.LG

Liam Dugan , Andrew Zhu , Firoj Alam , Preslav Nakov , Marianna Apidianaki , Chris Callison-Burch This is my paper · ORCID
classification cs.CLcs.LG
keywords textdetectiondomainsmanymodelssharedtaskable
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recently there have been many shared tasks targeting the detection of generated text from Large Language Models (LLMs). However, these shared tasks tend to focus either on cases where text is limited to one particular domain or cases where text can be from many domains, some of which may not be seen during test time. In this shared task, using the newly released RAID benchmark, we aim to answer whether or not models can detect generated text from a large, yet fixed, number of domains and LLMs, all of which are seen during training. Over the course of three months, our task was attempted by 9 teams with 23 detector submissions. We find that multiple participants were able to obtain accuracies of over 99% on machine-generated text from RAID while maintaining a 5% False Positive Rate -- suggesting that detectors are able to robustly detect text from many domains and models simultaneously. We discuss potential interpretations of this result and provide directions for future research.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Fine-tuning LLMs with DPO to push generated news and abstracts toward human style substantially reduces the F1 scores of state-of-the-art machine-generated text detectors.

  2. Characterizing Human-Likeness in AI Generated Poetry: A Zero-shot Classification Study

    cs.CL 2026-07 conditional novelty 5.0 of 10

    Zero-shot Gemma 4 classification separates AI from human poems at 90% weighted F1, while human evaluators reach only 45% accuracy, and the study catalogs the linguistic attributes behind correct and incorrect detections.

Pith tools