Pith. sign in

REVIEW 13 cited by

Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.12070 v3 pith:7SNFBXEI submitted 2024-01-22 cs.CL cs.AIcs.LG

Abhimanyu Hans , Avi Schwarzschild , Valeriia Cherepanova , Hamid Kazemi , Aniruddha Saha , Micah Goldblum , Jonas Geiping , Tom Goldstein This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords llmstextbinocularsrangechatgptdatageneratedlanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Detecting text generated by modern large language models is thought to be hard, as both LLMs and humans can exhibit a wide range of complex behaviors. However, we find that a score based on contrasting two closely related language models is highly accurate at separating human-generated and machine-generated text. Based on this mechanism, we propose a novel LLM detector that only requires simple calculations using a pair of pre-trained LLMs. The method, called Binoculars, achieves state-of-the-art accuracy without any training data. It is capable of spotting machine text from a range of modern LLMs without any model-specific modifications. We comprehensively evaluate Binoculars on a number of text sources and in varied situations. Over a wide range of document types, Binoculars detects over 90% of generated samples from ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being trained on any ChatGPT data.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 16 citations worldwide. Full citation record

  1. Attacks on Machine-Text Detectors Retain Stylistic Fingerprints

    cs.CL 2025-05 conditional novelty 7.0 of 10

    A style-aware paraphrasing attack evades all nine tested AI-text detectors at the single-document level, but multi-document analysis makes the attack detectable again.

  2. UTS at ELOQUENT 2026 Voight-Kampff: structural shifts in AI writing bypass state-of-the-art detectors

    cs.CR 2026-07 conditional novelty 6.5 of 10

    Structural register and narrative-form shifts make AI-written text evade adversarially retrained detectors, winning the ELOQUENT 2026 Voight-Kampff competition.

  3. AEyeDE: An Attention-Based Attribution Framework for AI-Generated Text Detection

    cs.CL 2026-04 conditional novelty 6.0 of 10

    Attention attribution maps from a white-box proxy Transformer, classified by a lightweight CNN, provide a competitive and interpretable signal for AI-generated text detection.

  4. MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark

    cs.CL 2026-01 conditional novelty 6.0 of 10

    Adding human-alignment augmentation (roleplaying, BPO, self-refine, RLDF) to machine-generated text both fools existing detectors and improves the generalization of detectors fine-tuned on it.

  5. DEER: Disentangled Mixture of Experts with Instance-Adaptive Routing for Generalizable Machine-Generated Text Detection

    cs.CL 2025-11 conditional novelty 6.0 of 10

    DEER, a disentangled mixture-of-experts detector with RL-based instance routing, reports F1 gains of about 1.4 in-domain and 5.3 points out-of-domain over prior MGT detectors.

  6. Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry

    cs.CL 2025-09 conditional novelty 6.0 of 10

    A new modern Chinese poetry detection benchmark shows most current AI-text detectors are unreliable, particularly when LLMs imitate a human style.

  7. WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipedia

    cs.CL 2025-07 conditional novelty 6.0 of 10

    WETBench shows that existing machine-generated text detectors, particularly zero-shot methods, underperform on task-specific Wikipedia editing scenarios, with supervised detectors averaging 78% accuracy and zero-shot ...

  8. PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning

    cs.CL 2025-06 conditional novelty 6.0 of 10

    PhantomHunter detects text from privately fine-tuned LLMs by learning shared token-probability traits within LLaMA, Gemma and Mistral families, reporting F1 above 96% on held-out derivatives.

  9. Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Fine-tuning LLMs with DPO to push generated news and abstracts toward human style substantially reduces the F1 scores of state-of-the-art machine-generated text detectors.

  10. Human-LLM Coevolution: Evidence from Academic Writing

    cs.CL 2025-02 conditional novelty 6.0 of 10

    After ChatGPT-style words were publicly flagged in early 2024, their frequency in arXiv abstracts dropped, while other common LLM-favored words kept rising, suggesting authors are adapting their writing to avoid detection.

  11. GPT Editors, Not Authors: The Stylistic Footprint of LLMs in Academic Preprints

    cs.CL 2025-05 reject novelty 5.0 of 10

    Across 2,408 arXiv preprints, LLM-typical word usage does not cluster in any section, indicating that AI assistance, when used, is uniform rather than limited to specific parts of a paper.

  12. DP-MGTD: Privacy-Preserving Machine-Generated Text Detection via Adaptive Differentially Private Entity Sanitization

    cs.CR 2026-01 reject novelty 4.0 of 10

    DP-MGTD claims that applying differential-privacy entity sanitization amplifies human-vs-machine text separability, reaching F1 > 0.99 on MGTBench-2.0 while satisfying an epsilon-DP guarantee.

  13. A Comprehensive Dataset for Human vs. AI Generated Text Detection

    cs.CL 2025-10 reject novelty 4.0 of 10

    A dataset of ~58k NYT articles plus AI rewrites from six LLMs, evaluated with a rewrite-distance baseline reaching 58.35% detection and 8.92% attribution accuracy.

Pith tools