REVIEW 4 major objections 5 minor 24 references
Watermarking Needs Input Repetition Masking
T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Through ordinary conversation, an unwatermarked model can begin producing text that watermark detectors attribute to the watermarked model, because it reuses n-grams from the prompt.
desk verdict Real but narrow false-positive mechanism (small models reusing watermarked prompt n-grams), wrapped in an abstract that oversells it as general 'mimicry'. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is n-gram reuse: a response inherits a watermark when it reuses token sequences from a watermarked prompt, because the detector scores n-gram statistics against a secret key. The experiments use two standard watermarking schemes (a green/red-list sampling bias and a hash-based scheme), the Binoculars zero-shot synthetic-text detector, and a de-duplication step that strips n-grams shared between input and output to isolate the repetition effect.
What would settle it
For the human claim, count the fraction of human turns whose Binoculars score exceeds the calibrated 1% false-positive threshold across conversation length; if that fraction stays flat while raw score variance grows, the mimicry signal is not real. For the model claim, rerun the model-model conversations after removing every n-gram that appears in the watermarked prompt from the unwatermarked response; if watermarked detections persist, input repetition is not the whole mechanism.
Extended reading notes
Core claim
The central claim is that mimicry of watermark signals is real and is driven by input repetition. In model-to-model conversations using two standard watermarking schemes, a small unwatermarked model (Guanaco-7B or 13B) responding to a watermarked prompt produced a watermarked response in a measurable fraction of cases — up to about 13% of five-response blocks in one configuration and more in some cross-size settings — and the effect grew with conversation length. Crucially, when the authors removed (de-duplicated) the n-grams shared between the watermarked prompt and the unwatermarked response, the mimicry disappeared, which identifies repetition of input tokens as the mechanism. In human-chatbot conversations, they observed that the variance of Binoculars detector scores increased over turns for humans, which they interpret as humans adapting toward LLM-like language; the same analysis shows the opposite trend for the LLM side. The paper concludes that watermarking schemes must assume lower false-positive rates and longer seeding sequences, and that repetition masking should be extended to prompts.
Load-bearing premise
The human-mimicry conclusion rests on interpreting a widening spread of detector scores as humans becoming more machine-like, without showing that more human turns actually cross the detector's false-positive threshold.
Editorial extensions
If this is right
- An unwatermarked model can be falsely attributed as the source of watermarked text after only a few turns of conversation, so real-world false-positive rates may be much higher than academic watermarking evaluations suggest.
- Because the effect grows as more responses are concatenated, long conversations are where misattribution risk concentrates.
- Removing shared n-grams between prompt and response eliminates the observed mimicry in the tested setups, making input repetition masking a direct defensive measure.
- Longer n-grams reduce mimicry but weaken watermark robustness, so watermark designers face a trade-off rather than a free fix.
- Larger, more capable models showed little or no mimicry in the tested setting, suggesting the vulnerability is most pressing for smaller or less capable deployed models.
Reading between the lines
- If input repetition is the mechanism, then a straightforward attack is to ask an unwatermarked model to paraphrase or echo a watermarked sample; any token-level watermark that survives copying becomes a false-attribution channel.
- The de-duplication result implies a cheap detector-side fix: ignore n-grams already present in the prompt or conversation context before computing the watermark score, which would suppress this class of false positives without altering generators.
- The human result could be tested cleanly by measuring the fraction of human turns crossing a fixed detector threshold; if that fraction stays flat while score variance rises, the practical risk to third-party detectors may be smaller than the paper's variance-based reading suggests.
- Conversation logs may act like training data: if chat transcripts containing watermarked text are later used for fine-tuning, the watermark statistics could propagate further, extending the paper's in-context finding to fine-tuning pipelines.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the concept of conversational 'mimicry' of LLM watermarks and machine-text detectors. In model-model experiments, a Guanaco model watermarked with either the Aaronson or Kirchenbauer scheme produces prompts, and an unwatermarked Guanaco model responds; the paper reports that unwatermarked responses are watermarked at rates above the random baseline (e.g., Table 1: Kirchenbauer P&R reaches 18.2% with five concatenated responses for the 7b(w)-vs-13b pair). The authors report that GPT-4o does not exhibit this behavior, and a de-duplication control (Figure 8) is said to remove the observed mimicry, supporting input repetition as the mechanism. In human experiments, the paper uses Binoculars scores and claims that the variance of human scores increases over long dialogues, which it interprets as humans becoming more LLM-like. The paper recommends lower false-positive thresholds, longer n-grams for watermark seeding, and input repetition masking to mitigate the effect.
Significance. If the small-model effect and the de-duplication causal claim hold, the paper identifies a concrete, previously underappreciated failure mode for current watermarking configurations in long conversational interactions, along with a sensible mitigation. The use of published watermark implementations, controlled baselines, and the de-duplication control are strengths; the paper also gives falsifiable measurements rather than deriving conclusions from fitted parameters. However, the significance as currently stated is much broader than the evidence: the GPT-4o rows show at most a weak effect, the human experiments never involve a watermark, and the de-duplication control is not quantitatively reported. The paper would be a useful contribution if its claims were re-scoped and the key control were quantified; in its present form, the abstract and introduction overstate the findings.
major comments (4)
- [Abstract, §4.1, Table 1] The paper's central claim that 'both humans and LLMs end up mimicking, including the watermarking signal' is not supported by the evidence for capable models. In Table 1, the GPT-4o rows for ngram=4 at the stated 1% false-positive threshold show P&R rates of 0.1% for Aaronson and 0.6% for Kirchenbauer at #R=1, and the text itself states that 'the mimicry observed for smaller models does not hold.' The abstract and introduction nevertheless make a general claim without quantifying this negative result or testing it against the 0.01% independence baseline with confidence intervals. Please re-scope the claims to small, weaker models, or reframe the conclusion so that the GPT-4o negative result is explicitly part of the central message.
- [§4.2, Figure 3] The human-mimicry result is inferred solely from an increase in the variance of Binoculars scores over conversation turns. The paper states that the detector is calibrated for a 1% false-positive rate but never reports the fraction of human turns that cross that fixed threshold, nor any significance test or error bars for the variance trend. A variance shift could be produced by topic drift, turn-length changes, or other confounds, none of which constitute mimicry. Moreover, the human conversations contain no watermark, so the abstract's claim that humans mimic 'the watermarking signal' is not operationalized in this experiment. Please report threshold-crossing rates and include a human baseline to validate the variance interpretation.
- [Figure 8, §4.1] The de-duplication control is the load-bearing evidence for input repetition as the mechanism and for the title's proposed remedy, but the figure reports no quantitative results: no de-duplicated P&R or P<R rates, no confidence intervals, and no test statistics are given in the text or caption. The three subpanels also appear visually indistinguishable, and the caption only asserts that de-duplication 'removes the observed mimicry behavior.' Without the de-duplicated numbers, the reader cannot tell whether the effect is fully eliminated, reduced to baseline, or merely attenuated. Please provide the corresponding Table 1 columns after masking n-grams shared between the watermarked input and the unwatermarked output.
- [§5, Figure 2] The recommendation to use longer word sequences for watermark seeding is not quantitatively supported as a mitigation in this paper. Figure 2 shows that response-side watermark rates decrease with n-gram length for the Guanaco pairs, but the paper does not report how the same n-gram lengths affect watermark robustness, such as detection power on genuinely watermarked text, in the same experimental setup. The claimed trade-off between reduced mimicry and reduced robustness is therefore not measured. Please add robustness numbers for the same n-gram values, or soften the recommendation accordingly.
minor comments (5)
- [Throughout] The model name is spelled inconsistently as 'Guanco' and 'Guanaco'; please use a single spelling.
- [Table 1] Standard deviations are reported only for the GPT-4o rows; please provide variance estimates for all rows, since the text acknowledges large variance across the three hash keys.
- [Figure 3] The y-axis label '% of text below given threshold (3 std.)' is not defined in the text; please clarify how the three-standard-deviation bands are computed and what the plotted quantity represents.
- [Appendix B] The original conversation prompt appears to be truncated in the extracted text; please include the complete prompt used in the experiments.
- [Reproducibility] The paper does not state whether code, exact dataset versions, and detector configurations will be released; please add a reproducibility statement, as the main table depends on precise ThreeBricks settings and hash keys.
Circularity Check
No significant circularity: the mimicry results come from external watermark detectors and independent de-duplication controls, with only minor non-load-bearing self-citations.
full rationale
The paper's central claim is empirical: unwatermarked Guanaco models, when prompted with watermarked text, sometimes produce responses that the Kirchenbauer/Aaronson detectors flag as watermarked. The detectors are taken from published implementations (ThreeBricks), and the P&R/P<R quantities are operational outcome measures, not quantities fitted to the data. No parameter is fit and then renamed as a prediction. The de-duplication control in Figure 8 is an independent manipulation rather than a refit: the paper states that de-duplicating n-grams between watermarked input and non-watermarked output removes the observed mimicry, and the examples in Appendix C show verbatim reuse of prompt phrases. Even though the de-duplicated numbers are not reported quantitatively, that is a reporting weakness, not circularity. The human-side Binoculars analysis infers mimicry from an increase in score variance rather than from a fixed false-positive threshold, but this is a construct-validity concern, not a circular derivation. The paper does cite prior work by one of its authors (Shumailov et al., 2023; Wyllie et al., 2024; Dathathri et al., 2024a/b), but these citations are contextual background and do not carry the argument; no uniqueness theorem from the authors is invoked, and no ansatz is smuggled in via self-citation. The recommendation to use longer n-grams follows directly from the measured n-gram trend in Figure 2 and is not a fitted input. Overall, the derivation chain is self-contained against external watermark detectors and benchmarks, so the circularity burden is low.
Assumptions & free parameters
free parameters (2)
- Watermark n-gram length =
4 for main table; varied 2,4,6,8 in Figure 2
- Watermark detection threshold =
0.01
assumptions (4)
- domain assumption Humans and LLMs adapt their language to their conversational partner.
- domain assumption ThreeBricks implementations of Aaronson and Kirchenbauer watermarks faithfully realize the published schemes.
- domain assumption ShareGPT and WildChat human turns are genuine, unedited human text suitable for adaptation measurements.
- ad hoc to paper Increasing variance of Binoculars scores over dialogue indicates human mimicry of LLM-like text.
Cite this review
Pith. "Pith review of Watermarking Needs Input Repetition Masking." pith.science (2026). https://pith.science/paper/WKJKDDEQ
@misc{pith2026250412229,
author = {Pith},
title = {Pith review of: Watermarking Needs Input Repetition Masking},
year = {2026},
howpublished = {\url{https://pith.science/paper/WKJKDDEQ}},
note = {Machine review of arXiv:2504.12229}
}
abstract
Recent advancements in Large Language Models (LLMs) raised concerns over potential misuse, such as for spreading misinformation. In response two counter measures emerged: machine learning-based detectors that predict if text is synthetic, and LLM watermarking, which subtly marks generated text for identification and attribution. Meanwhile, humans are known to adjust language to their conversational partners both syntactically and lexically. By implication, it is possible that humans or unwatermarked LLMs could unintentionally mimic properties of LLM generated text, making counter measures unreliable. In this work we investigate the extent to which such conversational adaptation happens. We call the concept $\textit{mimicry}$ and demonstrate that both humans and LLMs end up mimicking, including the watermarking signal even in seemingly improbable settings. This challenges current academic assumptions and suggests that for long-term watermarking to be reliable, the likelihood of false positives needs to be significantly lower, while longer word sequences should be used for seeding watermarking mechanisms.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Scott Aaronson. 2022. My ai safety lecture for ut effective altruism. Shtetl-Optimized: The blog of Scott Aaronson. Retrieved on September, 11:2023
work page 2022
-
[2]
Scott Aaronson and Hendrik Kirchner. 2023. Watermarking gpt outputs
work page 2023
-
[3]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...
arXiv 2020
-
[4]
Franklin Chang, Marius Janciauskas, and Hartmut Fitz. 2012. Language adaptation and learning: Getting explicit about implicit learning. Language and Linguistics Compass, 6(5):259--278
work page 2012
-
[5]
Miranda Christ, Sam Gunn, and Or Zamir. 2023. https://arxiv.org/abs/2306.09194 Undetectable watermarks for language models . Preprint, arXiv:2306.09194
arXiv 2023
-
[6]
Sumanth Dathathri, Pushmeet Kohli, Vandana Bachani, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Abi See, and Johannes Welbl. 2024 a . Watermarking ai-generated text and video with synthid. Google DeepMind blog
work page 2024
-
[7]
Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. 2024 b . Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818--823
work page 2024
-
[8]
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. https://arxiv.org/abs/2305.14314 Qlora: Efficient finetuning of quantized llms . Preprint, arXiv:2305.14314
arXiv 2023
Show all 24 references
-
[9]
Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. 2023. Three bricks to consolidate watermarks for large language models. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1--6. IEEE
2023
-
[10]
Gemini-Team Gemini-Team, Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry, Lepikhin, Timothy Lillicrap, Jean baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew Dai, Katie Milli...
2024 arXiv
-
[11]
Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. 2024. https://arxiv.org/abs/2312.04469 On the learnability of watermarks for language models . Preprint, arXiv:2312.04469
2024 arXiv
-
[12]
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. https://arxiv.org/abs/2401.12070 Spotting llms with binoculars: Zero-shot detection of machine-generated text . Preprint, arXiv:2401.12070
2024 arXiv
-
[13]
Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2023. https://arxiv.org/abs/2310.10669 Unbiased watermark for large language models . Preprint, arXiv:2310.10669
2023 arXiv
-
[14]
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. https://proceedings.mlr.press/v202/kirchenbauer23a.html A watermark for large language models . In Proceedings of the 40th International Conference on Machine Learning, volume 202 o...
2023
-
[15]
Manning, and Chelsea Finn
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. https://arxiv.org/abs/2301.11305 Detectgpt: Zero-shot machine-generated text detection using probability curvature . Preprint, arXiv:2301.11305
2023 arXiv
-
[16]
Lisette Mol, Emiel Krahmer, Alfons Maes, and Marc Swerts. 2012. Adaptation in gesture: Converging hands or converging minds? Journal of Memory and Language, 66(1):249--264
2012
-
[17]
OpenAI. 2024. https://openai.com/index/disrupting-malicious-uses-of-ai-by-state-affiliated-threat-actors/ Disrupting malicious uses of ai by state-affiliated threat actors . OpenAI blog
2024
-
[18]
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. https://arxiv.org/abs/2306.01116 The refinedweb dataset for falcon llm: Outperforming curated corpora wit...
2023 arXiv
-
[19]
Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. 2024. https://arxiv.org/abs/2402.14904 Watermarking makes language models radioactive . Preprint, arXiv:2402.14904
2024 arXiv
-
[20]
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. 2023. https://arxiv.org/abs/2305.17493 The curse of recursion: Training on generated data makes models forget . Preprint, arXiv:2305.17493
2023 arXiv
-
[21]
Sierra Wyllie, Ilia Shumailov, and Nicolas Papernot. 2024. Fairness feedback loops: Training on synthetic data amplifies bias. ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT 2024)
2024
-
[22]
KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. 2024. https://arxiv.org/abs/2308.00221 Advancing beyond identification: Multi-bit watermark for large language models . Preprint, arXiv:2308.00221
2024 arXiv
-
[23]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[24]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.