Pith. sign in

REVIEW 4 major objections 5 minor 24 references

Watermarking Needs Input Repetition Masking

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read Through ordinary conversation, an unwatermarked model can begin producing text that watermark detectors attribute to the watermarked model, because it reuses n-grams from the prompt.

desk verdict Real but narrow false-positive mechanism (small models reusing watermarked prompt n-grams), wrapped in an abstract that oversells it as general 'mimicry'. read the letter →

arxiv 2504.12229 v1 pith:WKJKDDEQ submitted 2025-04-16 cs.LG cs.CLcs.CR

classification cs.LGcs.CLcs.CR
keywords LLMwatermarkingmimicryinputrepetitionmaskingconversationaladaptationsynthetictextdetectionBinocularsfalsepositivesn-gram
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that LLM watermarking has a vulnerability that current evaluations overlook: conversation itself transfers the watermark. When an unwatermarked model responds to text produced by a watermarked model, it reuses chunks of that input, and those reused n-grams carry the watermark signal into otherwise unwatermarked output. Across multiple turns the effect grows, so an unwatermarked generator can produce responses that watermark detectors attribute to the watermarked model, and occasionally the response watermark is even stronger than the prompt's. The paper also reports that humans in long conversations with chatbots begin producing text that a state-of-the-art detector scores as more machine-like. If the claim is right, reliable watermarking needs input repetition masking and substantially lower false-positive thresholds.

What carries the argument

The load-bearing mechanism is n-gram reuse: a response inherits a watermark when it reuses token sequences from a watermarked prompt, because the detector scores n-gram statistics against a secret key. The experiments use two standard watermarking schemes (a green/red-list sampling bias and a hash-based scheme), the Binoculars zero-shot synthetic-text detector, and a de-duplication step that strips n-grams shared between input and output to isolate the repetition effect.

What would settle it

For the human claim, count the fraction of human turns whose Binoculars score exceeds the calibrated 1% false-positive threshold across conversation length; if that fraction stays flat while raw score variance grows, the mimicry signal is not real. For the model claim, rerun the model-model conversations after removing every n-gram that appears in the watermarked prompt from the unwatermarked response; if watermarked detections persist, input repetition is not the whole mechanism.

Watch

Extended reading notes

Core claim

The central claim is that mimicry of watermark signals is real and is driven by input repetition. In model-to-model conversations using two standard watermarking schemes, a small unwatermarked model (Guanaco-7B or 13B) responding to a watermarked prompt produced a watermarked response in a measurable fraction of cases — up to about 13% of five-response blocks in one configuration and more in some cross-size settings — and the effect grew with conversation length. Crucially, when the authors removed (de-duplicated) the n-grams shared between the watermarked prompt and the unwatermarked response, the mimicry disappeared, which identifies repetition of input tokens as the mechanism. In human-chatbot conversations, they observed that the variance of Binoculars detector scores increased over turns for humans, which they interpret as humans adapting toward LLM-like language; the same analysis shows the opposite trend for the LLM side. The paper concludes that watermarking schemes must assume lower false-positive rates and longer seeding sequences, and that repetition masking should be extended to prompts.

Load-bearing premise

The human-mimicry conclusion rests on interpreting a widening spread of detector scores as humans becoming more machine-like, without showing that more human turns actually cross the detector's false-positive threshold.

Editorial extensions

If this is right

  • An unwatermarked model can be falsely attributed as the source of watermarked text after only a few turns of conversation, so real-world false-positive rates may be much higher than academic watermarking evaluations suggest.
  • Because the effect grows as more responses are concatenated, long conversations are where misattribution risk concentrates.
  • Removing shared n-grams between prompt and response eliminates the observed mimicry in the tested setups, making input repetition masking a direct defensive measure.
  • Longer n-grams reduce mimicry but weaken watermark robustness, so watermark designers face a trade-off rather than a free fix.
  • Larger, more capable models showed little or no mimicry in the tested setting, suggesting the vulnerability is most pressing for smaller or less capable deployed models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If input repetition is the mechanism, then a straightforward attack is to ask an unwatermarked model to paraphrase or echo a watermarked sample; any token-level watermark that survives copying becomes a false-attribution channel.
  • The de-duplication result implies a cheap detector-side fix: ignore n-grams already present in the prompt or conversation context before computing the watermark score, which would suppress this class of false positives without altering generators.
  • The human result could be tested cleanly by measuring the fraction of human turns crossing a fixed detector threshold; if that fraction stays flat while score variance rises, the practical risk to third-party detectors may be smaller than the paper's variance-based reading suggests.
  • Conversation logs may act like training data: if chat transcripts containing watermarked text are later used for fine-tuning, the watermark statistics could propagate further, extending the paper's in-context finding to fine-tuning pipelines.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces the concept of conversational 'mimicry' of LLM watermarks and machine-text detectors. In model-model experiments, a Guanaco model watermarked with either the Aaronson or Kirchenbauer scheme produces prompts, and an unwatermarked Guanaco model responds; the paper reports that unwatermarked responses are watermarked at rates above the random baseline (e.g., Table 1: Kirchenbauer P&R reaches 18.2% with five concatenated responses for the 7b(w)-vs-13b pair). The authors report that GPT-4o does not exhibit this behavior, and a de-duplication control (Figure 8) is said to remove the observed mimicry, supporting input repetition as the mechanism. In human experiments, the paper uses Binoculars scores and claims that the variance of human scores increases over long dialogues, which it interprets as humans becoming more LLM-like. The paper recommends lower false-positive thresholds, longer n-grams for watermark seeding, and input repetition masking to mitigate the effect.

Significance. If the small-model effect and the de-duplication causal claim hold, the paper identifies a concrete, previously underappreciated failure mode for current watermarking configurations in long conversational interactions, along with a sensible mitigation. The use of published watermark implementations, controlled baselines, and the de-duplication control are strengths; the paper also gives falsifiable measurements rather than deriving conclusions from fitted parameters. However, the significance as currently stated is much broader than the evidence: the GPT-4o rows show at most a weak effect, the human experiments never involve a watermark, and the de-duplication control is not quantitatively reported. The paper would be a useful contribution if its claims were re-scoped and the key control were quantified; in its present form, the abstract and introduction overstate the findings.

major comments (4)
  1. [Abstract, §4.1, Table 1] The paper's central claim that 'both humans and LLMs end up mimicking, including the watermarking signal' is not supported by the evidence for capable models. In Table 1, the GPT-4o rows for ngram=4 at the stated 1% false-positive threshold show P&R rates of 0.1% for Aaronson and 0.6% for Kirchenbauer at #R=1, and the text itself states that 'the mimicry observed for smaller models does not hold.' The abstract and introduction nevertheless make a general claim without quantifying this negative result or testing it against the 0.01% independence baseline with confidence intervals. Please re-scope the claims to small, weaker models, or reframe the conclusion so that the GPT-4o negative result is explicitly part of the central message.
  2. [§4.2, Figure 3] The human-mimicry result is inferred solely from an increase in the variance of Binoculars scores over conversation turns. The paper states that the detector is calibrated for a 1% false-positive rate but never reports the fraction of human turns that cross that fixed threshold, nor any significance test or error bars for the variance trend. A variance shift could be produced by topic drift, turn-length changes, or other confounds, none of which constitute mimicry. Moreover, the human conversations contain no watermark, so the abstract's claim that humans mimic 'the watermarking signal' is not operationalized in this experiment. Please report threshold-crossing rates and include a human baseline to validate the variance interpretation.
  3. [Figure 8, §4.1] The de-duplication control is the load-bearing evidence for input repetition as the mechanism and for the title's proposed remedy, but the figure reports no quantitative results: no de-duplicated P&R or P<R rates, no confidence intervals, and no test statistics are given in the text or caption. The three subpanels also appear visually indistinguishable, and the caption only asserts that de-duplication 'removes the observed mimicry behavior.' Without the de-duplicated numbers, the reader cannot tell whether the effect is fully eliminated, reduced to baseline, or merely attenuated. Please provide the corresponding Table 1 columns after masking n-grams shared between the watermarked input and the unwatermarked output.
  4. [§5, Figure 2] The recommendation to use longer word sequences for watermark seeding is not quantitatively supported as a mitigation in this paper. Figure 2 shows that response-side watermark rates decrease with n-gram length for the Guanaco pairs, but the paper does not report how the same n-gram lengths affect watermark robustness, such as detection power on genuinely watermarked text, in the same experimental setup. The claimed trade-off between reduced mimicry and reduced robustness is therefore not measured. Please add robustness numbers for the same n-gram values, or soften the recommendation accordingly.
minor comments (5)
  1. [Throughout] The model name is spelled inconsistently as 'Guanco' and 'Guanaco'; please use a single spelling.
  2. [Table 1] Standard deviations are reported only for the GPT-4o rows; please provide variance estimates for all rows, since the text acknowledges large variance across the three hash keys.
  3. [Figure 3] The y-axis label '% of text below given threshold (3 std.)' is not defined in the text; please clarify how the three-standard-deviation bands are computed and what the plotted quantity represents.
  4. [Appendix B] The original conversation prompt appears to be truncated in the extracted text; please include the complete prompt used in the experiments.
  5. [Reproducibility] The paper does not state whether code, exact dataset versions, and detector configurations will be released; please add a reproducibility statement, as the main table depends on precise ThreeBricks settings and hash keys.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the mimicry results come from external watermark detectors and independent de-duplication controls, with only minor non-load-bearing self-citations.

full rationale

The paper's central claim is empirical: unwatermarked Guanaco models, when prompted with watermarked text, sometimes produce responses that the Kirchenbauer/Aaronson detectors flag as watermarked. The detectors are taken from published implementations (ThreeBricks), and the P&R/P<R quantities are operational outcome measures, not quantities fitted to the data. No parameter is fit and then renamed as a prediction. The de-duplication control in Figure 8 is an independent manipulation rather than a refit: the paper states that de-duplicating n-grams between watermarked input and non-watermarked output removes the observed mimicry, and the examples in Appendix C show verbatim reuse of prompt phrases. Even though the de-duplicated numbers are not reported quantitatively, that is a reporting weakness, not circularity. The human-side Binoculars analysis infers mimicry from an increase in score variance rather than from a fixed false-positive threshold, but this is a construct-validity concern, not a circular derivation. The paper does cite prior work by one of its authors (Shumailov et al., 2023; Wyllie et al., 2024; Dathathri et al., 2024a/b), but these citations are contextual background and do not carry the argument; no uniqueness theorem from the authors is invoked, and no ansatz is smuggled in via self-citation. The recommendation to use longer n-grams follows directly from the measured n-gram trend in Figure 2 and is not a fitted input. Overall, the derivation chain is self-contained against external watermark detectors and benchmarks, so the circularity burden is low.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper is empirical and its central result does not require fitted constants. The key unstated premises are the validity of the published watermark and detector implementations and the interpretive leap from Binoculars-score variance to human mimicry. No new entities are introduced.

free parameters (2)
  • Watermark n-gram length = 4 for main table; varied 2,4,6,8 in Figure 2
    Chosen by authors as the watermark configuration; the paper shows mimicry weakens at longer n-grams, so this is an experimental variable, not a fitted constant.
  • Watermark detection threshold = 0.01
    Standard false-positive threshold used for both watermarking schemes; affects all reported rates but is not fitted to the mimicry result.
assumptions (4)
  • domain assumption Humans and LLMs adapt their language to their conversational partner.
    Borrowed from cited psycholinguistics literature (Chang et al., 2012; Mol et al., 2012) and extended to LLMs.
  • domain assumption ThreeBricks implementations of Aaronson and Kirchenbauer watermarks faithfully realize the published schemes.
    All watermark experiments rely on this library and its detection scores.
  • domain assumption ShareGPT and WildChat human turns are genuine, unedited human text suitable for adaptation measurements.
    Section 4.2 uses these public datasets; the paper itself lists this as a limitation.
  • ad hoc to paper Increasing variance of Binoculars scores over dialogue indicates human mimicry of LLM-like text.
    Section 4.2 interprets variance growth as humans becoming more machine-like, but no fixed-threshold false-positive analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Watermarking Needs Input Repetition Masking." pith.science (2026). https://pith.science/paper/WKJKDDEQ

@misc{pith2026250412229,
  author       = {Pith},
  title        = {Pith review of: Watermarking Needs Input Repetition Masking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WKJKDDEQ}},
  note         = {Machine review of arXiv:2504.12229}
}
abstract

Recent advancements in Large Language Models (LLMs) raised concerns over potential misuse, such as for spreading misinformation. In response two counter measures emerged: machine learning-based detectors that predict if text is synthetic, and LLM watermarking, which subtly marks generated text for identification and attribution. Meanwhile, humans are known to adjust language to their conversational partners both syntactically and lexically. By implication, it is possible that humans or unwatermarked LLMs could unintentionally mimic properties of LLM generated text, making counter measures unreliable. In this work we investigate the extent to which such conversational adaptation happens. We call the concept $\textit{mimicry}$ and demonstrate that both humans and LLMs end up mimicking, including the watermarking signal even in seemingly improbable settings. This challenges current academic assumptions and suggests that for long-term watermarking to be reliable, the likelihood of false positives needs to be significantly lower, while longer word sequences should be used for seeding watermarking mechanisms.

Figures

Figures reproduced from arXiv: 2504.12229 by the authors.

Figure 1
Figure 1. An intuitive description of watermark mimicry. Here, a watermarked prompt is used together with an unwatermarked model. During a conversation parts of the original watermerk ( green ) prompt are reused by the model, leading to watermark mimicry ( red ), resulting in an unwatermarked model outputting watermarked response. Importantly, the watermark can even be stronger in the response, since it can by a coincidence p… view at source ↗
Figure 2
Figure 2. Aaronson (2022) scheme, changing ngram size, blue shows percentage of watermarked prompts, orange shows percentage of watermarked responses green shows percentage of watermarked responses where response watermark is stronger than in the prompt, red shows percentage of cases with both prompt and response watermarked. 100 200 300 Conversation turns 1.0 0.5 0.0 0.5 1.0 1.5 2.0 % of text below given threshold (3 std.) B… view at source ↗
Figure 3
Figure 3. Human–LLM dialogues (split in Figure [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Aaronson. watermarking with Guanco-13b and varying temperatures 0 10 20 30 40 50 Turn 25 20 15 10 5 0 5 p-value log10 guanaco-13b (temp=0.0) w/ Kirchenbauer et al.; avg. over 20 conversations, grouped by 3 water: 0.062*x + -10.0 normal: -0.027*x + -0.2 watermark thres.…
Figure 7
Figure 7. Figure 7: Kirchenbauer et al. (2023) watermarking with Guanco-13b and varying temperatures. B Original conversation prompt 1 promptstart = " A chat between two people talking about everyday topics such [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Kirchenbauer et al. (2023) watermarking with Guanco-13b and varying temperatures, de-duplicating ngrams between the watermarked input and the non￾watermarked output. We note that this de-duplication removes the observed mimicry behavior. as Weather , Family , Work / Sc…
Figure 9
Figure 9. Figure 9: Aaronson (2022) with Guanco-7b (w) v 7b and varying ngrams, T=1 2 4 6 8 ngram 2 0 2 4 6 8 Percentage watermarked 30 40 50 60 70 80 Percentage watermarked guanaco-7b w/ Aaronson et al. vs. guanaco-13b avg. over 20 conversations, grouped by 1 Prompt Response P < R (and P…
Figure 10
Figure 10. Figure 10: Aaronson (2022) with Guanco-7b (w) v 13b and varying ngrams, T=1 2 4 6 8 ngram 0 2 4 6 8 10 Percentage watermarked 10 15 20 25 30 35 40 45 Percentage watermarked guanaco-13b w/ Aaronson et al. vs. guanaco-7b avg. over 20 conversations, grouped by 1 Prompt Response P <…
Figure 11
Figure 11. Figure 11: Aaronson (2022) with Guanco-13b (w) v 7b and varying ngrams, T=1 2 4 6 8 ngram 0 2 4 6 8 Percentage watermarked 15 20 25 30 35 40 45 50 Percentage watermarked guanaco-13b w/ Aaronson et al. vs. guanaco-13b avg. over 20 conversations, grouped by 1 Prompt Response P < R…
Figure 12
Figure 12. Figure 12: Aaronson (2022) watermarking with Guanco-13b (w) v 13b and varying ngrams, T=1 [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: Datasets split with Binoculars scoring. Showing three standard deviations. [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 8 canonical work pages

  1. [1]

    Scott Aaronson. 2022. My ai safety lecture for ut effective altruism. Shtetl-Optimized: The blog of Scott Aaronson. Retrieved on September, 11:2023

  2. [2]

    Scott Aaronson and Hendrik Kirchner. 2023. Watermarking gpt outputs

  3. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

  4. [4]

    Franklin Chang, Marius Janciauskas, and Hartmut Fitz. 2012. Language adaptation and learning: Getting explicit about implicit learning. Language and Linguistics Compass, 6(5):259--278

  5. [5]

    Miranda Christ, Sam Gunn, and Or Zamir. 2023. https://arxiv.org/abs/2306.09194 Undetectable watermarks for language models . Preprint, arXiv:2306.09194

  6. [6]

    Sumanth Dathathri, Pushmeet Kohli, Vandana Bachani, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Abi See, and Johannes Welbl. 2024 a . Watermarking ai-generated text and video with synthid. Google DeepMind blog

  7. [7]

    Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. 2024 b . Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818--823

  8. [8]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. https://arxiv.org/abs/2305.14314 Qlora: Efficient finetuning of quantized llms . Preprint, arXiv:2305.14314

Show all 24 references
  1. [9]

    Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. 2023. Three bricks to consolidate watermarks for large language models. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1--6. IEEE

  2. [10]

    Gemini-Team Gemini-Team, Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry, Lepikhin, Timothy Lillicrap, Jean baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew Dai, Katie Milli...

  3. [11]

    Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. 2024. https://arxiv.org/abs/2312.04469 On the learnability of watermarks for language models . Preprint, arXiv:2312.04469

  4. [12]

    Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. https://arxiv.org/abs/2401.12070 Spotting llms with binoculars: Zero-shot detection of machine-generated text . Preprint, arXiv:2401.12070

  5. [13]

    Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2023. https://arxiv.org/abs/2310.10669 Unbiased watermark for large language models . Preprint, arXiv:2310.10669

  6. [14]

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. https://proceedings.mlr.press/v202/kirchenbauer23a.html A watermark for large language models . In Proceedings of the 40th International Conference on Machine Learning, volume 202 o...

  7. [15]

    Manning, and Chelsea Finn

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. https://arxiv.org/abs/2301.11305 Detectgpt: Zero-shot machine-generated text detection using probability curvature . Preprint, arXiv:2301.11305

  8. [16]

    Lisette Mol, Emiel Krahmer, Alfons Maes, and Marc Swerts. 2012. Adaptation in gesture: Converging hands or converging minds? Journal of Memory and Language, 66(1):249--264

  9. [17]

    OpenAI. 2024. https://openai.com/index/disrupting-malicious-uses-of-ai-by-state-affiliated-threat-actors/ Disrupting malicious uses of ai by state-affiliated threat actors . OpenAI blog

  10. [18]

    Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. https://arxiv.org/abs/2306.01116 The refinedweb dataset for falcon llm: Outperforming curated corpora wit...

  11. [19]

    Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. 2024. https://arxiv.org/abs/2402.14904 Watermarking makes language models radioactive . Preprint, arXiv:2402.14904

  12. [20]

    Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. 2023. https://arxiv.org/abs/2305.17493 The curse of recursion: Training on generated data makes models forget . Preprint, arXiv:2305.17493

  13. [21]

    Sierra Wyllie, Ilia Shumailov, and Nicolas Papernot. 2024. Fairness feedback loops: Training on synthetic data amplifies bias. ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT 2024)

  14. [22]

    KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. 2024. https://arxiv.org/abs/2308.00221 Advancing beyond identification: Multi-bit watermark for large language models . Preprint, arXiv:2308.00221

  15. [23]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  16. [24]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.