REVIEW 6 major objections 6 minor 22 references
Let Watermarks Speak: A Robust and Unforgeable Watermark for Language Models
T0 review · 6 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper claims the first undetectable, robust, single-bit watermark for language models, and chains hash-linked blocks so that prompt forgery and tampering are verifiable.
desk verdict A neat bit-level watermarking idea, but the formal write-up is broken at the sampler definition and detector equations, and the experiments are absent. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Dual Inverse Transform Sampling (DITS) rule, which replaces the model's sampler. For a next-bit distribution $(p(0), p(1))$ and a shared pseudorandom $r \in [0,1]$, DITS outputs a bit by checking which side of the predictor's threshold $r$ falls on, taking the complementary arrangement when the watermark bit is $1$; the detector reads each bit as $\hat{m}' = \mathbf{1}(r < \frac{1}{2}) \oplus b$ and aggregates the votes over a block. DITS does the work of making the watermark invisible: the marginal distribution of the output bit equals the predictor's distribution regardless of $m$, so the watermarked model is computationally indistinguishable from the original. Aggregation does the work of making the watermark reliable and robust: the vote mean concentrates around $\frac{1}{2} \pm \mathrm{gap}$, and a majority of flipped votes can be tolerated up to a Hamming radius. The multi-bit scheme adds a hash chain called a watermark link, where the payload of each block is one bit of the hash of the prompt or of the preceding content, and Verify recomputes the chain; this is the mechanism that turns robustness into unforgeability.
What would settle it
Measure $\min\{p(0),p(1)\}$ from a real token-level language model on ordinary prompts, compute the block length $N$ needed to make $\exp\!\left(-2N\,\mathrm{gap}^2\right)$ negligible where $\mathrm{gap}=\frac{1}{N}\sum_i \min\{p_i(0),p_i(1)\}$, and check whether that $N$ fits inside a realistic model response; if the measured gap is so small that $N$ exceeds practical response lengths, the central claim of practical effectiveness is falsified.
Extended reading notes
Core claim
The paper's central claim is that a one-bit watermark can be both statistically undetectable and robust, and that chaining such bits makes the watermark prefix-unforgeable. The key identity is the pair of complementary sampling and detection rules: a bit is generated by comparing a shared pseudorandom number $r$ against the predictor's thresholds under one arrangement for watermark bit $m=0$ and the mirror arrangement for $m=1$, and it is recovered by testing whether $r$ falls below $\frac{1}{2}$ and XORing the result with the observed bit. Because the sample mean of these per-bit votes sits at $\frac{1}{2} \pm \frac{1}{N}\sum_i \min\{p_i(0), p_i(1)\}$, the three states $0$, $1$, and $\bot$ (no watermark) are separated by a gap that Hoeffding's inequality makes overwhelming once enough bits are gathered. Repeating this within a block yields one watermark signal, and using the bits of $\mathrm{Hash}(\text{prompt} \| \text{previous block})$ as the signals for successive blocks gives a chain whose first link binds the text to its actual prompt and whose later links expose modification. The paper proves correctness, computational undetectability, and substitution robustness for the single-bit scheme, and prefix-unforgeability for the multi-bit chain under collision resistance of the hash.
Load-bearing premise
The whole construction stands on the assumption that the model's bit-level next-bit predictor keeps both bit values reasonably likely over enough generated positions; if real token-level models are often near-deterministic at the bit level, the statistical gap between watermarked and unwatermarked output shrinks and the block length needed becomes impractical.
Editorial extensions
If this is right
- A detector can now distinguish three states from a watermarked text: signal 0, signal 1, and no signal, so a single generated response can carry a meaningful payload rather than only a yes/no flag.
- Because the first watermark link carries the hash of the actual prompt, claiming a watermarked output was produced under a different prompt fails verification with overwhelming probability.
- Because later links carry hashes of the preceding content, any substitution that flips enough bits to change the content is exposed when Verify recomputes the chain, even though the watermark signal itself is still detectable.
- Watermarked text remains computationally indistinguishable from ordinary model output, so the scheme does not degrade text quality or change the sampling distribution.
- The scheme's substitution robustness is proportional to the square root of the block length, so longer responses tolerate more edits before detection fails.
Reading between the lines
- Beyond the paper, the same DITS construction could be applied per token rather than per bit by encoding token IDs into bit lists, but the effective minimum probability would then be governed by the least likely prefix across the vocabulary, which may be far smaller than the bit-level values the paper assumes; measuring this on real models is the natural next check.
- Beyond the paper, the robustness guarantee is stated only against substitution flips; deletions and insertions would desynchronize the shared randomness between embedder and detector, so an edit-robust extension would need an alignment or reset mechanism the paper does not provide.
- Beyond the paper, the hash-chain verification is keyed and private; turning it into a publicly verifiable scheme would require a signature or zero-knowledge component, which the paper explicitly avoids because bit payloads are expensive, so a public-verifiability version may need a different trade-off.
- Beyond the paper, prefix-unforgeability leaves the final open-ended link unchecked; a truncation attack that cuts the chain at a complete link boundary would still pass Verify, so the scheme detects modification only where a successor link exists.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a single-bit watermarking scheme for language models, claiming undetectability, robustness, and extensibility to a multi-bit unforgeable scheme that hashes the prompt and previous content into watermark links. The core construction is a 'Wat-Sampler' (Eq. 5) and a detector (Eq. 7), together with aggregation via Hoeffding bounds and a multi-bit chain built on hash values. The abstract also promises 'sufficient experiments' on popular language models. As written, however, the central sampler definition is self-referential and not computable, the detection formula is inconsistent with the paper's own figures and analysis, the decision rule in the algorithm is reversed relative to the derived expectations, the unforgeability proof is only a one-sentence appeal to collision resistance, and Section 5 contains no actual experiments or results.
Significance. The intended idea, a dual inverse-transform sampling that embeds a bit without changing the marginal output distribution, is a plausible and potentially useful building block for language-model watermarking. If the construction were correctly specified and the gaps in the analysis filled, the paper could be a valuable contribution to the undetectable- and unforgeable-watermark literature. However, the manuscript as submitted does not contain a well-defined central algorithm: Eq. (5) is not a sampling rule, and the detection rule is inverted with respect to the analysis. Because the formal claims are about an object different from the one actually defined, the current version does not establish its main results. The absence of any experimental content further weakens the paper's practical claims.
major comments (6)
- [§4.1, Eq. (5) and Algorithm 1] The definition b = 1(r < p(b)) ⊕ m is a fixed-point equation, not a computable sampling rule, because b appears on both sides. For p(0)=0.7, p(1)=0.3, m=0, and r=0.5, neither b=0 nor b=1 satisfies the equation, so the sampler is undefined on a positive-measure set of random inputs. The subsequent proofs (Claim 1, Figure 1, and the games in Figure 3) use a different, well-defined dual inverse-transform rule. The manuscript must replace Eq. (5) and Algorithm 1 with the explicitly sequential rule used in the proofs, or the central construction is absent.
- [§4.1, Eq. (7), Algorithm 2, and Figure 2] The detection formula m' = 1(r < 1/2) ⊕ b is the inverse of the rule that the paper's own analysis uses. With the construction in Figure 1, for p(0)=p(1)=1/2 and m=0, r<1/2 gives b=0, and Eq. (7) yields m'=1; the paper's text immediately after Eq. (7) claims the opposite. Figure 2 and Eqs. (8)-(13) are consistent with the rule m' = 1(r ≥ 1/2) ⊕ b, not with Eq. (7). This is not a typographical issue: the detection algorithm as written would output the wrong watermark bit.
- [§4.1, Eqs. (11)-(12) and Algorithm 4] Even if Eq. (7) were corrected to match Figure 2, the decision rule in Algorithm 4 and the bullet list in Section 4.1 is reversed. Equations (11) and (12) put E[X | m=0] below 1/2 and E[X | m=1] above 1/2, yet the algorithm outputs m=0 when X > 1/2 and m=1 when X < 1/2. The detector as specified would systematically misclassify the embedded bit.
- [§4.1 and §4.2 (Claim 2)] The stopping condition in Algorithm 3 relies on the empirical mean X deviating from 1/2 enough to make exp(-i(X-1/2)^2/2) negligible. The required block length depends on the gap δ = (1/N) Σ min{p_i(0), p_i(1)}. The paper only assumes δ > 0; it does not show that δ is non-negligible, nor that Embed terminates in expected polynomial time. For real token-level models converted to bits, min{p(0), p(1)} can be extremely small, and the necessary block length could be impractically large. The paper provides no measurement of this quantity, so the claimed practical effectiveness is unsupported.
- [§4.4, Claim 6] The proof of prefix-unforgeability consists of a single sentence citing collision resistance of the hash. The game in Definition 11 allows the adversary to modify the transcript arbitrarily, and the reduction is not established: an adversary need not find a hash collision, but could truncate, insert, or overwrite bits so that the decoded watermark bits pass Verify without any hash collision. The proof must explicitly model the decoding procedure, the robustness threshold, and the exact role of the hash, and show how any winning strategy leads to a collision or a second-preimage.
- [§5] The paper claims in the abstract to provide 'sufficient experiments' and that the results show the method is 'practically effective and robust,' but Section 5.1 ends after a single sentence ('We set the negl(λ) as e^{-λ}.') and no experiments, datasets, models, baselines, or result tables are included. The empirical claims are thus entirely unverified in the manuscript.
minor comments (6)
- [§2.2] 'gumble-softmax' should be 'Gumbel-softmax'.
- [§4.1, Lemma 1 title] 'Ineqality' should be 'Inequality'.
- [Algorithm 6, line 4] The condition 'if m_i ≠ ⊥' should be 'if m' ≠ ⊥', since the variable m_i is not defined at that point.
- [Definition 5 title] 'Single W atermark Output' should be 'Single Watermark Output'.
- [Figure 3] The pseudocode for G2 and G3 outputs 1 when r ≤ p(0), which is the inverse of the inverse-transform sampling described in the text; although the distribution is preserved up to relabeling, the code should match the intended mapping.
- [§3.2] The conversion from token-level language models to the bit-level Model is described only informally; Section 5.1 offers no additional implementation details despite its title.
Circularity Check
No significant circularity; the main derivation is not an input-output tautology, although the central sampler equation (5) is self-referential and ill-posed as written.
full rationale
The derivation chain is not circular in the sense of the rubric. The detector threshold 1/2 comes from the fixed symmetry of Detect-1bit (Eq. 7), not from a fitted parameter; block length is chosen adaptively by the embedder until a Hoeffding bound is negligible; and the entropy assumption on the Predictor is stated as an explicit assumption. Unforgeability of the multi-bit construction is reduced to collision resistance of Hash, an external standard assumption, and correctness and undetectability are inherited from the single-bit scheme rather than assumed. There are no load-bearing self-citations. What the manuscript does contain is a formal defect: Eq. (5), b = 1(r < p(b)) XOR m, defines the output bit in terms of p(b), so the right-hand side depends on the value being defined; for example, with p(0)=0.7, p(1)=0.3, m=0 and r=0.5, neither candidate bit satisfies the equation. The proof of Claim 1 instead uses the case split of Fig. 1, so the formal object supporting the central theorems is not well-defined as written. That is a correctness and well-definedness problem, not a circular derivation: the theorems are not made true by definition, and no result is rebuilt from its own conclusion. The experimental section is also missing its actual results, but that is a completeness issue.
Assumptions & free parameters
assumptions (5)
- domain assumption PRG/PRF output is computationally indistinguishable from uniform random bits.
- domain assumption Collision-resistant hash functions exist.
- domain assumption The model's next-bit distribution has non-negligible entropy, i.e., the Predictor does not always output deterministic distributions.
- domain assumption Token-level LMs can be converted to a bit-level Model by recursive token-bit sampling.
- standard math Hoeffding's inequality applies to the aggregated detector outputs X_i, which are treated as independent random variables.
Cite this review
Pith. "Pith review of Let Watermarks Speak: A Robust and Unforgeable Watermark for Language Models." pith.science (2026). https://pith.science/paper/3ECOUWTT
@misc{pith2026241219603,
author = {Pith},
title = {Pith review of: Let Watermarks Speak: A Robust and Unforgeable Watermark for Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/3ECOUWTT}},
note = {Machine review of arXiv:2412.19603}
}
read the original abstract
Watermarking is an effective way to trace model-generated content. Current watermark methods cannot resist forgery attacks, such as a deceptive claim that the model-generated content is a response to a fabricated prompt. None of them can be made unforgeable without degrading robustness. Unforgeability demands that the watermarked output is not only detectable but also verifiable for integrity, indicating whether it has been modified. This underscores the necessity and significance of a multi-bit watermarking scheme. Recent works try to build multi-bit scheme based on existing zero-bit watermarking scheme, but they either degrades the robustness or brings a significant computational burden. We aim to design a novel single-bit watermark scheme, which provides the ability to embed 2 different watermark signals. This paper's main contribution is that we are the first to propose an undetectable, robust, single-bit watermarking scheme. It has a comparable robustness to the most advanced zero-bit watermarking schemes. Then we construct a multi-bit watermarking scheme to use the hash value of prompt or the newest generated content as the watermark signals, and embed them into the following content, which guarantees the unforgeability. Additionally, we provide sufficient experiments on some popular language models, while the other advanced methods with provable guarantees do not often provide. The results show that our method is practically effective and robust.
Figures
Reference graph
Works this paper leans on
-
[1]
Simons institute talk on watermarking of large language models, 2023
Scott Aaronson. Simons institute talk on watermarking of large language models, 2023
work page 2023
-
[2]
Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast- detectGPT: Efficient zero-shot detection of machine-generated text via condi- tional probability curvature. In The Twelfth International Conference on Learning Representations, 2024
work page 2024
-
[3]
Pseudorandom error-correcting codes
Miranda Christ and Sam Gunn. Pseudorandom error-correcting codes. In Annual International Cryptology Conference, pages 325–347. Springer, 2024
work page 2024
-
[4]
Undetectable watermarks for language models
Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. In The Thirty Seventh Annual Conference on Learning Theory, pages 1125–
-
[5]
Watermarking language models for many adaptive users
Aloni Cohen, Alexander Hoover, and Gabe Schoenbach. Watermarking language models for many adaptive users. In 2025 IEEE Symposium on Security and Privacy (SP), pages 84–84. IEEE Computer Society, 2024
work page 2025
-
[6]
Publicly-detectable watermarking for language models, 2024
Jaiden Fairoze, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mah- moody, and Mingyuan Wang. Publicly-detectable watermarking for language models, 2024
work page 2024
-
[7]
Edit distance robust watermarks for language models, 2024
Noah Golowich and Ankur Moitra. Edit distance robust watermarks for language models, 2024
work page 2024
-
[8]
Spotting LLMs with binoculars: Zero-shot detection of machine-generated text
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting LLMs with binoculars: Zero-shot detection of machine-generated text. In Forty- first International Conference on Machine Learning , 2024
work page 2024
Show all 22 references
-
[9]
Carnegie Mellon University, 2004
Nicholas J Hopper.Toward a theory of Steganography. Carnegie Mellon University, 2004
2004
-
[10]
Radar: Robust ai-text detection via adversarial learning
Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. Radar: Robust ai-text detection via adversarial learning. Advances in Neural Information Processing Systems , 36:15077–15095, 2023
2023
-
[11]
Categorical reparametrization with gumble-softmax
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparametrization with gumble-softmax. In International Conference on Learning Representations (ICLR 2017). OpenReview. net, 2017
2017
-
[12]
Jois, Matthew Green, and Aviel D
Gabriel Kaptchuk, Tushar M. Jois, Matthew Green, and Aviel D. Rubin. Meteor: Cryptographically secure steganography for realistic distributions. InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , CCS ’21, page 1529–1548, New York, NY, USA...
2021
-
[13]
A watermark for large language models
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061–17084. PMLR, 2023
2023
-
[14]
On the reliability of watermarks for large language models, 2024
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models, 2024
2024
-
[15]
Ro- bust distortion-free watermarks for language models
Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Ro- bust distortion-free watermarks for language models. Transactions on Machine Learning Research, 2024
2024
-
[16]
Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and Philip S. Yu. An unforgeable publicly verifiable watermark for large language models. In The Twelfth International Conference on Learning Representations , 2024
2024
-
[17]
Detectgpt: Zero-shot machine-generated text detection using probability curvature
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning , pages 24950–24962. PMLR, 2023
2023
-
[18]
Provably robust multi-bit watermarking for ai-generated text, 2024
Wenjie Qu, Wengrui Zheng, Tianyang Tao, Dong Yin, Yanze Jiang, Zhihua Tian, Wei Zou, Jinyuan Jia, and Jiaheng Zhang. Provably robust multi-bit watermarking for ai-generated text, 2024
2024
-
[19]
SeqXGPT: Sentence-level AI-generated text detection
Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. SeqXGPT: Sentence-level AI-generated text detection. In The 2023 Conference on Empirical Methods in Natural Language Processing , 2023
2023
-
[20]
Zero-shot detection of machine-generated codes, 2023
Xianjun Yang, Kexun Zhang, Haifeng Chen, Linda Petzold, William Yang Wang, and Wei Cheng. Zero-shot detection of machine-generated codes, 2023
2023
-
[21]
Excuse me, sir? your language model is leaking (information)
Or Zamir. Excuse me, sir? your language model is leaking (information). arXiv preprint arXiv:2401.10360, 2024
2024 arXiv
-
[22]
Prov- able robust watermarking for AI-generated text
Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Prov- able robust watermarking for AI-generated text. In The Twelfth International Conference on Learning Representations , 2024
2024
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.