Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

SimMark: A Robust Sentence-Level Similarity-Based Watermarking Algorithm for Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SimMark claims that LLM output can be watermarked with no access to model internals: rejection sampling forces consecutive-sentence embedding similarities into a fixed interval, and a soft z-test detects the pattern even after paraphrasing.

desk verdict SimMark is a genuinely new, clearly specified sentence-level watermark, but its state-of-the-art claim rests on baseline numbers the authors could not reproduce and single-run evaluations, so the headline needs a direct rerun before it holds. read the letter →

arxiv 2502.02787 v2 pith:O2U4ZQ4D submitted 2025-02-05 cs.CL cs.CRcs.CYcs.LG

Amirhossein Dabiriaghdam , Lele Wang This is my paper · ORCID
classification cs.CLcs.CRcs.CYcs.LG
keywords LLMwatermarkingsentence-levelwatermarkparaphraserobustnesssemanticembeddingsimilarityrejectionsamplingsoftz-testblack-boxdetectiontextprovenance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces SimMark, a method for watermarking the text of any large language model — including API-only models whose internal token probabilities are hidden — by controlling the semantic similarity between consecutive sentences. Generation uses rejection sampling: candidate sentences are requested repeatedly until the embedding similarity of a new sentence with its predecessor falls inside a fixed interval $[a,b]$, and detection runs a soft z-test that counts how much of the text obeys the interval and compares that count with the distribution of human text. The paper claims SimMark beats the previous sentence-level methods SemStamp and k-SemStamp in paraphrase robustness and sampling efficiency, while keeping perplexity, trigram entropy, and semantic entropy close to unwatermarked text. If the claim holds, reliable provenance detection for closed, API-only models becomes available at sentence granularity rather than only at token granularity.

What carries the argument

The load-bearing object is the interval-restricted consecutive-sentence similarity combined with soft counting. For consecutive sentence embeddings $e_i, e_{i+1}$, the similarity $s_{i+1}=\mathrm{sim}(e_i,e_{i+1})$ is compared with a predefined interval $[a,b]$; a pair counts fully if $s_{i+1}\in[a,b]$ and partially otherwise via $c_{i+1}=\exp(-K\min\{|a-s_{i+1}|,|b-s_{i+1}|\})$ with decay $K=250$. The detection statistic is $z_{\mathrm{soft}}=(N_{\mathrm{valid\_soft}}-p_0 N)/\sqrt{p_0(1-p_0)N}$, where $N_{\mathrm{valid\_soft}}=\sum_i c_i$, $p_0$ is the area of the human-text similarity distribution inside $[a,b]$, and the threshold $\beta$ is calibrated on human text to fix the false-positive rate. The interval is what rejection sampling enforces during generation, and the exponential tail is what keeps a paraphrased pair that drifts just outside the interval from being lost from the count entirely.

What would settle it

Measure the consecutive-sentence embedding similarity distribution of unwatermarked text from a recent instruction-tuned model writing in a domain outside the paper's three datasets, using the same embedding model, and compute the probability mass inside the paper's fixed intervals ($[0.68,0.76]$ cosine, $[0.28,0.36]$ Euclidean with PCA). If that mass differs substantially from the human-text $p_0$ used to calibrate the threshold, the false-positive rate on human text drifts away from the claimed 1% or 5% or the z-statistic stops separating watermarked from unwatermarked text; a concrete failure would be a domain where human text places almost no mass in the interval, forcing expensive rejection sampling and collapsing detection power.

Watch

Extended reading notes

Core claim

SimMark's central claim is that a detectable watermark can be embedded at the level of whole sentences using only the LLM's sampling interface and an off-the-shelf embedding model. The watermark is a statistical regularity imposed on consecutive-sentence embedding similarity: rejection sampling re-queries the LLM until the cosine similarity (or Euclidean distance, optionally after PCA) between the new sentence's embedding and the previous sentence's embedding lands in a predefined interval; detection then computes a soft count of consecutive pairs lying in or near the interval and feeds it into a one-proportion z-test whose null model is the human-text similarity distribution. The paper reports that on the RealNews, BookSum, and Reddit-TIFU datasets, against Pegasus, Parrot, and GPT-3.5-Turbo paraphrases in regular and bigram attack modes, cosine-SimMark achieves the highest average paraphrased detection performance of all compared methods, including the token-level baselines UW, KGW, and SIR, at fixed 1% and 5% false-positive rates, while matching the sentence-level baselines on text quality and improving on their sampling efficiency.

Load-bearing premise

The method assumes that a single fixed similarity interval $[a,b]$, chosen by inspecting the consecutive-sentence embedding similarity of human and unwatermarked LLM text, remains predictive when the model, domain, or embedder changes; the paper's own Gemma3-4B experiments required new intervals ($[0.86,0.90]$ for cosine and $[0.11,0.16]$ for Euclidean with PCA), and the paper states that the interval must be adjusted when the similarity distribution shifts.

Editorial extensions

If this is right

  • Any API-only LLM can be watermarked without logits or fine-tuning, because the method only prompts the model and embeds the resulting sentences; the paper demonstrates this on OPT-1.3B and Gemma3-4B.
  • Paraphrase robustness at sentence level is achievable without domain-specific embedding fine-tuning: cosine-SimMark posts the best average paraphrased-detection numbers across the three datasets, including under bigram paraphrase attacks designed to break the watermark.
  • The sampling overhead is practical: about 7.1 samples per sentence on BookSum versus 13.3 for k-SemStamp and 20.9 for SemStamp, with detection performance plateauing once roughly 25 rejection-sampling trials are allowed.
  • Text quality is essentially preserved, with perplexity, trigram entropy, and semantic entropy close to the unwatermarked baseline, in contrast to token-level methods that raise perplexity.
  • Under Paraphrase+Drop attacks SimMark outperforms every compared method across most drop probabilities, and under Paraphrase+Merge it leads the sentence-level methods even though the token-level UW baseline remains strongest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A strict head-to-head re-run would sharpen the comparison: the paper reports it could not fully reproduce the SemStamp and k-SemStamp numbers, so some baseline entries in its main table were extracted from the original papers rather than re-computed in one harness.
  • Because the attack protocol paraphrases sentence by sentence to avoid the summarization behavior of whole-text paraphrase prompts, robustness to a length-preserving whole-document rewrite — the attack a real user is more likely to apply — is not directly measured; the soft-count tolerance suggests partial resilience but the paper does not quantify it.
  • The fixed interval $[a,b]$ and decay factor $K$ function as shared-secret parameters: an adversary who can estimate the human similarity distribution or identify the embedder could strip the signal by pushing similarities outside the interval or spoof it by steering otherwise human text inside, a balance the paper acknowledges but does not quantify.
  • Making the interval adaptive per prompt or derived from the running similarity distribution would improve transfer across models — the Gemma3-4B results show the chosen intervals do not transfer — and would double as a defense against reverse engineering; the paper lists adaptive intervals only as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript introduces SimMark, a black-box sentence-level watermarking algorithm for LLMs. During generation, SimMark uses rejection sampling to force the cosine similarity or Euclidean distance between embeddings of consecutive sentences into a predefined interval [a,b]; during detection, it computes a soft count of valid sentence pairs and applies a soft z-test with an empirically calibrated threshold. The authors evaluate SimMark on RealNews, BookSum, and Reddit-TIFU under seven paraphrase attack settings, reporting ROC-AUC and TP@FP rates, and claim that SimMark surpasses prior sentence-level watermarking methods (SemStamp, k-SemStamp) in robustness and sampling efficiency while preserving text quality.

Significance. If the comparative results are reproducible, SimMark is an elegant and practical contribution: it requires no access to LLM logits, uses an off-the-shelf embedding model, and the soft-counting mechanism is an intuitive way to tolerate paraphrase-induced perturbations. The ablations for the smoothness factor K and for PCA are informative, and the theoretical sampling-efficiency estimate in Appendix G is a useful addition. The code release is a concrete asset, as is the additional Gemma3-4B evaluation. However, the central comparative claim of state-of-the-art performance is currently supported by baseline numbers extracted from prior papers rather than by a controlled rerun, and all experiments are single-run, so the claimed margins are not yet established.

major comments (3)
  1. [Section 4.4, Table 1, footnote 15] The headline claim that SimMark "surpasses prior sentence-level watermarking techniques" rests on SemStamp, k-SemStamp, KGW, and SIR numbers extracted directly from Hou et al. (2024a,b), yet the authors state in footnote 15 that they "were unable to reproduce their reported results fully" and observed only "minor discrepancies" between their own reproduction and the numbers they tabulate. Detection metrics such as ROC-AUC and TP@FP are sensitive to threshold calibration, sentence tokenization, and paraphrase prompts, so comparing SimMark numbers from one pipeline against baseline numbers from another pipeline is not a controlled comparison. The authors should rerun all baselines under the exact same evaluation pipeline used for SimMark and report those numbers, or temper the SOTA claim accordingly.
  2. [Footnote 4, Section 4] All experimental results in Tables 1, 3, and 4 come from a single run, with no error bars or multiple seeds. Several differences between SimMark and the baselines are small, for example in Table 1 the RealNews No-Paraphrase ROC-AUC of Cosine-SimMark is 99.6 versus 99.2 for SemStamp, and under the Parrot paraphraser the AUCs are 98.7 versus 93.3. Without variance estimates or significance tests, the observed margins cannot be distinguished from run-to-run noise. The authors should report results over multiple seeds or provide bootstrap confidence intervals for both SimMark and the rerun baselines.
  3. [Section 4, Appendices I, J, K, L] The interval [a,b], the decay factor K=250, the PCA dimension of 16, and the detection threshold beta are all selected using the evaluation distributions themselves, as described in Section 4 and Appendices I, J, K, and L. This makes the reported numbers in-sample and weakens the claim of "applicability across diverse domains." Appendix D underscores the concern: the Gemma3-4B experiments require new intervals ([0.86,0.90] for cosine similarity and [0.11,0.16] for Euclidean distance with PCA), so the fixed-interval assumption does not transfer across models. The authors should validate hyperparameter choices on a held-out calibration split and provide a sensitivity analysis showing how performance varies with the interval.
minor comments (5)
  1. [Section 3.2, Eq. (2)] The soft counts c_i are not independent Bernoulli variables with probability p0, because consecutive similarity scores share sentences and the variance of the soft counts differs from p0(1-p0). Since beta is calibrated empirically, this does not invalidate the detector's false-positive control, but the paper should state explicitly that the z-statistic is a heuristic score rather than a standard normal test statistic.
  2. [Appendix K and Limitations] The statement in Appendix K that "the interval must be adjusted accordingly" is in tension with the Limitations section's claim of "consistent, predefined intervals across all datasets"; please reconcile these statements.
  3. [Table 1 and Section 4.4] The Reddit-TIFU panel of Table 1 omits k-SemStamp, and the text explains this only in passing. Table 3 should clarify that the k-SemStamp average is computed only over RealNews and BookSum.
  4. [Appendix A heading] Appendix A's heading contains a typo: "Aditional" should be "Additional." The source-code URL mentioned in the abstract footnote should also be included in the final camera-ready version.
  5. [Section 4.2] The term "bigram paraphrase attack" is used without a formal definition; the description in the text (generating multiple paraphrases and selecting the one that disrupts the signal) should be stated explicitly as a definition, since it is a core evaluation condition.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: SimMark's detector tests the same similarity constraint its generator enforces, with null parameters calibrated from human text rather than derived from the target result.

full rationale

SimMark's derivation chain is self-contained in the sense relevant to circularity. The generation rule (Algorithm 1) forces consecutive-sentence similarities into a predefined interval [a,b], and the detection rule (Algorithm 2, Eq. (1)-(2)) counts how much of the observed text respects that interval. A watermark detector necessarily tests the statistical signature its generator inserts; that generator/detector identity is the mechanism of the method, not a circular derivation of the method's claims. The null parameters are calibrated, not assumed: p0 is estimated as the area under the human-written similarity distribution inside [a,b], and beta is chosen empirically on human text to meet a target false-positive rate (Appendix L). Thus the false-positive control is calibration, and the true-positive claim rests on the distribution shift introduced by rejection sampling, which is an independent empirical effect. The headline comparative claim does depend on baseline numbers extracted from Hou et al. (2024a,b), and the paper admits in footnote 15 that it could not fully reproduce those reported results; this is a reproducibility and evaluation-comparability weakness, not a circularity, because the baseline numbers are not the paper's own inputs and no equation reduces the comparison to itself. Similar considerations apply to the interval, decay factor K, and PCA settings: they are tuned or selected on data and ablation studies, and Appendix D and the Limitations section explicitly note that intervals may need adjustment for other models. That is an external-validity or overfitting concern, not a self-referential derivation. I find no self-definitional step, no fitted input renamed as prediction, no load-bearing self-citation, no imported uniqueness theorem, and no renaming of a known result. Score 0 is therefore appropriate.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on no new physical or mathematical entities. The load-bearing inputs are the interval, decay factor, PCA dimension, p0, beta, and max trials, all fitted or calibrated on data; the statistical test adds an independence assumption on soft counts. These are honest hyperparameters, but they mean the reported state-of-the-art numbers are partly a function of tuned choices rather than a parameter-free derivation.

free parameters (6)
  • Similarity interval [a,b] = cosine [0.68,0.76] (OPT), [0.86,0.90] (Gemma); Euclidean with PCA [0.28,0.36] (OPT), [0.11,0.16] (Gemma); Euclidean…
    Chosen from embedding-similarity distributions of the evaluation corpora; Section 3.1 and Appendix K. The paper calls the intervals near-optimal, so reported detection rates depend on this choice.
  • Soft-count decay factor K = 250
    Selected via ablation on RealNews with Pegasus (Appendix I); controls how much partial credit is given to similarities outside the interval.
  • PCA dimension = 16 (from 768)
    Fitted on 8000 C4/RealNews samples; tested from 512 down to 16, with 16 reported as best (Section 4, Appendix J).
  • Null proportion p0 = Area under human similarity histogram within [a,b], e.g., 0.194 for cosine interval [0.68,0.76] on OPT (Appendix G)
    Estimated by binning human-written text and used in the z-test denominator; Appendix L.
  • Detection threshold beta = Calibrated per dataset and target FP (1% or 5%) across [-10,10]
    Determined empirically on human-written text to enforce a fixed false-positive rate; Appendix L.
  • Maximum rejection-sampling trials = 100 in main experiments; 25 shown to be sufficient
    Set to align with Hou et al. (2024a,b); Appendix E shows performance plateaus near 25 trials.
assumptions (4)
  • domain assumption The semantic embedding model (Instructor-Large) represents sentence meaning such that cosine or Euclidean similarity between consecutive sentences is a stable proxy for semantic relatedness, including after paraphrasing.
    Used throughout Section 3; if paraphrases move embeddings out of the interval, soft counting only partially compensates.
  • domain assumption Human-written consecutive-sentence similarity distribution is stable across datasets so that a single p0 and beta, estimated on human text, transfer to test conditions.
    Appendix L relies on this for false-positive control; Appendix D shows intervals had to be re-tuned for Gemma, so transfer is not automatic.
  • ad hoc to paper Soft counts c_i can be treated as approximately independent observations for the one-proportion z-test in Eq. (2).
    Adjacent similarities share a sentence embedding, so observations are dependent; the paper does not test this assumption.
  • domain assumption Rejection sampling does not change the marginal distribution of generated text enough to affect human-likeness or the null distribution used for detection.
    Text quality is checked via perplexity and entropy, but the statistical test assumes the LLM's unwatermarked similarity distribution is unchanged aside from the interval bias.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SimMark: A Robust Sentence-Level Similarity-Based Watermarking Algorithm for Large Language Models." pith.science (2026). https://pith.science/paper/O2U4ZQ4D

@misc{pith2026250202787,
  author       = {Pith},
  title        = {Pith review of: SimMark: A Robust Sentence-Level Similarity-Based Watermarking Algorithm for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O2U4ZQ4D}},
  note         = {Machine review of arXiv:2502.02787}
}
read the original abstract

The widespread adoption of large language models (LLMs) necessitates reliable methods to detect LLM-generated text. We introduce SimMark, a robust sentence-level watermarking algorithm that makes LLMs' outputs traceable without requiring access to model internals, making it compatible with both open and API-based LLMs. By leveraging the similarity of semantic sentence embeddings combined with rejection sampling to embed detectable statistical patterns imperceptible to humans, and employing a soft counting mechanism, SimMark achieves robustness against paraphrasing attacks. Experimental results demonstrate that SimMark sets a new benchmark for robust watermarking of LLM-generated content, surpassing prior sentence-level watermarking techniques in robustness, sampling efficiency, and applicability across diverse domains, all while maintaining the text quality and fluency.

Figures

Figures reproduced from arXiv: 2502.02787 by the authors.

Figure 1
Figure 1. A high-level overview of SimMark detection algorithm. The input text is divided into individual sen￾tences X1 to XN , which are embedded using a semantic embedding model. The similarity between consecutive sentence embeddings is computed. Sentences with simi￾larities within a predefined interval [a, b] are considered valid, while those outside are invalid. A statistical test is performed using the count of valid sen… view at source ↗
Figure 2
Figure 2. Overview of SimMark. Top: Generation. For each newly generated sentence (Xi+1), its embedding (ei+1) is computed using a semantic text embedding model, optionally applying PCA for dimensionality reduction. The cosine similarity (or Euclidean distance) between ei+1 and the embedding of the previous sentence (ei), denoted as si+1, is calculated. If si+1 lies within the predefined interval [a, b], the sentence is marke… view at source ↗
Figure 3
Figure 3. Detection performance of different water [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: presents the ROC-AUC performance of UW (the best token-level method in our exper￾iments), SimMark, k-SemStamp, and SemStamp under Paraphrase+Drop and Paraphrase+Merge attacks, evaluated on the RealNews dataset. Under Paraphrase+Drop, across most parameter regimes, SimM…
Figure 5
Figure 5. Figure 5: Detection performance averaged across three datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Impact of the maximum number of rejec￾tion sampling trials on detection performance. In￾creasing max_trials improves both ROC-AUC ↑ and TP@1%FP ↑ (↑: higher is better), but the improvement plateaus around 25. Results are reported on the Re￾alNews dataset using cosine-S…
Figure 7
Figure 7. Figure 7: Example of text generated with and without cosine- [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Example of text generated with and without Euclidean- [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Distribution of Euclidean distances between [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Screenshot of the instructions and consent form shown to participants before the human evaluation study. [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SAMark: A Self-Anchored Text Watermarking with Paragraph-Level Paraphrase Robustness

    cs.CR 2026-05 unverdicted novelty 6.5 of 10

    SAMark uses self-anchored semantic green regions, multi-channel hyperbolic scoring, and diversity-aware filtering to reach 90.2% TP@FP1% detection under paragraph paraphrasing while preserving text quality.

  2. DP-NCB: Privacy Preserving Fair Bandits

    cs.LG 2025-08 unverdicted novelty 6.0 of 10

    DP-NCB is claimed to be the first bandit framework achieving differential privacy and order-optimal Nash regret simultaneously in both global and local privacy models.

Reference graph

Works this paper leans on

52 extracted references · 16 canonical work pages · cited by 2 Pith papers

  1. [1]

    Scott Aaronson and Hendrik Kirchner. 2022. https://www.scottaaronson.com/talks/watermark.ppt Watermarking gpt outputs

    work page 2022

  2. [2]

    Mikhail J Atallah, Victor Raskin, Michael Crogan, Christian Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. 2001. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In Information Hiding: 4th International Workshop, IH 2001 Pittsburgh, PA, USA, April 25--27, 2001 Proceedings 4, pages 185--200. Springer

    work page 2001

  3. [3]

    O'Reilly Media, Inc

    Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc."

    2009

  4. [4]

    Yapei Chang, Kalpesh Krishna, Amir Houmansadr, John Frederick Wieting, and Mohit Iyyer. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.506 P ost M ark: A robust blackbox watermark for large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8969--8987, Miami, Florida, USA. Association for ...

    doi:10.18653/v1/2024.emnlp-main.506 2024

  5. [5]

    Cheng-Han Chiang and Hung-yi Lee. 2024. Over-reasoning and redundant calculation of large language models. arXiv preprint arXiv:2401.11467

    arXiv 2024

  6. [6]

    Prithiviraj Damodaran. 2021. Parrot: Paraphrase generation for nlu

    work page 2021

  7. [7]

    Pierre Fernandez, Guillaume Couairon, Herv\'e J\'egou, Matthijs Douze, and Teddy Furon. 2023. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22466--22477

    work page 2023

  8. [8]

    Matthew Finlayson, Xiang Ren, and Swabha Swayamdipta. 2024. Logits of api-protected llms leak proprietary information. arXiv preprint arXiv:2403.09539

    arXiv 2024

Show all 52 references
  1. [9]

    Yu Fu, Deyi Xiong, and Yue Dong. 2024. https://doi.org/10.1609/aaai.v38i16.29756 Watermarking conditional text generation for ai detection: Unveiling challenges and a semantic-aware watermark remedy . Proceedings of the AAAI Conference on Artificial Intelligence, 38(16):18003--18011

    2024 doi

  2. [10]

    Futurism. 2023. https://futurism.com/cnet-ai-articles-label Cnet quietly deletes ai-generated articles amid backlash . Accessed: January 28, 2025

    2023

  3. [11]

    Hadsell, S

    R. Hadsell, S. Chopra, and Y. LeCun. 2006. https://doi.org/10.1109/CVPR.2006.100 Dimensionality reduction by learning an invariant mapping . In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), volume 2, pages 1735--1742

    2006 doi

  4. [12]

    Seungju Han, Beomsu Kim, and Buru Chang. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.66 Measuring and improving semantic diversity of dialogue generation . In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 934--950, Abu Dhabi, United Ara...

    2022 doi

  5. [13]

    Jifei Hao, Jipeng Qiang, Yi Zhu, Yun Li, Yunhao Yuan, and Xiaoye Ouyang. 2025. https://aclanthology.org/2025.coling-main.364/ Post-hoc watermarking for robust detection in text generated by large language models . In Proceedings of the 31st International Conference on Computat...

    2025

  6. [14]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. https://openreview.net/forum?id=XPZIaotutsD Deberta: Decoding-enhanced bert with disentangled attention . In International Conference on Learning Representations

    2021

  7. [15]

    Abe Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. 2024 a . https://doi.org/10.18653/v1/2024.naacl-long.226 S em S tamp: A semantic watermark with paraphrastic robustness for ...

    2024 doi

  8. [16]

    Abe Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.98 k- S em S tamp: A clustering-based semantic watermark for detection of machine-generated text . In Findings of the Association for Computational Ling...

    2024 doi

  9. [17]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  10. [18]

    Piotr Indyk and Rajeev Motwani. 1998. Approximate nearest neighbors: towards removing the curse of dimensionality. In Proceedings of the thirtieth annual ACM symposium on Theory of computing, pages 604--613

    1998

  11. [19]

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. 1977. Perplexity—a measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1):S63--S63

    1977

  12. [20]

    Ian T Jolliffe. 2002. Principal component analysis for special types of data. Springer

    2002

  13. [21]

    Byeongchang Kim, Hyunwoo Kim, and Gunhee Kim. 2019. https://doi.org/10.18653/v1/N19-1260 Abstractive summarization of R eddit posts with multi-level memory networks . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Ling...

    2019 doi

  14. [22]

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. In International Conference on Machine Learning, pages 17061--17084. PMLR

    2023

  15. [23]

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2024. https://openreview.net/forum?id=DEJIDCmWOz On the reliability of watermarks for large language models . In The Twelfth...

    2024

  16. [24]

    Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2024. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36

    2024

  17. [25]

    Wojciech Kryscinski, Nazneen Rajani, Divyansh Agarwal, Caiming Xiong, and Dragomir Radev. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.488 BOOKSUM : A collection of datasets for long-form narrative summarization . In Findings of the Association for Computational Lingu...

    2022 doi

  18. [26]

    Tharindu Kumarage, Paras Sheth, Raha Moraffah, Joshua Garland, and Huan Liu. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.94 How reliable are AI -generated-text detectors? an assessment framework using evasive soft prompts . In Findings of the Association for Computat...

    2023 doi

  19. [27]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

    2023

  20. [28]

    Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2023. A semantic invariant robust watermark for large language models. arXiv preprint arXiv:2310.06356

    2023 arXiv

  21. [29]

    Stuart Lloyd. 1982. Least squares quantization in pcm. IEEE transactions on information theory, 28(2):129--137

    1982

  22. [30]

    OpenAI. 2022. https://openai.com/blog/chatgpt ChatGPT

    2022

  23. [31]

    Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. 2024. https://doi.org/10.18653/v1/2024.emnlp-demo.7 M ark LLM : An open-source toolkit for LLM watermarking . In Proceedings of...

    2024 doi

  24. [32]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. http://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of Machine Lea...

    2020

  25. [33]

    Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156

    2023 arXiv

  26. [34]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, and 1 others. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172--180

    2023

  27. [35]

    Smith, Luke Zettlemoyer, and Tao Yu

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. https://doi.org/10.18653/v1/2023.findings-acl.71 One embedder, any task: Instruction-finetuned text embeddings . In Findings of the Asso...

    2023 doi

  28. [36]

    Maksym Taranukhin, Sahithya Ravi, Gabor Lukacs, Evangelos Milios, and Vered Shwartz. 2024. https://doi.org/10.18653/v1/2024.nllp-1.27 Empowering air travelers: A chatbot for C anadian air passenger rights . In Proceedings of the Natural Legal Language Processing Workshop 2024,...

    2024 doi

  29. [37]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram \'e , Morgane Rivi \`e re, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786

    2025 arXiv

  30. [38]

    Mohammadreza Teymoorianfard, Shiqing Ma, and Amir Houmansadr. 2025. https://arxiv.org/abs/2505.01406 Vidstamp: A temporally-aware watermark for ownership and integrity in video diffusion models . Preprint, arXiv:2505.01406

    2025

  31. [39]

    Mercan Topkara, Umut Topkara, and Mikhail J Atallah. 2006. Words are not enough: sentence level natural language watermarking. In Proceedings of the 4th ACM international workshop on Contents protection and security, pages 37--46

    2006

  32. [40]

    Yasaman Torabi, Shahram Shirani, and James P Reilly. 2025. Large language model-based nonnegative matrix factorization for cardiorespiratory sound separation. arXiv preprint arXiv:2502.05757

    2025 arXiv

  33. [41]

    Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, and 1 others. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359

    2021 arXiv

  34. [42]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and ...

    2020 doi

  35. [43]

    Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, and 1 others. 2025. Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs. arXiv preprint arXiv:2504.00993

    2025 arXiv

  36. [44]

    Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, and Nenghai Yu. 2023. Watermarking text generated by black-box language models. arXiv preprint arXiv:2305.08883

    2023 arXiv

  37. [45]

    Shunyu Yao, Qingqing Ke, Qiwei Wang, Kangtong Li, and Jie Hu. 2024. Lawyer gpt: A legal large language model with enhanced domain knowledge and reasoning capabilities. In Proceedings of the 2024 3rd International Symposium on Robotics, Artificial Intelligence and Information E...

    2024

  38. [46]

    Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020. https://proceedings.mlr.press/v119/zhang20ae.html PEGASUS : Pre-training with extracted gap-sentences for abstractive summarization . In Proceedings of the 37th International Conference on Machine Learning, volume ...

    2020

  39. [47]

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, and 1 others. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    2022 arXiv

  40. [48]

    Weinberger, and Yoav Artzi

    Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with bert . In International Conference on Learning Representations

    2020

  41. [49]

    Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018. Generating informative and diverse conversational responses via adversarial information maximization. Advances in Neural Information Processing Systems, 31

    2018

  42. [50]

    Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. 2023. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.17439

    2023 arXiv

  43. [51]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.