{"id":"2ee4ae12-ce33-4fd3-bcc7-447d7a772bd0","arxiv_id":"2506.20926","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A backdoor watermarking method for code models that selects embedding positions by self-attention and hides triggers as distributed Unicode homoglyphs.","lead":"CodeGuard embeds invisible backdoor watermarks into generative code models by swapping ordinary letters for visually identical Unicode characters at positions the model pays most attention to. The authors report near-perfect watermark verification across two datasets and two tasks, with very low detection by two standard defenses.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Verification uses an out-of-distribution concatenated Unicode trigger and the paper reports no false-positive or specificity tests, so near-100% WSR may reflect a generic Unicode response rather than a unique watermark.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: the verification trigger is not drawn from the training distribution, and the method may respond to any Unicode homoglyph rather than to the exact learned trigger. I agree that this is the central point on which the paper's main claim depends. Some additional concerns exist, such as the absence of a trivial non-ASCII scanner in the stealth evaluation and the reported CodeBLEU value of 0.4685 not appearing in Table II, but neither is as fundamental as the unproven specificity of the verification signal. If the watermark fires for arbitrary Unicode characters, then the method is not a watermark at all; it is a generic Unicode backdoor, and the near-100% WSR numbers in Table I would be measuring the wrong thing. The paper also acknowledges in its conclusion that adversarial testing and robustness against deliberate detection/removal require further validation, which supports treating stealth and uniqueness claims as conditional. Since the missing evidence is clearly obtainable by running a clean model through the verification protocol and by ablating the trigger construction, the appropriate disposition remains CONDITIONAL rather than outright rejection. I therefore recommend UNCHANGED relative to the reader's verdict.","tokens_in":19620,"tokens_out":4897,"duration_ms":56129,"concrete_test":"Run a clean, unwatermarked CodeT5 model through the exact Section III-B verification procedure: insert the concatenated mapping-table Unicode string at the attention-selected position and check whether the output contains Fw. Repeat on the watermarked model with (i) a single random homoglyph from M, (ii) a non-ASCII character outside M, and (iii) a distributed trigger matching the training-time embedding pattern. If clean models produce Fw at a non-negligible rate, or if a single homoglyph triggers the watermarked model as often as the full concatenated trigger, then the verification is not watermark-specific and the central ownership claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"CodeGuard's ownership claim reduces to a single binary test: does the model's output contain Fw when the verification trigger is inserted? The paper reports near-perfect WSR, but only for verification triggers built by concatenating all Unicode characters from mapping table M and inserting them into one high-attention position (Section III-B). Training, by contrast, embeds one homoglyph per selected semantic unit, dispersed across positions, so the verification input is drawn from a different distribution than the training triggers. If the model has learned a broad association such as 'non-ASCII character at the attended position implies Fw' rather than the exact dispersed homoglyph pattern, then benign code containing Unicode characters, or an adversary inserting a single Cyrillic letter, would also elicit Fw. The paper reports no false-positive rate on clean models, no specificity test with partial or alternative Unicode triggers, and no ablation that verifies using the training-distributed pattern. Without such measurements, the 100% WSR cannot support the conclusion that the watermark is a unique, verifiable identifier.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CodeGuard, a backdoor watermarking method for generative code models. CodeGuard selects watermark embedding positions using self-attention scores computed from a Transformer model, then embeds triggers by replacing selected ASCII characters with visually similar Unicode homoglyphs, distributing the trigger characters across positions. At verification time, the method concatenates all characters from the homoglyph mapping table, inserts this concatenated string at an attention-selected position, and checks whether the model output contains a predefined watermark feature F_w. Experiments with CodeT5 on CodeXGLUE and CodeSearchNet across code summarization and code generation report near-100% watermark success rates at 5-15% poison rates, minimal harmlessness impact, and low ONION and spectral-signature detection rates compared with CoProtector, CodeMark, and ModMark.","tokens_in":19728,"tokens_out":7797,"duration_ms":77923,"significance":"If the empirical claims hold, CodeGuard would be a useful contribution: it combines attention-based position selection, dispersed trigger embedding, and homoglyph substitution to address the generalization and stealth limitations of prior code-model watermarks. The paper reports a broad set of comparisons across two tasks, two datasets, three poison rates, and two detection methods, which is a strength. However, the central ownership claim depends on a verification protocol that does not match the training-time trigger distribution, and the paper provides no false-positive or specificity measurements. The paper also leaves the watermark feature F_w undefined and does not specify which model's attention is used for position selection. These issues are load-bearing because they bear directly on whether near-100% WSR demonstrates a unique, reproducible watermark rather than a generic response to out-of-distribution Unicode input. The conclusion itself acknowledges that broader task/LLM applicability and adversarial robustness remain to be validated, which is an appropriate limitation statement.","major_comments":[{"comment":"The verification trigger is out of distribution relative to the training triggers. Training embeds one homoglyph per selected semantic unit, dispersed across positions (Section III-B), but verification concatenates all Unicode characters from the mapping table M and inserts that concatenated string into a single high-attention position. This input distribution was not seen during training. The paper reports no false-positive rate on clean models, no control experiments with partial or alternative Unicode triggers, and no verification run that uses the training-distributed trigger pattern. If the model responds to any novel Unicode sequence at an attended position, the near-100% WSR in Table I would not establish a unique watermark. Please add specificity tests: clean-model false positives, verification with single homoglyphs, verification with homoglyphs not in M, and verification using the dispersed training pattern.","section":"Section III-B and Figure 3"},{"comment":"The watermark feature F_w is never defined. The verification criterion is stated as 'checking whether the output O contains the predefined watermark feature Fw,' but the paper does not specify what F_w is, how it is generated, whether it is a fixed string, a set of allowed substrings, or a pattern, nor how the output is matched against it. Without this definition, the WSR values in Table I cannot be reproduced or interpreted. Please specify F_w precisely and describe the matching rule used in the experiments.","section":"Sections III-B and IV-B"},{"comment":"It is unclear which model's self-attention is used for position selection. The paper says a 'pre-trained model' is used to compute attention weights but does not state whether this is the same CodeT5 model being watermarked, a separate pre-trained model, or the fine-tuned watermarked model. If the same model is used for both position selection and verification, the method's generalizability to other model architectures is untested, and black-box verification by an owner without access to the exact architecture may fail. If a separate model is used, it should be identified and the sensitivity of WSR to that model choice should be reported.","section":"Section III-A"},{"comment":"The harmlessness table is internally inconsistent with the text. The text states that CodeXGLUE code generation EM scores are between 0.4373 and 0.4375, but the 'Ours 5%' row of Table II lists 0.6114 in that column; the same value 0.6114 also appears as the code summarization EM in the 'Ours 10%' row. The text also reports a CodeSearchNet code generation CodeBLEU maximum of 0.4685, but the 'Ours 5%' row lists 0.5402 for that column. Please correct the table alignment and reconcile all values with the narrative, because the harmlessness and even performance-improvement claims rest on these numbers.","section":"Table II and Section V-B"}],"minor_comments":[{"comment":"The phrase 'homomorphic character replacement' should be 'homoglyph character replacement' throughout; homomorphic has a different meaning in algebra and cryptography.","section":"Abstract and Introduction"},{"comment":"The WSR formula is malformed as printed: 'W SR= P xi∈X Mb(xi) = τP xi∈X xicontains triggers .' Please rewrite the formula with clear definitions of M_b, τ, and the set X, and ensure the denominator and numerator are explicit.","section":"Section IV-B"},{"comment":"The DSR@β formula uses α, β, and N without defining α in the text near the equation; please define all symbols in the metric definition.","section":"Section IV-B"},{"comment":"The second block of Table III is labeled 'Task Code Summarization' but appears to contain different numbers from the first block; based on the surrounding text, one block should likely be the code generation task. Please correct the column/row labels.","section":"Table III"},{"comment":"The homoglyph mapping table M is shown only as an example in Figure 3 and is not formally listed in the text. For reproducibility, please include the exact mapping table used in the experiments.","section":"Section III-B"},{"comment":"The text cites 'Theoretical work by Tran et al. [32]' when the spectral signature reference is [33], not [32]; please check all reference numbers in the appendix.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and an extensive empirical comparison, but the verification protocol mismatch and the absence of false-positive/specificity tests mean the central ownership claim is not yet established. The requested experiments (clean-model false positives, alternative Unicode controls, verification with the training-distributed pattern, and a precise F_w definition) are within the scope of a revision. I recommend major revision rather than rejection. The table inconsistencies in Table II should also be corrected before any further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CodeGuard is a plausible and well-described extension of backdoor watermarking for generative code models. Its actual new contribution is the combination: using self-attention scores to pick embedding positions, then dispersing homoglyph substitutions across those positions. That combination is not in CodeMark, CoProtector, or ModMark, and the two-task/two-dataset evaluation is more thorough than most papers in this niche. The WSR and harmlessness numbers are strong, and the stealth numbers beat the baselines on ONION and spectral signature. Give credit where due: this is a real empirical effort.\n\nThe soft spots are also real. The verification protocol in Section III-B concatenates all Unicode characters from the mapping table and inserts that concatenated string into a single high-attention position. Training, by contrast, embeds one homoglyph per selected semantic unit, dispersed across positions. So the verification input is out-of-distribution relative to training triggers. That matters because the near-100% WSR could just mean 'model sees any non-ASCII characters and emits the watermark feature,' rather than a unique response to the trained trigger pattern. The paper reports no false-positive rate on clean models and no specificity test with partial or alternative Unicode triggers. Without that, the ownership claim is not established. This is not a minor omission; it is load-bearing for a watermarking paper.\n\nOther issues are smaller. No code or training configuration is provided, so independent reproduction is hard. Stealth is only tested against ONION and spectral signature; a trivial non-ASCII scanner would flag homoglyph characters, so the 'stealthy' claim should be scoped more carefully. There is also a small numeric inconsistency in the CodeBLEU improvement (0.4685 in text vs 0.4675 in Table II); this is minor but should be fixed.\n\nIn sum, the core idea is worth pursuing and the experiments are substantial. The paper deserves peer review, but a serious referee should require a specificity analysis, a verification protocol that matches the training distribution (or an ablation showing why the concatenated trigger works), and artifact release. I would not desk-reject this; I would send it out with expectations of a major revision.","headline":"A useful extension of backdoor watermarking for code models, but the verification trigger is out-of-distribution and the missing false-positive tests undercut the ownership claim until fixed.","tokens_in":20313,"tokens_out":3598,"would_cite":true,"duration_ms":36869,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Unicode backdoor marks verify code models at near 100% rate","keywords":["backdoor watermark","generative code model","code summarization","code generation","homoglyph substitution","attention mechanism","copyright protection","stealth watermark"],"falsifier":"Run the verification protocol on a clean, unwatermarked model: if inserting the concatenated homoglyph string at the high-attention position produces the watermark feature with non-negligible probability, the watermark is not unique and false ownership claims become possible. Conversely, test a watermarked model with the concatenated string inserted at a low-attention position instead of the top-scoring one; if verification still succeeds, the attention-based position selection is not the operative mechanism, and the method reduces to a generic Unicode-trigger backdoor.","tokens_in":19349,"feed_emoji":"🛡️","tokens_out":8262,"duration_ms":80178,"temperature":0.7,"pith_summary":"Generative code models are expensive to train, and their owners need a way to prove when a stolen copy is theirs. This paper proposes CodeGuard, a backdoor watermarking method that plants hidden triggers in a code model during fine-tuning, so a later black-box query can reveal the owner's mark. The central claim is that CodeGuard reaches nearly 100% watermark verification with only 10% of training samples carrying triggers, across code summarization and code generation tasks on two different benchmarks, without harming task quality and sometimes improving it. The paper also claims the marks are stealthy: the ONION detector finds them at rates of at most 0.078, far below baseline methods. If these claims hold, model owners get a task-general copyright tool that resists both human inspection and automated detection.","feed_headline":"Unicode backdoor marks verify code models at near 100% rate","feed_subtitle":"A homoglyph trigger spread across samples stays invisible and verifies ownership on any code task.","key_machinery":"The load-bearing mechanism is the pairing of attention-based trigger placement with distributed homoglyph embedding. The attention side picks a pivot token that the model already treats as semantically central, so a small perturbation there is more likely to be learned as a watermark feature. The homoglyph side substitutes visually identical Unicode characters, such as Latin 'a' to Cyrillic 'а', so the trigger is invisible to human readers; splitting the trigger into individual characters placed one per sample makes per-sample anomalies statistically small, which is what keeps automated detectors like ONION and spectral signature from flagging them. For natural-language inputs, a perplexity check rejects replacements that would read unnaturally.","core_discovery":"The central claim is that watermark verifiability in generative code models can be made both task-general and stealthy by choosing trigger positions through the model's own self-attention scores and by spreading each trigger across many characters via homoglyph substitution. CodeGuard first extracts non-keyword identifiers from code, or words with stop-words removed from natural language, vectorizes them, and scores each by summed attention in the encoder's last layer; the top-scoring identifier or word becomes the embedding site. During training, one character of the trigger is replaced with a visually similar Unicode homoglyph at that site, and different trigger characters are spread across many samples, so no single sample shows a conspicuous pattern. At verification, the owner concatenates every homoglyph from the mapping table into a single trigger string, inserts it at the attention-selected position, and checks whether the model output contains a predefined watermark feature. On CodeSearchNet and CodeXGLUE, across code summarization and code generation, the paper reports watermark success rates up to 100% at a 10% poison rate, with BLEU, EM, and CodeBLEU scores at or above the clean model, and ONION trigger detection rates between 0.035 and 0.078.","pith_inferences":["A natural stress test the paper does not run is watermark removal: fine-tuning the watermarked model on clean data for a few epochs could erase the scattered character-level trigger, since the trigger is distributed so thinly.","The verification trigger's out-of-distribution nature may be a liability against a sophisticated adversary who detects the concatenated homoglyph string by simple Unicode normalization; defenders could then remove the mark without knowing the mapping table.","The attention-scoring selection is model-specific, so extending CodeGuard to other architectures or to larger language models would require recomputing attention maps with the target model, which the paper leaves for future work.","The same dispersed-embedding recipe might be adapted to other transformer-based modalities, such as plain text or structured data, wherever a top-attention token can be modified."],"forward_implications":["Owners can verify a suspect model in a black-box setting with as little as 10% of training data watermarked, making theft detectable without access to model internals.","Because the same attention-based position selection works on both code and natural-language inputs, the watermark transfers across code summarization and code generation tasks without redesign.","Stealth metrics are low enough that standard perplexity-based and spectral detectors would not single out watermarked samples, so the mark is likely to survive routine screening.","Task quality does not drop and sometimes rises slightly, suggesting the scattered homoglyph perturbations act as mild regularization rather than degradation."],"supporting_citations":[{"why":"Supplies the CodeMark baseline: semantic-preserving transformations used as trigger features in the comparison experiments.","marker":"[8]"},{"why":"Supplies the CoProtector baseline: a fixed-vocabulary backdoor watermark whose verification rates vary strongly across datasets.","marker":"[9]"},{"why":"Supplies the ModMark baseline: a model-level watermark for code summarization, against which CodeGuard's transfer to code generation is compared.","marker":"[10]"},{"why":"CodeSearchNet dataset providing one of the two data distributions for both tasks.","marker":"[11]"},{"why":"CodeXGLUE benchmark providing the second data distribution for both tasks.","marker":"[12]"},{"why":"CodeT5, the pretrained model used for fine-tuning and for computing the self-attention scores that select trigger positions.","marker":"[21]"},{"why":"ONION, the perplexity-based detector used to measure trigger stealth in the stealthiness experiments.","marker":"[32]"},{"why":"Spectral signature detection, the representation-based detector used as the second stealth evaluation.","marker":"[33]"}],"fun_headline_variants":["Homoglyph watermarks verify code models at 100% rate","Attention-picked triggers make code watermarks undetectable","CodeGuard: Near-perfect watermark verification for code models","Distributed Unicode triggers hide watermarks from detectors","Stealthy code watermarks hit perfect verification via homoglyphs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the verification trigger, every homoglyph character run together into one long string, will activate the same learned watermark as the single-character replacements the model saw during training, even though the model never encountered that exact concatenated string.","fun_headline_variants_meta":{"raw":{"variants":["Homoglyph watermarks verify code models at 100% rate","Attention-picked triggers make code watermarks undetectable","CodeGuard: Near-perfect watermark verification for code models","Distributed Unicode triggers hide watermarks from detectors","Stealthy code watermarks hit perfect verification via homoglyphs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000215,"raw_usage":{"total_tokens":1464,"prompt_tokens":1015,"completion_tokens":449,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":366}},"tokens_in":631,"tokens_out":449,"duration_ms":5338,"temperature":1.0,"reasoning_tokens":366,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:38:36.429071+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the verification protocol on a clean, unwatermarked model: if inserting the concatenated homoglyph string at the high-attention position produces the watermark feature with non-negligible probability, the watermark is not unique and false ownership claims become possible. Conversely, test a watermarked model with the concatenated string inserted at a low-attention position instead of the top-scoring one; if verification still succeeds, the attention-based position selection is not the operative mechanism, and the method reduces to a generic Unicode-trigger backdoor.","supporting_citations":[{"cited_title":"Codemark: Imperceptible wa- termarking for code datasets against neural code completion models,","cited_arxiv_id":null,"evidence_quote":"Supplies the CodeMark baseline: semantic-preserving transformations used as trigger features in the comparison experiments."},{"cited_title":"Coprotector: Protect open- source code against unauthorized training usage with data poisoning,","cited_arxiv_id":null,"evidence_quote":"Supplies the CoProtector baseline: a fixed-vocabulary backdoor watermark whose verification rates vary strongly across datasets."},{"cited_title":"Beyond dataset watermarking: Model-level copyright protection for code summarization models,","cited_arxiv_id":null,"evidence_quote":"Supplies the ModMark baseline: a model-level watermark for code summarization, against which CodeGuard's transfer to code generation is compared."},{"cited_title":"Onion: A simple and effective defense against textual backdoor attacks,","cited_arxiv_id":null,"evidence_quote":"ONION, the perplexity-based detector used to measure trigger stealth in the stealthiness experiments."}],"review_version":1}