{"id":"5038b660-c6f3-4946-9570-d37ac47f16a9","arxiv_id":"2412.08549","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Injected audio watermarks can be detected in music generated by a fine-tuned MusicGen model; simple tones work best, and a neural watermark requires dozens of repeated embeddings.","lead":"The authors fine-tuned a music generation model on audio injected with watermarks and tested whether the watermarks reappear in generated music. Simple tones, including one below human hearing, could be detected in outputs, while a state-of-the-art neural watermark needed to be embedded dozens of times before detection worked.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'imperceptible watermark' claim rests on Tone 10, whose imperceptibility is unvalidated by human listening and which is removed by standard high-pass filtering; without it, only perceptible watermark distortions yield high AUC.","rationale":"I read the paper in good faith and credit its genuine strengths: matched training runs, three seeds, a random-guess baseline, a frequency sweep, and a specificity control (Section 5.4). The core experimental finding that clearly audible tone watermarks and heavily repeated AudioSeal watermarks produce detectable output shifts is supported. However, the single most load-bearing assertion in the abstract is that imperceptible watermarks can be detected, because that is what would make the method practically useful for copyright protection. The only imperceptible success is Tone 10, and its imperceptibility is not demonstrated by the metrics used. PESQ is designed for speech, the paper itself states there is no threshold, and no human evaluation was conducted. Moreover, the 10 Hz tone is fragile: it is below the conventional audio band and the authors acknowledge it is removed by high-pass filtering. Since real music distribution and preprocessing pipelines frequently include high-pass filtering, the Tone 10 result may not transfer to deployed settings. The reader's weakest_assumption identified exactly this issue, so I agree with the conditional verdict. I do not think the paper should be rejected, because the controlled experiments are a valid proof of concept for the tone and repeated-AudioSeal results, but the headline imperceptibility claim needs substantial qualification or additional human and robustness evidence.","tokens_in":13960,"tokens_out":5845,"duration_ms":69445,"concrete_test":"Run a MUSHRA/ABX listening test in which human listeners compare original MusicCaps clips against Tone 10-watermarked clips, and separately train the same watermarked model after applying a 20 Hz high-pass filter to the watermarked training audio and to the validation prompts. If listeners cannot reliably distinguish Tone 10 watermarked audio from clean audio (e.g., d' < 1) and the rule-based detector still achieves AUC above approximately 0.7 after high-pass filtering, the concern is resolved. If either condition fails, the 'imperceptible watermark' conclusion should be removed or explicitly restricted to unfiltered pipelines.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim that watermarking techniques 'including some that are imperceptible to humans' can be detected in model outputs rests almost entirely on the Tone 10 result (AUC 0.7112). That result is not sufficient to support the claim as stated, for two concrete reasons. First, imperceptibility is asserted from PESQ and SI-SNR, but PESQ is a speech-quality metric, the authors explicitly state there is no established threshold for either metric, and no human listening test was run (Section 3.5). Second, a 10 Hz tone, while below the nominal 20 Hz hearing limit, is precisely in the range that audio playback systems and music preprocessing pipelines typically high-pass filter; the authors concede this in Section 7: 'These tones are however subject to removal through high-pass filtering.' In the experiments, no such filtering is applied, so the Tone 10 AUC measures a setting that may not survive even routine dataset preprocessing. The other high-AUC results are not imperceptible: Tone 440 has PESQ 2.12 and Multi 50 has PESQ 1.86, both clearly audible distortions. Therefore, if the imperceptibility and real-world robustness of Tone 10 is not established, the paper's headline conclusion is reduced to a demonstration that audible corruption of training data can shift model outputs, which is a much weaker and less surprising claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper asks whether audio watermarks embedded in the training data of a music generation model can be detected in the model's generated outputs. The authors fine-tune MusicGen-small on MusicCaps, watermarking 50% of the training data with either tone-based watermarks or AudioSeal (including repeated AudioSeal embeddings), and compare outputs of the watermarked model against a matched clean model using the watermark's own detector as a binary classifier. They report that Tone 440 achieves AUC 0.7787, Tone 10—claimed to be imperceptible—achieves AUC 0.7112, and repeated AudioSeal (Multi 50) achieves AUC 0.7113, while vanilla AudioSeal does not beat random guessing. A different-tone control yields AUC 0.51, and a scaling experiment with Tone 440 shows increasing AUC with the proportion of watermarked data.","tokens_in":14160,"tokens_out":7054,"duration_ms":75767,"significance":"If the results hold, this is a useful first step toward auditing whether a music generation model was trained on protected content, and the matched clean/watermarked model design is a strength: both models use the same training procedure, three sampling seeds are used, a random-guess baseline is reported, and the different-tone control in Section 5.4 helps establish specificity to the trained watermark. The study of AudioSeal's robustness to the Encodec32 tokenizer, including the repeated-embedding idea, is also valuable. However, the paper's central 'imperceptible watermark' claim rests on a single 10 Hz tone condition whose imperceptibility is asserted from objective metrics that the authors themselves say have no established threshold, and whose real-world survival is questionable. Until that condition is substantiated by human listening or robustness experiments, the significance of the main claim is substantially weaker.","major_comments":[{"comment":"The abstract's claim that watermarking techniques 'including some that are imperceptible to humans' cause detectable shifts rests almost entirely on the Tone 10 result (AUC 0.7112). The only imperceptibility evidence offered is PESQ 4.59 and SI-SNR 3.00, but the authors state in Section 3.5 that there is no established threshold for either metric and that human evaluations are out of scope. PESQ was designed for speech quality, not music, and SI-SNR 3.00 does not by itself establish perceptual transparency. Without a human listening test or a music-appropriate perceptual evaluation, the manuscript does not support the 'imperceptible' part of the headline claim; please add such evidence or remove/qualify the claim.","section":"§3.5, Table 1"},{"comment":"The conclusion acknowledges that Tone 10 'is subject to removal through high-pass filtering,' yet no high-pass filtering is applied in the experiments. Because real music datasets and playback chains routinely high-pass filter at 20–30 Hz, the Tone 10 AUC of 0.7112 may not survive standard preprocessing. Please report the AUC after high-pass filtering, or explicitly scope all claims to unfiltered audio; as written, the conclusion that a model trained on Tone 10 watermarks produces significantly different outputs is not robust to a threat the authors themselves identify.","section":"§7"},{"comment":"The high-AUC AudioSeal result (Multi 50, AUC 0.7113) is obtained under clear audible degradation (PESQ 1.86, SI-SNR 3.59) and therefore is not an imperceptible watermark. The paper should not be read as showing that an imperceptible state-of-the-art watermark survives training; at best it shows that repeated, perceptible embeddings do. Please separate the imperceptible and perceptible regimes in the abstract and conclusion, and be explicit that the only claimed imperceptible condition is Tone 10.","section":"Table 1, §5.2"}],"minor_comments":[{"comment":"The row label 'Secret 5' is not defined in Section 4.2.1, which defines Switch d and Alternate d but not a watermark called Secret 5. Please define it or rename it to Switch 5.","section":"Table 1"},{"comment":"The Fréchet Audio Distance formula should use the squared L2 norm of the mean difference, ||μ_r − μ_g||², not the unsquared norm as written; please correct the formula.","section":"Eq. (2)"},{"comment":"There is a duplicated phrase in the text: 'the rule-based classifier the classifier captures a change in distribution' should read 'the rule-based classifier captures a change in distribution.'","section":"§5.2"},{"comment":"The phrase 'Lest Significant Bit' should be 'Least Significant Bit.'","section":"§6.1"},{"comment":"Figure 4 reports mean AUC values without error bars, while Table 1 reports ± standard deviations for the same type of measurement; please add error bars or confidence intervals for consistency.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the experimental design is generally careful, with matched models, multiple seeds, a random-guess baseline, and a useful control in Section 5.4. My main reservation is that the headline 'imperceptible' claim depends on a single 10 Hz tone whose imperceptibility is asserted from PESQ/SI-SNR rather than demonstrated perceptually, and whose real-world survival is questionable; the authors themselves note that high-pass filtering removes it. If the authors can add a human listening test or high-pass filtering experiments, or scale back the claims, I would support publication. I would also encourage the editor to request code and data release, as the paper does not currently state availability."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a well-run proof-of-concept for a real problem, but the headline claim about imperceptible watermarks is weaker than the abstract suggests.\n\nWhat's actually new: it's the first study I've seen that applies the radioactive-data trick to a music generation model—embed a watermark in fine-tuning data and check whether the model's outputs betray it. They use MusicGen, fine-tune on MusicCaps, and compare clean vs watermarked models with three random seeds and a rule-based detector for tones plus AudioSeal's detector. The control in Section 5.4 (a different tone gives AUC 0.51) is a nice specificity check.\n\nThe strongest empirical contribution is the tokenizer robustness story: vanilla AudioSeal does not survive EnCodec compression, but applying it 10–50 times pushes detection accuracy up to 0.89 on compressed audio, and that translates into output separation (AUC 0.71 at Multi 50). That is a concrete, reproducible observation even if the watermark is no longer imperceptible at that level.\n\nNow the soft spots. The abstract says 'including some that are imperceptible to humans'—that refers to Tone 10. That claim rests on PESQ 4.59 and no human listening. PESQ is a speech metric; the paper even says there's no established threshold. And Section 7 concedes the tone is subject to removal by high-pass filtering, which is routine in music pipelines. So the imperceptible result likely doesn't survive contact with real-world preprocessing. Tone 440 and Multi 50, which give the highest AUCs, are clearly audible. This is a proportionate criticism: the authors are transparent about both issues, but the abstract overstates.\n\nSecondary issues: the frequency sweep is not corrected for multiple comparisons, and there are no code/data artifacts. The multiple-comparisons point is minor because the goal is existence, not best-frequency estimation. Missing artifacts is more annoying—this is an empirical paper, and the exact training/download pipeline would be valuable.\n\nBottom line: this is a honest, useful proof-of-concept for people working on training-data auditing or audio watermarking. It deserves a serious referee. I'd want to see a human listening test for Tone 10, a high-pass filtering experiment, and a toned-down abstract before accepting. If you're working on data provenance, it's worth citing for the tokenizer robustness finding.","headline":"A honest proof-of-concept for watermark-based auditing of music generation models, but the 'imperceptible watermark' claim rests on a 10 Hz tone that won't survive high-pass filtering.","tokens_in":14781,"tokens_out":4577,"would_cite":true,"duration_ms":42620,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Watermarks embedded in released audio can leak into the output of a music-generation model fine-tuned on that audio, allowing detection of unauthorized training use.","keywords":["audio watermarking","membership inference","music generation","training data detection","MusicGen","AudioSeal","fine-tuning","imperceptible watermark"],"falsifier":"Play Tone-10-watermarked and clean clips to listeners in an ABX test; if listeners reliably identify the watermarked clips, the imperceptibility premise fails. Alternatively, high-pass filter the watermarked training audio above 20 Hz before fine-tuning and check whether the output-classification AUC drops from 0.7112 toward 0.5.","tokens_in":13689,"feed_emoji":"🎵","tokens_out":5066,"duration_ms":50377,"temperature":0.7,"pith_summary":"This paper asks whether a musician can prove that their songs were used to fine-tune a music-generation model, by watermarking the audio before releasing it. The authors fine-tune MusicGen on clean audio and on audio carrying a watermark, then ask the watermark's own detector to classify which model produced each continuation. The central result is that the watermark changes generation behavior enough for detection: a continuous 440 Hz tone gives an AUC of 0.7787, and a 10 Hz tone, claimed to be in the imperceptible range, gives 0.7112. Detection also works when only 10% of the training data is watermarked, with an AUC of 0.6794. The main limitation is that the watermark must survive the model's audio tokenizer, and the perceptual claims rest on metrics with no established threshold.","feed_headline":"Watermarks in training audio leak into AI music output","feed_subtitle":"Fine-tuning MusicGen on watermarked songs makes its continuations identifiable, even with a 10 Hz tone.","key_machinery":"The load-bearing objects are the model's audio tokenizer and the watermark detectors. MusicGen compresses audio into discrete tokens with Encodec32 before the transformer predicts continuations, so a watermark only influences generation if it survives that compression; this is the bottleneck that makes vanilla AudioSeal undetectable. The paper's main technical move is repeated watermarking, applying AudioSeal $k$ times to the same sample, to push the watermark through the tokenizer, and its evaluation uses the watermark's own detector—a rule-based mel-band energy sum for tones and AudioSeal's trained detector for AudioSeal—to classify generated outputs.","core_discovery":"The paper's central claim is that audio watermarks embedded in a music model's fine-tuning data leak into the model's generated output, so that the watermark's own detector can tell a watermarked model from a clean one. This holds for simple tones, including a 10 Hz tone in the sub-audible band, and for AudioSeal only when the watermark is applied many times so that it survives Encodec32 compression: vanilla AudioSeal is indistinguishable from random guessing (AUC 0.4914), but 50 repeated applications reach AUC 0.7113 at heavy perceptual cost. The effect scales with the fraction of watermarked data and is frequency-specific—a model trained on Tone 440 does not produce detectable continuations of Tone 880 prompts—and generating multiple continuations per prompt improves extraction for frequency-switching secrets.","pith_inferences":["In a deployed setting, high-pass filtering—a common audio preprocessing step—would likely erase the 10 Hz tone before training, so the imperceptible-watermark result probably requires a pipeline that preserves sub-20 Hz content; the paper explicitly notes this vulnerability.","A human listening test on Tone-10-watermarked clips would settle whether the 'imperceptible' premise holds; the paper's PESQ and SI-SNR metrics were designed for speech and have no accepted threshold for music.","The tokenizer-bottleneck finding suggests that training a watermark end-to-end against Encodec32, rather than applying AudioSeal repeatedly, could yield watermarks that are both imperceptible and detectable in generated output.","Because the method only needs access to a model's outputs, the same audit strategy could plausibly transfer to other autoregressive audio models, while diffusion-based music models may respond differently to watermarked training data."],"forward_implications":["A content creator can audit a suspect music model by watermarking released audio, generating continuations from the model, and running the watermark detector on those outputs; a clearly above-chance AUC indicates the model was likely fine-tuned on the watermarked audio.","Increasing the fraction of watermarked training samples makes detection easier: with 10% of the fine-tuning data watermarked by a 440 Hz tone, the detector still reaches an AUC of 0.6794.","Watermark robustness to the model's audio tokenizer is decisive; making watermarks tokenizer-resistant, rather than merely imperceptible to humans, is what enables detection in generated output.","For temporal 'secret' watermarks, generating many continuations per prompt raises detection from an AUC of 0.569 to 0.7294, so an auditor can trade generation budget for stronger evidence.","A watermark in the nominal imperceptible frequency range (10 Hz) can shift outputs detectably, but that result depends on the audio pipeline preserving sub-20 Hz content."],"supporting_citations":[{"why":"MusicCaps provides the 10-second music clips with text captions used as the fine-tuning and evaluation dataset.","marker":"[1]"},{"why":"MusicGen is the music generation model being fine-tuned and tested throughout the paper.","marker":"[8]"},{"why":"EnCodec is the audio tokenizer whose compression destroys vanilla AudioSeal watermarks and motivates repeated watermarking.","marker":"[13]"},{"why":"AudioSeal is the learned watermarking technique and detector whose robustness to the tokenizer is tested and extended.","marker":"[28]"},{"why":"Copyright traps for LLMs is the analogous training-data watermarking idea this work transfers to the audio domain.","marker":"[22]"}],"fun_headline_variants":["Watermark leaks from training data into AI music","Sub-audible tones mark AI-generated music","Inaudible watermark detectable in model output","Training audio watermarks fingerprint AI music","AI music reveals hidden training watermarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result that imperceptible watermarks shift model outputs assumes the 10 Hz tone really is inaudible; the paper's only evidence is PESQ and SI-SNR scores, which have no established threshold for music and were not validated by human listening, and the tone disappears under common high-pass filtering.","fun_headline_variants_meta":{"raw":{"variants":["Watermark leaks from training data into AI music","Sub-audible tones mark AI-generated music","Inaudible watermark detectable in model output","Training audio watermarks fingerprint AI music","AI music reveals hidden training watermarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000324,"raw_usage":{"total_tokens":1782,"prompt_tokens":875,"completion_tokens":907,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":841}},"tokens_in":491,"tokens_out":907,"duration_ms":8646,"temperature":1.0,"reasoning_tokens":841,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:45:08.152151+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Play Tone-10-watermarked and clean clips to listeners in an ABX test; if listeners reliably identify the watermarked clips, the imperceptibility premise fails. Alternatively, high-pass filter the watermarked training audio above 20 Hz before fine-tuning and check whether the output-classification AUC drops from 0.7112 toward 0.5.","supporting_citations":[{"cited_title":"Simple and controllable music generation.Advances in Neural Information Processing Systems, 36, 2024","cited_arxiv_id":null,"evidence_quote":"MusicGen is the music generation model being fine-tuned and tested throughout the paper."},{"cited_title":"Proactive detection of voice cloning with localized watermarking","cited_arxiv_id":null,"evidence_quote":"AudioSeal is the learned watermarking technique and detector whose robustness to the tokenizer is tested and extended."},{"cited_title":"Copyright traps for large language models","cited_arxiv_id":null,"evidence_quote":"Copyright traps for LLMs is the analogous training-data watermarking idea this work transfers to the audio domain."}],"review_version":1}