{"id":"064fe5d9-4e11-4453-b94a-d94441da8d12","arxiv_id":"2501.18919","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Whisper encoder features plus a ResNet34 classifier detect AI-generated singing voices on the Singfake dataset, beating standard audio features and a Wav2vec2-based baseline in reported tests.","lead":"The authors found that features extracted from OpenAI's Whisper speech model can distinguish real singers from AI-generated singing voices, with the medium model performing best on the Singfake benchmark. This matters because unauthorized AI vocal clones are a growing problem for musicians, and better automatic detection tools could help platforms and rights holders catch them.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison rests on an undocumented modification to Wav2vec2+AASIST; without a same-protocol rerun, the reported SOTA improvement is not substantiated.","rationale":"The reader's weakest assumption exactly identifies the most load-bearing uncertainty. The central claim of the paper is that Whisper encodings outperform the previous best system on Singfake; but this comparison is only valid if the Wav2vec2+AASIST numbers were produced under the same protocol with RawBoost removed. The manuscript's single sentence in Section III-C is ambiguous and lacks any rerun details, making the baseline result unverifiable. The same problem affects the LFCC comparisons in Section IV-A, where results are taken from [7] rather than reproduced. The additional issue of test-set-based model selection compounds the problem, as the proposed approach has a selection advantage over the fixed baseline. However, these are all empirical verification gaps rather than internal contradictions; the within-paper comparisons among Whisper sizes are consistent, and the proposed system may well be strong. Therefore the appropriate verdict remains CONDITIONAL, pending a reproducible baseline rerun. My read does not change the reader's verdict.","tokens_in":5927,"tokens_out":6152,"duration_ms":53492,"concrete_test":"Rerun the Wav2vec2+AASIST baseline without RawBoost in the same codebase and on the exact same data partitions, metric, and hyperparameter-selection procedure used for the proposed system (i.e., choose baseline hyperparameters on the validation set, not the test set). Recompute the average EER across T01-T04 and the T04 EER for both vocals and mixtures. Then compare these numbers with the values used in Section IV-C; if the baseline EERs do not match the values implied by [7], the reported 28.94 and 24.52 point improvements are inflated and the 'outperforms SOTA' claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-C says 'in this baseline work [7], the RawBoost data augmentation module has been removed for fair comparisons between methods.' This is the only description of how the SOTA baseline was obtained. The paper never states whether the authors reran Wav2vec2+AASIST in their own codebase, on the same train/validation/test partitions, with the same hyperparameters, or whether the reported numbers come from [7] with a presumed adjustment. Since Section IV-C reports only the EER differences (28.94 and 24.52 points on T04) and not the absolute baseline EERs, the reader cannot audit the comparison. If the published [7] numbers actually include RawBoost and the authors simply subtracted its effect, or if the baseline used different input segmentation or classifier training, the claimed superiority of Whisper encodings over the previous SOTA is not established. The same issue applies to the LFCC results taken from [7] in Section IV-A, which underpin the claim that Whisper encodings outperform standard features. A further confound is that the proposed system selects the Whisper model size and classifier based on the test partitions (Figures 3-6 report 'testing' EERs and W(Med.) is chosen as best), so the best reported number benefits from test-set selection, whereas the baseline has no such selection advantage. The load-bearing assumption is that removing RawBoost yields a directly comparable baseline; the paper provides no evidence for it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a singing voice deepfake detection (SVDD) system that extracts encoder representations from OpenAI's Whisper model at four sizes (tiny, base, small, medium) and feeds them to a CNN or ResNet34 classifier. Using the Singfake dataset, it evaluates the system on vocals and mixtures across test conditions T01-T04 and reports average EERs of 4.86% and 9.45% for Whisper-medium with ResNet34 on vocals and mixtures, respectively. It also reports large EER improvements over a Wav2vec2+AASIST baseline, especially on T04, and concludes that Whisper encodings outperform standard MFCC/CQCC/LFCC features.","tokens_in":6226,"tokens_out":5357,"duration_ms":48652,"significance":"If the reported results are reproducible under a common protocol, the paper would provide a useful empirical data point: an off-the-shelf frozen ASR encoder combined with a small classifier can be a competitive detector for singing voice deepfakes. The conceptual hook that Whisper's encodings are not noise-invariant and may carry discriminative cues is interesting, and the within-paper ordering across Whisper sizes and standard features is consistent with the main claim. However, the comparison to prior state of the art is not currently auditable because the baseline modification is undocumented, and the best configuration is selected using test-partition results. These issues must be addressed before the headline claim can be accepted.","major_comments":[{"comment":"The comparison against the Wav2vec2+AASIST baseline is load-bearing and not substantiated. Section III-C says only that in the baseline work [7] 'the RawBoost data augmentation module has been removed for fair comparisons between methods,' but it does not state whether the authors reran Wav2vec2+AASIST in their own codebase, with the same train/validation/test partitions and the same hyperparameters, or whether published numbers from [7] were used with an adjustment. Section IV-C then reports only absolute EER differences (28.94 and 24.52 percentage points on T04) and no absolute baseline EERs, so the reader cannot audit the comparison. If the published [7] numbers include RawBoost or were obtained under a different input segmentation or classifier training protocol, the claimed superiority over prior state of the art is not established. Please rerun the baseline under the common protocol and report its absolute EERs alongside the proposed system.","section":"III-C and IV-C"},{"comment":"The model configuration appears to be selected on the test partitions. The paper identifies W(Med.) as the best based on average 'testing' EERs in Figures 3-6, and Section IV-B then restricts all subsequent experiments to ResNet34 after observing that it gave the best performance. The baseline from [7] is not given the same selection advantage, so the reported T04 differences mix algorithmic gain with test-set selection. Please specify a validation-based model selection rule and report test EERs for the selected configuration, or provide results for all configurations so that the selection protocol is transparent.","section":"IV-A, Figures 3-6"},{"comment":"All EER values appear to be from a single run with no error bars, confidence intervals, or significance tests, and the paper does not mention releasing code or experimental details sufficient for reproduction. Because some reported differences between Whisper sizes and between feature sets may be small, the reader cannot determine which pairwise gaps are statistically meaningful. Please provide multiple seeds or bootstrap confidence intervals, and release code or a detailed reproducibility description.","section":"IV-A"},{"comment":"The comparison with standard features mixes protocols. LFCC results are taken directly from [7], while MFCC and CQCC experiments are performed by the authors, but the text does not specify whether the same CNN/ResNet34 classifiers, input pipelines, and train/validation splits were used for these standard features as for the Whisper encodings. If the LFCC numbers come from a different classifier or protocol, the claim that Whisper encodings outperform standard representations is not an apples-to-apples comparison. Please report the standard-feature EERs under the same protocol or clearly state the differences.","section":"III-C and IV-A"}],"minor_comments":[{"comment":"The value '24.52' should read '24.52%' for consistency with the adjacent '28.94%'.","section":"IV-C"},{"comment":"There is a typo: 'T o4' should be 'T04'.","section":"IV-B.1"},{"comment":"The title contains an erroneous space in 'V oices'; this is likely a LaTeX formatting artifact that should be corrected in the final version.","section":"Title"},{"comment":"The claim that Whisper's last hidden state has 'fixed dimensions' is imprecise, because the encoder output length depends on the input mel-spectrogram duration. The paper should state how variable-length clips are converted to the fixed-size input expected by ResNet34/CNN (for example, padding, truncation, or pooling).","section":"II-A"},{"comment":"The noise-variance property is attributed to [9] for general audio, but the paper does not directly measure whether the encodings for singing vocals are more noise-sensitive than standard features. This is a framing issue rather than a flaw in the performance comparison, but it should be phrased as a hypothesis supported by the prior work.","section":"II-B"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claim is plausible, but the manuscript currently overstates its evidence because the baseline comparison is unauditable and the best configuration is selected on test data. A successful revision should rerun the baseline under one common protocol, report absolute baseline EERs with uncertainty estimates, release code, and make model selection validation-based. These are fixable within the manuscript's scope, so major revision rather than rejection seems appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid empirical study of an obvious-in-hindsight idea: frozen Whisper encodings plus a small classifier work well for singing-voice deepfake detection on Singfake. What is genuinely new is the systematic coverage—four Whisper sizes, two classifiers, vocals vs. mixtures, and the full T01–T04 test suite. The within-paper comparisons are consistent, and the Whisper-AT noise-variance motivation is used sensibly. I believe the within-paper result: Whisper encodings beat MFCC/CQCC/LFCC for this task on this dataset, with W(Med.)+ResNet34 clearly the best configuration. The paper also honestly flags T04 as a limitation.\n\nThe soft spots are real and they sit exactly where the reader's report puts them. The headline outperformance against Wav2vec2+AASIST depends on removing RawBoost from the baseline, but the paper never says whether the baseline was rerun on the same splits, with the same training recipe, or whether the numbers were taken from the Singfake paper after a mental adjustment. Since Section IV-C reports only EER differences (28.94 and 24.52 points on T04) and not the absolute baseline EERs, the comparison cannot be audited. That is a load-bearing problem, not a nitpick. Second, the model size and classifier are selected based on the testing partitions: the figures report \"testing\" EERs across the four Whisper sizes, and W(Med.) is then presented as the best. This gives the proposed system a selection advantage that the baseline does not get. Third, all EERs are single runs with no variance; with a 4.86% vs 9.45% average, that is not disqualifying, but it weakens the precision of the claims. No code is released, which makes the undocumented baseline change worse, not better.\n\nNone of this refutes the central empirical finding. The paper is coherent on its own terms, and the main result is plausible. But the SOTA claim needs a same-protocol rerun of Wav2vec2+AASIST (or a clearly stated reuse of published numbers with justification), and the model-selection protocol needs to be changed to validation-based or explicitly reported as test-selected. The limiting-condition and error-bar issues are minor by comparison.\n\nWho is this for? Researchers in audio spoofing and singing-voice deepfake detection who want a quick, practical baseline using Whisper features. It is worth citing once the baseline comparison is cleaned up, and it certainly deserves a serious referee. I would send it to peer review, but with a required revision on the baseline and selection protocol.","headline":"Useful first systematic look at Whisper encodings for singing-voice deepfake detection, but the SOTA-over-baseline claim is not yet auditable because the baseline was modified off-book and the best model is picked on test partitions.","tokens_in":6745,"tokens_out":1504,"would_cite":true,"duration_ms":17250,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Whisper encoder representations, especially the medium variant with a ResNet34 classifier, detect deepfake singing voices in both isolated vocals and full mixtures, outperforming standard audio features and the Wav2vec2+AASIST baseline on…","keywords":["singing voice deepfake detection","Whisper encodings","anti-spoofing","transfer learning","Singfake dataset","voice synthesis detection","Equal Error Rate","audio forensics"],"falsifier":"Run the Wav2vec2+AASIST baseline with RawBoost removed on the same train/validation splits and test sets used for the proposed system; if the resulting T04 equal error rates are substantially lower than the numbers the paper compares against, the reported 28.94- and 24.52-percentage-point improvements would not hold.","tokens_in":5721,"feed_emoji":"🎤","tokens_out":7800,"duration_ms":66784,"temperature":0.7,"pith_summary":"This paper tries to establish that the internal audio encodings of the Whisper speech-recognition model, despite coming from a system trained to be robust to noise, are in fact sensitive to background and non-speech content, and that this sensitivity can be turned into a detector for deepfake singing voices. Using the Singfake dataset, the authors feed Whisper encoder outputs into CNN and ResNet34 classifiers and report that the medium Whisper variant with ResNet34 achieves a 4.86% average equal error rate on vocals and 9.45% on mixtures, outperforming MFCC, CQCC, LFCC, and the Wav2vec2+AASIST baseline. The results matter because unauthorized synthetic vocals threaten artists' likeness and royalties, and singing voices behave differently from speech, so speech anti-spoofing systems may not carry over. If the claim is right, a readily available ASR encoder plus a small classifier is a practical starting point for singing-voice deepfake detection.","feed_headline":"Whisper encodings catch fake singing voices at 4.86% EER","feed_subtitle":"A medium Whisper encoder plus ResNet34 beats standard audio features and a Wav2vec2 baseline on Singfake vocals and mixtures.","key_machinery":"The central object is the last hidden state of the Whisper encoder, a sequence of fixed-dimension vectors produced from input mel-spectrograms by a Transformer encoder with a number of blocks that varies by model size. Whisper is trained on hundreds of thousands of hours of diverse web audio, and prior work has shown that, unlike typical ASR front-ends, it encodes background sound rather than discarding it. That noise-variant representation is the mechanism: it retains acoustic context such as the abrupt transition from silence into a synthesized vocal and the separation of formants from instrumentals, giving a CNN or ResNet34 classifier enough discriminative signal to separate bonafide from deepfake singing voices without data augmentation. In the paper's experiments the medium Whisper variant performs best with both classifiers.","core_discovery":"The paper's central claim is that Whisper's audio representation is not a noise-invariant abstraction but a noise-conditioned one, and that very property is useful for anti-spoofing. Concretely, the authors show that feeding the encoder output of the medium Whisper variant through a ResNet34 classifier yields a 4.86% average EER on the Singfake vocals and 9.45% on mixtures, outperforming standard feature sets and the Wav2vec2+AASIST baseline. On the hardest test condition, T04 with unseen languages and musical contexts, the reported absolute EER improvements over the baseline are 28.94 percentage points for vocals and 24.52 percentage points for mixtures. Spectrographic differences illustrated in the paper indicate that deepfake vocals show sharp onsets after silence and formants that remain distinct from background instruments, while bonafide recordings fade gradually and blend with the accompaniment; the Whisper encodings are said to preserve these synthesis artifacts.","pith_inferences":["The same noise-variant encoding principle could be tested on speech deepfakes and non-vocal audio deepfakes: if Whisper's representations preserve the acoustic context that synthesis models reconstruct imperfectly, a similar encoder-plus-classifier pipeline may transfer without retraining the encoder.","A practical deployment would freeze the Whisper encoder and train only the small classification head, making the detector cheap to update as new synthesis methods appear; this is not tested in the paper but follows from its architecture.","The sharp T04 drop suggests the model's cues are partly tied to language and musical context seen in training; covering more languages in the training partition or adding augmentation that simulates unseen contexts would test whether the artifacts the encodings capture are truly universal.","Combining Whisper encodings with the best standard feature set, such as LFCC, or with augmentation might close the remaining T04 gap, an avenue the paper itself names as future work."],"forward_implications":["A single Whisper encoder variant with a ResNet34 head is enough to reach 4.86% average EER on Singfake vocals and 9.45% on mixtures, so a singing-voice deepfake detector need not start from speech-specific countermeasures.","The detector keeps working when the singing voice is mixed with instrumental accompaniment, though mixtures are consistently harder to detect than isolated vocals.","The method transfers across the Singfake testing conditions with a clear difficulty gradient: seen-singer unseen-song clips are easiest at 1.09% EER, while unseen languages and musical contexts are hardest, localizing the remaining robustness gap.","Whisper encodings outperform MFCC, CQCC, and LFCC for both vocals and mixtures with both classifiers, indicating the gain comes from the pre-trained representation rather than from the classifier choice.","Because the system uses no data augmentation, the discriminative power is carried by the encodings themselves rather than by input perturbation."],"supporting_citations":[{"why":"Introduces the Singfake dataset, the Wav2vec2+AASIST baseline, and LFCC results that the proposed system is compared against.","marker":"[7]"},{"why":"Provides the Whisper model whose encoder's last hidden states are used as the proposed feature representation.","marker":"[8]"},{"why":"Supplies the empirical result that Whisper encodings are noise-variant and carry non-speech information, the premise the SVDD method exploits.","marker":"[9]"},{"why":"Performs the source separation used to create the vocals subset from full songs in the Singfake dataset.","marker":"[15]"},{"why":"Provides voice activity detection used to segment songs into clips that define the evaluation units.","marker":"[16]"},{"why":"Provides the voice activity detection library used alongside [16] to segment both vocals and mixture clips.","marker":"[17]"}],"fun_headline_variants":["Whisper's noise-variant encodings unmask fake singing at 4.86% EER","Whisper encodings, not noise-proof, catch fake singers at 4.86% EER","Whisper's noise sensitivity exposes deepfake vocals at 4.86% EER","Fake singing unmasked by Whisper's noise-variant encodings: EER 4.86%","Whisper's noise trick exposes deepfake singers: 4.86% EER"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the previous best system's scores after removing its data-augmentation module were computed on exactly the same train/test splits and settings as the proposed system; the paper does not document this.","fun_headline_variants_meta":{"raw":{"variants":["Whisper's noise-variant encodings unmask fake singing at 4.86% EER","Whisper encodings, not noise-proof, catch fake singers at 4.86% EER","Whisper's noise sensitivity exposes deepfake vocals at 4.86% EER","Fake singing unmasked by Whisper's noise-variant encodings: EER 4.86%","Whisper's noise trick exposes deepfake singers: 4.86% EER"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001259,"raw_usage":{"total_tokens":5127,"prompt_tokens":886,"completion_tokens":4241,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":4112}},"tokens_in":502,"tokens_out":4241,"duration_ms":45570,"temperature":1.0,"reasoning_tokens":4112,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T21:57:16.883145+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Wav2vec2+AASIST baseline with RawBoost removed on the same train/validation splits and test sets used for the proposed system; if the resulting T04 equal error rates are substantially lower than the numbers the paper compares against, the reported 28.94- and 24.52-percentage-point improvements would not hold.","supporting_citations":[{"cited_title":"Singfake: Singing voice deepfake detection,","cited_arxiv_id":null,"evidence_quote":"Introduces the Singfake dataset, the Wav2vec2+AASIST baseline, and LFCC results that the proposed system is compared against."},{"cited_title":"Whisper-AT: Noise- robust automatic speech recognizers are also strong general audio event taggers,","cited_arxiv_id":null,"evidence_quote":"Supplies the empirical result that Whisper encodings are noise-variant and carry non-speech information, the premise the SVDD method exploits."},{"cited_title":"Computationally-efficient voice activity detection based on deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides voice activity detection used to segment songs into clips that define the evaluation units."},{"cited_title":"Pyannote. audio: neu- ral building blocks for speaker diarization,","cited_arxiv_id":null,"evidence_quote":"Provides the voice activity detection library used alongside [16] to segment both vocals and mixture clips."}],"review_version":1}