Pith. sign in

REVIEW 4 major objections 8 minor 62 references

Robust AI-Synthesized Speech Detection Using Feature Decomposition Learning and Synthesizer Feature Augmentation

T0 review · 4 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A dual-stream classifier learns synthesizer-independent content features, reaching 97.79% AUC on unseen vocoders.

desk verdict A well-ablated robustness recipe for audio deepfake detection whose central pseudo-labeling mechanism is underspecified; worth refereeing, but the authors must clarify the pipeline and release code. read the letter →

arxiv 2411.09167 v1 pith:CTPT25QO submitted 2024-11-14 cs.SD cs.CReess.AS

Kuiyuan Zhang , Zhongyun Hua , Yushu Zhang , Yifang Guo , Tao Xiang This is my paper · ORCID
classification cs.SDcs.CReess.AS
keywords AI-synthesizedspeechdetectiondeepfakefeaturedecompositionsynthesizer-independentfeaturespseudo-labelingadversariallearningaugmentationcross-methodgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deepfake speech detectors usually key on the specific artifacts left by the synthesizer that made the audio, so their accuracy drops when a new or unseen synthesizer appears. This paper proposes a dual-stream design that explicitly splits the learned speech representation into a synthesizer stream and a content stream, and claims this decomposition keeps detection accurate on unseen generators. The content stream is trained with pseudo-labels produced by randomly changing speech speed and compression, while an adversarial loss pushes its synthesizer predictions toward random guessing; the synthesizer stream keeps the artifact-specific signal. On cross-method evaluation with LibriSeVoc the method reports 97.79% average AUC and 6.12% EER, versus 91.14% AUC and 13.12% EER for the best compared baseline, and on ASVspoof2021 DF it reports 14.79% EER on the whole test subset. If these results hold, deepfake speech screening would become more durable against the steady stream of new synthesis methods.

What carries the argument

The load-bearing mechanism is a dual-stream feature decomposition applied to the representation from a ResNet18 backbone operating on log-scale spectrograms. A synthesizer stream, one convolutional block with average pooling and a linear synthesizer classifier, is trained with cross-entropy on synthesizer labels plus a contrastive loss; a content stream of the same structure is trained with cross-entropy on pseudo-labels generated by random compression and speed changes, plus an adversarial loss that drives synthesizer prediction from content features toward a uniform distribution. Feature blending and feature shuffle then augment the concatenated features used for the final binary classification. The pseudo-labeling tasks are what make the content stream learn dataset-agnostic properties, and the adversarial loss is intended to remove residual artifact information.

What would settle it

Train the proposed model, then measure how much synthesizer identity can be recovered from the content features Fc with a linear probe on held-out synthesizer methods; if probe accuracy stays far above chance while the adversarial loss is active, the claimed synthesizer independence of the content stream is falsified. A second check is to confirm that the speed and compression pseudo-labels are computed from the exact input the content stream sees; if the transformed audio is not what is fed through the network, the pretext supervision is not well-posed for unmodified inputs.

Watch

Extended reading notes

Core claim

The paper's central claim is that the poor generalization of deepfake speech detectors to unseen synthesizers can be reduced by decomposing the learned representation into two complementary parts: synthesizer-specific features, trained with synthesizer labels and a contrastive loss, and synthesizer-independent content features, trained by predicting pseudo-labels from speech speed and compression transformations. The final real/fake decision is made on the concatenation of the two feature vectors. An adversarial loss, applied only to the content stream, penalizes accurate prediction of the synthesizer identity from content features, encouraging that stream to discard artifact information. A synthesizer feature augmentation scheme that blends styles within real and fake groups and randomly shuffles synthesizer and content features across samples is claimed to improve robustness further. The reported inner, cross-method, cross-dataset, and cross-language experiments on WaveFake, LibriSeVoc, DECRO, and ASVspoof2021 DF are offered as evidence.

Load-bearing premise

The load-bearing premise is that predicting random speed and compression labels, together with an adversarial loss that randomizes synthesizer prediction, makes the content stream learn genuinely synthesizer-independent features rather than a different set of artifact-correlated cues; the paper does not verify that independence directly, only via downstream detection accuracy.

Editorial extensions

If this is right

  • Detectors built this way should degrade less sharply when new TTS or voice-conversion methods appear, because the content stream is not optimized to latch onto artifact-specific cues.
  • On the ASVspoof2021 DF test subset, the method reports the lowest whole-subset EER among the compared detectors, 14.79%, including on seen and unseen synthesizer categories.
  • Cross-dataset transfer improves: trained on LibriSeVoc and tested on WaveFake gives 2.18% average EER, and on DECRO English 6.88% and Chinese 17.77%.
  • Cross-language transfer gives the best average EER of 22.48% across WaveFake EN→JP and DECRO ZH→EN and EN→ZH tasks.
  • Ablation studies indicate both streams contribute: removing either stream, or replacing the speed/compression pretext with F0 prediction, raises EER, supporting the claim that decomposition and the chosen pretext tasks drive the gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: because the content stream's supervision comes from transforming the input, the method should be stress-tested on audio that is already compressed, resampled, or band-limited; the paper's future-work section acknowledges that pseudo-labeling may give limited guidance on such data.
  • Extension: a direct test of the decomposition claim is to measure how much synthesizer identity remains recoverable from the content features; a linear probe that still reads synthesizer identity from Fc on held-out generators would show the claimed independence is only approximate.
  • Extension: the feature-shuffle operation creates mixed real/fake feature pairs labeled by an intersection rule, which suggests the same machinery could be adapted to partially manipulated or locally synthesized utterances by treating the shuffle as a noisy-label mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. This paper proposes a robustness-oriented deepfake speech detection method built on a dual-stream decomposition of a shared ResNet18 representation. A synthesizer stream is supervised with synthesizer-identity classification and a contrastive loss; a content stream is trained with two pseudo-labeling tasks (random speed change and random compression) plus an adversarial loss that pushes synthesizer prediction from content features toward a uniform distribution, with gradients restricted to the content stream. The final real/fake classifier uses the concatenation of the two streams, and a feature augmentation strategy randomly blends feature statistics within real/fake classes and randomly shuffles content features with synthesizer features from other samples, using focal loss for the shuffled pairs. The method is evaluated on WaveFake, LibriSeVoc, DECRO, and a modified subset of ASVspoof2021 DF under intra-dataset, cross-method, cross-dataset, and cross-language protocols, and is reported to achieve the best average AUC/EER on most internal comparisons, with ablations of the loss weights, streams, pseudo-label objectives, and augmentation components.

Significance. The work targets a real limitation of current detectors — collapse onto synthesizer-specific artifacts — and its central idea of learning content features that are orthogonal to synthesizer identity is a reasonable and potentially useful contribution. The empirical evidence is extensive: twelve methods compared across four datasets, an external ASVspoof2021 DF benchmark whose test data were not used in training, component-wise ablations, and model complexity/throughput analysis. Hyperparameters are fixed before testing, the paper reports ten-run averages, and no free parameters appear to be fitted to the test sets. If the pseudo-label training procedure is implemented as the text implies (and clarified as requested below), the cross-method improvements in Table III (97.79% versus 91.14% average AUC over ASDG) would be a meaningful advance. The main weaknesses are the underspecified pseudo-label pipeline, missing standard deviations, the modified ASVspoof test protocol, and the overbroad cross-language "state-of-the-art" claim; none of these invalidates the contribution, but all should be fixed or qualified.

major comments (4)
  1. [Section III-D.1, Eqs. (6)-(8), Table XII] The manuscript never states whether the speed/compression-transformed audio is the signal fed into the network when computing the content features Fc for the pseudo-label losses. Eq. (7) computes the predictions from Fc, Eq. (6) defines Fc from the hidden states FH of the input speech, and the text in Section III-D.1 describes only how the labels are generated ("randomly change the speech speed ... and randomly compress the speech"). If Fc is computed from the original, untransformed clip, then the speed/compression labels are independent of the features, the cross-entropy terms in Eq. (8) cannot be minimized by any learned representation, and the proposed feature-decomposition mechanism is not actually trained. Table XII shows this is load-bearing: removing the pseudo-labeling losses raises Task1 EER from 6.12% to 12.52%. The authors should specify unambiguously whether the transformed audio is the forward-pass input for Eq. (8), how the transformed and original signals are batched and separated across the multiple losses (including whether the main real/fake loss in Eq. (10) is computed on original or transformed audio), and how the resampled signals are re-trimmed or padded to the fixed 48000-sample length, since speed resampling changes signal duration.
  2. [Section V-A, Tables X-XII] The paper says "we train it with ten runs in each task, where each run utilizes a different global random seed ... Then, we report the average values on the ten runs," but no standard deviation, min-max range, or per-seed results are given anywhere. The training pipeline is stochastic (random crops, random speed/compression transforms, feature blending noise), and several of the ablation gaps are small — e.g., 7.06 vs. 6.12 on Task1 in Table XI and 2.18 vs. 2.23 on Task2 in Table X — so without variance information the reader cannot tell whether the reported improvements are stable across seeds. Reporting standard deviations or per-run results is necessary to support the ablation-based attribution of gains to each component.
  3. [Section V-B.2, Table VI] The ASVspoof2021 DF evaluation uses a modified test protocol: "we use only a portion of fake samples in the test subset" so that the number of fake samples per synthesizer category matches the number of test real samples. This changes the test-set class balance and category composition relative to the official ASVspoof2021 protocol, and the resulting 14.79% "Whole Testing" EER is therefore not directly comparable to published ASVspoof2021 scores unless this deviation is clearly flagged at the point where the number is presented. I recommend stating the modification in the table caption and main text and reporting official-protocol numbers as well, since the current presentation risks being read as official-protocol results.
  4. [Abstract, Section V-D, Table VIII] The claim of "state-of-the-art robust detection performance across various evaluation scenarios, including cross-method, cross-dataset, and cross-language evaluations" is stronger than the data support. On WaveFake EN→JP the method achieves 23.26% EER, below LCNN (6.74%), ASDG (7.25%), WaveLM (10.22%), RawGAT (12.86%), AASIST (13.04%), and Wave2Vec2 (16.90%), and on DECRO EN→ZH it is behind Wav2Clip (27.54% vs. 21.66%). Only on DECRO ZH→EN is the proposed method best. The method has the best average EER, but it is not state of the art on two of the three cross-language tasks; the abstract and Section V-D should be qualified accordingly.
minor comments (8)
  1. [Section IV-D] N1=10 is stated, but "three codecs (aac, ops, mp3) and three bitrates (16000, 32000, 64000)" yields nine combinations; please clarify the tenth setting and correct "ops" to "opus" if that is intended.
  2. [Table I caption] "EN, JP, and ZH donate the English, Japanese, and Chinese subsets" should read "denote".
  3. [Throughout] The dataset name is spelled inconsistently ("LibriSeVoc" in captions and "LibriseV oc" in the text); please unify the spelling.
  4. [Reference [59]] Reference [59] is cited for the Adam optimizer but is the Gradient Centralization paper; the original Adam reference (Kingma and Ba, 2015) should be cited instead.
  5. [Eq. (13)] The focal loss is written only for the positive class; the full two-class form (including the negative-class term with the 1-alpha weight) should be given so that alpha=0.25 is unambiguous.
  6. [Section III-C] "By training the synthesizer stream with the losses Lcls s and Lcon c" should read "Lcon s".
  7. [Section VI-G] The first sentence, "We method can effectively address the growing threat of malicious deepfake speech," contains a grammar error and should be rewritten.
  8. [Eq. (10) and Section I] The text after Eq. (10) says "with weighs size of 1 x 2N"; "weighs" should be "weights", and the introduction's reference to "Sections VI" does not match the actual section numbering used in the rest of the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are empirical results on held-out and external benchmarks, with no parameter fitted to the test sets and no load-bearing self-citation chain.

full rationale

The paper's central claim is an empirical generalization claim, not a derivation: the method is trained on fixed splits, selected hyperparameters before testing, and evaluated on unseen synthesizers, unseen datasets, and an external ASVspoof2021 DF test subset that was not used during training. The pseudo-labeling speed/compression losses in Section III-D are pretext tasks whose labels are generated by speech transformations, not by the deepfake labels being predicted, so the detection result is not defined in terms of the loss objectives. The adversarial loss and feature augmentation are regularizers whose effects are measured through ablations, not by construction of the final AUC/EER numbers. There is no self-citation chain carrying a uniqueness theorem or a forced ansatz, and the synthetic-feature decomposition is a proposed architecture rather than a renaming of an existing result. The ambiguity about whether transformed audio is actually fed to the content branch is a reproducibility and specification concern, not a circularity: even if the pretext loss were inert, the reported numbers would be empirical outcomes of the remaining training components, not quantities made equal to their inputs by definition. Accordingly, no equation-level or self-citation-level circularity is exhibited.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical ML submission without a mathematical derivation. Its central claim rests on the effectiveness of the pretext tasks, adversarial decorrelation, and feature augmentation, plus many hand-selected hyperparameters. No code or data artifacts are released, so these premises cannot be independently tested from the preprint.

free parameters (7)
  • Loss weights beta0-beta3 = 1.0, 0.5, 0.5, 0.5
    Selected from four empirically tested combinations in Section VI-E; ablation shows EER changes by about 1-2% with different settings.
  • Contrastive margin alpha = 0.4
    Hyperparameter in the synthesizer-stream and final-feature contrastive losses (Eq. 4); set by hand.
  • Feature blending noise level eta = 10
    Controls the Beta and Gaussian noise added to blended features in Algorithm 1; set by hand.
  • Focal loss parameters alpha and gamma = 0.25, 2
    Used in the feature-shuffle classification loss (Eq. 13); these are defaults from Lin et al.
  • Pretext task categories N1, N2 = N1=10 compression settings, N2=16 speed settings
    Number of pseudo-label classes for the content stream; chosen by the authors and affects the difficulty of the pretext task.
  • STFT window size and hop length = window 512, hop 187
    Input feature extraction settings in Eq. (1); they determine the time-frequency resolution of the log-scale spectrogram.
  • Input audio length and sampling rate = 3 seconds at 16 kHz
    All clips are truncated or padded to 48k samples; this controls how much of each utterance the model sees.
assumptions (4)
  • domain assumption Predicting speed and compression pseudo-labels yields features that are synthesizer-independent and useful for real/fake detection.
    Section III-D-1 assumes this proxy task transfers to detection; no proof is given that the learned content features are decorrelated from synthesizer identity.
  • domain assumption An adversarial loss that makes synthesizer prediction from content features close to uniform removes synthesizer-related information without discarding detection-relevant cues.
    Section III-D-2 relies on the optimization finding a useful solution; this is a standard but unproven assumption in adversarial feature decorrelation.
  • domain assumption Feature blending and shuffling with the stated label rule (real only when both source samples are real) provides valid training supervision.
    Section III-F assumes the composed labels for shuffled features are meaningful for learning a robust decision boundary.
  • domain assumption The benchmark datasets and the balanced subset of ASVspoof2021 DF represent meaningful evaluations of robustness.
    Section IV and Section V-B-2; the paper modifies the official ASVspoof2021 test set by downsampling fake samples to match real counts, so absolute EERs are not comparable to official challenge results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust AI-Synthesized Speech Detection Using Feature Decomposition Learning and Synthesizer Feature Augmentation." pith.science (2026). https://pith.science/paper/CTPT25QO

@misc{pith2026241109167,
  author       = {Pith},
  title        = {Pith review of: Robust AI-Synthesized Speech Detection Using Feature Decomposition Learning and Synthesizer Feature Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CTPT25QO}},
  note         = {Machine review of arXiv:2411.09167}
}
read the original abstract

AI-synthesized speech, also known as deepfake speech, has recently raised significant concerns due to the rapid advancement of speech synthesis and speech conversion techniques. Previous works often rely on distinguishing synthesizer artifacts to identify deepfake speech. However, excessive reliance on these specific synthesizer artifacts may result in unsatisfactory performance when addressing speech signals created by unseen synthesizers. In this paper, we propose a robust deepfake speech detection method that employs feature decomposition to learn synthesizer-independent content features as complementary for detection. Specifically, we propose a dual-stream feature decomposition learning strategy that decomposes the learned speech representation using a synthesizer stream and a content stream. The synthesizer stream specializes in learning synthesizer features through supervised training with synthesizer labels. Meanwhile, the content stream focuses on learning synthesizer-independent content features, enabled by a pseudo-labeling-based supervised learning method. This method randomly transforms speech to generate speed and compression labels for training. Additionally, we employ an adversarial learning technique to reduce the synthesizer-related components in the content stream. The final classification is determined by concatenating the synthesizer and content features. To enhance the model's robustness to different synthesizer characteristics, we further propose a synthesizer feature augmentation strategy that randomly blends the characteristic styles within real and fake audio features and randomly shuffles the synthesizer features with the content features. This strategy effectively enhances the feature diversity and simulates more feature combinations.

Figures

Figures reproduced from arXiv: 2411.09167 by the authors.

Figure 1
Figure 1. Network architecture of our method. A main stream is used to learn robust speech representation from the log-scale frequency [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. T-SNE visualization in the cross-evaluation task on the LibriseVoc dataset. For each deepfake speech detection method, we extract [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. T-SNE visualization in the cross-evaluation task on the DECRO ZH subset. For each deepfake speech detection method, we extract [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Grad-CAM visualization on the LibriseVoc dataset. From top to bottom of each column, the three images are raw log-scale [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Ablation results (EER (%)) of the contrastive losses on two [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 47 canonical work pages

  1. [1]

    Deepfake detection: Current challenges and next steps,

    S. Lyu, “Deepfake detection: Current challenges and next steps,” in 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2020, pp. 1–6

    work page 2020

  2. [2]

    Deepfake generation, detection and datasets: a rapid-review,

    A. KoC ¸ ak and M. Alkan, “Deepfake generation, detection and datasets: a rapid-review,” in 2022 15th International Conference on Information Security and Cryptography (ISCTURKEY) , 2022, pp. 86–91

    work page 2022

  3. [3]

    Listen to This Deepfake Audio Impersonating a CEO in Brazen Fraud Attempt,

    L. Franceschi-Bicchierai, “Listen to This Deepfake Audio Impersonating a CEO in Brazen Fraud Attempt,” Jul

    work page

  4. [4]

    Telegram Still Hasn’t Removed an AI Bot That’s Abusing Women,

    M. Burgess, “Telegram Still Hasn’t Removed an AI Bot That’s Abusing Women,” Wired, 2020. [Online]. Available: https://www.wired.co.uk/ article/porn-bots-in-telegram-deepfake

    work page 2020

  5. [5]

    WaveFake: A Data Set to Facilitate Audio Deepfake Detection,

    J. Frank and L. Sch ¨onherr, “WaveFake: A Data Set to Facilitate Audio Deepfake Detection,” in Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2021

    2021

  6. [6]

    P-flow: A fast and data-efficient zero-shot tts through speech prompting,

    S. Kim, K. Shih, J. F. Santos, E. Bakhturina, M. Desta, R. Valle, S. Yoon, B. Catanzaro et al. , “P-flow: A fast and data-efficient zero-shot tts through speech prompting,” Advances in Neural Information Processing Systems, vol. 36, 2024

    work page 2024

  7. [7]

    Phoneme hallucinator: One-shot voice conversion via set expansion,

    S. Shan, Y . Li, A. Banerjee, and J. B. Oliva, “Phoneme hallucinator: One-shot voice conversion via set expansion,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 13, 2024, pp. 14 910–14 918

    work page 2024

  8. [8]

    A review of modern audio deepfake de- tection methods: challenges and future directions,

    Z. Almutairi and H. Elgibreen, “A review of modern audio deepfake de- tection methods: challenges and future directions,” Algorithms, vol. 15, no. 5, p. 155, 2022

    work page 2022

Show all 62 references
  1. [9]

    AI-Synthesized V oice Detection Using Neural V ocoder Artifacts,

    C. Sun, S. Jia, S. Hou, and S. Lyu, “AI-Synthesized V oice Detection Using Neural V ocoder Artifacts,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 904– 912

    2023

  2. [10]

    Spoofing speech detection using modified relative phase information,

    L. Wang, S. Nakagawa, Z. Zhang, Y . Yoshida, and Y . Kawakami, “Spoofing speech detection using modified relative phase information,” IEEE Journal of selected topics in signal processing , vol. 11, no. 4, pp. 660–670, 2017

    2017

  3. [11]

    Robustness of speech spoofing detectors against adversarial post-processing of voice conversion,

    Y .-Y . Ding, H.-J. Lin, L.-J. Liu, Z.-H. Ling, and Y . Hu, “Robustness of speech spoofing detectors against adversarial post-processing of voice conversion,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3415–3426, 2021

    2021

  4. [12]

    Detecting spoofed speeches via segment-based word cqcc and average zcr for embedded systems,

    J. Zhan, Z. Pu, W. Jiang, J. Wu, and Y . Yang, “Detecting spoofed speeches via segment-based word cqcc and average zcr for embedded systems,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 41, no. 11, pp. 3862–3873, 2022

    2022

  5. [13]

    Detecting ai-synthesized speech using bispectral analysis

    E. A. AlBadawy, S. Lyu, and H. Farid, “Detecting ai-synthesized speech using bispectral analysis.” in CVPR workshops, 2019, pp. 104–109

    2019

  6. [14]

    Fake Audio Detection Based On Unsupervised Pretraining Models,

    Z. Lv, S. Zhang, K. Tang, and P. Hu, “Fake Audio Detection Based On Unsupervised Pretraining Models,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2022, pp. 9231–9235

    2022

  7. [15]

    Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks,

    J.-w. Jung, H.-S. Heo, H. Tak, H.-j. Shim, J. S. Chung, B.-J. Lee, H.-J. Yu, and N. Evans, “Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (IC...

    2022

  8. [16]

    A robust audio deepfake detection system via multi-view feature,

    Y . Yang, H. Qin, H. Zhou, C. Wang, T. Guo, K. Han, and Y . Wang, “A robust audio deepfake detection system via multi-view feature,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 13 131–13 135

    2024

  9. [17]

    Towards end-to-end synthetic speech detection,

    G. Hua, A. B. J. Teoh, and H. Zhang, “Towards end-to-end synthetic speech detection,” IEEE Signal Processing Letters , vol. 28, pp. 1265– 1269, 2021

    2021

  10. [18]

    Does audio deepfake detection generalize?

    N. M ¨uller, P. Czempin, F. Diekmann, A. Froghyar, and K. B ¨ottinger, “Does audio deepfake detection generalize?” Interspeech 2022, 2022

    2022

  11. [19]

    Sedeptts: Enhancing the naturalness via semantic dependency and local convolution for text-to-speech synthesis,

    C. Jiang, Y . Gao, W. W. Ng, J. Zhou, J. Zhong, and H. Zhen, “Sedeptts: Enhancing the naturalness via semantic dependency and local convolution for text-to-speech synthesis,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 11, 2023, pp. 12 959– 12 967

    2023

  12. [20]

    Drvc: A frame- work of any-to-any voice conversion with self-supervised learning,

    Q. Wang, X. Zhang, J. Wang, N. Cheng, and J. Xiao, “Drvc: A frame- work of any-to-any voice conversion with self-supervised learning,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2022, pp. 3184–3188

    2022

  13. [21]

    Wavenet: A gener- ative model for raw audio,

    A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A gener- ative model for raw audio,” arXiv preprint arXiv:1609.03499 , 2016

    2016 arXiv

  14. [22]

    Efficient neural audio synthesis,

    N. Kalchbrenner, E. Elsen, K. Simonyan, S. Noury, N. Casagrande, E. Lockhart, F. Stimberg, A. Oord, S. Dieleman, and K. Kavukcuoglu, “Efficient neural audio synthesis,” in International Conference on Ma- chine Learning, 2018, pp. 2410–2419

    2018

  15. [23]

    Guided-tts: A diffusion model for text-to- speech via classifier guidance,

    H. Kim, S. Kim, and S. Yoon, “Guided-tts: A diffusion model for text-to- speech via classifier guidance,” in International Conference on Machine Learning, 2022, pp. 11 119–11 133

    2022

  16. [24]

    Grad- tts: A diffusion probabilistic model for text-to-speech,

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, and M. Kudinov, “Grad- tts: A diffusion probabilistic model for text-to-speech,” in International Conference on Machine Learning , 2021, pp. 8599–8608

    2021

  17. [25]

    Melgan: Generative adversarial networks for conditional waveform synthesis,

    K. Kumar, R. Kumar, T. De Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. De Brebisson, Y . Bengio, and A. C. Courville, “Melgan: Generative adversarial networks for conditional waveform synthesis,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  18. [26]

    Dspgan: a gan-based universal vocoder for high-fidelity tts by time- frequency domain supervision from dsp,

    K. Song, Y . Zhang, Y . Lei, J. Cong, H. Li, L. Xie, G. He, and J. Bai, “Dspgan: a gan-based universal vocoder for high-fidelity tts by time- frequency domain supervision from dsp,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (I...

    2023

  19. [27]

    Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech,

    J. Kim, J. Kong, and J. Son, “Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech,” in Proceedings of the 38th International Conference on Machine Learning , Jul. 2021, pp. 5530–5540

    2021

  20. [28]

    Hierspeech: Bridging the gap between text and speech by hierarchical variational inference using self-supervised representations for speech synthesis,

    S.-H. Lee, S.-B. Kim, J.-H. Lee, E. Song, M.-J. Hwang, and S.-W. Lee, “Hierspeech: Bridging the gap between text and speech by hierarchical variational inference using self-supervised representations for speech synthesis,” in Advances in Neural Information Processing Systems ,...

    2022

  21. [29]

    Unisyn: an end-to-end unified model for text-to-speech and singing voice synthesis,

    Y . Lei, S. Yang, X. Wang, Q. Xie, J. Yao, L. Xie, and D. Su, “Unisyn: an end-to-end unified model for text-to-speech and singing voice synthesis,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 11, 2023, pp. 13 025–13 033

    2023

  22. [30]

    ” hello, it’s me

    E. Wenger, M. Bronckers, C. Cianfarani, J. Cryan, A. Sha, H. Zheng, and B. Y . Zhao, “” hello, it’s me”: Deep learning-based speech synthesis attacks in the real world,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , 2021, pp. 235– 251

    2021

  23. [31]

    Audio replay attack detection with deep learning frameworks

    G. Lavrentyeva, S. Novoselov, E. Malykh, A. Kozlov, O. Kudashev, and V . Shchemelinin, “Audio replay attack detection with deep learning frameworks.” in Interspeech, 2017, pp. 82–86

    2017

  24. [32]

    Improved rawnet with feature map scaling for text-independent speaker verification using raw waveforms,

    J.-w. Jung, S.-b. Kim, H.-j. Shim, J.-h. Kim, and H.-J. Yu, “Improved rawnet with feature map scaling for text-independent speaker verification using raw waveforms,” Proc. Interspeech, pp. 3583–3587, 2020

    2020

  25. [33]

    Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,

    X. Liu, X. Wang, M. Sahidullah, J. Patino, H. Delgado, T. Kinnunen, M. Todisco, J. Yamagishi, N. Evans, A. Nautsch, and K. A. Lee, “Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. ...

    2021

  26. [34]

    Stc antispoofing systems for the asvspoof2019 challenge,

    G. Lavrentyeva, S. Novoselov, A. Tseren, M. V olkova, A. Gorlanov, and A. Kozlov, “Stc antispoofing systems for the asvspoof2019 challenge,” Interspeech 2019, 2019

    2019

  27. [35]

    Domain Generalization via Aggregation and Separation for Audio Deepfake Detection,

    Y . Xie, H. Cheng, Y . Wang, and L. Ye, “Domain Generalization via Aggregation and Separation for Audio Deepfake Detection,” IEEE Transactions on Information Forensics and Security , vol. 19, pp. 344– 358, 2024

    2024

  28. [36]

    Fully automated end-to-end fake audio detection,

    C. Wang, J. Yi, J. Tao, H. Sun, X. Chen, Z. Tian, H. Ma, C. Fan, and R. Fu, “Fully automated end-to-end fake audio detection,” in Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia , 2022, p. 27–33

    2022

  29. [37]

    Deepfake audio detection with vision transformer based method,

    G. Ulutas, G. Tahaoglu, and B. Ustubioglu, “Deepfake audio detection with vision transformer based method,” in 2023 46th International Conference on Telecommunications and Signal Processing (TSP) , 2023, pp. 244–247

    2023

  30. [38]

    Audio spectrogram transformer for synthetic speech detection via speech formant analysis,

    L. Cuccovillo, M. Gerhardt, and P. Aichroth, “Audio spectrogram transformer for synthetic speech detection via speech formant analysis,” in 2023 IEEE International Workshop on Information Forensics and Security (WIFS) , pp. 1–6, ISSN: 2157-4774. [Online]. Available: https://ie...

    2023

  31. [39]

    Audio transformer for synthetic speech detection via multi-formant analysis,

    L. Cuccovillo, M. Gerhardt, and P. Aichroth, “Audio transformer for synthetic speech detection via multi-formant analysis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4409–4417

    2024

  32. [40]

    Clad: Robust audio deepfake detection against manipulation attacks with contrastive learning,

    H. Wu, J. Chen, R. Du, C. Wu, K. He, X. Shang, H. Ren, and G. Xu, “Clad: Robust audio deepfake detection against manipulation attacks with contrastive learning,” arXiv preprint arXiv:2404.15854 , 2024

    2024 arXiv

  33. [41]

    Towards attention-based contrastive learning for audio spoof detection,

    C. Goel, S. Koppisetti, B. Colman, A. Shahriyari, and G. Bharaj, “Towards attention-based contrastive learning for audio spoof detection,” in INTERSPEECH 2023, 2023, pp. 2758–2762

    2023

  34. [42]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  35. [43]

    How does batch normalization help optimization?

    S. Santurkar, D. Tsipras, A. Ilyas, and A. Madry, “How does batch normalization help optimization?” Advances in neural information pro- cessing systems, vol. 31, 2018

    2018

  36. [44]

    Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization,

    Z. Cai, K. Stefanov, A. Dhall, and M. Hayat, “Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization,” in 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Nov. ...

    2022

  37. [45]

    Focal loss for dense object detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2980–2988

    2017

  38. [46]

    Transferring Audio Deepfake Detection Capability across Languages,

    Z. Ba, Q. Wen, P. Cheng, Y . Wang, F. Lin, L. Lu, and Z. Liu, “Transferring Audio Deepfake Detection Capability across Languages,” in Proceedings of the ACM Web Conference 2023, Apr. 2023, pp. 2033– 2044

    2023

  39. [47]

    Parallel Wavegan: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram,

    R. Yamamoto, E. Song, and J.-M. Kim, “Parallel Wavegan: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 20...

    2020

  40. [48]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” Advances in Neural Information Processing Systems , vol. 33, pp. 17 022–17 033, 2020

    2020

  41. [49]

    The lj speech dataset,

    K. Ito and L. Johnson, “The lj speech dataset,” https://keithito.com/ LJ-Speech-Dataset/, 2017

    2017

  42. [50]

    JSUT corpus: free large- scale japanese speech corpus for end-to-end speech synthesis,

    R. Sonobe, S. Takamichi, and H. Saruwatari, “JSUT corpus: free large- scale japanese speech corpus for end-to-end speech synthesis,” CoRR, vol. abs/1711.00354, 2017

    2017 arXiv

  43. [51]

    Conformer: Convolution- augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented Transformer for Speech Recognition,” May 2020

    2020

  44. [52]

    Libritts: A corpus derived from librispeech for text-to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” Interspeech 2019, 2019

    2019

  45. [53]

    End-to-end spectro-temporal graph attention networks for speaker ver- ification anti-spoofing and speech deepfake detection,

    H. Tak, J. weon Jung, J. Patino, M. Kamble, M. Todisco, and N. Evans, “End-to-end spectro-temporal graph attention networks for speaker ver- ification anti-spoofing and speech deepfake detection,” in Proc. 2021 Edition of the Automatic Speaker Verification and Spoofing Counter...

    2021

  46. [54]

    wav2vec 2.0: A frame- work for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A frame- work for self-supervised learning of speech representations,” Advances in Neural Information Processing Systems , vol. 33, pp. 12 449–12 460, 2020

    2020

  47. [55]

    WavLM: Large-Scale Self-Supervised Pre- Training for Full Stack Speech Processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “WavLM: Large-Scale Self-Supervised Pre- Training for Full Stack Speech Processing,” IEEE Journal of Select...

    2022

  48. [56]

    Wav2CLIP: Learning Robust Audio Representations from Clip,

    H.-H. Wu, P. Seetharaman, K. Kumar, and J. P. Bello, “Wav2CLIP: Learning Robust Audio Representations from Clip,” in ICASSP 2022 - 15 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 4563–4567

    2022

  49. [57]

    Audioclip: Extending Clip to Image, Text and Audio,

    A. Guzhov, F. Raue, J. Hees, and A. Dengel, “Audioclip: Extending Clip to Image, Text and Audio,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 976–980

    2022

  50. [58]

    ESResNe(X)t-fbsp: Learning Robust Time-Frequency Transformation of Audio,

    A. Guzhov, F. Raue, J. Hees, and A. Dengel, “ESResNe(X)t-fbsp: Learning Robust Time-Frequency Transformation of Audio,” in 2021 International Joint Conference on Neural Networks (IJCNN) , Jul. 2021, pp. 1–8

    2021

  51. [59]

    Gradient centralization: A new optimization technique for deep neural networks,

    H. Yong, J. Huang, X. Hua, and L. Zhang, “Gradient centralization: A new optimization technique for deep neural networks,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16 , 2020, pp. 635–652

    2020

  52. [60]

    Visualizing data using t-SNE

    L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE.” Journal of machine learning research , vol. 9, no. 11, 2008

    2008

  53. [61]

    Grad-cam: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 618–626

    2017

  54. [2020]

    Available: https://www.vice.com/en/article/pkyqvb/ deepfake-audio-impersonating-ceo-fraud-attempt

    [Online]. Available: https://www.vice.com/en/article/pkyqvb/ deepfake-audio-impersonating-ceo-fraud-attempt

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.