Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

ShiftySpeech: A Large-Scale Synthetic Speech Dataset with Distribution Shifts

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 3,000-hour benchmark shows every tested distribution shift degrades state-of-the-art synthetic speech detectors.

desk verdict A genuinely useful large synthetic-speech benchmark, but the paper's blanket claim that all tested shifts degrade detection is contradicted by its own Table 8, and the 'controlled' shift labels are muddier than advertised. read the letter →

arxiv 2502.05674 v4 pith:IH5FNOSU submitted 2025-02-08 eess.AS cs.SD

Ashi Garg , Zexin Cai , Lin Zhang , Henry Li Xinyuan , Leibny Paola García-Perera , Kevin Duh , Sanjeev Khudanpur , Matthew Wiesner
show 1 more author
Nicholas Andrews
This is my paper · ORCID
classification eess.AScs.SD
keywords syntheticspeechdetectiondistributionshiftdeepfakeaudiobenchmarkdatasettext-to-speechvocoderself-supervisedlearninggeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ShiftySpeech is a large benchmark of over 3,000 hours of synthetic speech built from seven source domains, six text-to-speech systems, twelve vocoders, and three languages. The paper uses it to ask whether state-of-the-art synthetic speech detectors trained on standard data keep working when test conditions drift: new speakers, new emotion, new language, background noise, or a new synthesis system. Across all tested shifts, detection performance degrades substantially, with the noisiest YouTube-style domain being the hardest and newer, higher-quality vocoders harder to detect than older ones. The authors' claim is that current self-supervised-feature detectors are not robust to realistic test-time shifts, and that a controlled benchmark is the right tool for measuring and eventually fixing this.

What carries the argument

The load-bearing object is the ShiftySpeech benchmark itself: parallel real and synthetic samples generated by re-vocoding and text-to-speech synthesis from seven carefully chosen source corpora. Its mechanism is controlled distribution shift; each source corpus is selected to isolate a different real-world factor, such as JSUT for language, VoxCeleb for speaking-style variation, LibriSpeech for reading style, MSP-Podcast for emotion, GigaSpeech-YouTube for background noise and spontaneity, and CommonVoice for age, accent, and gender, while the same set of twelve test vocoders and six TTS systems is applied across all of them. Detection is scored by the equal error rate of the SSL-AASIST countermeasure, which runs raw audio through a wav2vec2.0 XLSR-53 self-supervised front-end and a spectro-temporal graph attention network backend.

What would settle it

Take one TTS system and one vocoder, re-synthesize a single set of transcripts recorded under controlled studio conditions, and then add only one factor at a time, such as the same speakers re-recording with emotion or the same spontaneous content recorded clean versus with added noise. If EER gaps between ShiftySpeech domains largely disappear under this matched-content comparison, the benchmark's attribution of degradation to specific distribution shifts would be called into question; if they persist, the controlled-shift design is supported.

Watch

Extended reading notes

Core claim

The paper's central discovery is empirical: for an SSL-AASIST detector using wav2vec2.0 XLSR-53 features, every distribution shift it was able to isolate—language, celebrity and spontaneous speaking style, emotion, background noise, near and far-field condition, age, accent, gender, and synthesis system—raises the equal error rate relative to in-domain performance. Training alone on a single vocoder is not enough: with WaveFake-style vocoded LJSpeech training, average EER over six test domains ranges from 9.14% for the best training vocoder (HiFiGAN) to 32.37% for the worst (WaveGlow), and even leave-one-out multi-vocoder training leaves an average EER around 14%. Newer vocoders such as BigVGAN are markedly harder to detect, and detectors trained on one end-to-end TTS system generalize poorly to other TTS systems. The paper presents ShiftySpeech as the enabling instrument: a single consistent pipeline that keeps TTS and vocoder systems fixed while varying only the intended source-domain factor.

Load-bearing premise

The controlled-shift analysis assumes each source corpus varies only the intended factor—for example that MSP-Podcast changes only emotion or GigaSpeech-YouTube only background noise—so that EER differences can be attributed to that single factor; in reality several domains blend multiple acoustic and content differences.

Editorial extensions

If this is right

  • Production deployment of current SSL-based synthetic speech detectors should be conditioned on the expected test distribution; a detector that looks strong on ASVspoof-style benchmarks can have EER above 30% on noisy or spontaneous speech.
  • Detectors trained on HiFiGAN-vocoded speech generalize best among single-vocoder training sets, and dropping HiFiGAN from multi-vocoder training hurts the most, so HiFiGAN-like artifacts are a disproportionate source of transferable signal.
  • A detector trained on one end-to-end TTS system can fail badly on another TTS system, with EERs of roughly 38-44% when Grad-TTS, VITS, or Glow-TTS are swapped, so synthesis-system shift is itself a major degradation.
  • Including more speakers in training helps up to about four speakers, after which gains are small, and Mandarin (AISHELL) training can generalize as well as or better than English training to unseen languages such as Japanese, Polish, and Russian.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension suggested by the UTMOS correlation results is that automatic speech-quality scores, which correlate 0.58-0.89 with EER, might serve as cheap, no-training proxies for how hard a new synthesis system will be to detect, though the paper only reports correlation, not causal use.
  • The benchmark's controlled-shift claim rests on source corpora being factor-pure; since GigaSpeech-YouTube bundles noise, spontaneity, and near and far-field conditions, the high EER on that domain cannot by itself tell which factor causes the degradation.
  • If the trend that newer-generation vocoders are harder to detect continues, any detector deployed now will age, and periodically adding newly released vocoders to training data, as the paper's incremental experiment suggests, may be a practical mitigation.
  • The language experiments suggest that training on non-English data may be a useful regularization for multilingual deployment, an implication the authors state cautiously.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces ShiftySpeech, a large-scale synthetic speech dataset built by re-vocoding and re-synthesizing speech from seven source corpora using six TTS systems and twelve vocoders across three languages. The authors train SSL-AASIST with an XLSR-53 front-end on vocoded or TTS-generated speech and evaluate equal error rates under a series of labeled distribution shifts (vocoder identity, language, speaking style, emotion, noise, age/accent/gender). They report that certain shifts, especially the GigaSpeech-YouTube domain, consistently degrade detection performance, while other shifts such as language mismatch are often handled well. The paper also includes analyses of training-vocoder choice, leave-one-out training, incremental addition of new vocoders, UTMOS-EER correlation, and the number of training speakers.

Significance. If the claims were properly qualified, ShiftySpeech would be a valuable community resource: it is large (>3,000 hours), spans current TTS and vocoder models, includes parallel real-synthetic pairs, and ships with code and dataset links that support reproducibility. The central finding that vocoder and acoustic shifts (notably YouTube-style noise and spontaneity) degrade an SSL-based detector is supported by the aggregate tables and is practically important. The speaker-count saturation analysis and the UTMOS-EER correlation are useful auxiliary contributions. However, the paper's headline conclusion that 'all tested distribution shifts significantly degrade detection performance' is directly contradicted by the paper's own Table 8, and the 'controlled' single-factor interpretation of the domain shifts is not supported by the source-corpus design. The significance of the work is therefore contingent on a substantial revision of the empirical claims.

major comments (3)
  1. [Abstract and §4.3/Table 8] The abstract states that 'All tested distribution shifts significantly degrade detection performance of state-of-the-art detection approaches based on self-supervised features,' but Table 8 reports 0.00% EER for the AISHELL-trained XTTS model on AISHELL and JSUT and for the VCTK-trained model on AISHELL, and the text on page 9 explicitly notes 'perfect detection performance' for the AISHELL-trained model on JSUT. The blanket claim is false unless restricted to vocoder/acoustic shifts (Tables 5-7) and to the single detector actually used. This is a load-bearing conclusion and must be revised with explicit caveats.
  2. [§3.2 and Table 3] The 'controlled distribution shifts' claim is not supported by the source-corpus design. Each row of Table 3 identifies a shift with an entire corpus that varies along multiple axes simultaneously: GigaSpeech-YouTube combines background noise, spontaneity, and near/far-field conditions; VoxCeleb adds celebrity speaking style on top of YouTube-derived recording conditions; MSP-Podcast couples emotive speech with spontaneity; and CommonVoice bundles age, accent, and gender. Since re-vocoding leaves content, speaker set, language, and recording environment unchanged from the source corpus, the EER differences in Tables 5-7 cannot be uniquely attributed to the named factor. The authors should either reframe these as multi-factor 'domain shifts,' provide matched-content or crossed-speaker controls to isolate single factors, or explicitly discuss the confounding.
  3. [§4.1 and Limitations (page 9)] The abstract and introduction refer to 'detection approaches' and 'SOTA synthetic speech detectors' in the plural, but every experiment trains the single SSL-AASIST architecture with the XLSR-53 front-end. The Limitations paragraph concedes this ('we choose to focus on a small number of state-of-the-art detector architectures... specifically SSL-AASIST'), which shows an awareness of the gap, but the mismatch between the plural framing and the single-architecture evidence remains in the abstract, Section 1, and Section 5. The claims should be narrowed to SSL-AASIST, or at least one additional countermeasure should be run to substantiate the plural wording.
minor comments (5)
  1. [§4.3] The text contains 'ShitySpeech-TTS' where the dataset name is spelled without the letter 'f'; this typo should be fixed.
  2. [Appendix C.1] The phrase beginning 'Table 4.2 Notes for various distribution shifts, certain vocoders...' is malformed; it should reference 'Section 4.2' rather than a nonexistent 'Table 4.2' and be rewritten as a complete sentence.
  3. [Appendix B and Appendix C.1] There are several typographical errors, including 'Archictetural details' and 'inclemently trained' (presumably 'incrementally trained'); these should be corrected.
  4. [Appendix F] The heading 'Co-relation score between UTMOS and EER' uses 'Co-relation' instead of 'Correlation,' and similar misspellings appear elsewhere in the appendix.
  5. [Table 33 and surrounding text] The UTMOS table is referenced as 'Table 33' in two places with different content (once in Section 3.3 text referring to Table 3, and once in Appendix F); the numbering and cross-references should be checked and harmonized.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark is external to the detectors, and the evaluation protocol does not fit or relabel its own outputs.

full rationale

ShiftySpeech is an empirical benchmark paper rather than a derivation chain. The EER numbers in Tables 5–8 are produced by training SSL-AASIST on WaveFake or LibriTTS-derived re-vocoded speech and evaluating on held-out ShiftySpeech samples; the distribution-shift labels are defined by source corpora, not by detector failures, so there is no fitted parameter that is later renamed as a prediction. The paper's self-citations are not load-bearing: [64] is a future-direction pointer, and the author-affiliated GigaSpeech and LibriSpeech corpora are used as data sources, not as evidence for the conclusions. No uniqueness theorem, ansatz, or known-result renaming is invoked. Two non-circular correctness risks should be noted: the abstract's blanket claim that all tested shifts degrade detection is contradicted by the paper's own Table 8 (e.g., 0.00% EER for AISHELL-trained XTTS on JSUT), and the 'controlled' shifts in Table 3 conflate multiple factors (e.g., GigaSpeech-YouTube is labeled 'Background noise; Spontaneous; Near and far field'), so EER differences cannot be attributed to a single factor. Additionally, the Limitations section states that only SSL-AASIST was used, so the plural 'approaches' is unsupported. These are validity concerns, not circularity, because the measurements are not constructed to equal their inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no free parameters and no new physical or theoretical entities. Its central claims rest on domain assumptions about the representativeness of source corpora, the representativeness of a single detector architecture, and the content-preservation of re-vocoding. The dataset itself is the contribution, not a fitted model.

assumptions (4)
  • domain assumption Source corpora are representative of their labeled distribution shifts (e.g., AISHELL for Mandarin, MSP-Podcast for emotion, GigaSpeech-YouTube for noise/spontaneity).
    Used in Section 3.2 to define the shift axes; the paper does not validate acoustic or linguistic separation between corpora.
  • domain assumption SSL-AASIST is representative of the class of state-of-the-art self-supervised speech detectors.
    Only this architecture is evaluated; Section 4.1 states the choice and the limitations section acknowledges it.
  • domain assumption Re-vocoding preserves utterance content so that real/fake pairs differ only in synthesis artifacts.
    Assumed in Section 3.3 for the vocoder subset; no perceptual or alignment check is reported.
  • domain assumption Training on WaveFake is a reasonable proxy for a deployed detector's training distribution.
    Used in Section 4.2 for all main vocoder experiments; WaveFake is clean English LJSpeech re-vocoded by older GAN vocoders.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ShiftySpeech: A Large-Scale Synthetic Speech Dataset with Distribution Shifts." pith.science (2026). https://pith.science/paper/IH5FNOSU

@misc{pith2026250205674,
  author       = {Pith},
  title        = {Pith review of: ShiftySpeech: A Large-Scale Synthetic Speech Dataset with Distribution Shifts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IH5FNOSU}},
  note         = {Machine review of arXiv:2502.05674}
}
read the original abstract

The problem of synthetic speech detection has enjoyed considerable attention, with recent methods achieving low error rates across several established benchmarks. However, to what extent can low error rates on academic benchmarks translate to more realistic conditions? In practice, while the training set is fixed at one point in time, test-time conditions may exhibit distribution shifts relative to the training conditions, such as changes in speaker characteristics, emotional expressiveness, language and acoustic conditions, and the emergence of novel synthesis methods. Although some existing datasets target subsets of these distribution shifts, systematic analysis remains difficult due to inconsistencies between source data and synthesis systems across datasets. This difficulty is further exacerbated by the rapid development of new text-to-speech (TTS) and vocoder systems, which continually expand the diversity of synthetic speech. To enable systematic benchmarking of model performance under distribution shifts, we introduce ShiftySpeech, a large-scale benchmark comprising over 3,000 hours of synthetic speech across 7 source domains, 6 TTS systems, 12 vocoders, and 3 languages. ShiftySpeech is specifically designed to evaluate model generalization under controlled distribution shifts while ensuring broad coverage of modern synthetic speech generation techniques. It fills a key gap in current benchmarks by supporting fine-grained, controlled analysis of generalization robustness. All tested distribution shifts significantly degrade detection performance of state-of-the-art detection approaches based on self-supervised features. Overall, our findings suggest that reliance on synthetic speech detection methods in production environments should be carefully evaluated based on anticipated distribution shifts.

Figures

Figures reproduced from arXiv: 2502.05674 by the authors.

Figure 2
Figure 2. ShiftySpeech data generation pipeline illustrating three synthetic speech generation pro [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Average EERs reported with models trained on increasing number of speakers with HFG [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗
Figure 4
Figure 4. Average EERs reported with models trained on increasing number of speakers with HFG [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating AI Models' Capability to Automate Voice Phishing Attacks

    cs.CR 2026-07 conditional novelty 7.0 of 10

    AI voice models already drive self-reported vishing compliance up to 36% and make automated attacks economically viable at U.S. scale while human operators are not.

  2. Rapidly Adapting to New Voice Spoofing: Few-Shot Detection of Synthesized Speech Under Distribution Shifts

    eess.AS 2025-08 unverdicted novelty 5.0 of 10

    Few-shot adaptation with a self-attentive prototypical network reduces spoofed-speech detection errors by up to 32% relative under distribution shifts.

Reference graph

Works this paper leans on

84 extracted references · 75 canonical work pages · cited by 2 Pith papers

  1. [1]

    Does Audio Deepfake Detection Generalize? In Interspeech, pages 2783–2787, 2022

    Nicolas Müller, Pavel Czempin, Franziska Diekmann, Adam Froghyar, and Konstantin Böttinger. Does Audio Deepfake Detection Generalize? In Interspeech, pages 2783–2787, 2022

    2022

  2. [2]

    Generalization of Audio Deepfake Detection

    Tianxiang Chen, Avrosh Kumar, Parav Nagarsheth, Ganesh Sivaraman, and Elie Khoury. Generalization of Audio Deepfake Detection. In The Speaker and Language Recognition Workshop (Odyssey 2020), pages 132–137, 2020

    2020

  3. [3]

    Breaking Security-Critical V oice Authentication

    Andre Kassis and Urs Hengartner. Breaking Security-Critical V oice Authentication. InIEEE Symposium on Security and Privacy, pages 951–968, 2023

    2023

  4. [4]

    StreamVC: Real-Time Low-Latency V oice Conversion

    Yang Yang, Yury Kartynnik, Yunpeng Li, Jiuqiang Tang, Xing Li, George Sung, and Matthias Grundmann. StreamVC: Real-Time Low-Latency V oice Conversion. InIEEE International Conference on Acoustics, Speech and Signal Processing, pages 11016–11020, 2024

    2024

  5. [5]

    Kameoka, T

    H. Kameoka, T. Kaneko, K. Tanaka, and N. Hojo. StarGAN-VC: Non-Parallel Many-to-Many V oice Conversion Using Star Generative Adversarial Networks. InIEEE Spoken Language Technology Workshop, pages 266–273, 2018

    work page 2018

  6. [6]

    DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust V oice Conversion

    Ha-Yeong Choi, Sang-Hoon Lee, and Seong-Whan Lee. DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust V oice Conversion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17862–17870, 2024

    work page 2024

  7. [7]

    YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot V oice Conversion for Everyone

    Edresson Casanova, Julian Weber, Christopher D Shulby, Arnaldo Candido Junior, Eren Gölge, and Moacir A Ponti. YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot V oice Conversion for Everyone. In International Conference on Machine Learning, pages 2709–2720, 2022

    work page 2022

  8. [8]

    AutoVC: Zero-Shot V oice Style Transfer with Only Autoencoder Loss

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. AutoVC: Zero-Shot V oice Style Transfer with Only Autoencoder Loss. In International Conference on Machine Learning, volume 97, pages 5210–5219, 2019

    work page 2019

Show all 84 references
  1. [9]

    CycleGAN-VC2: Improved CycleGAN-Based Non-Parallel V oice Conversion

    Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, and Nobukatsu Hojo. CycleGAN-VC2: Improved CycleGAN-Based Non-Parallel V oice Conversion. InIEEE International Conference on Acoustics, Speech and Signal Processing, pages 6820–6824, 2019

    2019

  2. [10]

    Reimagining Speech: A Scoping Review of Deep Learning-Powered V oice Conversion.arXiv preprint arXiv:2311.08104, 2023

    Anders R Bargum, Stefania Serafin, and Cumhur Erkut. Reimagining Speech: A Scoping Review of Deep Learning-Powered V oice Conversion.arXiv preprint arXiv:2311.08104, 2023

    2023 arXiv

  3. [11]

    Sahidullah, Héctor Delgado, Andreas Nautsch, Junichi Yamagishi, Nicholas Evans, Tomi H

    Massimiliano Todisco, Xin Wang, Ville Vestman, Md. Sahidullah, Héctor Delgado, Andreas Nautsch, Junichi Yamagishi, Nicholas Evans, Tomi H. Kinnunen, and Kong Aik Lee. ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection. In Interspeech, pages 1008–1012, 2019

    2019

  4. [12]

    ASVspoof 2021: Accelerating Progress in Spoofed and Deepfake Speech Detection

    Junichi Yamagishi, Xin Wang, Massimiliano Todisco, Md Sahidullah, Jose Patino, Andreas Nautsch, Xuechen Liu, Kong Aik Lee, Tomi Kinnunen, Nicholas Evans, et al. ASVspoof 2021: Accelerating Progress in Spoofed and Deepfake Speech Detection. In ASVspoof 2021 Workshop-Automatic S...

    2021

  5. [13]

    WaveFake: A Data Set to Facilitate Audio Deepfake Detection

    Joel Frank and Lea Schönherr. WaveFake: A Data Set to Facilitate Audio Deepfake Detection. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021

    2021

  6. [14]

    Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing, 2023

    Hye jin Shim, Jee weon Jung, and Tomi Kinnunen. Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing, 2023

    2023

  7. [15]

    Deep Feature Engineering for Noise Robust Spoofing Detection

    Yanmin Qian, Nanxin Chen, Heinrich Dinkel, and Zhizheng Wu. Deep Feature Engineering for Noise Robust Spoofing Detection. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(10): 1942–1955, 2017

    1942

  8. [16]

    Investigating Raw Wave Deep Neural Networks for End-to-End Speaker Spoofing Detection

    Heinrich Dinkel, Yanmin Qian, and Kai Yu. Investigating Raw Wave Deep Neural Networks for End-to-End Speaker Spoofing Detection. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26 (11):2002–2014, 2018

    2002

  9. [17]

    Peinado, Jose A

    Alejandro Gomez-Alanis, Antonio M. Peinado, Jose A. Gonzalez, and Angel M. Gomez. A Gated Recurrent Convolutional Neural Network for Robust Spoofing Detection. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(12):1985–1999, 2019

    1985

  10. [18]

    ResNet and Model Fusion for Automatic Spoofing Detection

    Zhuxin Chen, Zhifeng Xie, Weibin Zhang, and Xiangmin Xu. ResNet and Model Fusion for Automatic Spoofing Detection. In Interspeech, pages 102–106, 2017. 10

    2017

  11. [19]

    Investigating Self-Supervised Front Ends for Speech Spoofing Coun- termeasures

    Xin Wang and Junichi Yamagishi. Investigating Self-Supervised Front Ends for Speech Spoofing Coun- termeasures. In The Speaker and Language Recognition Workshop (Odyssey 2022) , pages 100–106, 2022

    2022

  12. [20]

    Exploring generalization to unseen audio data for spoofing: insights from SSL models

    Atharva Kulkarni, Hoan My Tran, Ajinkya Kulkarni, Sandipana Dowerah, Damien Lolive, and Mathew Maginai Doss. Exploring generalization to unseen audio data for spoofing: insights from SSL models. In The Automatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof ...

    2024

  13. [21]

    WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):...

    2022

  14. [22]

    HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:...

    2021

  15. [23]

    Wav2vec 2.0: A Framework for Self-supervised Learning of Speech Representations

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. Wav2vec 2.0: A Framework for Self-supervised Learning of Speech Representations. Advances in Neural Information Processing Systems, 33:12449–12460, 2020

    2020

  16. [24]

    Müller, Nicholas Evans, Hemlata Tak, Philip Sperl, and Konstantin Böttinger

    Nicolas M. Müller, Nicholas Evans, Hemlata Tak, Philip Sperl, and Konstantin Böttinger. Harder or Different? Understanding Generalization of Audio Deepfake Detection. In Interspeech, pages 2705–2709, 2024

    2024

  17. [25]

    Audio Deepfake Detection: A Survey, 2023

    Jiangyan Yi, Chenglong Wang, Jianhua Tao, Xiaohui Zhang, Chu Yuan Zhang, and Yan Zhao. Audio Deepfake Detection: A Survey, 2023. URL https://arxiv.org/abs/2308.14970

    2023 arXiv

  18. [26]

    Open Challenges in Synthetic Speech Detec- tion

    Luca Cuccovillo, Christoforos Papastergiopoulos, Anastasios Vafeiadis, Artem Yaroshchuk, Patrick Aichroth, Konstantinos V otis, and Dimitrios Tzovaras. Open Challenges in Synthetic Speech Detec- tion. In IEEE International Workshop on Information Forensics and Security, page 1–6, 2022

    2022

  19. [27]

    SpoofCeleb: Speech Deepfake Detection and SASV In The Wild

    Jee weon Jung, Yihan Wu, Xin Wang, Ji-Hoon Kim, Soumi Maiti, Yuta Matsunaga, Hye jin Shim, Jinchuan Tian, Nicholas Evans, Joon Son Chung, Wangyou Zhang, Seyun Um, Shinnosuke Takamichi, and Shinji Watanabe. SpoofCeleb: Speech Deepfake Detection and SASV In The Wild. IEEE Open J...

    2025

  20. [28]

    Generalisation in Humans and Deep Neural Networks, 2018

    Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in Humans and Deep Neural Networks, 2018

    2018

  21. [29]

    Covariate Shift Adaptation by Impor- tance Weighted Cross Validation

    Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate Shift Adaptation by Impor- tance Weighted Cross Validation. Journal of Machine Learning Research, 8(35):985–1005, 2007

    2007

  22. [30]

    Analysis of Representations for Domain Adaptation

    Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of Representations for Domain Adaptation. volume 19, 2006

    2006

  23. [31]

    A Survey on Concept Drift Adaptation

    João Gama, Indr˙e Žliobait˙e, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A Survey on Concept Drift Adaptation. ACM computing surveys (CSUR), 46(4):1–37, 2014

    2014

  24. [32]

    Kinnunen, Nicholas Evans, Kong Aik Lee, and Junichi Yamagishi

    Xin Wang, Héctor Delgado, Hemlata Tak, Jee weon Jung, Hye jin Shim, Massimiliano Todisco, Ivan Kukanov, Xuechen Liu, Md Sahidullah, Tomi H. Kinnunen, Nicholas Evans, Kong Aik Lee, and Junichi Yamagishi. Asvspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks a...

    2024

  25. [33]

    MLS: A Large- Scale Multilingual Dataset for Speech Research

    Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. MLS: A Large- Scale Multilingual Dataset for Speech Research. In Interspeech 2020, 2020

    2020

  26. [34]

    Xin Wang and Junichi Yamagishi. Can Large-Scale V ocoded Spoofed Data Improve Speech Spoofing Countermeasure with a Self-Supervised Front End? In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 10311–10315, 2024

    2024

  27. [35]

    Spoofed Training Data for Speech Spoofing Countermeasure Can Be Efficiently Created Using Neural V ocoders

    Xin Wang and Junichi Yamagishi. Spoofed Training Data for Speech Spoofing Countermeasure Can Be Efficiently Created Using Neural V ocoders. InIEEE International Conference on Acoustics, Speech and Signal Processing, 2023

    2023

  28. [36]

    AI-Synthesized V oice Detection Using Neural V ocoder Artifacts

    Chengzhe Sun, Shan Jia, Shuwei Hou, and Siwei Lyu. AI-Synthesized V oice Detection Using Neural V ocoder Artifacts. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 904–912, 2023. 11

    2023

  29. [37]

    A Cross-V ocoder Study of Speaker Independent Synthetic Speech Detection using Phase Information

    Jon Sanchez, Ibon Saratxaga, Inma Hernaez, Eva Navas, and Daniel Erro. A Cross-V ocoder Study of Speaker Independent Synthetic Speech Detection using Phase Information. In Interspeech, pages 1663–1667, 2014

    2014

  30. [38]

    Robust AI-Synthesized Speech Detection Using Feature Decomposition Learning and Synthesizer Feature Augmentation

    Kuiyuan Zhang, Zhongyun Hua, Yushu Zhang, Yifang Guo, and Tao Xiang. Robust AI-Synthesized Speech Detection Using Feature Decomposition Learning and Synthesizer Feature Augmentation. IEEE Transactions on Information Forensics and Security, 20:871–885, 2025

    2025

  31. [39]

    Anomaly Detection of Deepfake Audio Based on Real Audio Using Generative Adversarial Network Model

    Daeun Song, Nayoung Lee, Jiwon Kim, and Eunjung Choi. Anomaly Detection of Deepfake Audio Based on Real Audio Using Generative Adversarial Network Model. IEEE Access, 12:184311–184326, 2024

    2024

  32. [40]

    The LJ Speech Dataset

    Keith Ito. The LJ Speech Dataset. https://keithito.com/LJ-Speech-Dataset/ , 2017

    2017

  33. [41]

    Jsut corpus: free large-scale japanese speech corpus for end-to-end speech synthesis

    Ryosuke Sonobe, Shinnosuke Takamichi, and Hiroshi Saruwatari. Jsut corpus: free large-scale japanese speech corpus for end-to-end speech synthesis. arXiv preprint arXiv:1711.00354, 2017

    2017 arXiv

  34. [42]

    Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu

    Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech. In Interspeech, pages 1526–1530, 2019

    2019

  35. [43]

    Wanying Ge, Xin Wang, Junichi Yamagishi, Massimiliano Todisco, and Nicholas Evans. Spoofing Attack Augmentation: Can Differently-Trained Attack Models Improve Generalisation? In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 12531–12535, 2024

    2024

  36. [44]

    FoR: A Dataset for Synthetic Speech Detection

    Ricardo Reimao and Vassilios Tzerpos. FoR: A Dataset for Synthetic Speech Detection. In International Conference on Speech Technology and Human-Computer Dialogue, pages 1–10, 2019

    2019

  37. [45]

    MLAAD: The Multi-Language Audio Anti-Spoofing Dataset

    Nicolas M Müller, Piotr Kawa, Wei Herng Choong, Edresson Casanova, Eren Gölge, Thorsten Müller, Piotr Syga, Philip Sperl, and Konstantin Böttinger. MLAAD: The Multi-Language Audio Anti-Spoofing Dataset. 2024

    2024

  38. [46]

    V oxceleb: A large-scale speaker identification dataset

    Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. V oxceleb: A large-scale speaker identification dataset. In Interspeech, pages 2616–2620, 2017

    2017

  39. [47]

    V oxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation

    Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. V oxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation. In Proceedings o...

    2021

  40. [48]

    GigaSpeech: An Evolving, Multi-Domain ASR Corpus with 10,000 Hours of Transcribed Audio

    Guoguo Chen, Shuzhou Chai, Guan-Bo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Zhao You, and Zhiyong Yan. GigaSpeech...

    2021

  41. [49]

    AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline

    Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, and Hao Zheng. AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline. In 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Asses...

    2017

  42. [50]

    Common V oice: A Massively-Multilingual Speech Corpus

    Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. Common V oice: A Massively-Multilingual Speech Corpus. In Language Resources and Evaluation Conference, pages 4218–4222, 2020

    2020

  43. [51]

    Lotfian and C

    R. Lotfian and C. Busso. Building Naturalistic Emotionally Balanced Speech Corpus by Retrieving Emotional Speech From Existing Podcast Recordings. IEEE Transactions on Affective Computing, 10(4): 471–483, 2019

    2019

  44. [52]

    V oxCeleb2: Deep Speaker Recognition

    Joon Son Chung, Arsha Nagrani, and Andrew Zisserman. V oxCeleb2: Deep Speaker Recognition. In Interspeech 2018, pages 1086–1090, 2018

    2018

  45. [53]

    Librispeech: An ASR Corpus Based on Public Domain Audio Books

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: An ASR Corpus Based on Public Domain Audio Books. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 5206–5210, 2015

    2015

  46. [54]

    Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

    Jaehyeon Kim, Jungil Kong, and Juhee Son. Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech. In International Conference on Machine Learning , pages 5530–5540, 2021. 12

    2021

  47. [55]

    Fastpitch: Parallel Text-to-Speech with Pitch Prediction

    Adrian Ła ´ncucki. Fastpitch: Parallel Text-to-Speech with Pitch Prediction. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6588–6592, 2021

    2021

  48. [56]

    Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search

    Jaehyeon Kim, Sungwon Kim, Jungil Kong, and Sungroh Yoon. Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search. volume 33, pages 8067–8077, 2020

    2020

  49. [57]

    Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech

    Vadim Popov, Ivan V ovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech. In International Conference on Machine Learning , pages 8599–8608, 2021

    2021

  50. [58]

    XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model

    Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, et al. XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model. arXiv preprint arXiv:2406.04904, 2024

    2024 arXiv

  51. [59]

    Superseded-CSTR VCTK Corpus: English Multi-Speaker Corpus for CSTR V oice Cloning Toolkit.University of Edinburgh, The Centre for Speech Technology Research, 2016

    Christophe Veaux, Junichi Yamagishi, Kirsten MacDonald, et al. Superseded-CSTR VCTK Corpus: English Multi-Speaker Corpus for CSTR V oice Cloning Toolkit.University of Edinburgh, The Centre for Speech Technology Research, 2016

    2016

  52. [60]

    Automatic Speaker Verification Spoofing and Deepfake Detection Using Wav2vec 2.0 and Data Augmenta- tion

    Hemlata Tak, Massimiliano Todisco, Xin Wang, Jee weon Jung, Junichi Yamagishi, and Nicholas Evans. Automatic Speaker Verification Spoofing and Deepfake Detection Using Wav2vec 2.0 and Data Augmenta- tion. In Speaker and Language Recognition Workshop, pages 112–119, 2022

    2022

  53. [61]

    XLS-R: Self-Supervised Cross-Lingual Speech Representation Learning at Scale

    Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, Alexei Baevski, Alexis Conneau, and Michael Auli. XLS-R: Self-Supervised Cross-Lingual Speech Representation Learning at Scale. In...

    2022

  54. [62]

    AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks

    Jee-weon Jung, Hee-Soo Heo, Hemlata Tak, Hye-jin Shim, Joon Son Chung, Bong-Jin Lee, Ha-Jin Yu, and Nicholas Evans. AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks. In IEEE International Conference on Acoustics, Speech and Signal Processi...

    2022

  55. [63]

    The M-AILABS Speech Dataset

    Imdat Solak. The M-AILABS Speech Dataset. https://www.caito.de/2019/01/03/ the-m-ailabs-speech-dataset/ , 2023

    2019

  56. [64]

    Chen, Marcus Bishop, and Nicholas Andrews

    Rafael Alberto Rivera Soto, Kailin Koch, Aleem Khan, Barry Y . Chen, Marcus Bishop, and Nicholas Andrews. Few-Shot Detection of Machine-Generated Text using Style Representations. In International Conference on Learning Representations, 2024

    2024

  57. [65]

    A simple fix to mahalanobis distance for improving near-ood detection

    Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint arXiv:2106.09022, 2021

    2021 arXiv

  58. [66]

    WaveNet: A Generative Model for Raw Audio

    Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. WaveNet: A Generative Model for Raw Audio. In ISCA Speech Synthesis Workshop, pages 125–125, 2016

    2016

  59. [67]

    SampleRNN: An Unconditional End-to-End Neural Audio Generation Model

    Soroush Mehri, Kundan Kumar, Ishaan Gulrajani, Rithesh Kumar, Shubham Jain, Jose Sotelo, Aaron Courville, and Yoshua Bengio. SampleRNN: An Unconditional End-to-End Neural Audio Generation Model. In International Conference on Learning Representations, 2017

    2017

  60. [68]

    Efficient Neural Audio Synthesis

    Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aaron Oord, Sander Dieleman, and Koray Kavukcuoglu. Efficient Neural Audio Synthesis. In International Conference on Machine Learning, pages 2410–2419, 2018

    2018

  61. [69]

    Generative Adversarial Networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Networks. Communications of the ACM, 63(11): 139–144, 2020

    2020

  62. [71]

    Parallel Wavegan: A Fast Waveform Genera- tion Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram

    Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim. Parallel Wavegan: A Fast Waveform Genera- tion Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6199–6203, 2020. 13

    2020

  63. [72]

    HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

    Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. In Advances in Neural Information Processing Systems, volume 33, pages 17022–17033, 2020

    2020

  64. [73]

    Multi-Band Melgan: Faster Waveform Generation For High-Quality Text-To-Speech

    Geng Yang, Shan Yang, Kai Liu, Peng Fang, Wei Chen, and Lei Xie. Multi-Band Melgan: Faster Waveform Generation For High-Quality Text-To-Speech. In IEEE Spoken Language Technology Workshop, pages 492–498, 2021

    2021

  65. [74]

    StyleMelGAN: An Efficient High-Fidelity Adversarial V ocoder with Temporal Adaptive Normalization

    Ahmed Mustafa, Nicola Pia, and Guillaume Fuchs. StyleMelGAN: An Efficient High-Fidelity Adversarial V ocoder with Temporal Adaptive Normalization. InIEEE International Conference on Acoustics, Speech and Signal Processing, pages 6034–6038, 2021

    2021

  66. [75]

    UnivNet: A Neural V ocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation

    Won Jang, Dan Lim, Jaesam Yoon, Bongwan Kim, and Juntae Kim. UnivNet: A Neural V ocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation. In Interspeech, pages 2207–2211, 2021

    2021

  67. [76]

    BigVGAN: A Universal Neural V ocoder with Large-Scale Training

    Sang gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, and Sungroh Yoon. BigVGAN: A Universal Neural V ocoder with Large-Scale Training. In The Eleventh International Conference on Learning Representations, 2023

    2023

  68. [77]

    BigVSAN: Enhancing GAN-based Neural V ocoders with Slicing Adversarial Network

    Takashi Shibuya, Yuhta Takida, and Yuki Mitsufuji. BigVSAN: Enhancing GAN-based Neural V ocoders with Slicing Adversarial Network. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 10121–10125, 2024

    2024

  69. [78]

    SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer

    Yuhta Takida, Masaaki Imaizumi, Takashi Shibuya, Chieh-Hsin Lai, Toshimitsu Uesaka, Naoki Murata, and Yuki Mitsufuji. SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer. In The Twelfth International Conference on Learning Representations, 2024

    2024

  70. [79]

    ISTFTNET: Fast and Lightweight Mel-Spectrogram V ocoder Incorporating Inverse Short-Time Fourier Transform

    Takuhiro Kaneko, Kou Tanaka, Hirokazu Kameoka, and Shogo Seki. ISTFTNET: Fast and Lightweight Mel-Spectrogram V ocoder Incorporating Inverse Short-Time Fourier Transform. InIEEE International Conference on Acoustics, Speech and Signal Processing, pages 6207–6211, 2022

    2022

  71. [80]

    V ocos: Closing the Gap between Time-Domain and Fourier-Based Neural V ocoders for High-Quality Audio Synthesis

    Hubert Siuzdak. V ocos: Closing the Gap between Time-Domain and Fourier-Based Neural V ocoders for High-Quality Audio Synthesis. In The Twelfth International Conference on Learning Representations, 2024

    2024

  72. [81]

    APNet: An All-Frame-Level Neural V ocoder Incorporating Direct Prediction of Amplitude and Phase Spectra

    Yang Ai and Zhen-Hua Ling. APNet: An All-Frame-Level Neural V ocoder Incorporating Direct Prediction of Amplitude and Phase Spectra. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2145–2157, 2023

    2023

  73. [82]

    APNet2: High-quality and High-efficiency Neural V ocoder with Direct Prediction of Amplitude and Phase Spectra

    Hui-Peng Du, Ye-Xin Lu, Yang Ai, and Zhen-Hua Ling. APNet2: High-quality and High-efficiency Neural V ocoder with Direct Prediction of Amplitude and Phase Spectra. InNational Conference on Man-Machine Speech Communication, pages 66–80

  74. [83]

    Waveglow: A Flow-Based Generative Network for Speech Synthesis

    Ryan Prenger, Rafael Valle, and Bryan Catanzaro. Waveglow: A Flow-Based Generative Network for Speech Synthesis. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 3617–3621, 2019

    2019

  75. [84]

    Weiss, Mohammad Norouzi, and William Chan

    Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. WaveGrad: Estimating Gradients for Waveform Generation. In International Conference on Learning Representations, 2021

    2021

  76. [85]

    UTMOS: UTokyo-SaruLab System for V oiceMOS Challenge 2022

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. UTMOS: UTokyo-SaruLab System for V oiceMOS Challenge 2022. In Interspeech 2022, pages 4521–4525, 2022. 14 A Identifying best train-time vocoders The following tables provide...

    2022

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.