Pith. sign in

REVIEW 3 major objections 6 minor 31 references

Detecting Spoof Voices in Asian Non-Native Speech: An Indonesian and Thai Case Study

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Spoofing countermeasures trained only on native English fail on Indonesian and Thai accented speech, while adding non-native training data brings the best system's EER from above 35% down to 8.56%.

desk verdict Useful new dataset and a striking in-domain result, but the accent-generalization claim is confounded by attack-generator overlap; needs a held-out-attack experiment and error bars. read the letter →

arxiv 2412.01040 v1 pith:D3Z3ZFTF submitted 2024-12-02 eess.AS

Aulia Adila , Candy Olivia Mawalim , Masashi Unoki This is my paper · ORCID
classification eess.AS
keywords spoofingcountermeasurenon-nativespeechdetectionIndonesianaccentEnglishThaivoiceconversionattacktext-to-speechcepstralfeaturesgradientboosting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a spoofing countermeasure trained only on native English can detect text-to-speech and voice-conversion attacks in English spoken with Indonesian and Thai accents. It constructs a new dataset, ENIT, with native and non-native bonafide recordings and spoof utterances generated by SpeechT5, FreeVC, and WORLD, then trains three feature-classifier combinations on either native-only or native-plus-non-native data. The answer it defends is that native-only training performs poorly on non-native speech, with minDCF above 0.79 and EER mostly above 35%, while adding non-native speech to training improves non-native detection dramatically: the best system (CQCC features with CatBoost) reaches minDCF 0.19 and EER 8.56% at a small cost to native accuracy. The paper's central claim is that accent-domain coverage in the training data, rather than a more sophisticated model, is what drives non-native spoof detection performance here.

What carries the argument

The load-bearing object is the ENIT dataset, which pairs English-native speech from ASVspoof 5 with newly recorded Indonesian and Thai accented English, and generates spoof utterances using three systems: SpeechT5 (speech synthesis and voice conversion), FreeVC (one-shot voice conversion), and WORLD (vocoder-based synthesis). The argument works by comparing two training regimes, a Native CM trained only on native speech and a Combined CM trained on native plus non-native speech, while keeping front-end features (MFCC, LFCC, CQCC) and classifiers (CatBoost, XGBoost, GMM) identical. The performance gap between the two regimes on the non-native evaluation set is the evidence for the paper's claim.

What would settle it

Run the Combined CM on non-native test spoofs produced by an attack algorithm that was absent from the non-native training set (leave-one-generator-out), using non-native speakers whose voices were never used to generate training spoofs; if minDCF and EER rise back toward Native CM levels, the paper's conclusion that adding non-native data is sufficient would be undermined.

Watch

Extended reading notes

Core claim

The central discovery is that the population of training speakers, not the choice of feature or classifier, is the dominant factor in detecting spoofed non-native speech in this setup. A countermeasure trained solely on native English degrades sharply when tested on Indonesian and Thai accented English, with error rates above 35% even for the best classifiers; the same classifiers trained on a combined native and non-native corpus detect non-native spoofs far more reliably, and the best configuration, CQCC with CatBoost, cuts the non-native EER to 8.56% while only slightly hurting native performance. The paper interprets this as demonstrating the necessity of domain-specific training data for non-native accents.

Load-bearing premise

The claim assumes that detecting spoofs generated by the same three systems (SpeechT5, FreeVC, WORLD) used to build the non-native training spoof, with largely the same non-native speaker pool feeding the SpeechT5 features, predicts how the countermeasure will perform against unseen attack algorithms and different non-native speakers.

Editorial extensions

If this is right

  • Deploying a countermeasure built only on native English speech on Indonesian or Thai accented users risks error rates above 35%; such systems should be retrained or augmented with accent-matched data.
  • Adding non-native speech to training yields large gains on non-native evaluation (average minDCF improvement around 0.41 and EER improvement around 30% relative), with only a small native-side penalty.
  • Simple hand-crafted cepstral features with gradient-boosted tree classifiers are sufficient to obtain these gains; the best configuration is CQCC with CatBoost.
  • ENIT provides a baseline dataset for future work on non-native spoofing countermeasures, including deep-learning models that the paper leaves to future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the non-native test spoofs are generated by the same three attack systems used in training, the measured advantage of Combined over Native is an in-domain comparison; a leave-one-generator-out test could show a smaller gap.
  • If the accent-coverage interpretation is right, the same recipe, adding target-accent data to training without changing the model, should transfer to other under-resourced accents such as Vietnamese or Filipino English, as long as attack diversity is kept similar.
  • The small native-side degradation suggests a data-balancing or domain-adaptation approach might recover native accuracy while keeping non-native gains, a direction the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper addresses spoofing countermeasure (CM) performance on non-native English speech from Indonesian and Thai speakers. The authors construct the ENIT dataset, containing 7,990 bonafide utterances from native English speakers and 7,990 from non-native speakers, plus spoof utterances generated from ASVspoof5 native data and, for non-native data, from SpeechT5, FreeVC, and WORLD. They train two types of CMs (Native and Combined) using MFCC/LFCC/CQCC features with CatBoost, XGBoost, and GMM classifiers. Experiment 1 shows that Native CMs perform poorly on non-native evaluation (minDCF > 0.79, EER > 35% for tree-based models). Experiment 2 shows that Combined CMs substantially improve non-native detection (best CQCC+CatBoost: minDCF 0.19, EER 8.56%) with only modest native degradation. The paper concludes that incorporating non-native speech data into training is necessary for robust spoofing detection in non-native contexts.

Significance. If the central claim is sustained, the paper would make a useful contribution: a new non-native spoofing dataset, a clear demonstration of the domain gap, and strong baselines for future work. The strengths are the explicit focus on an understudied population, the use of standard features/classifiers and challenge metrics (minDCF, EER), and the large effect sizes that are easy to reproduce. However, the current evidence does not yet separate accent-domain coverage from attack-algorithm familiarity or from increased training data quantity. The dataset contribution and baseline numbers remain valuable regardless, but the interpretation in the abstract and conclusions needs additional controlled experiments. No formal circularity is present; the concern is external validity of the measurement.

major comments (3)
  1. [III-B / V-B, Table II] The central Native-vs-Combined comparison on non-native speech is confounded by attack-algorithm overlap. The non-native test spoof is generated with the same three systems (SpeechT5, FreeVC, WORLD) used to create the non-native training spoof; for SpeechT5, roughly 2,000 training utterances use x-vector and bottleneck features extracted from the collected non-native bonafide speaker data (Section III-B). The Native CM, by contrast, was trained only on ASVspoof5 native spoof attacks (Section V-A). Therefore the non-native test attacks are unseen algorithms for Native CM but seen for Combined CM, so the improvement could reflect attack familiarity rather than accent-domain capability. To support the claim that adding non-native speech data is what drives the gain, the authors should add a held-out attack condition (e.g., generate non-native test spoof with unseen TTS/VC systems) or an attack-matched ablation (e.g., train Native CM on the same three attack algorithms using native speech only). Without such a control, the conclusion in the abstract and Section VII is an overinterpretation of the in-domain results.
  2. [V-B / V-C] The Combined CM is trained on both native and non-native training sets, roughly doubling the number of training utterances relative to Native CM. The reported improvement could therefore be an effect of training-set size rather than of the accent domain. The authors should control for data quantity, for example by subsampling an equal-size native training set for Native CM or by adding an equivalent amount of native-only data to the Native CM, to isolate the contribution of non-native data. This is a second confound for the central claim and should be addressed in the revised manuscript.
  3. [V-A, Table II] All experiments use a single random seed (42) and report point estimates without error bars, confidence intervals, or significance tests. Claims such as "significantly improved" in Section VI and the identification of a "best" system are therefore not statistically supported. At minimum, the authors should run multiple seeds for the tree-based classifiers and report mean and standard deviation, or use a paired test across seeds. This is important for between-feature and between-classifier comparisons, though the main Native-vs-Combined gap is large enough that it may survive such analysis.
minor comments (6)
  1. [III-A] The description of native bonafide sampling is ambiguous: the text first says "randomly selected 4,000 utterances" and then Table I reports 7,990 native bonafide utterances. Please clarify the sampling procedure and reconcile the numbers.
  2. [Table I] The table caption should explain the role of the separator '/' in the speaker-count columns; the text defines it only later in Section III-B.
  3. [IV-B, Table II] The GMM classifier description says separate GMMs are learned for bonafide and spoof data, but it is not stated whether the likelihood ratio is score-normalized; the 100% EER for MFCC with GMM in Table II suggests a possible implementation or thresholding issue. Please clarify or check the GMM scoring pipeline.
  4. [VI] The sentence "This behavior may be due to the increased variability introduced by the non-native speech data within the training sets" is speculative; consider removing it or supporting it with a small analysis of native performance variability.
  5. [II] Reference [5], "ASVspoof 5: Crowdsourced data, deepfakes and adversarial attacks at scale," is listed as submitted; please update to the published version if available.
  6. [III / VII] The authors should state whether the ENIT dataset and generation code will be made publicly available, as this is central to the dataset contribution and to reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the paper reports empirical measurements with a generalization confound, not a derivation that reduces to its own inputs.

full rationale

The paper makes no formal derivation claim; it reports measured minDCF and EER for spoofing countermeasures trained on native and combined native/non-native data. The central comparison (Native CM vs. Combined CM on non-native test speech) is an empirical result, not an identity. The non-native test spoof is generated with the same three attack systems (SpeechT5, FreeVC, WORLD) used to create the non-native training spoof, and for SpeechT5 some x-vector and bottleneck features come from the collected non-native speaker data. This means the improvement of Combined CM over Native CM may partly reflect familiarity with specific attack algorithms rather than accent generalization alone, and the Combined CM also trains on more utterances. These are external-validity or experimental-design concerns, not circularity by construction: the evaluation protocol does not define the outcome, no fitted parameter is renamed as a prediction, and no load-bearing result depends on a self-citation chain. The paper is self-contained in the sense that all reported numbers are genuine measurements under the stated in-domain protocol. Therefore the circularity score is 0, with the generalization caveat noted as a correctness risk rather than circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the representativeness of the constructed ENIT corpus and of the three chosen spoof generators, and on the adequacy of cepstral features plus boosting/GMM classifiers as a probe of domain mismatch. No free parameters are introduced beyond the standard model hyperparameters used in the comparison (these do not enter any derivation), and no new physical or theoretical entities are posited.

assumptions (5)
  • domain assumption ASVspoof 5 training data (MLS-based native English) is a representative source of native English bonafide and spoof speech for building a baseline CM.
    Invoked in Section III-A to select native utterances; the Native CM in Experiment 1 is trained on this data, so the comparison assumes the data quality is adequate.
  • domain assumption The three generation systems (SpeechT5, FreeVC, WORLD) produce spoof utterances whose artifacts are representative of real TTS/VC attacks on non-native English.
    Used in Section III-B to create the non-native spoof data; the central improvement claim is measured against these same generators.
  • domain assumption The ENIT recordings (21 speakers, soundproof room, read style) capture the relevant non-native speech characteristics for target applications such as language proficiency testing.
    Section III-A describes the recording protocol; the conclusions about non-native speech handling presuppose this sample represents the broader population.
  • domain assumption Cepstral features (MFCC, LFCC, CQCC) retain sufficient discriminative information about spoof artifacts in accented speech.
    Section IV-A selects these features; the experiment compares CM performance but does not test alternative feature families.
  • domain assumption Hyperparameters chosen (GMM with two components, default CatBoost/XGBoost settings, seed 42) are adequate for a fair comparison between Native and Combined CMs.
    Section V-C fixes these settings; no sensitivity analysis is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Detecting Spoof Voices in Asian Non-Native Speech: An Indonesian and Thai Case Study." pith.science (2026). https://pith.science/paper/D3Z3ZFTF

@misc{pith2026241201040,
  author       = {Pith},
  title        = {Pith review of: Detecting Spoof Voices in Asian Non-Native Speech: An Indonesian and Thai Case Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D3Z3ZFTF}},
  note         = {Machine review of arXiv:2412.01040}
}
read the original abstract

This study focuses on building effective spoofing countermeasures (CMs) for non-native speech, specifically targeting Indonesian and Thai speakers. We constructed a dataset comprising both native and non-native speech to facilitate our research. Three key features (MFCC, LFCC, and CQCC) were extracted from the speech data, and three classic machine learning-based classifiers (CatBoost, XGBoost, and GMM) were employed to develop robust spoofing detection systems using the native and combined (native and non-native) speech data. This resulted in two types of CMs: Native and Combined. The performance of these CMs was evaluated on both native and non-native speech datasets. Our findings reveal significant challenges faced by Native CM in handling non-native speech, highlighting the necessity for domain-specific solutions. The proposed method shows improved detection capabilities, demonstrating the importance of incorporating non-native speech data into the training process. This work lays the foundation for more effective spoofing detection systems in diverse linguistic contexts.

Figures

Figures reproduced from arXiv: 2412.01040 by the authors.

Figure 1
Figure 1. Block diagrams of our proposed method for building spoofing countermeasures [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 27 canonical work pages

  1. [1]

    ASVspoof 2019: Spoofing Countermeasures for the Detection of Synthesized, Converted and Replayed Speech,

    A. Nautsch, X. Wang, N. Evans, et al. , “ASVspoof 2019: Spoofing Countermeasures for the Detection of Synthesized, Converted and Replayed Speech,” IEEE Trans. on Biometrics, Behavior, and Identity Science , vol. 3, no. 2, pp. 252–265, Apr. 2021, ISSN : 2637-6407

    work page 2019

  2. [2]

    HABLA: A Dataset of Latin American Spanish Accents for V oice Anti-spoofing,

    P. A. Tamayo Fl ´orez, R. Manrique, and B. Pereira Nunes, “HABLA: A Dataset of Latin American Spanish Accents for V oice Anti-spoofing,” in Proc. of INTER- SPEECH 2023, 2023, pp. 1963–1967

    work page 2023

  3. [3]

    ADD 2022: the first Audio Deep Synthesis Detection Challenge,

    J. Yi, R. Fu, J. Tao, et al. , “ADD 2022: the first Audio Deep Synthesis Detection Challenge,” in Proc. of ICASSP 2022 , IEEE, 2022, pp. 9216–9220. 5

    work page 2022

  4. [4]

    ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge,

    Z. Wu, T. Kinnunen, N. Evans, et al. , “ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge,” in Proc. of INTER- SPEECH2015, ISCA, 2015, pp. 2037–2041

    work page 2015

  5. [5]

    ASVspoof 5: Crowdsourced data, deep- fakes and adversarial attacks at scale,

    X. W. et al., “ASVspoof 5: Crowdsourced data, deep- fakes and adversarial attacks at scale,” inASVspoof 2024 workshop (submitted), 2024

    work page 2024

  6. [6]

    Audio Anti- Spoofing Detection: A Survey,

    M. Li, Y . Ahmadiadli, and X.-P. Zhang, “Audio Anti- Spoofing Detection: A Survey,” CoRR, 2024

    work page 2024

  7. [7]

    A compar- ison of features for synthetic speech detection,

    M. Sahidullah, T. Kinnunen, and C. Hanilc ¸i, “A compar- ison of features for synthetic speech detection,” in Proc. of INTERSPEECH2015, ISCA, 2015, pp. 2087–2091

    work page 2015

  8. [8]

    The ASVspoof 2017 Challenge: Assessing the Limits of Replay Spoofing Attack Detection,

    T. Kinnunen, M. Sahidullah, H. Delgado, et al. , “The ASVspoof 2017 Challenge: Assessing the Limits of Replay Spoofing Attack Detection,” in Proc. of INTER- SPEECH2017, ISCA, 2017, pp. 2–6

    work page 2017

Show all 31 references
  1. [9]

    ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection,

    J. Yamagishi, X. Wang, M. Todisco, et al., “ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection,” CoRR, vol. abs/2109.00537, 2021

    2021 arXiv

  2. [10]

    ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech,

    X. Wang, J. Yamagishi, M. Todisco, et al., “ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech,” Comput. Speech Lang., vol. 64, p. 101 114, 2020

    2019

  3. [11]

    Speechocean762: An open-source non-native english speech corpus for pronunciation assessment,

    J. Zhang, Z. Zhang, Y . Wang, et al., “Speechocean762: An open-source non-native english speech corpus for pronunciation assessment,” CoRR, vol. abs/2104.01378, 2021

    2021 arXiv

  4. [12]

    The ISLE corpus of non- native spoken English,

    W. Menzel, E. Atwell, P. Bonaventura, D. Herron, P. Howarth, and R. Morton, “The ISLE corpus of non- native spoken English,” Apr. 2000

    2000

  5. [13]

    Mls: A large-scale multilingual dataset for speech research,

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “Mls: A large-scale multilingual dataset for speech research,” in Proc. of INTERSPEECH2020 , ISCA, 2020, pp. 2757–2761

    2020

  6. [14]

    SpeechT5: Unified- Modal Encoder-Decoder Pre-Training for Spoken Lan- guage Processing,

    J. Ao, R. Wang, L. Zhou, et al. , “SpeechT5: Unified- Modal Encoder-Decoder Pre-Training for Spoken Lan- guage Processing,” in Proc. of ACL 2022 , Association for Computational Linguistics, 2022, pp. 5723–5738

    2022

  7. [15]

    Exploring the limits of transfer learning with a unified text-to- text transformer,

    C. Raffel, N. Shazeer, A. Roberts, et al. , “Exploring the limits of transfer learning with a unified text-to- text transformer,” J. Mach. Learn. Res. , vol. 21, 140:1– 140:67, 2020

    2020

  8. [16]

    X-vectors: Robust DNN embeddings for speaker recognition,

    D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust DNN embeddings for speaker recognition,” in Proc. of ICASSP 2018 , IEEE, 2018, pp. 5329–5333

    2018

  9. [17]

    The CMU Arctic speech databases,

    J. Kominek and A. W. Black, “The CMU Arctic speech databases,” in Fifth ISCA ITRW on Speech Synthesis, Pittsburgh, PA, USA, June 14-16, 2004 , A. W. Black and K. A. Lenzo, Eds., ISCA, 2004, pp. 223–224

    2004

  10. [18]

    Yamagishi, C

    J. Yamagishi, C. Veaux, and K. MacDonald, CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92) , 2019

    2019

  11. [19]

    FreeVC: Towards High- Quality Text-Free One-Shot V oice Conversion,

    J. Li, W. Tu, and L. Xiao, “FreeVC: Towards High- Quality Text-Free One-Shot V oice Conversion,” inProc. of ICASSP 2023 , IEEE, 2023, pp. 1–5

    2023

  12. [20]

    Conditional variational au- toencoder with adversarial learning for end-to-end text- to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational au- toencoder with adversarial learning for end-to-end text- to-speech,” in Proc. of ICML 2021 , vol. 139, PMLR, 2021, pp. 5530–5540

    2021

  13. [21]

    WavLM: Large- Scale Self-Supervised Pre-Training for Full Stack Speech Processing,

    S. Chen, C. Wang, Z. Chen, et al. , “WavLM: Large- Scale Self-Supervised Pre-Training for Full Stack Speech Processing,” IEEE J. Sel. Top. Signal Process. , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  14. [22]

    WORLD: A vocoder-based high-quality speech synthesis system for real-time applications,

    M. Morise, F. Yokomori, and K. Ozawa, “WORLD: A vocoder-based high-quality speech synthesis system for real-time applications,” IEICE Trans. Inf. Syst., vol. 99- D, no. 7, pp. 1877–1884, 2016

    2016

  15. [23]

    Cheaptrick, a spectral envelope estimator for high-quality speech synthesis,

    M. Morise, “Cheaptrick, a spectral envelope estimator for high-quality speech synthesis,” Speech Commun. , vol. 67, pp. 1–7, 2015

    2015

  16. [24]

    Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,

    S. Davis and P. Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,” IEEE Trans. Audio, Speech, and Lang. Proc. , vol. 28, no. 4, pp. 357–366, 1980

    1980

  17. [25]

    A one-class classification approach to generalised speaker verifica- tion spoofing countermeasures using local binary pat- terns,

    F. Alegre, A. Amehraye, and N. Evans, “A one-class classification approach to generalised speaker verifica- tion spoofing countermeasures using local binary pat- terns,” in 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS) , 2013, pp. 1–8

    2013

  18. [26]

    An Explainability Study of the Constant Q Cepstral Coefficient Spoofing Countermeasure for Automatic Speaker Verification,

    H. Tak, J. Patino, A. Nautsch, N. W. D. Evans, and M. Todisco, “An Explainability Study of the Constant Q Cepstral Coefficient Spoofing Countermeasure for Automatic Speaker Verification,” in Proc of Odyssey 2020, K. Lee, T. Koshinaka, and K. Shinoda, Eds., ISCA, 2020, pp. 333–340

    2020

  19. [27]

    CatBoost: unbiased boosting with categorical features,

    L. O. Prokhorenkova, G. Gusev, A. V orobev, A. V . Dorogush, and A. Gulin, “CatBoost: unbiased boosting with categorical features,” in Proc. of NeurIPS 2018 , 2018, pp. 6639–6649

    2018

  20. [28]

    XGBoost: A Scalable Tree Boosting System,

    T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in Proc. of ACM SIGKDD 2016 , ser. KDD ’16, ACM, Aug. 2016

    2016

  21. [29]

    CatBoost: gradient boosting with categorical features support,

    A. V . Dorogush, V . Ershov, and A. Gulin, “CatBoost: gradient boosting with categorical features support,” CoRR, vol. abs/1810.11363, 2018

    2018 arXiv

  22. [30]

    Scikit- learn: Machine learning in Python,

    F. Pedregosa, G. Varoquaux, A. Gramfort, et al., “Scikit- learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011

    2011

  23. [31]

    V oice spoofing countermeasures: Taxonomy, state-of-the-art, experimental analysis of generalizabil- ity, open challenges, and the way forward,

    A. Khan, K. M. Malik, J. Ryan, and M. Sara- vanan, “V oice spoofing countermeasures: Taxonomy, state-of-the-art, experimental analysis of generalizabil- ity, open challenges, and the way forward,” CoRR, vol. abs/2210.00417, 2022. 6

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.