Pith. sign in

REVIEW 4 major objections 6 minor 34 references

Enhancing In-Domain and Out-Domain EmoFake Detection via Cooperative Multilingual Speech Foundation Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multilingual speech foundation models, fused by THAMA, outperform single-language models at detecting emotion-manipulated fake audio.

desk verdict The multilingual-over-monolingual pattern for EmoFake detection looks real; the 0.89%/1.03% EER 'SOTA' does not, because the best fusion pair was picked on the test set. read the letter →

arxiv 2507.12595 v1 pith:WMHWQFNF submitted 2025-07-16 eess.AS

Orchid Chetia Phukan , Mohd Mujtaba Akhtar , Girish , Arun Balaji Buduru This is my paper · ORCID
classification eess.AS
keywords EmoFakedetectionspeechfoundationmodelsmultilingualpre-trainingTuckerdecompositionHadamardproductfusioncross-lingualaudio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that speech foundation models trained on many languages are better at detecting EmoFake audio—recordings whose emotional tone has been altered while the words stay the same—than models trained on one language or on music. It tests thirteen models in same-language and cross-lingual settings on the only public EmoFake dataset, covering English and Chinese, and reports that multilingual models consistently give the lowest error rates. The paper also introduces THAMA, a fusion method that combines Tucker decomposition with a Hadamard product to merge representations from two models. Fusing the two strongest multilingual models, XLS-R and MMS, with THAMA yields 0.89% and 1.03% equal-error rates on English and Chinese, against 3.65% and 8.34% for the prior benchmark. If these results hold, they suggest that broad linguistic pre-training captures the pitch, tone, and intensity cues that emotional manipulation changes.

What carries the argument

THAMA (Tucker-HAdaMArd Conjunction) is the paper's fusion mechanism. It takes the flattened representations of two foundation models, projects them into a shared space, and combines them with a Tucker decomposition, in which a small core tensor models the high-order interactions between the two feature sets; the result is then squared element-wise with a Hadamard product, which sharpens the fused features before a fully connected classifier. The method is what turns two individually strong multilingual models into a single detector that outperforms either alone and outperforms simple concatenation-based fusion.

What would settle it

Run the same pipeline with model selection done on a held-out development fold and report test EER for every fusion pair; if XLS-R+MMS is not the selected pair, or its EER no longer beats the 3.65%/8.34% baseline with a clear margin, the central SOTA claim is not supported. Reporting per-seed variability would let a reader see whether the top fusion is statistically distinct from the runner-ups.

Watch

Extended reading notes

Core claim

The central claim is that multilingual speech foundation models are intrinsically better suited to EmoFake detection and that fusing them with THAMA sets a new state of the art. The paper argues that pre-training across languages gives the models a more refined grasp of prosodic variation, letting them spot the subtle acoustic edits left by emotion voice conversion. In the author's own comparison, the multilingual models XLS-R and MMS beat every monolingual, speaker-recognition, and music model in both in-domain and cross-lingual tests, and their THAMA fusion improves further still. The reported outcome is that the XLS-R + MMS pair with THAMA reaches 0.89% EER on English and 1.03% on Chinese, versus 3.65% and 8.34% for the previous best method on the same dataset.

Load-bearing premise

The headline error rates assume the test set was not used to choose which two models to fuse; if the best pair was selected by reading test results, the 0.89%/1.03% figures are optimistic.

Editorial extensions

If this is right

  • If correct, THAMA-style fusion of XLS-R and MMS becomes the default baseline to beat for future EmoFake detection research.
  • Cross-lingual EmoFake detection is feasible: a system trained on English transfers to Chinese and vice versa, with EERs in the 3-5% range, which the paper reports as a new benchmark for out-domain evaluation.
  • Multilingual pre-training matters more than model size alone for this task: MMS's advantage over smaller multilingual models is attributed partly to scale, but all multilingual models beat much larger monolingual and music alternatives.
  • Music foundation models are not useful for EmoFake detection; the paper's results show they systematically perform worse, so resources for this task should go to speech models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's comparison is limited to one dataset with two languages; a natural extension is to test whether the same THAMA fusion generalizes to other language pairs or to emotion-manipulated samples generated by unseen voice-conversion systems.
  • Because the official test split was used to select the best fusion pair, the reported SOTA numbers should be read as optimistic upper bounds; a held-out validation protocol would give a fairer estimate.
  • The THAMA fusion mechanism is task-agnostic in principle and could be evaluated for other audio deepfake tasks such as singing-voice or scene-fake detection, where complementary multilingual representations may also help.
  • Re-running with different random seeds and reporting variance would clarify whether the small EER differences between top fusion pairs are meaningful.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates EmoFake Detection (EFD) using frozen speech foundation models. It compares 13 foundation models (monolingual, multilingual, speaker-recognition, and music models) on the EmoFake dataset in both same-language (English/Chinese) and cross-lingual settings, and proposes THAMA, a fusion method combining Tucker decomposition with a Hadamard product. The authors report that multilingual speech foundation models consistently outperform other families, and that THAMA fusing XLS-R and MMS achieves the best results, with 0.89% and 1.03% EER on English and Chinese, which they claim is a new state of the art compared with the 3.65% and 8.34% EER reported by prior work [10].

Significance. If validated, the paper would provide a useful empirical benchmark: it is, to my knowledge, the first systematic comparison of multiple speech foundation model families for EFD, the first cross-lingual EFD evaluation on this dataset, and it proposes a fusion method that improves over simple concatenation. The breadth of the comparison (13 individual models and 78 fusion pairs) and the consistent multilingual advantage across languages are genuine strengths. However, the central empirical claims currently rest on an evaluation protocol that selects the best model on the test set and on a cross-family comparison that is confounded by model scale and architecture; these issues must be resolved before the state-of-the-art claim can be accepted.

major comments (4)
  1. [Section 4.2, Tables 1-2] The reported state-of-the-art numbers are selected from the test set rather than estimated on held-out data. Table 2 reports EER for 78 fusion pairs evaluated on the official test split, and the headline XLS-R+MMS THAMA results (0.89%/1.03%) are the minimum of that table. No development-set results, repeated runs, or confidence intervals are reported, and the same test set also appears to have guided the choice between CNN and FCN downstream heads. Selecting the minimum of many correlated estimates biases the reported EER downward relative to the true EER of any fixed model, and the magnitude of this bias is unknown. Because the entire comparison to prior work [10] (3.65%/8.34%) depends on these two single-run numbers, the state-of-the-art claim is not yet supported. Please select the fusion pair and downstream architecture on a held-out validation split, and report mean and standard deviation over multiple training seeds, or use nested cross-validation.
  2. [Section 3.1, THAMA definition] The core fusion equation is internally inconsistent as written. The text defines Z = sum_i sum_j sum_k T_ijk * F1_i * F2_j, where the sum runs over all three indices; since no output index appears, Z is a scalar. The subsequent operations H = Z ⊙ Z and the FCN block require a vector. The intended formula is presumably Z_k = sum_i sum_j T_ijk * F1_i * F2_j, with the tensor contraction leaving one free index. Please correct the equation and specify the tensor contraction explicitly; the current definition does not match Figure 2 or the downstream processing.
  3. [Section 2/Table 1] The main hypothesis, that multilingual pre-training causes the observed superiority, is confounded with model scale and architecture. The multilingual models are MMS (1B parameters), XLS-R (300M), and Whisper base (74M), while the monolingual baselines are all roughly 95M. The paper itself attributes MMS's top individual performance to its larger model size, which undercuts the claim that multilinguality per se is the cause. The Whisper result (74M multilingual outperforming 95M monolingual models) suggests scale is not the whole story, but a controlled comparison is needed, e.g., XLS-R 300M versus wav2vec2 300M, or MMS 1B versus a monolingual 1B model, before the attribution to multilingual pre-training can be accepted.
  4. [Section 4.2, out-domain evaluation] The out-domain evaluation is narrower than the claims suggest. Only the in-domain best pair (XLS-R+MMS) is carried to the cross-lingual setting, and that pair was selected from the same test split via Table 2, so the same selection-bias concern applies. In addition, 'out-domain' here means training on one language and testing on the other within the same EmoFake corpus; it does not include unseen datasets, unseen speakers, or unseen emotional voice conversion systems. The contribution statement claiming 'the first-ever out-domain assessment' should be scoped accordingly, and the selection protocol for the out-domain model should be described explicitly.
minor comments (6)
  1. [Abstract and throughout] There are several typographical and grammatical errors, e.g., 'Our results shows', 'To our end', 'tehcnique', 'resarch', and 'Hardmard' in Figure 2. These should be corrected.
  2. [Table 2] The table does not state which downstream head (CNN or FCN) was used for the fusion experiments. The text says CNN generally performed better for individual models, but it should be explicit for Table 2 and for the out-domain results.
  3. [Section 4.2, comparison to SOTA] The statement 'Due to space constraint we are unable to present the comparison with SOTA in tabular format' is not acceptable in a full paper; the comparison to [10] should be shown in a table with the same evaluation protocol, including the training/development/test split details.
  4. [Section 3.1] No ablation isolates the contributions of the Tucker core, the Hadamard product, and the learned shared projection. The comparison to concatenation is useful, but an ablation would strengthen the claim that each component of THAMA is necessary.
  5. [Section 4.1] The dataset description says training, development, and test contain 27300, 9100, and 17500 samples 'for both English and Chinese', but it is not clear whether these counts are per language or combined; please clarify.
  6. [References] Reference [7] is cited in two consecutive sentences with the same description; the duplication should be removed. Also, the paper says models and code will be open-sourced after double-blind review, but the current submission contains no code or data release, which limits reproducibility checks of the reported EER values.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the EFD results are an empirical benchmark with independent external comparison; the only self-citation is motivational and not load-bearing.

full rationale

This paper is an empirical benchmark and architecture study, so there is no derivation chain whose conclusion equals its own input. The central claim that multilingual speech foundation models outperform other families for EmoFake detection is established by experiments on the fixed EmoFake official splits, not by a fitted parameter, a definition, or an imported uniqueness theorem. THAMA is presented as a proposed fusion architecture in Section 3.1, and its description does not assume the reported EER values. The only self-citation is reference [16], used in Section 1 as motivation for combining foundation models and for x-vector relevance; it does not supply the headline EER numbers or the choice of XLS-R plus MMS. The state-of-the-art comparison against Zhao et al. [10] is a direct external benchmark on the same dataset. A potential validity caveat is that the best fusion pair appears to be selected from the same test split used for the final reported numbers, with no development-set selection or confidence intervals, but that is an evaluation-soundness and statistical-bias concern rather than circular reasoning. Therefore, applying the stated circularity criteria, no circular step can be exhibited from the paper's equations or citations.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

This is an empirical machine-learning benchmark, so no derivation introduces scientific constants or new physical entities. The central numbers rest on the data-split assumption, the use of frozen features, the attribution of multilingual superiority, and hand-chosen training hyperparameters.

free parameters (2)
  • THAMA shared-space dimension df
    The dimension of the fused space is a hand-chosen capacity knob; it is not reported and sensitivity to it is not tested. It determines the size of the core tensor and the stated 5.5M-10M parameter range.
  • Downstream training hyperparameters = Adam lr=1e-3, batch size=32, 100 epochs
    Chosen by hand with no ablation study; all reported results depend on these choices, including early stopping and dropout settings that are mentioned but not quantified.
assumptions (4)
  • domain assumption The official EmoFake train/dev/test split is used correctly and test labels are not used to select the final fusion model.
    The paper reports evaluations on the official split from [10], but the best fusion pair appears to be selected from test-set EER values in Table 2, so this assumption is likely violated; see Sections 4.1 and 4.2.
  • domain assumption Differences in single-run EER values reflect model capability rather than chance.
    No confidence intervals, standard deviations, or significance tests are reported, so small differences such as 0.89% versus 1.00% EER cannot be distinguished from noise; see Tables 1 and 2.
  • ad hoc to paper Multilingual pre-training, rather than model scale or architecture, causes the observed superiority of Whisper, XLS-R, and MMS.
    The paper attributes the gains to diverse linguistic pre-training, but model sizes differ widely: MMS 1B, XLS-R 300M, and Whisper-base 74M versus roughly 95M monolingual bases; see Section 4.2 and Table 1.
  • domain assumption Average-pooled final-layer representations of frozen foundation models retain enough emotional and timbre information for EmoFake detection.
    All experiments rely on frozen representations with average pooling over the final hidden layer; no fine-tuning or layer-selection study is provided; see Sections 2 and 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing In-Domain and Out-Domain EmoFake Detection via Cooperative Multilingual Speech Foundation Models." pith.science (2026). https://pith.science/paper/WMHWQFNF

@misc{pith2026250712595,
  author       = {Pith},
  title        = {Pith review of: Enhancing In-Domain and Out-Domain EmoFake Detection via Cooperative Multilingual Speech Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WMHWQFNF}},
  note         = {Machine review of arXiv:2507.12595}
}
read the original abstract

In this work, we address EmoFake Detection (EFD). We hypothesize that multilingual speech foundation models (SFMs) will be particularly effective for EFD due to their pre-training across diverse languages, enabling a nuanced understanding of variations in pitch, tone, and intensity. To validate this, we conduct a comprehensive comparative analysis of state-of-the-art (SOTA) SFMs. Our results shows the superiority of multilingual SFMs for same language (in-domain) as well as cross-lingual (out-domain) evaluation. To our end, we also propose, THAMA for fusion of foundation models (FMs) motivated by related research where combining FMs have shown improved performance. THAMA leverages the complementary conjunction of tucker decomposition and hadamard product for effective fusion. With THAMA, synergized with cooperative multilingual SFMs achieves topmost performance across in-domain and out-domain settings, outperforming individual FMs, baseline fusion techniques, and prior SOTA methods.

Figures

Figures reproduced from arXiv: 2507.12595 by the authors.

Figure 1
Figure 1. Demonstration of EmoFake: Person A’s neutral speech is manipulated to synthesize an angry emotional tone in the audio while maintaining the same spoken content (’How are you?’) evidence in legal contexts could undermine judicial integrity, while fabricating emotionally persuasive messages could exploit listeners’ psychological vulnerabilities at scale. These adverse implications highlight the urgent need for advance… view at source ↗
Figure 2
Figure 2. Proposed Framework: THAMA element-wise multiplication. The Hadamard product further en￾hances fusion by preserving element-wise interactions within the fused representation, strengthening feature dependencies and allowing the model to better capture complementary and discrim￾inative information. Tucker Decomposition with Hadamard Product provides a novel and effective approach that leverages high-order fusion while … view at source ↗
Figure 3
Figure 3. t-SNE Plots- (a) MERT-v1-330M(b) WavLM (c) MMS (d) XLS-R 5. Conclusion In this work, we show the effectiveness of multilingual SFMs for EFD in comparison to other SOTA SFMs and MFMs and demonstrating their ability to capture subtle emotional variations through their diverse linguistic pre-training. Our comprehen￾sive analysis confirms their superiority in both in-domain and cross-lingual settings. Also, we introduce… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 28 canonical work pages

  1. [16]

    Scenefake: An initial dataset and benchmarks for scene fake audio detection,

    J. Yi, C. Wang, J. Tao, C. Y . Zhang, C. Fan, Z. Tian, H. Ma, and R. Fu, “Scenefake: An initial dataset and benchmarks for scene fake audio detection,” Pattern Recognition, vol. 152, p. 110468, 2024

    work page 2024

  2. [10]

    The ASVspoof 2017 Challenge: Assessing the Limits of Replay Spoofing Attack Detection,

    T. Kinnunen, M. Sahidullah, H. Delgado et al., “The ASVspoof 2017 Challenge: Assessing the Limits of Replay Spoofing Attack Detection,” in Proc. Interspeech 2017, 2017, pp. 2–6

    work page 2017

  3. [1]

    Introduction Driven by the development of comprehensive datasets and rigor- ous benchmarking challenges, the field of fake audio detection has seen significant development. Early efforts, ASVspoof [1], laid the foundation for detecting spoofed speech, with subse- quent iterations like ASVspoof 2017 [2] and ASVspoof 2019 [3] addressing replay attacks and s...

    work page 2017

  4. [2]

    Speech Foundation Models (SFMs): For monolingual SFMs, we consider WavLM1 [17], Unispeech-SAT2 [18], Wav2vec23 [19], and HuBERT 4 [20]

    Foundation Models In this section, we present the SOTA FMs used in our study. Speech Foundation Models (SFMs): For monolingual SFMs, we consider WavLM1 [17], Unispeech-SAT2 [18], Wav2vec23 [19], and HuBERT 4 [20]. WavLM and Unispeech-SAT are SOTA SFMs in SUPERB benchmark. WavLM is trained to solve speech denoising together with masked modeling whereas Uni...

    work page

  5. [3]

    We use Fully Connected Network (FCN) and CNN as downstream networks with individual FMs

    Modeling In this section, we present the downstream modeling approaches with individual FMs followed by the proposed framework,THAMA for combining the FMs. We use Fully Connected Network (FCN) and CNN as downstream networks with individual FMs. CNN starts with three 1D convolutional layers with 64, 128, 256 filters with a kernel size of 3 and ReLU activat...

    work page

  6. [4]

    Benchmark Dataset We use the only dataset available for EFD by Zhao et al

    Experiments 4.1. Benchmark Dataset We use the only dataset available for EFD by Zhao et al. [ 10]. It comprises recordings in both English and Chinese, featuring audio samples from multiple speakers across five core emotions: Neutral, Happy, Angry, Sad, and Surprise. Fake emotional sam- ples are generated using seven open-source Emotional V oice Conversio...

    work page

  7. [5]

    The projected features are then fused using Tucker Decomposi- tion, where a core tensor T ∈ Rdf ×df ×df models the high- order interactions between features

    Next, these transformed representations are pro- jected into a shared space as F1 = W1X′ 1 and F2 = W2X′ 2, where W1 ∈ Rd′ 1×df and W2 ∈ Rd′ 2×df are trainable weight matrices, and df is the dimensionality of the fused space. The projected features are then fused using Tucker Decomposi- tion, where a core tensor T ∈ Rdf ×df ×df models the high- order inte...

    work page

  8. [6]

    Add 2023: the second audio deepfake detection challenge,

    J. Yi, J. Tao, R. Fuet al., “Add 2023: the second audio deepfake detection challenge,” arXiv preprint arXiv:2305.13774, 2023

    arXiv 2023

Show all 34 references
  1. [7]

    However, despite these advancements, an under- explored yet critical dimension remains: scenarios where the emotional state of speech is artificially modified i.e

    explores multi-lingual TTS-generated audio across diverse languages. However, despite these advancements, an under- explored yet critical dimension remains: scenarios where the emotional state of speech is artificially modified i.e. EmoFake (EF) (Figure 1). While such technolo...

    2025 arXiv

  2. [8]

    Our comprehen- sive analysis confirms their superiority in both in-domain and cross-lingual settings

    Conclusion In this work, we show the effectiveness of multilingual SFMs for EFD in comparison to other SOTA SFMs and MFMs and demonstrating their ability to capture subtle emotional variations through their diverse linguistic pre-training. Our comprehen- sive analysis confirms...

  3. [9]

    ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures chal- lenge,

    Z. Wu, T. Kinnunen, N. Evans et al., “ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures chal- lenge,” in Proc. Interspeech 2015, 2015, pp. 2037–2041

    2015

  4. [11]

    ASVspoof 2019: Fu- ture Horizons in Spoofed and Fake Audio Detection,

    M. Todisco, X. Wang, V . Vestmanet al., “ASVspoof 2019: Fu- ture Horizons in Spoofed and Fake Audio Detection,” in Proc. Interspeech 2019, 2019, pp. 1008–1012

    2019

  5. [12]

    ASVspoof 2021: ac- celerating progress in spoofed and deepfake speech detection,

    J. Yamagishi, X. Wang, M. Todiscoet al., “ASVspoof 2021: ac- celerating progress in spoofed and deepfake speech detection,” in Proc. 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge, 2021, pp. 47–54

    2021

  6. [13]

    Add 2022: the first audio deep synthesis detection challenge,

    J. Yi, R. Fu, J. Tao et al., “Add 2022: the first audio deep synthesis detection challenge,” in ICASSP 2022, 2022, pp. 9216–9220

    2022

  7. [14]

    Mlaad: The multi-language audio anti-spoofing dataset,

    N. M. Müller, P. Kawa, W. H. Choong, E. Casanova, E. Gölge, T. Müller, P. Syga, P. Sperl, and K. Böttinger, “Mlaad: The multi-language audio anti-spoofing dataset,” arXiv preprint arXiv:2401.09512, 2024

    2024 arXiv

  8. [15]

    Singfake: Singing voice deepfake detection,

    Y . Zang, Y . Zhang, M. Heydari, and Z. Duan, “Singfake: Singing voice deepfake detection,” in ICASSP 2024 - 2024 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 12 156–12 160

    2024

  9. [17]

    Emofake: An initial dataset for emotion fake audio detection,

    Y . Zhao, J. Yi, J. Tao, C. Wang, and Y . Dong, “Emofake: An initial dataset for emotion fake audio detection,” inChina National Conference on Chinese Computational Linguistics . Springer, 2024, pp. 419–433

    2024

  10. [18]

    Im- proved deepfake detection using whisper features,

    P. Kawa, M. Plata, M. Czuba, P. Szyma ´nski, and P. Syga, “Im- proved deepfake detection using whisper features,” in Interspeech 2023, 2023, pp. 4009–4013

    2023

  11. [19]

    Detection of deepfake environmental audio,

    H. Ouajdi, O. Hadder, M. Tailleur, M. Lagrange, and L. M. Heller, “Detection of deepfake environmental audio,” in2024 32nd Euro- pean Signal Processing Conference (EUSIPCO), 2024, pp. 196– 200

    2024

  12. [20]

    Exploring wavlm back-ends for speech spoofing and deepfake detection,

    T. Stourbe, V . Miara, T. Lepage, and R. Dehak, “Exploring wavlm back-ends for speech spoofing and deepfake detection,” in The Au- tomatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof 2024), 2024, pp. 72–78

    2024

  13. [21]

    Investi- gation of ensemble features of self-supervised pretrained models for automatic speech recognition,

    A. Arunkumar, V . Nileshkumar Sukhadia, and S. Umesh, “Investi- gation of ensemble features of self-supervised pretrained models for automatic speech recognition,” in Interspeech 2022, 2022, pp. 5145–5149

    2022

  14. [22]

    Investigation of ensemble of self-supervised models for speech emotion recognition,

    Y . Wu, P. Yue, C. Cheng, and T. Li, “Investigation of ensemble of self-supervised models for speech emotion recognition,” in 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 2023, pp. 988–995

    2023

  15. [23]

    Heterogeneity over homogeneity: Investigating multilingual speech pre-trained models for detecting audio deepfake,

    O. Chetia Phukan, G. Kashyap, A. B. Buduru, and R. Sharma, “Heterogeneity over homogeneity: Investigating multilingual speech pre-trained models for detecting audio deepfake,” in Findings of the Association for Computational Linguistics: NAACL 2024, K. Duh, H. Gomez, and S. Be...

    2024

  16. [24]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505– 1518, 2022

    2022

  17. [25]

    Unispeech-sat: Universal speech repre- sentation learning with speaker aware pre-training,

    S. Chen, Y . Wu, C. Wang, Z. Chen, Z. Chen, S. Liu, J. Wu, Y . Qian, F. Wei, J. Li, and X. Yu, “Unispeech-sat: Universal speech repre- sentation learning with speaker aware pre-training,” ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processi...

    2022

  18. [26]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems , vol. 33, pp. 12 449–12 460, 2020

    2020

  19. [27]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM trans- actions on audio, speech, and language processing , vol. 29, pp. 3451–3460, 2021

    2021

  20. [28]

    Xls-r: Self-supervised cross-lingual speech representation learning at scale,

    A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y . Saraf, J. Pino, A. Baevski, A. Con- neau, and M. Auli, “Xls-r: Self-supervised cross-lingual speech representation learning at scale,” in Interspeech 2022, 2022, pp. 2278–2282

    2022

  21. [29]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning . PMLR, 2023, pp. 28 492–28 518

    2023

  22. [30]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al., “Scaling speech technology to 1,000+ languages,” Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    2024

  23. [31]

    X-vectors: Robust dnn embeddings for speaker recognition,

    D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust dnn embeddings for speaker recognition,”2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5329–5333, 2018

    2018

  24. [32]

    X- vectors meet emotions: A study on dependencies between emotion and speaker recognition,

    R. Pappagari, T. Wang, J. Villalba, N. Chen, and N. Dehak, “X- vectors meet emotions: A study on dependencies between emotion and speaker recognition,” in ICASSP 2020 - 2020 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 7169–7173

    2020

  25. [33]

    Mert: Acoustic music understanding model with large-scale self-supervised training,

    Y . Li, R. Yuan, G. Zhang, Y . Ma, X. Chen, H. Yin, C. Xiao, C. Lin, A. Ragni, E. Benetos et al., “Mert: Acoustic music understanding model with large-scale self-supervised training,” arXiv preprint arXiv:2306.00107, 2023

    2023 arXiv

  26. [34]

    Map- music2vec: A simple and effective baseline for self-supervised music audio representation learning,

    Y . Li, R. Yuan, G. Zhang, Y . Ma, C. Lin, X. Chen, A. Ragni, H. Yin, Z. Hu, H. He, E. Benetos, N. Gyenge, R. Liu, and J. Fu, “Map- music2vec: A simple and effective baseline for self-supervised music audio representation learning,” ArXiv, vol. abs/2212.02508, 2022

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.