Towards Generalized Source Tracing for Codec-Based Deepfake Speech

Haibin Wu; Hung-yi Lee; I-Ming Lin; Jyh-Shing Roger Jang; Lin Zhang; Xuanjun Chen

REVIEW 3 major objections 5 minor 53 references

Towards Generalized Source Tracing for Codec-Based Deepfake Speech

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that codec-based deepfake speech can be traced to its generating codec even when training data is only resynthesized, provided a semantic content anchor is fused with class-specific acoustic reconstruction, and that…

desk verdict SASTNet reports large, plausible gains on codec attribution, but its main silence-overfitting diagnosis rests on an unspecified preprocessing step, so the paper deserves review with a request for details. read the letter →

arxiv 2506.07294 v3 pith:CYJ5AYXJ submitted 2025-06-08 cs.SD cs.CRcs.LGeess.AS

Xuanjun Chen , I-Ming Lin , Lin Zhang , Haibin Wu , Hung-yi Lee , Jyh-Shing Roger Jang This is my paper

classification cs.SDcs.CRcs.LGeess.AS

keywords anti-spoofingsourcetracingaudiodeepfakedetectionneuralcodeccodec-basedspeechgenerationsemantic-acousticfusionexplainability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deepfake speech detectors can often tell fake from real but not which generator made a fake. This paper takes on the harder task of source tracing for speech produced by neural audio codec systems, and argues that models trained on codec-resynthesized audio fail on real codec-generated speech because they latch onto non-speech regions, especially boundary silence, and because they cannot handle content never seen in training. To fix this, the paper proposes SASTNet, which anchors acoustic analysis with semantic content embeddings while also refining codec fingerprints through class-specific masked reconstruction. On the CodecFake+ evaluation, SASTNet reports the highest F1 scores on the CoSG subset among the compared baselines for all three codec-attribute tracing tasks. If those results hold, useful source attribution is possible without large amounts of real codec-generated training data.

What carries the argument

The load-bearing object is the Semantic-Acoustic Source Tracing Network (SASTNet). It pairs a semantic encoder built from a frozen Whisper speech-recognition transformer with a coarse-to-fine acoustic encoder: a fine-tuned Wav2Vec2 2.0 XLS-R model feeds a multi-decoder AudioMAE, where each source class has its own masked-autoencoder decoder and the decoder that reconstructs best identifies the class. Two cross-modal transformer encoders let semantic features attend to acoustic features and vice versa, a fusion transformer combines the two conditioned maps, and an AASIST classifier outputs the source label. The semantic anchor forces the network to compare like content with like content; the multi-decoder reconstruction strips away noise and leaves the codec's residual fingerprint.

What would settle it

Re-run the Table II and Table III experiments with a fully specified voice-activity detector and a published threshold, then compare F1 with and without silence on the same utterances; if the large drop disappears or tracks speech-onset clipping rather than true silence, the paper's overfitting-to-silence diagnosis is not supported. A second check: train SASTNet on the same CoRS data after removing silence at training time, since the silence-overfitting story predicts little loss while a pure-architecture story predicts a large loss.

Watch

Extended reading notes

Core claim

The paper's central claim is that the generalization gap between codec-resynthesized (CoRS) and codec-generated (CoSG) speech can be substantially closed by joining two complementary signals: semantic content and codec-specific acoustic residuals. It first shows that baseline models trained only on CoRS drop roughly 27 F1 points when silent segments are removed from test audio and drop further on unseen utterance content, which it reads as overfitting to silence and to familiar words and speakers rather than to codec fingerprints. It then introduces SASTNet, whose frozen semantic encoder provides a content anchor so the acoustic branch must explain the same words in the same way, while a coarse-to-fine acoustic encoder (a self-supervised speech encoder followed by a multi-decoder masked autoencoder) reconstructs embeddings class by class to amplify the codec's own fingerprint. Evaluated on CodecFake+ CoSG (All), SASTNet improves F1 over the S-VQ, S-AUX, and S-DEC baselines by 12.9, 24.0, and 13.5 points respectively, and more than doubles baseline accuracy on unseen real speech in the confusion-matrix analysis.

Load-bearing premise

The central diagnostic result rests on an unstated silence-removal procedure: the paper reports that cutting the silent segments costs baseline models about 27 F1 points, but it never specifies the detector, threshold, or trimming method, so the drop attributed to silence could come partly from the trimming itself, for example by clipping speech onsets or leaving residual noise.

Editorial extensions

If this is right

If SASTNet's numbers hold, codec-based deepfake attribution is feasible from resynthesized-only training data, avoiding the need to collect large labeled sets of real codec-generated speech.
The silence diagnosis implies that any source-tracing model trained on untrimmed speech should either be trained or evaluated with silence handled explicitly, and that silence-aware preprocessing could close part of the CoRS-to-CoSG gap.
The semantic anchor should make the model less sensitive to unseen words, speakers, and languages than acoustic-only systems, since matching content is required before acoustic differences count as codec artifacts.
The gains on unseen real speech in CoSG (All), where accuracy improves by roughly 2.6-3.4 times over the baseline, suggest the same architecture may strengthen out-of-distribution bona fide detection, not just codec attribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next experiment the paper does not run: train SASTNet on CoRS with silence removed at training time and measure CoSG F1; if the gains vanish or shrink, the silence-overfitting story would need revision in favor of an architecture effect.
The same recipe, a content-invariant semantic anchor plus class-conditional reconstruction of embeddings, transfers in principle to other provenance tasks where content variation confounds source attribution, such as vocoder identification or camera-model attribution on images.
The reported correlation between silent-sample proportion and CoSG F1 (0.60-0.75) could be sharpened by an explicit regression on measured silence duration per utterance, turning a diagnosis into a predictive quantity, but that regression is not established in this paper.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: this is a useful empirical paper, not a conceptual breakthrough. The new piece is the combination of a semantic stream (Whisper) with a coarse-to-fine acoustic stream (W2V2 + multi-decoder AudioMAE), fused via cross-attention, for attributing codec-based synthetic speech to generator attributes. On the CodecFake+ CoSG test set, the gains over the authors' own baselines are substantial — about 13–24 F1 points across the three tasks. Those numbers, if they hold, make SASTNet the best published result on this benchmark, and the paper's diagnosis of silence overfitting is an interesting empirical claim.

What it does well: the problem is real and understudied; the experimental design with seen/unseen utterance splits is a reasonable way to study content generalization; the ablations cleanly isolate the contribution of each module; and the failure-case analysis is informative. I also appreciate the honesty about remaining gaps — the paper does not claim to solve CoSG generalization, just to improve on it.

The soft spots are real but not fatal. The most load-bearing is the silence experiment. The paper reports 'rm sil' drops of ~24–27 F1 points, but never specifies the silence detector, threshold, or trimming method. Without that, the drop could be partly an artifact of truncation — cutting into onsets or changing input length — rather than evidence of silence overfitting. The text also says 'the presence of silence has a detrimental effect' when the experiment actually shows the opposite: removing silence hurts performance, meaning the model relies on silence. That wording undermines the diagnostic even if the intended inference is right. The stress-test note is correct on this point.

Secondary issues: no error bars or multiple seeds; the baseline set contains only the authors' prior models, so the 'state-of-the-art' claim is narrower than stated; and the margin m=0.1 in the reconstruction loss is set from a single observed behavior, not from a sweep. These are minor in comparison.

Overall: the central architectural result is likely sound, and the silence-overfitting hypothesis is plausible but not fully supported as stated. The paper deserves a serious referee — the numbers are important enough that the community needs them checked. My recommendation: send to peer review, and ask for the silence-removal protocol, error bars, and at least one independent baseline before acceptance.

Referee Report

3 major / 5 minor

Summary. The paper addresses source tracing of neural-codec-based speech generation (CoSG) in the CodecFake+ dataset. The authors argue that models trained on codec-resynthesized speech (CoRS) overfit to silence and to seen content, and they propose SASTNet, which combines a frozen Whisper semantic encoder with a coarse Wav2Vec2 plus multi-decoder AudioMAE acoustic encoder and a cross-modal transformer/AASIST backend. On the three CodecFake+ attribute tasks (VQ, AUX, DEC), SASTNet reports F1 gains over the S-VQ/S-AUX/S-DEC baselines on CoSG (All) of 12.9, 24.0, and 13.5 points, respectively, and ablation experiments attribute part of the generalization gap to silence and unseen content.

Significance. If the results hold, SASTNet is the strongest published source-tracing method on the CodecFake+ CoSG benchmark, and the analysis of silence and unseen content would be a useful contribution to understanding CoRS-to-CoSG generalization. The paper is transparent about its intended code release, provides per-task and per-condition F1 tables, and includes confusion-matrix and attention-map analyses. The main caveat is that the central diagnostic claim relies on an unspecified silence-removal procedure and a partially reversed statement of the observed effect, so the explanation for the gains is not yet fully supported.

major comments (3)

[V-A1, Tables II and III] The 'rm sil' operation is never defined, yet it carries the paper's central diagnostic claim that models trained on CoRS 'overfit to non-speech regions.' Please specify the silence detector, threshold, and trimming method, and state whether only leading/trailing silence is removed or all silent segments are removed. Without this information, the 24-27 point F1 drops attributed to silence could equally result from a test-time input-length distribution shift or from cropping into speech onsets/offsets, and the conclusion is not reproducible. A length-matched control (for example, removing or trimming an equivalent number of non-silent frames, or replacing silence with noise) would separate the two effects.
[V-A1] The inference in this paragraph is stated in reverse. Tables II and III show that removing silence decreases F1 score; the supported statement is that models rely on silence, not that 'the presence of silence has a detrimental effect' on baseline performance. The abstract and conclusion repeat the overfitting claim, so the wording should be corrected and reconciled with the reported direction of the effect.
[IV-A and V-A2] The 'utt. ID <= 250' split used for the seen/unseen content experiments is not defined. In VCTK, utterance identifiers are of the form pXXX_NNN; please clarify whether the threshold applies to speaker IDs, utterance numbers, or something else, and state how disjointness of spoken content is guaranteed. As written, the unseen-content analysis is not reproducible.

minor comments (5)

[Tables I, III, IV] All reported results appear to come from single runs with no error bars or significance testing. The 2-4 point CoSG (All) differences in the ablations would be more persuasive with mean and standard deviation over at least three seeds, or with an explicit statement that only one seed was used.
[Figure 2] The sentence 'it can achieve perfect source-tracing F1 scores on samples with long silences as shown in Figure 2(b)' does not match the figure's displayed examples (UniAudio, USLM, MaskGCT); please clarify what is shown and how it supports the claim about long versus short silences.
[V-A1] The correlation analysis does not define how the 'proportion of silent samples' is computed; please specify the silence detection method and the aggregation unit (per CoSG model or per utterance) for the reported rho values.
[Table I and related text] The state-of-the-art claim is supported only against the authors' prior S-VQ/S-AUX/S-DEC baselines and internal MAE variants; if other published source-tracing systems can be adapted to the CodecFake+ protocol, a comparison would strengthen the claim.
[III-A] The phrase 'truncate s_pad to the first 256 non-padding time steps' should be defined more precisely, since the input is zero-padded to 30 seconds; please state how the end of the actual speech is detected or otherwise justify the fixed truncation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CoSG test F1 is never used to set training constants, and the self-cited taxonomy and dataset are external scaffolding rather than fitted inputs.

full rationale

The paper's core quantitative claims rest on Table I, where SASTNet is trained on CoRS labels and evaluated on the held-out CoSG test set of CodecFake+; the test-set F1 values are not used to tune the model or to select hyperparameters. The margin m=0.1 is chosen from the training-set behavior of the single-decoder AudioMAE reconstruction MSE (Section III-D), not from CoSG classification F1, so the reported CoSG results are not forced by construction. The taxonomy, task definitions, and S-VQ/S-AUX/S-DEC baselines are inherited from the same group's prior work [23] and the dataset from [22], but these are public, externally reproducible resources; using them as comparison points does not make the improvement circular. The diagnostic claim about silence overfitting is supported by the paper's own controlled comparisons in Section V-A (Tables II-III). Although the 'rm sil' operation is never specified (detector, threshold, or trimming method), that is a reproducibility and attribution weakness, not a circular reduction. No equation in the paper defines a claimed output in terms of the target metric, and no load-bearing argument reduces to an unverified self-citation. The 'state-of-the-art' wording is stronger than the comparison supports because only same-group baselines are tabulated, but that is an overclaim, not circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the public CodecFake+ dataset and taxonomy from the authors' prior work, on the suitability of Whisper as a content anchor, and on an underspecified silence-removal procedure. No new physical entities are introduced. The main hand-set constants are the reconstruction margin and the AudioMAE masking ratio.

free parameters (4)

margin m in reconstruction loss = 0.1
Set in Section III-D from the observation that single-decoder AudioMAE MSE converges to 0.02-0.05; it enforces a minimum separation between target and other decoder reconstruction errors, and is a hand-chosen constant rather than derived.
AudioMAE masking ratio = 0.4
Chosen in Section IV-A; no sensitivity analysis is provided, so the central results may depend on this choice.
Whisper truncation length = 256 time steps (128 after pooling)
Section III-A truncates Whisper features to the first 256 non-padding steps to align with acoustic features; the choice is motivated by time-span matching, but no ablation varies it.
Utterance ID threshold for seen/unseen split = 250
Section IV-A defines CoRS Train (utt. ID <= 250) to create seen/unseen splits; the threshold is arbitrary and the paper does not justify it.

assumptions (4)

domain assumption The VQ/AUX/DEC taxonomy from Chen et al. [23] is a sufficient and meaningful description of codec fingerprints for source tracing.
Section II adopts the taxonomy without validating that these three axes capture the distinguishing information in CoSG audio.
domain assumption Whisper's frozen encoder supplies content-robust semantic features even when inputs are zero-padded to 30 seconds and truncated to 256 time steps.
Section III-A relies on this to anchor acoustic features; no experiment validates the padding/truncation behavior of Whisper on short utterances.
domain assumption The silence-removal procedure ('rm sil') removes silence without removing codec-related acoustic content.
Section V-A and Tables II-III depend on this for the overfitting diagnosis, but the procedure is not specified.
domain assumption Self-supervised features (Wav2Vec2, AudioMAE) generalize better than mel-spectrograms across content and codec types.
Used to explain ablations in Table IV; supported only indirectly by the reported F1 numbers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Generalized Source Tracing for Codec-Based Deepfake Speech." pith.science (2026). https://pith.science/paper/CYJ5AYXJ

@misc{pith2026250607294,
  author       = {Pith},
  title        = {Pith review of: Towards Generalized Source Tracing for Codec-Based Deepfake Speech},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CYJ5AYXJ}},
  note         = {Machine review of arXiv:2506.07294}
}

read the original abstract

Recent attempts at source tracing for codec-based deepfake speech (CodecFake), generated by neural audio codec-based speech generation (CoSG) models, have exhibited suboptimal performance. However, how to train source tracing models using simulated CoSG data while maintaining strong performance on real CoSG-generated audio remains an open challenge. In this paper, we show that models trained solely on codec-resynthesized data tend to overfit to non-speech regions and struggle to generalize to unseen content. To mitigate these challenges, we introduce the Semantic-Acoustic Source Tracing Network (SASTNet), which jointly leverages Whisper for semantic feature encoding and Wav2vec2 with AudioMAE for acoustic feature encoding. Our proposed SASTNet achieves state-of-the-art performance on the CoSG test set of the CodecFake+ dataset, demonstrating its effectiveness for reliable source tracing.

Figures

Figures reproduced from arXiv: 2506.07294 by the authors.

**Figure 2.** The failure case study of codec-based deepfake source tracing. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 4.** The Semantic-Acoustic Interaction Attention Maps for Case Study. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 36 canonical work pages

[1]

A survey on speech deepfake detection,

M. Li, Y . Ahmadiadli, and X.-P. Zhang, “A survey on speech deepfake detection,” ACM Comput. Surv., vol. 57, no. 7, 2025

work page 2025
[2]

The defender's perspective on automatic speaker verification: An overview

H. Wu, J. Kang, L. Meng, H. Meng, and H.-y. Lee, “The defender’s perspective on automatic speaker verification: An overview,” arXiv preprint arXiv:2305.12804 , 2023

work page Pith review arXiv 2023
[3]

Asvspoof 2015: the first automatic speaker verification spoofing and countermea- sures challenge,

Z. Wu, T. Kinnunen, N. Evans, J. Yamagishi, C. Hanilc ¸i, M. Sahidullah, and A. Sizov, “Asvspoof 2015: the first automatic speaker verification spoofing and countermea- sures challenge,” in Proc. Interspeech, 2015

work page 2015
[4]

The asvspoof 2017 challenge: Assessing the limits of replay spoofing attack detection,

T. Kinnunen, M. Sahidullah, H. Delgado, M. Todisco, N. Evans, J. Yamagishi, and K. A. Lee, “The asvspoof 2017 challenge: Assessing the limits of replay spoofing attack detection,” in Proc. Interspeech, 2017

work page 2017
[5]

ASVspoof 2019: future horizons in spoofed and fake audio detection,

M. Todisco, X. Wang, V . Vestman, M. Sahidullah, H. Delgado, A. Nautsch, J. Yamagishi, N. Evans, T. H. Kinnunen, and K. A. Lee, “ASVspoof 2019: future horizons in spoofed and fake audio detection,” in Proc. Interspeech, 2019

work page 2019
[6]

Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,

X. Liu, X. Wang, M. Sahidullah, J. Patino, H. Del- gado, T. Kinnunen, M. Todisco, J. Yamagishi, N. Evans, A. Nautsch, and K. A. Lee, “Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,”IEEE Transactions on Audio, Speech and Language Process- ing, vol. 31, 2023

work page 2021
[7]

ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,

X. Wang, H. Delgado, H. Tak, J.-w. Jung, H.-j. Shim, M. Todisco et al. , “ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,” in Proc. ASVspoof Workshop, 2024

work page 2024
[8]

Add 2022: the first audio deep synthesis detection challenge,

J. Yi, R. Fu, J. Tao, S. Nie, H. Ma, C. Wang, T. Wang et al., “Add 2022: the first audio deep synthesis detection challenge,” in Proc. ICASSP, 2022

work page 2022

Show all 53 references

[9]

Add 2023: Towards audio deep- fake detection and analysis in the wild,

J. Yi, C. Y . Zhang, J. Tao, C. Wang, X. Yan, Y . Ren, H. Gu, and J. Zhou, “Add 2023: Towards audio deep- fake detection and analysis in the wild,” arXiv preprint arXiv:2408.04967, 2024

2023 arXiv
[10]

Synthetic speech detection through short- term and long-term prediction traces,

C. Borrelli, P. Bestagini, F. Antonacci, A. Sarti, and S. Tubaro, “Synthetic speech detection through short- term and long-term prediction traces,” EURASIP Journal on Information Security , vol. 2021, 2021

2021
[11]

An initial investigation for detecting vocoder fingerprints of fake audio,

X. Yan, J. Yi, J. Tao, C. Wang, H. Ma, T. Wang, S. Wang, and R. Fu, “An initial investigation for detecting vocoder fingerprints of fake audio,” Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia , 2022

2022
[12]

Distinguishing neural speech synthesis models through fingerprints in speech waveforms,

C. Y . Zhang, J. Yi, J. Tao, C. Wang, and X. Yan, “Distinguishing neural speech synthesis models through fingerprints in speech waveforms,” in China National Conference on Chinese Computational Linguistics , 2023

2023
[13]

Source tracing: detecting voice spoofing,

T. Zhu, X. Wang, X. Qin, and M. Li, “Source tracing: detecting voice spoofing,” in Proc. APSIPA ASC , 2022

2022
[14]

Source tracing of audio deepfake systems,

N. Klein, T. Chen, H. Tak, R. Casal, and E. Khoury, “Source tracing of audio deepfake systems,” in Proc. Interspeech, 2024

2024
[15]

On the landscape of spoken lan- guage models: A comprehensive survey,

S. Arora, K.-W. Chang, C.-M. Chien, Y . Peng, H. Wu, Y . Adi, E. Dupoux, H.-Y . Lee, K. Livescu, and S. Watanabe, “On the landscape of spoken lan- guage models: A comprehensive survey,” arXiv preprint arXiv:2504.08528, 2025

2025 arXiv
[16]

Neural codec source tracing: Toward comprehensive attribution in open-set condition,

Y . Xie, X. Wang, Z. Wang, R. Fu, Z. Wen, S. Cao, L. Ma, C. Li, H. Cheng, and L. Ye, “Neural codec source tracing: Toward comprehensive attribution in open-set condition,” arXiv preprint arXiv:2501.06514 , 2025

2025 arXiv
[17]

Towards explainable spoofed speech attribution and detection: a probabilistic approach for character- izing speech synthesizer components,

J. Mishra, M. Chhibber, H.-j. Shim, and T. H. Kin- nunen, “Towards explainable spoofed speech attribution and detection: a probabilistic approach for character- izing speech synthesizer components,” arXiv preprint arXiv:2502.04049, 2025

2025 arXiv
[18]

Source verification for speech deepfakes,

V . Negroni, D. Salvi, P. Bestagini, and S. Tubaro, “Source verification for speech deepfakes,” arXiv preprint arXiv:2505.14188, 2025

2025 arXiv
[19]

Listen, analyze, and adapt to learn new attacks: An exemplar-free class incremental learning method for audio deepfake source tracing,

Y . Xiao and R. K. Das, “Listen, analyze, and adapt to learn new attacks: An exemplar-free class incremental learning method for audio deepfake source tracing,” arXiv preprint arXiv:2505.14601 , 2025

2025 arXiv
[20]

Towards neural audio codec source parsing,

O. C. Phukan, M. M. Akhtar, A. B. Buduru, R. Sharma et al. , “Towards neural audio codec source parsing,” arXiv preprint arXiv:2506.12627 , 2025

2025 arXiv
[21]

CodecFake: En- hancing anti-spoofing models against deepfake audios from codec-based speech synthesis systems,

H. Wu, Y . Tseng, and H. yi Lee, “CodecFake: En- hancing anti-spoofing models against deepfake audios from codec-based speech synthesis systems,” in Proc. Interspeech, 2024

2024
[22]

Codec- Fake+: A large-scale neural audio codec-based deepfake speech dataset,

X. Chen, J. Du, H. Wu, L. Zhang, I. Lin, I. Chiu, W. Ren, Y . Tseng, Y . Tsao, J.-S. R. Jang, and H.-y. Lee, “Codec- Fake+: A large-scale neural audio codec-based deepfake speech dataset,” arXiv preprint arXiv:2501.08238 , 2025

2025 arXiv
[23]

Codec-based deepfake source trac- ing via neural audio codec taxonomy,

X. Chen, I. Lin, L. Zhang, J. Du, H. Wu, H.-y. Lee, J.-S. R. Jang et al. , “Codec-based deepfake source trac- ing via neural audio codec taxonomy,” arXiv preprint arXiv:2505.12994, 2025

2025 arXiv
[24]

Recent advances in discrete speech tokens: A review,

Y . Guo, Z. Li, H. Wang, B. Li, C. Shao, H. Zhang, C. Du, X. Chen, S. Liu, and K. Yu, “Recent advances in discrete speech tokens: A review,” arXiv preprint arXiv:2502.06490, 2025

2025
[25]

Towards au- dio language modeling-an overview,

H. Wu, X. Chen, Y .-C. Lin, K.-w. Chang, H.-L. Chung, A. H. Liu, and H.-y. Lee, “Towards au- dio language modeling-an overview,” arXiv preprint arXiv:2402.13236, 2024. 8

2024 arXiv
[26]

Codec-SUPERB: An in-depth analysis of sound codec models,

H. Wu, H.-L. Chung, Y .-C. Lin, Y .-K. Wu, X. Chen, Y .-C. Paiet al. , “Codec-SUPERB: An in-depth analysis of sound codec models,” in Findings Assoc. Comput. Linguist., 2024

2024
[27]

Codec-SUPERB@ SLT 2024: A lightweight benchmark for neural audio codec models,

H. Wu, X. Chen, Y .-C. Lin, K. Chang, J. Du, K.-H. Lu et al. , “Codec-SUPERB@ SLT 2024: A lightweight benchmark for neural audio codec models,” in Proc. IEEE Spoken Lang. Technol. Workshop , 2024

2024
[28]

Improved deepfake detection using whisper features,

P. Kawa, M. Plata, M. Czuba, P. Szyma ´nski, and P. Syga, “Improved deepfake detection using whisper features,” in Interspeech 2023, 2023

2023
[29]

Aasist: Audio anti- spoofing using integrated spectro-temporal graph atten- tion networks,

J.-w. Jung, H.-S. Heo, H. Tak, H.-j. Shim, J. S. Chung, B.-J. Lee, H.-J. Yu, and N. Evans, “Aasist: Audio anti- spoofing using integrated spectro-temporal graph atten- tion networks,” in ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing ...

2022
[30]

Robust speech recog- nition via large-scale weak supervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recog- nition via large-scale weak supervision,” 2022

2022
[31]

Xls-r: Self- supervised cross-lingual speech representation learning at scale,

A. Babu, C. Wang, A. Tjandra et al. , “Xls-r: Self- supervised cross-lingual speech representation learning at scale,” arXiv preprint arXiv:2111.09296 , 2021

2021 arXiv
[32]

Masked au- toencoders that listen,

P.-Y . Huang, H. Xu, J. Li, A. Baevski, M. Auli, W. Galuba, F. Metze, and C. Feichtenhofer, “Masked au- toencoders that listen,” Advances in Neural Information Processing Systems, vol. 35, pp. 28 708–28 720, 2022

2022
[33]

Mae: Masked autoencoders are scalable vision learners,

K. He, X. Chen, S. Xie, Y . Li, P. Doll´ar, and R. Girshick, “Mae: Masked autoencoders are scalable vision learners,” arXiv preprint arXiv:2111.06377 , 2021

2021 arXiv
[34]

Adversarial sample detection for speaker verification by neural vocoders,

H. Wu, P.-C. Hsu, J. Gao, S. Zhang, S. Huang, J. Kang, Z. Wu, H. Meng, and H.-Y . Lee, “Adversarial sample detection for speaker verification by neural vocoders,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2022

2022
[35]

Neural codec-based adversarial sample detection for speaker verification,

X. Chen, J. Du, H. Wu, J.-S. R. Jang, and H. yi Lee, “Neural codec-based adversarial sample detection for speaker verification,” in Interspeech 2024 , 2024

2024
[36]

To- wards out-of-distribution detection in vocoder recogni- tion via latent feature reconstruction,

R. Du, J. Yao, Q. Kong, and Y . Cao, “To- wards out-of-distribution detection in vocoder recogni- tion via latent feature reconstruction,” arXiv preprint arXiv:2406.02233, 2024

2024 arXiv
[37]

Push-pull: Characterizing the adversarial robustness for audio-visual active speaker detection,

X. Chen, H. Wu, H. Meng, H.-y. Lee, and J.-S. R. Jang, “Push-pull: Characterizing the adversarial robustness for audio-visual active speaker detection,” in 2022 IEEE Spoken Language Technology Workshop (SLT) , 2023

2022
[38]

Multimodal transformer distillation for audio- visual synchronization,

X. Chen, H. Wu, C.-C. Wang, H.-Y . Lee, and J.-S. R. Jang, “Multimodal transformer distillation for audio- visual synchronization,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP) , 2024

2024
[39]

Localizing audio- visual deepfakes via hierarchical boundary modeling,

X. Chen, S.-P. Cheng, J. Du et al. , “Localizing audio- visual deepfakes via hierarchical boundary modeling,” arXiv preprint arXiv:2508.02000 , 2025

2025 arXiv
[40]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural informa- tion processing systems , vol. 30, 2017

2017
[41]

Auxiliary tasks in multi-task learning,

L. Liebel and M. K ¨orner, “Auxiliary tasks in multi-task learning,” arXiv preprint arXiv:1805.06334 , 2018

2018 arXiv
[42]

Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation,

H. Tak, M. Todisco, X. Wang, J.-w. Jung, J. Yamagishi, and N. Evans, “Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation,” in Proc. Odyssey Speaker Lang. Recognit. Workshop, 2022

2022
[43]

Rawboost: A raw data boosting and augmentation method applied to automatic speaker verification anti- spoofing,

H. Tak, M. Kamble, J. Patino, M. Todisco, and N. Evans, “Rawboost: A raw data boosting and augmentation method applied to automatic speaker verification anti- spoofing,” in Proc. ICASSP, 2022

2022
[44]

Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92),

J. Yamagishi, C. Veaux, K. MacDonald et al. , “Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92),” Univ. of Edinburgh, The Centre for Speech Technology Research (CSTR) , 2019

2019
[45]

UniAudio: Towards universal audio gen- eration with large language models,

D. Yang, J. Tian, X. Tan, R. Huang, S. Liu, H. Guo, X. Chang, J. Shi, S. Zhao, J. Bian, Z. Zhao, X. Wu, and H. M. Meng, “UniAudio: Towards universal audio gen- eration with large language models,” in Proceedings of the 41st International Conference on Machine Learning , 2024

2024
[46]

MaskGCT: Zero-shot text-to-speech with masked generative codec transformer,

Y . Wang, H. Zhan, L. Liu, R. Zeng, H. Guo, J. Zheng et al., “MaskGCT: Zero-shot text-to-speech with masked generative codec transformer,” in Proc. ICLR, 2025

2025
[47]

The impact of silence on speech anti- spoofing,

Y . Zhang, Z. Li, J. Lu, H. Hua, W. Wang, and P. Zhang, “The impact of silence on speech anti- spoofing,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 3374–3389, 2023

2023
[48]

The effect of silence and dual-band fusion in anti-spoofing system,

Y . Zhang, W. Wang, and P. Zhang, “The effect of silence and dual-band fusion in anti-spoofing system,” in Proc. Interspeech, 2021, pp. 4279–4283

2021
[49]

Genuine-focused learning using mask autoencoder for generalized fake audio detection,

X. Wang, R. Fu, Z. Wen, Z. Wang, Y . Xie, Y . Liu, J. Tao, X. Liu, Y . Li, X. Qi et al. , “Genuine-focused learning using mask autoencoder for generalized fake audio detection,” arXiv preprint arXiv:2406.03247, 2024

2024 arXiv
[50]

Dynamic-superb: Towards a dynamic, collabo- rative, and comprehensive instruction-tuning benchmark for speech,

C.-y. Huang, K.-H. Lu, S.-H. Wang, C.-Y . Hsiao, C.-Y . Kuan, H. Wu, S. Arora, K.-W. Chang, J. Shi, Y . Peng et al. , “Dynamic-superb: Towards a dynamic, collabo- rative, and comprehensive instruction-tuning benchmark for speech,” in ICASSP 2024-2024 IEEE International Confere...

2024
[51]

Dynamic-SUPERB phase-2: A collaboratively expanding benchmark for measuring the capabilities of spoken language models with 180 tasks,

C.-Y . Huang et al. , “Dynamic-SUPERB phase-2: A collaboratively expanding benchmark for measuring the capabilities of spoken language models with 180 tasks,” in Proc. ICLR, 2025

2025
[52]

Towards holistic evaluation of large audio-language models: A compre- hensive survey,

C.-K. Yang, N. S. Ho, and H.-y. Lee, “Towards holistic evaluation of large audio-language models: A compre- hensive survey,” arXiv preprint arXiv:2505.15957 , 2025

2025 arXiv
[53]

A preliminary exploration with gpt-4o voice mode,

Y .-X. Lin, C.-K. Yang, W.-C. Chenet al., “A preliminary exploration with gpt-4o voice mode,” arXiv preprint arXiv:2502.09940, 2025

2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

A survey on speech deepfake detection,

M. Li, Y . Ahmadiadli, and X.-P. Zhang, “A survey on speech deepfake detection,” ACM Comput. Surv., vol. 57, no. 7, 2025

work page 2025

[2] [2]

The defender's perspective on automatic speaker verification: An overview

H. Wu, J. Kang, L. Meng, H. Meng, and H.-y. Lee, “The defender’s perspective on automatic speaker verification: An overview,” arXiv preprint arXiv:2305.12804 , 2023

work page Pith review arXiv 2023

[3] [3]

Asvspoof 2015: the first automatic speaker verification spoofing and countermea- sures challenge,

Z. Wu, T. Kinnunen, N. Evans, J. Yamagishi, C. Hanilc ¸i, M. Sahidullah, and A. Sizov, “Asvspoof 2015: the first automatic speaker verification spoofing and countermea- sures challenge,” in Proc. Interspeech, 2015

work page 2015

[4] [4]

The asvspoof 2017 challenge: Assessing the limits of replay spoofing attack detection,

T. Kinnunen, M. Sahidullah, H. Delgado, M. Todisco, N. Evans, J. Yamagishi, and K. A. Lee, “The asvspoof 2017 challenge: Assessing the limits of replay spoofing attack detection,” in Proc. Interspeech, 2017

work page 2017

[5] [5]

ASVspoof 2019: future horizons in spoofed and fake audio detection,

M. Todisco, X. Wang, V . Vestman, M. Sahidullah, H. Delgado, A. Nautsch, J. Yamagishi, N. Evans, T. H. Kinnunen, and K. A. Lee, “ASVspoof 2019: future horizons in spoofed and fake audio detection,” in Proc. Interspeech, 2019

work page 2019

[6] [6]

Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,

X. Liu, X. Wang, M. Sahidullah, J. Patino, H. Del- gado, T. Kinnunen, M. Todisco, J. Yamagishi, N. Evans, A. Nautsch, and K. A. Lee, “Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,”IEEE Transactions on Audio, Speech and Language Process- ing, vol. 31, 2023

work page 2021

[7] [7]

ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,

X. Wang, H. Delgado, H. Tak, J.-w. Jung, H.-j. Shim, M. Todisco et al. , “ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,” in Proc. ASVspoof Workshop, 2024

work page 2024

[8] [8]

Add 2022: the first audio deep synthesis detection challenge,

J. Yi, R. Fu, J. Tao, S. Nie, H. Ma, C. Wang, T. Wang et al., “Add 2022: the first audio deep synthesis detection challenge,” in Proc. ICASSP, 2022

work page 2022