Pith. sign in

REVIEW 2 major objections 5 minor 78 references

TokAN: Accent Normalization Using Self-Supervised Speech Tokens

T0 review · 2 major / 5 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TokAN converts non-native accented speech into native-like speech by translating self-supervised discrete tokens, cutting word error rate to 9.23% without real parallel L1–L2 recordings.

desk verdict Clean, well-executed journal extension that actually moves token-based accent normalization forward with joint VQ and GRPO; the gains are real and the checks are thorough. read the letter →

arxiv 2607.03928 v1 pith:GMWTKY3L submitted 2026-07-04 cs.SD cs.AIeess.AS

Qibing Bai , Shuai Wang , Yuhan Du , Bohan Li , Yannan Wang , Haizhou Li This is my paper
classification cs.SDcs.AIeess.AS
keywords accentconversionnormalizationdiscretespeechtokensvectorquantizationflowmatchingreinforcementlearningdurationcontrolself-supervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TokAN shows that accent normalization can be done by mapping speech into self-supervised discrete tokens and translating those tokens from non-native to native form. A jointly trained vector-quantization tokenizer already encodes strong phonetic content; an autoregressive encoder-decoder then rewrites L2 token sequences into L1 ones. Supervised fine-tuning on semi-synthetic token pairs lowers word error rate from a 12.40% baseline to 9.89%, and a subsequent reinforcement-learning stage that rewards lower word error and higher native-accent classifier scores pushes it to 9.23% while also producing the strongest subjective accent reduction among compared systems. A flow-matching synthesizer, optionally conditioned on total duration, recovers the waveform while keeping the source speaker, so the same pipeline can serve dubbing and live casting. A sympathetic reader cares because the method removes the need for scarce natural parallel L1–L2 recordings yet still improves both intelligibility and nativeness across seven English accents.

What carries the argument

The jointly trained VQ tokenizer (WavLM-Large layer 22, codebook 1024) co-optimized with flow-matching synthesis and CTC recognition; an accent-universal autoregressive encoder-decoder that rewrites L2 tokens into L1 tokens; GRPO post-training with complementary WER and accent-classifier rewards; and a flow-matching duration predictor that supports total-duration-aware synthesis.

What would settle it

On held-out real L2–L1 parallel recordings never used in training, measure phoneme-level divergence of TokAN output from true native targets; if that divergence does not fall relative to the source while the paper’s native-only ASR WER still drops, the claim of genuine phonetic normalization is falsified.

Watch

Extended reading notes

Core claim

Accent normalization can be performed as token-to-token translation over self-supervised discrete speech tokens from a jointly trained VQ tokenizer, without naturally recorded parallel L1–L2 speech. On seven English accents the system reduces native-only ASR word error rate from a strong baseline of 12.40% to 9.89% after supervised fine-tuning and further to 9.23% after GRPO post-training that optimizes word-error and accent-classifier rewards, while also achieving the lowest subjective accentedness and near-100% native-accent probability among the baselines tested.

Load-bearing premise

The method assumes that the jointly trained discrete tokenizer plus TTS-generated native targets strip away enough synthetic artifacts and duration mismatch that token-level training and automatic rewards truly learn phonetic accent normalization rather than residual synthesis patterns.

Editorial extensions

If this is right

  • Accent-normalization systems can be trained without scarce natural parallel L1–L2 recordings if semi-synthetic token pairs plus unpaired multi-accent RL are sufficient.
  • Task-level RL with WER and accent-classifier rewards can close the gap between next-token cross-entropy and the true goals of intelligibility and accent reduction.
  • Total-duration-aware flow-matching synthesis makes the same pipeline usable for dubbing and live casting where output length must match the source.
  • The modular split (tokenizer/synthesizer on native data, converter on semi-synthetic and unpaired data) lets each stage be scaled independently.
  • Because the tokenizer already performs partial accent reduction, conversion and RL mainly refine an already-improved representation rather than inventing nativeness from scratch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same token-plus-GRPO recipe may transfer to other scarce-data paralinguistic attributes such as emotion or speaking style.
  • The observed trade-off between accent reduction and speaker similarity suggests adding a speaker-similarity term to the GRPO reward could rebalance identity preservation without paired data.
  • As SSL tokenizers improve, the residual gain from conversion may shrink; tracking that residual would diagnose how much work is already done by the codebook itself.
  • Extending total-duration conditioning to real-time adaptive constraints could support live casting where length must track a changing source.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. TokAN is a modular accent-normalization pipeline that maps L2-accented speech to native-like speech via discrete self-supervised tokens. A WavLM-Large VQ tokenizer is jointly trained with a flow-matching Mel synthesizer and CTC ASR on native data; an accent-universal autoregressive encoder–decoder converts L2 token sequences to L1-like sequences after BART-style pre-training and semi-synthetic SFT; GRPO post-training then optimizes WER and accent-classifier rewards on unpaired multi-accent speech; a duration-aware flow-matching synthesizer recovers Mel spectrograms conditioned on the source speaker embedding. On L2-ARCTIC (seven English accents), TokAN-1 reports WER 9.23% (vs. CosyAccent-1 12.40%, FramAN 17.55%, VEVO 28.94%), lowest subjective accentedness, and highest L1-Prob, with ablations isolating pre-training, SFT, CTC, and GRPO contributions and a phonemic KL analysis of Mandarin-transfer corrections.

Significance. If the reported gains hold under independent reimplementation, TokAN is a solid systems contribution to reference-free foreign accent conversion: it combines a carefully selected jointly trained VQ tokenizer, an accent-universal token converter, the first reported GRPO post-training stage for accent conversion, and total-duration-aware synthesis useful for dubbing. Strengths include systematic tokenizer selection (Tables I–III), stage-wise ablations (Table VII), accent-wise WERs (Table V), MUSHRA/BWS with 95% CIs, phonemic distribution analysis (Fig. 5), public samples and code, and modular training that separates native synthesis data from conversion data. The work is a substantial extension of the authors’ Interspeech 2025 precursor rather than a wholly new paradigm, but the empirical package is complete enough for a journal archive of the method.

major comments (2)
  1. [Abstract; Sec. V-B; Sec. VIII-A] Abstract vs. body on synthetic supervision: the abstract states that TokAN operates “without the need of synthetic supervisory speech,” yet Sec. V-B and VIII-A explicitly fine-tune on semi-synthetic pairs whose native targets are TTS-synthesized (Matcha-TTS / L2-LibriTTSR). Token-level supervision reduces the impact of cloning artifacts (as argued), but synthetic targets are still used for SFT. Please revise the abstract (and any parallel claims in the introduction) to state accurately that natural parallel L1–L2 recordings are not required, while semi-synthetic / TTS targets are used for SFT and unpaired real speech for GRPO.
  2. [Abstract; Sec. IV] Abstract vs. body on the tokenizer: the abstract describes “a L1–L2 jointly trained vector-quantization (VQ) tokenizer,” but Sec. IV trains the VQ codebook jointly with the synthesizer and CTC recognizer on native English only (LibriTTS-R / Emilia-EN). L2 speech enters later via conversion SFT and RL. Correct the abstract phrasing to “jointly trained VQ tokenizer (with synthesis and ASR objectives on native speech)” so the contribution is not overstated.
minor comments (5)
  1. [Table IV; Sec. IX-B] Table IV SIM column: Resynthesis is strongly positive (0.417) while all conversion systems are near zero or negative under BWS counting; a one-sentence note in Sec. IX-B that BWS is relative within each case (and thus can be negative) would help readers not familiar with the scoring.
  2. [Sec. VI-C; Sec. VIII-C] Sec. VI-C / evaluation metrics: GRPO reward uses only US+England accent labels, while L1-Prob evaluation sums five native labels. The text already notes the difference; consider stating the two label sets once in a single place (e.g., Sec. VIII-C) to avoid rereading.
  3. [Fig. 4; Table VI; Sec. IV-C] Fig. 4 caption and surrounding text refer to “TokAN-1 with direct duration scaling” vs. TokAN-2; Table VI uses the same distinction. Ensure the main text always uses the TokAN-1 / TokAN-2 definitions from Sec. IV-C so the figure is self-contained.
  4. [Sec. II; Sec. VIII-B] Related work and baselines: CosyAccent and the conference precursor share first authorship; a brief explicit statement that CosyAccent is a concurrent/related system from the same group (and that training data overlap is controlled) would improve transparency for readers.
  5. [References; throughout] Minor typos / polish: “V oice” / “V oice-preserving” spacing artifacts in references; “arXiv:2607.03928v1” date line is fine for preprint but should be cleaned for the journal version; ensure consistent hyphenation of “self-attention-only” and “total-duration-aware.”

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical ML system whose WER/L1-Prob gains are measured by external models on held-out data, not forced by construction or self-citation.

full rationale

TokAN is an engineering paper whose central claims are empirical performance numbers (WER drop from 12.40% to 9.23%, lowest ACT, highest L1-Prob) obtained by training a VQ tokenizer + AR converter + flow-matching synthesizer on public corpora (LibriTTS-R, Emilia-EN, L2-LibriTTSR, L2-ARCTIC, GLOBE) and evaluating with independent native-only ASR (s2t-medium-librispeech-asr / Whisper) and an off-the-shelf accent classifier. The tokenizer selection (Tables I–III), SFT objective (Eq. 9), GRPO rewards (Eqs. 14–16), and ablations (Table VII) are all open-loop with respect to the final test metrics; nothing equates a reported quantity to a fitted parameter or to a definition. Self-citations of the Interspeech 2025 precursor [25] and of CosyAccent [28] (overlapping authors) merely locate the work relative to prior systems that are treated as baselines and outperformed; they do not supply a uniqueness theorem or close the evaluation loop. Duration control, phonemic KL analysis (Fig. 5), and speaker-similarity trade-offs are likewise independent observations. Consequently the derivation chain contains no self-definitional step, no fitted-input-as-prediction, and no load-bearing self-citation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The paper is an empirical systems paper. Its claims rest on standard SSL and flow-matching machinery plus a collection of hand-chosen hyperparameters and modeling choices that are not derived from first principles. No new physical or mathematical entities are postulated; the free parameters and domain assumptions listed below are the main non-data ingredients that must hold for the reported gains to transfer.

free parameters (5)
  • VQ codebook size V = 1024
    Selected by reconstruction WER sweep (Table II); final value 1024 is a free design choice that directly affects token expressiveness and normalization.
  • Joint-training loss weights λ1,λ2,λ3 = 2.0 / 2.0 / 0.5
    Set to 2.0, 2.0, 0.5 without theoretical derivation; control the balance among VQ, synthesis and CTC objectives.
  • GRPO reward weights w_WER, w_Acc = 1.0 / 0.5
    Hand-set to 1.0 and 0.5; directly shape the post-training objective that produces the final 9.23% WER.
  • GRPO group size G, top-p, top-k, KL weight = 12 / 0.85 / 25 / 0.02
    G=12, p=0.85, k=25, λ_kl=0.02 chosen for training stability; affect exploration and policy deviation.
  • CFG strengths w1,w2,w3 and Euler steps = 1.0 / 1.0 / 0.1 / 32
    Inference-time guidance and sampler steps (32) are free knobs that influence naturalness and duration fidelity.
assumptions (4)
  • domain assumption Self-supervised discrete tokens from a deep WavLM layer primarily encode phonetic content while largely factoring out speaker and duration.
    Stated in Sec. I and IV; underpins the entire token-conversion strategy. Supported by prior literature but not re-proven here.
  • domain assumption Semi-synthetic token pairs (real L2 + TTS native) are sufficiently free of voice-cloning and prosodic artifacts once quantized that cross-entropy training yields genuine accent mapping.
    Core justification for avoiding natural parallel data (Sec. V-B); if false, reported gains partly reflect synthesis cleanup rather than accent normalization.
  • domain assumption Native-only ASR WER and accent-classifier L1 probability are faithful proxies for human intelligibility and accentedness.
    Used both as GRPO rewards and as primary evaluation metrics (Sec. VI-C, VIII-C); circularity risk is low but proxy validity is assumed.
  • standard math Standard Transformer + RoPE + flow-matching + GRPO mathematics hold without modification.
    Background machinery imported from cited works; no novel proofs required.
invented entities (1)
  • TokAN pipeline (joint VQ tokenizer + accent-universal AR converter + GRPO + duration-aware FM synthesizer) independent evidence
    purpose: End-to-end accent normalization system that operates purely at the discrete-token level and supports total-duration control.
    The named system is the paper's central engineering contribution; it is a composition of existing components rather than a new physical or mathematical object. Independent evidence is the public code and samples, not an external physical prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TokAN: Accent Normalization Using Self-Supervised Speech Tokens." pith.science (2026). https://pith.science/paper/GMWTKY3L

@misc{pith2026260703928,
  author       = {Pith},
  title        = {Pith review of: TokAN: Accent Normalization Using Self-Supervised Speech Tokens},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GMWTKY3L}},
  note         = {Machine review of arXiv:2607.03928}
}
read the original abstract

Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity. The current techniques either require naturally recorded parallel L1-L2 speech for training, or suffer from quality degradation when supervised by synthesized targets. In this paper, we present TokAN, a token-based accent normalization framework that operates on self-supervised discrete speech tokens extracted from a L1-L2 jointly trained vector-quantization (VQ) tokenizer, without the need of synthetic supervisory speech. An autoregressive encoder-decoder model performs token-to-token conversion, translating L2-accented token sequences into the tokens of standard voice. We also introduce reinforcement learning (RL) post-training based on Group Relative Policy Optimization (GRPO), using word error rate and accent classifier confidence as complementary rewards. A non-autoregressive flow-matching synthesizer recovers the Mel-spectrogram from the converted tokens, conditioned on the source speaker embedding. We also develop a flow-matching duration predictor that supports total-duration-aware synthesis, making TokAN applicable to duration-critical tasks such as voice dubbing and live casting. Experiments on seven English accents demonstrate that TokAN reduced the word error rate from 12.40% to 9.89% after supervised fine-tuning, and further to 9.23% after RL post-training, consistently outperforming frame-to-frame, direct flow-matching, and prompt-based token-conversion baselines in terms of accent reduction and intelligibility.

Figures

Figures reproduced from arXiv: 2607.03928 by the authors.

Figure 1
Figure 1. Overview of TokAN. (a) The SSL tokenizer encodes L2-accented speech into discrete tokens via a jointly trained VQ codebook. (b) The autoregressive [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Architecture of the jointly trained SSL tokenizer and speech syn [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture of the token converter. The encoder consumes source [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Token-frame alignment comparison. Left: TokAN-1 with direct [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Token-based phoneme divergence from native US English targets. Each cell shows the KL divergence between the token distribution of a given phoneme [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

78 extracted references · 1 canonical work pages

  1. [1]

    Foreign accent conversion by synthesizing speech from phonetic posteriorgrams

    G. Zhao, S. Ding, and R. Gutierrez-Osuna, “Foreign accent conversion by synthesizing speech from phonetic posteriorgrams.” inProc. Inter- speech, 2019, pp. 2843–2847

    2019

  2. [2]

    Foreign accent conver- sion in computer assisted pronunciation training,

    D. Felps, H. Bortfeld, and R. Gutierrez-Osuna, “Foreign accent conver- sion in computer assisted pronunciation training,”Speech communica- tion, vol. 51, no. 10, pp. 920–932, 2009

    2009

  3. [3]

    Subband based voice conversion

    O. T ¨urk and L. M. Arslan, “Subband based voice conversion.” inProc. Interspeech, 2002, pp. 289–292

    2002

  4. [4]

    Personalized, cross- lingual tts using phonetic posteriorgrams

    L. Sun, H. Wang, S. Kang, K. Li, and H. M. Meng, “Personalized, cross- lingual tts using phonetic posteriorgrams.” inProc. Interspeech, 2016, pp. 322–326

    2016

  5. [5]

    Accent conversion using phonetic posteriorgrams,

    Z. Guanlong, S. Sinem, L. John, C.-H. Evgeny, and G.-O. Ricardo, “Accent conversion using phonetic posteriorgrams,” inProc. ICASSP, 2018, pp. 5314–5318

    2018

  6. [6]

    Improving accent conversion with reference encoder and end-to-end text-to-speech,

    W. Li, B. Tang, X. Yin, Y . Zhao, W. Li, K. Wang, H. Huang, Y . Wang, and Z. Ma, “Improving accent conversion with reference encoder and end-to-end text-to-speech,”arXiv preprint arXiv:2005.09271, 2020

    arXiv 2005

  7. [7]

    Accentron: Foreign accent conversion to arbitrary non-native speakers using zero-shot learning,

    S. Ding, G. Zhao, and R. Gutierrez-Osuna, “Accentron: Foreign accent conversion to arbitrary non-native speakers using zero-shot learning,” Computer Speech & Language, vol. 72, p. 101302, 2022

    2022

  8. [8]

    Vevo: Control- lable zero-shot voice imitation with self-supervised disentanglement,

    X. Zhang, X. Zhang, K. Peng, Z. Tang, V . Manohar, Y . Liu, J. Hwang, D. Li, Y . Wang, J. Chan, Y . Huang, Z. Wu, and M. Ma, “Vevo: Control- lable zero-shot voice imitation with self-supervised disentanglement,” in ICLR, 2025

    2025

Show all 78 references
  1. [9]

    Converting foreign accent speech without a reference,

    G. Zhao, S. Ding, and R. Gutierrez-Osuna, “Converting foreign accent speech without a reference,”TASLP, vol. 29, pp. 2367–2381, 2021

    2021

  2. [10]

    Accent conversion using pre-trained model and synthesized data from voice conversion

    T.-N. Nguyen, N.-Q. Pham, and A. Waibel, “Accent conversion using pre-trained model and synthesized data from voice conversion.” inProc. Interspeech, 2022, pp. 2583–2587

    2022

  3. [11]

    Zero-shot foreign accent conversion without a native reference,

    W. Quamer, A. Das, J. Levis, E. Chukharev-Hudilainen, and R. Gutierrez-Osuna, “Zero-shot foreign accent conversion without a native reference,” inProc. Interspeech, 2022, pp. 4920–4924

    2022

  4. [12]

    End-to-end accent conversion without using native utterances,

    S. Liu, D. Wang, Y . Cao, L. Sun, X. Wu, S. Kang, Z. Wu, X. Liu, D. Su, D. Yuet al., “End-to-end accent conversion without using native utterances,” inProc. ICASSP, 2020, pp. 6289–6293

    2020

  5. [13]

    V oice- preserving zero-shot multiple accent conversion,

    M. Jin, P. Serai, J. Wu, A. Tjandra, V . Manohar, and Q. He, “V oice- preserving zero-shot multiple accent conversion,” inProc. ICASSP, 2023

    2023

  6. [14]

    Tts-guided training for accent conversion without parallel data,

    Y . Zhou, Z. Wu, M. Zhang, X. Tian, and H. Li, “Tts-guided training for accent conversion without parallel data,”Signal Processing Letters, vol. 30, pp. 533–537, 2023

    2023

  7. [15]

    Transfer the linguistic representations from tts to accent conversion with non-parallel data,

    X. Chen, J. Pei, L. Xue, and M. Zhang, “Transfer the linguistic representations from tts to accent conversion with non-parallel data,” inProc. ICASSP, 2024

    2024

  8. [16]

    Diffusion-based method with tts guidance for foreign accent conver- sion,

    Q. Bai, S. Wang, Z. Liu, M. Zhang, W. Rao, Y . Wang, and H. Li, “Diffusion-based method with tts guidance for foreign accent conver- sion,” inProc. ISCSLP, 2024, pp. 284–288

    2024

  9. [17]

    Improving pronunciation and accent conversion through knowledge distillation and synthetic ground-truth from native tts,

    T. N. Nguyen, S. Akti, N. Q. Pham, and A. Waibel, “Improving pronunciation and accent conversion through knowledge distillation and synthetic ground-truth from native tts,” inICASSP, 2025

    2025

  10. [18]

    Convert and speak: Zero-shot accent conversion with minimum supervision,

    Z. Jia, H. Xue, X. Peng, and Y . Lu, “Convert and speak: Zero-shot accent conversion with minimum supervision,” inMultimedia, 2024

    2024

  11. [19]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”TASLP, vol. 29, pp. 3451–3460, 2021

    2021

  12. [20]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,”J-STSP, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  13. [21]

    Self-supervised speech representations are more phonetic than semantic,

    K. Choi, A. Pasad, T. Nakamura, S. Fukayama, K. Livescu, and S. Watanabe, “Self-supervised speech representations are more phonetic than semantic,” inProc. Interspeech, 2024

    2024

  14. [22]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”Transactions on Machine Learning Research, 2023

    2023

  15. [23]

    Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

    Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Maet al., “Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,”arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  16. [24]

    Accent conversion using discrete units with parallel data synthesized from controllable accented tts,

    T.-N. Nguyen, Q. Pham, and A. Waibel, “Accent conversion using discrete units with parallel data synthesized from controllable accented tts,” inSynthetic Data’s Transformative Role in Foundational Speech Models, 2024, pp. 51–55

    2024

  17. [25]

    Accent normalization using self-supervised discrete tokens with non-parallel data,

    Q. Bai, S. Inoue, S. Wang, Z. Jiang, Y . Wang, and H. Li, “Accent normalization using self-supervised discrete tokens with non-parallel data,” inInterspeech 2025, 2025, pp. 1618–1622

    2025

  18. [26]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . K. Li, Y . Wu, and D. Guo, “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,” Feb. 2024. [Online]. Available: https://arxiv.org/abs/2402.03300

    2024 arXiv

  19. [27]

    L2-ARCTIC: A Non-native English Speech Corpus,

    G. Zhao, S. Sonsaat, A. Silpachai, I. Lucic, E. Chukharev-Hudilainen, J. Levis, and R. Gutierrez-Osuna, “L2-ARCTIC: A Non-native English Speech Corpus,” inProc. Interspeech, 2018

    2018

  20. [28]

    Cosyaccent: Duration-controllable accent normalization using source-synthesis train- ing data,

    Q. Bai, S. Shi, S. Wang, Y . Ju, Y . Wang, and H. Li, “Cosyaccent: Duration-controllable accent normalization using source-synthesis train- ing data,”Proc. ICASSP 2026, 2026

    2026

  21. [29]

    Evaluating methods for ground-truth-free foreign accent conversion,

    W.-C. Huang and T. Toda, “Evaluating methods for ground-truth-free foreign accent conversion,” inAPSIPA ASC, 2023

    2023

  22. [30]

    Fac- facodec: Controllable zero-shot foreign accent conversion with factor- ized speech codec,

    Y . Halychanskyi, C. Churchwell, Y . Wen, and V . Kindratenko, “Fac- facodec: Controllable zero-shot foreign accent conversion with factor- ized speech codec,”Proc. ICASSP 2026, 2026

    2026

  23. [31]

    Any-to-one sequence-to- sequence voice conversion using self-supervised discrete speech rep- resentations,

    W.-C. Huang, Y .-C. Wu, and T. Hayashi, “Any-to-one sequence-to- sequence voice conversion using self-supervised discrete speech rep- resentations,” inProc. ICASSP, 2021

    2021

  24. [32]

    Speak, read and prompt: High-fidelity text-to-speech with minimal supervision,

    E. Kharitonov, D. Vincent, Z. Borsos, R. Marinier, S. Girgin, O. Pietquin, M. Sharifi, M. Tagliasacchi, and N. Zeghidour, “Speak, read and prompt: High-fidelity text-to-speech with minimal supervision,”Trans. ACL, vol. 11, pp. 1703–1718, 2023

    2023

  25. [33]

    On generative spoken language modeling from raw audio,

    K. Lakhotia, E. Kharitonov, W.-N. Hsu, Y . Adi, A. Polyak, B. Bolte, T.- A. Nguyen, J. Copet, A. Baevski, A. Mohamedet al., “On generative spoken language modeling from raw audio,”Trans. ACL, vol. 9, pp. 1336–1354, 2021

    2021

  26. [34]

    Direct speech-to- speech translation with discrete units,

    A. Lee, P.-J. Chen, C. Wang, J. Gu, S. Popuri, X. Ma, A. Polyak, Y . Adi, Q. He, Y . Tang, J. Pino, and W.-N. Hsu, “Direct speech-to- speech translation with discrete units,” inProc. ACL, 2022

    2022

  27. [35]

    Llama-omni: Seamless speech interaction with large language models,

    Q. Fang, S. Guo, Y . Zhou, Z. Ma, S. Zhang, and Y . Feng, “Llama-omni: Seamless speech interaction with large language models,”arXiv preprint arXiv:2409.06666, 2024

    2024 arXiv

  28. [36]

    W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training,

    Y .-A. Chung, Y . Zhang, W. Han, C.-C. Chiu, J. Qin, R. Pang, and Y . Wu, “W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training,” inASRU, 2021, pp. 244–250

    2021

  29. [37]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models,

    Z. Du, Y . Wang, Q. Chen, X. Shi, X. Lv, T. Zhao, Z. Gao, Y . Yang, C. Gao, H. Wanget al., “Cosyvoice 2: Scalable streaming speech synthesis with large language models,”arXiv preprint arXiv:2412.10117, 2024

    2024 arXiv

  30. [38]

    Flow matching for generative modeling,

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” inICLR, 2023

    2023

  31. [39]

    Matcha-tts: A fast tts architecture with conditional flow matching,

    S. Mehta, R. Tu, J. Beskow, ´E. Sz ´ekely, and G. E. Henter, “Matcha-tts: A fast tts architecture with conditional flow matching,” inProc. ICASSP, 2024, pp. 11 341–11 345

    2024

  32. [40]

    V oicebox: Text- guided multilingual universal speech generation at scale,

    M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V . Manohar, Y . Adi, J. Mahadeokar, and W.-N. Hsu, “V oicebox: Text- guided multilingual universal speech generation at scale,” inAdvances in NeurIPS, 2023, pp. 14 005–14 034

    2023

  33. [41]

    Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,

    Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, Y . Liu, Y . Leng, K. Song, S. Tanget al., “Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,”arXiv preprint arXiv:2403.03100, 2024

    2024 arXiv

  34. [42]

    Total- duration-aware duration modeling for text-to-speech systems,

    S. E. Eskimez, X. Wang, M. Thakker, C.-H. Tsai, C. Li, Z. Xiao, H. Yang, Z. Zhu, M. Tang, J. Li, S. Zhao, and N. Kanda, “Total- duration-aware duration modeling for text-to-speech systems,” inProc. Interspeech, 2024

    2024

  35. [43]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions...

    2022

  36. [44]

    Group relative policy optimization for speech recognition,

    P. G. Shivakumar, Y . Gu, A. Gandhe, and I. Bulyko, “Group relative policy optimization for speech recognition,” in2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2025

    2025

  37. [45]

    Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability,

    R. Liu, B. Sisman, and H. Li, “Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability,” in Interspeech 2021, 2021, pp. 4648–4652

    2021

  38. [46]

    Dmospeech 2: Reinforcement learning for duration prediction in metric-optimized speech synthesis,

    Y . A. Li, X. Jiang, F. Tao, C. Niu, K. Xu, J. Song, and N. Mesgarani, “Dmospeech 2: Reinforcement learning for duration prediction in metric-optimized speech synthesis,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 38, 2026, pp. 31 814– 31 822...

    2026 doi

  39. [47]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  40. [48]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in NeurIPS, vol. 30, 2017

    2017

  41. [49]

    Roformer: En- hanced transformer with rotary position embedding,

    J. Su, M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “Roformer: En- hanced transformer with rotary position embedding,”Neurocomputing, vol. 568, p. 127063, 2024

    2024

  42. [50]

    Hiftnet: A fast high- quality neural vocoder with harmonic-plus-noise filter and inverse short time fourier transform,

    Y . A. Li, C. Han, X. Jiang, and N. Mesgarani, “Hiftnet: A fast high- quality neural vocoder with harmonic-plus-noise filter and inverse short time fourier transform,”arXiv preprint arXiv:2309.09493, 2023

    2023 arXiv

  43. [51]

    Neural discrete representation learning,

    A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural discrete representation learning,” inProceedings of the 31st International Con- ference on Neural Information Processing Systems, 2017, p. 6309–6318

    2017

  44. [52]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” inProc. ICCV, 2023, pp. 4195–4205

    2023

  45. [53]

    Film: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “Film: Visual reasoning with a general conditioning layer,” inProceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018

    2018

  46. [54]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” inNeurIPS 2021 Workshop on Deep Generative Models and Downstream Applica- tions, 2021

    2021

  47. [55]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449– 12 460, 2020

    2020

  48. [56]

    LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus,

    Y . Koizumi, H. Zen, S. Karita, Y . Ding, K. Yatabe, N. Morioka, M. Bacchiani, Y . Zhang, W. Han, and A. Bapna, “LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus,” inProc. Interspeech, 2023

    2023

  49. [57]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,

    H. He, Z. Shang, C. Wang, X. Li, Y . Gu, H. Hua, L. Liu, C. Yang, J. Li, P. Shiet al., “Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,” in2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 885–890

    2024

  50. [58]

    Connection- ist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connection- ist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” inICML, 2006

    2006

  51. [59]

    Bart: Denoising sequence-to-sequence pre-training for natu- ral language generation, translation, and comprehension,

    M. Lewis, “Bart: Denoising sequence-to-sequence pre-training for natu- ral language generation, translation, and comprehension,”arXiv preprint arXiv:1910.13461, 2019

    1910 arXiv

  52. [60]

    DAPO: An open-source LLM reinforcement learning system at scale,

    Q. Yu, Z. Zhang, R. Zhu, Y . Yuan, X. Zuo, YuYue, W. Dai, T. Fan, G. Liu, J. Liu, L. Liu, X. Liu, H. Lin, Z. Lin, B. Ma, G. Sheng, Y . Tong, C. Zhang, M. Zhang, R. Zhang, W. Zhang, H. Zhu, J. Zhu, J. Chen, J. Chen, C. Wang, H. Yu, Y . Song, X. Wei, H. Zhou, J. Liu, W.-Y . Ma, ...

    2026

  53. [61]

    Robust speech recognition via large-scale weak super- vision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak super- vision,” inICML, 2023, pp. 28 492–28 518

    2023

  54. [62]

    Com- monAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common V oice,

    J. Zuluaga-Gomez, S. Ahmed, D. Visockas, and C. Subakan, “Com- monAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common V oice,” inInterspeech 2023, 2023, pp. 5291–5295

    2023

  55. [63]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” inProceedings of the Twelfth Language Resources and Evaluation Conference. Marseille, France: European Lan...

    2020

  56. [64]

    GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speech,

    W. Wang, Y . Song, and S. Jha, “GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speech,” inInterspeech 2024, 2024, pp. 1365–1369

    2024

  57. [65]

    The cmu arctic speech databases,

    J. Kominek and A. W. Black, “The cmu arctic speech databases,” in Fifth ISCA workshop on speech synthesis, 2004

    2004

  58. [66]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” inICLR, 2021

    2021

  59. [67]

    Bfa: Real- time multilingual text-to-speech forced alignment,

    A. Rehman, J. Cai, J.-J. Zhang, and X. Yang, “Bfa: Real- time multilingual text-to-speech forced alignment,” 2025. [Online]. Available: https://arxiv.org/abs/2509.23147

    2025

  60. [68]

    J. J. Louviere, T. N. Flynn, and A. A. J. Marley,Best-worst scaling: Theory, methods and applications. Cambridge University Press, 2015

    2015

  61. [69]

    A comparison of best-worst scaling and rating scale for timbre characterisation,

    A. M. V . Ravillion, “A comparison of best-worst scaling and rating scale for timbre characterisation,” 2020

    2020

  62. [70]

    The t05 system for the VoiceMOS Challenge 2024: Transfer learning from deep image classifier to naturalness MOS prediction of high-quality synthetic speech,

    K. Baba, W. Nakata, Y . Saito, and H. Saruwatari, “The t05 system for the VoiceMOS Challenge 2024: Transfer learning from deep image classifier to naturalness MOS prediction of high-quality synthetic speech,” inIEEE Spoken Language Technology Workshop (SLT), 2024, pp. 818–824

    2024

  63. [71]

    ECAPA-TDNN: Emphasized channel attention, propagation and aggregation in TDNN based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “ECAPA-TDNN: Emphasized channel attention, propagation and aggregation in TDNN based speaker verification,” inProc. Interspeech, 2020, pp. 3830–3834

    2020

  64. [72]

    High-fidelity neural phonetic posteriorgrams,

    C. Churchwell, M. Morrison, and B. Pardo, “High-fidelity neural phonetic posteriorgrams,” inICASSP 2024 Workshop on Explainable Machine Learning for Speech and Audio, 2024

    2024

  65. [73]

    Exploring ssl discrete tokens for multilingual asr,

    M. Cui, D. Tan, Y . Yang, D. Wang, H. Wang, X. Chen, X. Chen, and X. Liu, “Exploring ssl discrete tokens for multilingual asr,” inProc. ICASSP 2025, 2025

    2025

  66. [74]

    Towards universal speech discrete tokens: A case study for asr and tts,

    Y . Yang, F. Shen, C. Du, Z. Ma, K. Yu, D. Povey, and X. Chen, “Towards universal speech discrete tokens: A case study for asr and tts,” in2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 10 401–10 405

    2024

  67. [75]

    Exploring speech recognition, translation, and understanding with discrete speech units: A comparative study,

    X. Chang, B. Yan, K. Choi, J.-W. Jung, Y . Lu, S. Maiti, R. Sharma, J. Shi, J. Tian, S. Watanabe, Y . Fujita, T. Maekaku, P. Guo, Y .-F. Cheng, P. Denisov, K. Saijo, and H.-H. Wang, “Exploring speech recognition, translation, and understanding with discrete speech units: A com...

    2024

  68. [76]

    Codecmos-accent: A mos benchmark of resynthesized and tts speech from neural codecs across english accents,

    W.-C. Huang, N. Sanders, and E. Cooper, “Codecmos-accent: A mos benchmark of resynthesized and tts speech from neural codecs across english accents,”arXiv preprint arXiv:2603.14328, 2026

    2026

  69. [77]

    Montreal forced aligner: Trainable text-speech alignment using kaldi

    M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, “Montreal forced aligner: Trainable text-speech alignment using kaldi.” inProc. Interspeech, 2017, pp. 498–502

    2017

  70. [78]

    Duanmu,The Phonology of Standard Chinese, 2nd ed

    S. Duanmu,The Phonology of Standard Chinese, 2nd ed. Oxford: Oxford University Press, 2007

    2007

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.