Pith. sign in

REVIEW 4 major objections 5 minor 45 references

Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Streaming accent conversion now matches offline quality at a stable 0.8-second latency, the paper claims, by processing speech in small chunks with no padding and training on native-TTS-generated ideal ground truth.

desk verdict A credible streaming adaptation of the authors' own non-streaming accent conversion model with comparable quality, but the 'ideal ground truth' premise and ASR-selected test set keep it from being fully established. read the letter →

arxiv 2506.16580 v1 pith:AYNF3G2D submitted 2025-06-19 cs.CL cs.SDeess.AS

Tuan-Nam Nguyen , Ngoc-Quan Pham , Seymanur Akti , Alexander Waibel This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords streamingaccentconversionpronunciationimprovementEmformernon-autoregressivespeechsynthesisnativeTTSgroundtruthchunkedinferencelatencyknowledgedistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that accent conversion and pronunciation improvement can be done in a streaming fashion without sacrificing the quality of the best offline systems. The authors modify a previous non-autoregressive accent conversion model by replacing its Transformer encoder with an Emformer, processing audio in 0.08-second chunks with a controlled look-ahead, and caching internal states so that streaming output is identical to full-segment output. They also use a native text-to-speech model to synthesize 'ideal' ground truth for non-native speech, preserving the original speaker's duration, prosody, and identity while delivering native pronunciation. If correct, this would be the first accent conversion system that works in live settings like video conferencing, with stable latency around 0.8 seconds and word error rates comparable to the non-streaming baseline.

What carries the argument

The key mechanism is the streaming inference protocol: the model consumes 0.08-second chunks, waits for ten chunks (0.8 seconds total delay) before emitting the first output chunk, and then processes one new chunk at a time while updating and reusing cached hidden states from the Emformer, WaveNet-based component, and HiFi-GAN. Because the receptive field extends 0.64 seconds into the future, the model needs the look-ahead, but the authors design the caching so that no padding is introduced, which they argue guarantees chunk-wise output equals full-segment output. The second load-bearing machinery is the native TTS ground-truth generator: a VITS model trained on a multi-speaker native dataset, conditioned on MFA upsampled phonemes, F0, and speaker embeddings, which synthesizes the training target for each non-native utterance.

What would settle it

Take a set of non-native utterances, generate their synthetic ground-truth audio with the described native TTS, and run the accent classifier (ACC) on that synthetic audio: if the synthetic audio is labeled non-native at rates far above native speech, the 'ideal ground truth' premise fails. Separately, feed the same utterance to the model both chunk-by-chunk and as one full segment and compare the outputs sample-by-sample; any difference disproves the no-padding identical-output claim.

Watch

Extended reading notes

Core claim

The central claim is that a streaming, non-autoregressive accent conversion model can perform comparably to the best non-streaming models while maintaining stable latency, making it the first AC system capable of streaming. The model achieves this by swapping a global-attention Transformer encoder for an Emformer with a finite right look-ahead of eight frames, and by running a chunk-based inference procedure where all components (Emformer, WaveNet-based bottleneck extractor, and HiFi-GAN decoder) update their caches after each chunk and never pad the input. The authors assert that because no padding is used, the streaming output is identical to what the same model would produce on the full utterance. Training relies on synthetic ground truth generated by a native TTS model (VITS) that uses Montreal Forced Aligner upsampled transcripts, F0, and speaker embeddings from the original non-native audio, so that the target retains the speaker's voice and prosody but has native pronunciation.

Load-bearing premise

The load-bearing premise is that the native TTS generates ideal ground truth for every non-native utterance: native pronunciation with the original duration, prosody, and speaker identity, so the model learns a clean accent-to-native mapping rather than reproducing accent cues hidden in the synthetic target.

Editorial extensions

If this is right

  • Live video conferencing and other real-time applications could get accent conversion and pronunciation correction with about 0.8 seconds of delay, instead of waiting for the whole utterance.
  • The padding-free chunking design means the same model can serve both offline and streaming deployments with identical outputs, simplifying evaluation and deployment.
  • Latency can be further reduced by shrinking the look-ahead receptive field, as the authors state, though the causal variant's poor performance suggests a minimum look-ahead is needed for pronunciation improvement.
  • The custom Flex Attention implementation cuts training time by threefold and allows an eightfold larger batch size, making the streaming model practical to train.
  • If the native TTS ground truth is truly accent-free, the approach can be extended to other non-native accents and languages without collecting parallel accented data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The identical-output claim is directly testable: feeding the same utterance chunk-by-chunk versus all-at-once should produce bit-identical audio if the caching is exact; this comparison would also catch any subtle state-handling bugs.
  • The same cache-based, no-padding chunking recipe could transfer to other streaming speech transformation tasks built on non-causal convolutional decoders, such as real-time voice conversion or speech enhancement.
  • The residual 16.3% non-native classification after conversion hints that prosody carried by the preserved F0 and speaker embedding may leak accent cues; validating the synthetic ground truth with the accent classifier would isolate whether the leak originates in the TTS target.
  • The dramatic failure of the causal variant (WER 33.5 versus 14.1 for the streaming model) suggests a measurable lower bound on look-ahead for pronunciation correction, which could guide a latency-quality tradeoff curve.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a streaming accent conversion (AC) system that also aims at pronunciation improvement. The authors adapt a non-streaming AC architecture by replacing the Transformer-based ASR content encoder with an Emformer encoder, simplifying the conditional VAE into an autoencoder, and designing a chunk-based streaming inference with caching. They train a native TTS model on multi-speaker native data and use it to synthesize "ideal ground-truth" audio for non-native utterances, conditioning on F0 and speaker embeddings extracted from the original non-native audio. Experiments on L2-ARCTIC compare the streaming model with the non-streaming baseline and a causal streaming variant, reporting WER, accent classifier accuracy (ACC), speaker embedding cosine similarity (SECS), MOSNet, and subjective nativeness/similarity scores. The central claim is that the streaming model achieves performance comparable to the non-streaming model at about 0.8 s latency, constituting the first streaming AC system.

Significance. If the central claim holds, this is a useful contribution: it demonstrates a practical streaming formulation of a non-autoregressive accent converter, with an open-source streaming inference implementation and a Flex Attention-based Emformer optimization that yields substantial training speedups. The paper also gives concrete latency and real-time-factor numbers. However, the load-bearing evaluation has serious weaknesses: the test set is selected using the same ASR model that measures WER, objective results have no error bars or significance tests, subjective results come from only ten participants, and the synthetic ground truth itself shows signs of accent leakage and speaker-identity shift (ACC 11.5%, SECS 0.83). These issues prevent acceptance of the quantitative claims as stated, although the architectural contribution remains plausible and potentially valuable after revision.

major comments (4)
  1. [3.1 and 3.2.2] The test set is selected using the authors' own competitive ASR model [37] by keeping utterances with average WER greater than 10, and the same ASR model is then used to measure WER improvement in Table 2. This creates a partial circularity and a regression-to-the-mean effect: utterances selected for high WER on this particular ASR are likely to show lower WER on re-test even without any true pronunciation improvement. The WER comparison between original (18.3) and converted (14.1) speech may therefore be inflated. Please report results on the full L2-ARCTIC test speakers/utterances, or at least on a split that was not used for selection, and additionally report WER with an independent ASR model. Confidence intervals or bootstrap estimates should accompany the Table 2 numbers.
  2. [2.1.2, Eqs. (3)-(4)] The premise that the native TTS generates 'ideal ground-truth' audio is weakened by the paper's own design and results. The synthetic target is generated by conditioning the prior and HiFi-GAN decoder on F0 extracted from the non-native source audio, and the paper acknowledges that prosody can carry accent cues. Consistent with this, Table 2 shows the synthetic ground truth is judged non-native by the accent classifier 11.5% of the time, and its SECS is 0.83, below the paper's own 0.85 threshold for 'likely from the same speaker'. Because the AC model is trained to reproduce these imperfect targets, the claimed speaker-identity preservation and pronunciation-improvement ceiling are not established. Please provide direct validation of the synthetic ground truth's nativeness and speaker similarity (e.g., subjective ratings of synthetic targets against native references), or ablate the F0 conditioning to quantify how much accent prosody leaks into the target.
  3. [2.2.3] The assertion that 'streaming inference is performed without any paddings, ensuring that the streaming output remains identical to full-segment processing' is not demonstrated. While the chunking and cache design for the Emformer is plausible, the WaveNet-based bottleneck and HiFi-GAN decoder also have receptive fields, and the algorithmic description in Algorithm 1 does not show how all components' caches are initialized and updated to guarantee bit-exact (or even near-exact) equivalence. Please provide either a formal argument covering all three components or an empirical comparison of streaming versus full-segment outputs (e.g., maximum absolute waveform difference or mel-spectrogram error) on a held-out set.
  4. [Table 2] Table 2 reports objective metrics without error bars or significance testing, and Table 1's subjective confidence intervals overlap substantially (e.g., nativeness 3.78±0.18 for streaming vs. 3.87±0.20 for non-streaming). Given only ten participants and the small number of test speakers per accent, the claim that the streaming model 'performs comparably' to the non-streaming model is currently supported only by point estimates. Please report per-utterance or per-speaker standard deviations, confidence intervals, and, where appropriate, paired significance tests for WER, ACC, SECS, and MOSNet.
minor comments (5)
  1. [2.2.3] The pseudocode label reads 'Algorithm 0' but should be 'Algorithm 1'; the text in Section 2.2.3 also refers to 'Algorithm 0'.
  2. [References] Reference [26] cites 'EMF-former: An Efficient and Memory-Friendly Transformer', but the Emformer architecture used in the paper originates from Shi et al. (Interspeech 2021). Please cite the original Emformer work.
  3. [Eq. (1)] In Eq. (1), the MFA alignment A is computed but its role in Eq. (4) is implicit; clarify that A is used in Eq. (2) to construct cupsample, or rewrite the equations to make the dependency explicit.
  4. [3.3] The real-time factor is stated as 'achievable at around 0.25 on a GTX 1060 6GB' with no measurement methodology or variance; please specify how the RTF was computed and on which utterance lengths.
  5. [Throughout] There are several typos and formatting errors: 'kikt' in the correspondence email, 'mprocess' in the Introduction, 'Y AAP T' for YAAPT, 'Machifi-ne' in reference [18], and 'T; European Commission' in the acknowledgment. Please proofread.

Circularity Check

2 steps flagged · score 4.0 of 10

WER evaluation loop and self-cited 'ideal ground truth' premise create partial circularity, but the streaming architecture and subjective tests remain independent.

  1. fitted input called prediction [Section 3.1 (test set selection) and Section 3.2.2 (WER metric)]
    "The test set is selected based on our competitive ASR model [37], focusing on utterances with an average Word Error Rate (WER) greater than 10 across all speakers, under the assumption that higher WERs indicate stronger accents and more pronunciation issues. ... To assess pronunciation improvement, we measure WER using a competitive seq2seq Transformer ASR model [37]"

    The same ASR model [37] both selects the test utterances (requiring WER > 10) and computes the WER used to demonstrate pronunciation improvement. The evaluation set is therefore conditioned on the outcome metric, so the reported drop from 18.3 to 14.1 is not measured on an independent sample. This is not a derivation-level equivalence, but it makes the WER-based pronunciation-improvement claim partially an artifact of the evaluation loop.

  2. self citation load bearing [Section 1 and Section 2.1.2, Equations 1–4]
    "They hypothesize that a TTS system trained solely on native speech will produce accent-independent linguistic representations. Additionally, this native TTS system is able to generate ideal ground-truth data for non-native speakers... We adopt their approach of using native TTS to generate ideal ground truth. ... the synthetic ground truth y_ground−truth is generated from the transcripts with perfect native pronunciation, while utilizing the MFA alignment, F0 and g from the non-native audio to retain the original duration, prosody and speaker identity."

    The 'ideal ground-truth' premise is the central training target, and it is inherited from the authors' own prior work [17] rather than independently validated here. Worse, the paper's own operationalization contradicts the premise: Equations 3–4 inject F0 from the non-native source into the target, and Table 2 shows synthetic ground truth is judged non-native 11.5% of the time with SECS 0.83, below the 0.85 same-speaker threshold. The model is trained to reproduce these non-ideal targets, so the claimed pronunciation improvement is bounded by a self-cited, unmet premise.

full rationale

The paper's central streaming claims (Emformer-based chunked inference, stable 0.8s latency, and identical streaming/full-segment output) are architectural and algorithmic contributions that do not reduce to their inputs; they are supported by implementation details and runtime measurements. The subjective nativeness and similarity tests provide independent grounding. However, two load-bearing elements are partially circular. First, the pronunciation-improvement evaluation uses the same self-cited ASR [37] to select the test set (WER > 10) and to measure WER, so the reported improvement is not evaluated on an independent sample. Second, the 'ideal ground-truth' training target is adopted from the authors' own prior work [17] and, by the paper's own equations and results, is not ideal: F0 from the non-native source is injected into the synthetic target, and the synthetic ground truth itself is classified as non-native 11.5% of the time with speaker similarity below the paper's stated 0.85 threshold. These issues undermine the strength of the pronunciation-improvement and accent-conversion claims, but they do not invalidate the streaming architecture or the independent subjective evaluations. A score of 4 reflects partial circularity in evaluation and in a self-cited central premise, without the entire derivation collapsing by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the idealness of TTS-generated ground truth, the accent-independence of the Emformer content encoder, and the equivalence of streaming and full-segment inference. Several free parameters (prosody coefficient, data ratio, delay, WER threshold) are chosen by hand without ablations. No new physical or theoretical entities are introduced.

free parameters (4)
  • Prosody loss coefficient alpha = 0.2
    Hand-tuned hyperparameter in the content encoder loss (Section 2.2.2); no sensitivity analysis is reported.
  • Native/non-native data mixing ratio = 3:1 non-native to native
    Chosen for the fine-tuning stage (Section 2.2.2); no ablation justifies this ratio.
  • Streaming delay and chunk size = 0.8 s delay, 0.08 s chunks
    Engineering choices tied to the Emformer look-ahead and VAD (Section 2.2.3); they directly determine the central latency claim.
  • Test-set WER threshold = WER > 10
    Test utterances were selected using average WER greater than 10 from the authors' own ASR (Section 3.1); this is a hand-set evaluation criterion that can inflate apparent improvements.
assumptions (5)
  • domain assumption Native TTS yields ideal ground truth for non-native audio
    Section 2.1.2: synthetic ground truth is assumed to have perfect native pronunciation while retaining original duration, prosody, and speaker identity.
  • domain assumption Emformer content encoder captures accent-independent linguistic content and prosody
    Section 2.2.2: trained with CTC and log-F0 L1 loss, then frozen; no verification that accent and mispronunciations are absent from bottleneck features.
  • domain assumption A look-ahead of 0.64 s is necessary for accent conversion
    Section 3.4: causal variant performs worse, but no look-ahead sensitivity analysis supports the fixed 0.64 s value.
  • ad hoc to paper High WER indicates strong accent and pronunciation issues
    Section 3.1: test set selected using this criterion from the authors' own ASR; the assumption is not independently validated.
  • domain assumption Streaming chunked inference with caches and no padding exactly reproduces full-segment outputs
    Section 2.2.3: claimed design goal, but no empirical comparison of streaming versus full-segment outputs is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement." pith.science (2026). https://pith.science/paper/AYNF3G2D

@misc{pith2026250616580,
  author       = {Pith},
  title        = {Pith review of: Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AYNF3G2D}},
  note         = {Machine review of arXiv:2506.16580}
}
read the original abstract

We propose a first streaming accent conversion (AC) model that transforms non-native speech into a native-like accent while preserving speaker identity, prosody and improving pronunciation. Our approach enables stream processing by modifying a previous AC architecture with an Emformer encoder and an optimized inference mechanism. Additionally, we integrate a native text-to-speech (TTS) model to generate ideal ground-truth data for efficient training. Our streaming AC model achieves comparable performance to the top AC models while maintaining stable latency, making it the first AC system capable of streaming.

Figures

Figures reproduced from arXiv: 2506.16580 by the authors.

Figure 1
Figure 1. Native TTS with prosody and speaker preserving 2.1. Training Native TTS with prosody and speaker pre￾serving 2.1.1. Training We train the native TTS model using the VITS [18], a condi￾tional variational autoencoder (CAVE) enhanced with normal￾izing flow. VITS consists of three key components: a posterior encoder define qϕ(z|x), a prior encoder define pθ(z|c), and a waveform generator define pψ(y|z), shown in [PITH_… view at source ↗
Figure 2
Figure 2. Training streaming Accent Conversion Architecture with generated ideal ground-truth with CTC and prosody losses. This training allows the final layer’s output to capture linguistic content and prosody features. The prosody loss is computed as the L1 loss between the pre￾dicted and ground truth log-F0 values. Based on our observa￾tions, the optimal coefficient for the prosody loss is set to 0.2. The loss function is … view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 38 canonical work pages

  1. [37]

    Wavenet: A generative model for raw audio,

    A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” 2016. [Online]. Available: https://arxiv.org/abs/1609.03499

    arXiv 2016

  2. [1]

    AC modifies speech to enhance intelligibility while preserving content, emo- tion, and speaker identity

    Introduction Second-language L2 English learners often have accents and mispronunciations that impact communication. AC modifies speech to enhance intelligibility while preserving content, emo- tion, and speaker identity. Streaming AC enables streaming ap- plications like video conferencing [1], where seamless interac- tion are crucial. Previous AC mproce...

    work page

  3. [2]

    Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement

    Methodology This section presents the step-by-step training process for our streaming AC model. We begin by training the Native TTS model and demonstrating how it generates ideal ground-truth data for non-native speech. Next, we outline the architectural modifications needed for streaming AC and describe the train- ing process using synthetic ground truth...

    work page Pith review arXiv 2025

  4. [3]

    Data We use the LJSpeech dataset [34], which contains recordings from a single native speaker with consistent pronunciation

    Experiment and Result 3.1. Data We use the LJSpeech dataset [34], which contains recordings from a single native speaker with consistent pronunciation. To augment the dataset, we employ FreeVC [25], a voice con- version model, to generate multi-speaker native-accented utter- ances from the original voices. The augmented multi-speaker dataset used in our w...

    work page

  5. [4]

    Our results show that the streaming model achieves performance comparable to its non- streaming counterpart

    Conclusion This work presents the first streaming accent conversion model, demonstrating its capability to synthesize high-quality audio with native-like pronunciation. Our results show that the streaming model achieves performance comparable to its non- streaming counterpart. We introduced an effective streaming inference method for the non-autoregressiv...

    work page

  6. [5]

    Acknowledgment This research was supported by a grant from Zoom Video Communications , Inc. T; European Commission Project Meetween (101135798), the Federal Ministry of Education and Research (BMBF) of Germany under the number 01EF1803B (RELATER), and the pilot program Core-Informatics of the Helmholtz A and the HoreKa supercomputer funded by the Ministry...

    work page

  7. [6]

    Convert and speak: Zero-shot accent conversion with minimum supervision,

    Z. jia, H. Xue, X. Peng, and Y . Lu, “Convert and speak: Zero-shot accent conversion with minimum supervision,” in ACM Multime- dia 2024, 2024

    work page 2024

  8. [7]

    Syntacc : Synthe- sizing multi-accent speech by weight factorization,

    T.-N. Nguyen, N.-Q. Pham, and A. Waibel, “Syntacc : Synthe- sizing multi-accent speech by weight factorization,” in ICASSP 2023

    work page 2023

Show all 45 references
  1. [8]

    Simultaneous translation of open do- main lectures and speeches,

    A. Waibel and C. Fuegen, “Simultaneous translation of open do- main lectures and speeches,” Jan. 3 2012, uS Patent 8,090,570

    2012

  2. [9]

    Zero-Shot Foreign Accent Conversion with- out a Native Reference,

    W. Quamer, A. Das, J. Levis, E. Chukharev-Hudilainen, and R. Gutierrez-Osuna, “Zero-Shot Foreign Accent Conversion with- out a Native Reference,” in Proc. Interspeech 2022, 2022

    2022

  3. [10]

    Accent conversion us- ing discrete units with parallel data synthesized from controllable accented tts,

    T.-N. Nguyen, Q. Pham, and A. Waibel, “Accent conversion us- ing discrete units with parallel data synthesized from controllable accented tts,” in Synthetic Data’s Transformative Role in Founda- tional Speech Models, 2024, pp. 51–55

    2024

  4. [11]

    Converting foreign accent speech without a reference,

    G. Zhao, S. Ding, and R. Gutierrez-Osuna, “Converting foreign accent speech without a reference,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 2367–2381, 2021

    2021

  5. [12]

    Recent AC advancements combine both disentangling and mapping approaches into a unified framework[17]

    and Pseudo-Siamese networks [11] attempt to remove ac- cent influences, they struggle to improve pronunciation due to the lack of ground-truth references. Recent AC advancements combine both disentangling and mapping approaches into a unified framework[17]. They hy- pothesize ...

  6. [13]

    Accent Conversion using Pre-trained Model and Synthesized Data from V oice Con- version,

    T. N. Nguyen, N.-Q. Pham, and A. Waibel, “Accent Conversion using Pre-trained Model and Synthesized Data from V oice Con- version,” in Proc. Interspeech 2022, 2022, pp. 2583–2587

    2022

  7. [14]

    wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” in NEURIPS 2020, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33, 2020

    2020

  8. [15]

    Lvcsr-based language iden- tification,

    T. Schultz, I. Rogina, and A. Waibel, “Lvcsr-based language iden- tification,” in 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings

    1996

  9. [16]

    Code-switching with- out switching: Language agnostic end-to-end speech translation,

    C. Huber, E. Y . Ugan, and A. Waibel, “Code-switching with- out switching: Language agnostic end-to-end speech translation,” arXiv preprint arXiv:2210.01512, 2022

    2022 arXiv

  10. [17]

    Experiments on cross-language acous- tic modeling

    T. Schultz and A. Waibel, “Experiments on cross-language acous- tic modeling.” in INTERSPEECH, 2001, pp. 2721–2724

    2001

  11. [18]

    Zero-Shot Accent Conversion using Pseudo Siamese Disentanglement Network,

    D. Jia, Q. Tian, K. Peng, J. Li, Y . Chen, M. Ma, Y . Wang, and Y . Wang, “Zero-Shot Accent Conversion using Pseudo Siamese Disentanglement Network,” inProc. INTERSPEECH 2023, 2023

    2023

  12. [19]

    Our prior is conditioned on both the upsampled transcript and the F0 sequence

    to upsample text representations before training, ensuring they match the length of the audio. Our prior is conditioned on both the upsampled transcript and the F0 sequence. The F0 Encoder extracts frame-level F0 embeddings, which are combined with text embeddings before being...

  13. [21]

    Wavlm: Large- scale self-supervised pre-training for full stack speech process- ing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “Wavlm: Large- scale self-supervised pre-training for full stack speech process- ing,” IEEE Journal of Sele...

  14. [22]

    Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,”IEEE/ACM Trans. Audio, Speech and Lang. Proc., 2021

    2021

  15. [23]

    V oice-preserving zero-shot multiple accent conversion,

    M. Jin, P. Serai, J. Wu, A. Tjandra, V . Manohar, and Q. He, “V oice-preserving zero-shot multiple accent conversion,” in ICASSP 2023, 2023, pp. 1–5

    2023

  16. [24]

    Improving pronunciation and accent conversion through knowledge distilla- tion and synthetic ground-truth from native tts,

    T. N. Nguyen, S. Akti, N. Q. Pham, and A. Waibel, “Improving pronunciation and accent conversion through knowledge distilla- tion and synthetic ground-truth from native tts,” in ICASSP 2025

    2025

  17. [25]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Inter- national Conference on Machifi-ne Learning, 2021

    2021

  18. [26]

    Montreal forced aligner: Trainable text-speech align- ment using kaldi,

    M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Son- deregger, “Montreal forced aligner: Trainable text-speech align- ment using kaldi,” in Interspeech 2017, 2017, pp. 498–502

    2017

  19. [27]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garn...

    2017

  20. [28]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” in Ad- vances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., 2020

    2020

  21. [29]

    Comparing conventional pitch detection algorithms with a neural network approach,

    A. Kroon, “Comparing conventional pitch detection algorithms with a neural network approach,” 2022

    2022

  22. [30]

    Vqmivc: Vector quantization and mutual information-based un- supervised speech representation disentanglement for one-shot voice conversion,

    D. Wang, L. Deng, Y . T. Yeung, X. Chen, X. Liu, and H. Meng, “Vqmivc: Vector quantization and mutual information-based un- supervised speech representation disentanglement for one-shot voice conversion,” inInterspeech 2021, 2021, pp. 1344–1348

    2021

  23. [31]

    Any-to-many voice conversion with location-relative sequence- to-sequence modeling,

    S. Liu, Y . Cao, D. Wang, X. Wu, X. Liu, and H. Meng, “Any-to-many voice conversion with location-relative sequence- to-sequence modeling,” TASLP 2021, 2021

    2021

  24. [32]

    Freevc: Towards hifigagh-quality text- free one-shot voice conversion,

    J. Li, W. Tu, and L. Xiao, “Freevc: Towards hifigagh-quality text- free one-shot voice conversion,” inICASSP 2023, 2023, pp. 1–5

    2023

  25. [33]

    EMF-former: An Efficient and Memory-Friendly Transformer for Medical Image Segmentation ,

    Z. Hao, H. Quan, and Y . Lu, “ EMF-former: An Efficient and Memory-Friendly Transformer for Medical Image Segmentation ,” in proceedings of Medical Image Computing and Computer As- sisted Intervention – MICCAI 2024, 2024

    2024

  26. [34]

    Super-human per- formance in online low-latency recognition of conversational speech,

    T.-S. Nguyen, S. St ¨uker, and A. Waibel, “Super-human per- formance in online low-latency recognition of conversational speech,” in Interspeech 2021, 2021, pp. 1762–1766

    2021

  27. [35]

    Low-latency neural speech translation,

    J. Niehues, N.-Q. Pham, T.-L. Ha, M. Sperber, and A. Waibel, “Low-latency neural speech translation,” 2018

    2018

  28. [36]

    Dynamic transcription for low-latency speech translation

    J. Niehues, T. S. Nguyen, E. Cho, T.-L. Ha, K. Kilgour, M. M¨uller, M. Sperber, S. St¨uker, and A. Waibel, “Dynamic transcription for low-latency speech translation.” in Interspeech, 2016

    2016

  29. [38]

    Lib- rispeech: An asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An asr corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

  30. [39]

    Flex atten- tion: A programming model for generating optimized attention kernels,

    J. Dong, B. Feng, D. Guessous, Y . Liang, and H. He, “Flex atten- tion: A programming model for generating optimized attention kernels,” 2024

    2024

  31. [40]

    Speech activity de- tection based on multilingual speech recognition system,

    S. S. Sarfjoo, S. Madikeri, and P. Motlicek, “Speech activity de- tection based on multilingual speech recognition system,” in In- terspeech 2021, 2021, pp. 4369–4373

    2021

  32. [41]

    The lj speech dataset,

    K. Ito and L. Johnson, “The lj speech dataset,” https://keithito. com/LJ-Speech-Dataset/, 2017

    2017

  33. [42]

    CSTR VCTK Corpus: English multi- speaker corpus for CSTR voice cloning toolkit,

    J. Yamagishi and C. Veaux, “CSTR VCTK Corpus: English multi- speaker corpus for CSTR voice cloning toolkit,” 2019

    2019

  34. [43]

    L2-arctic: A non- native english speech corpus,

    G. Zhao, S. Sonsaat, A. Silpachai, I. Lucic, E. Chukharev- Hudilainen, J. Levis, and R. Gutierrez-Osuna, “L2-arctic: A non- native english speech corpus,” in Interspeech 2018, 2018

    2018

  35. [44]

    Relative positional encoding for speech recognition and direct translation,

    N.-Q. Pham, T.-L. Ha, T.-N. Nguyen, T.-S. Nguyen, S. St ¨uker, J. Niehues, and A. Waibel, “Relative positional encoding for speech recognition and direct translation,” in Interspeech 2020

    2020

  36. [45]

    Differ- entiable mean opinion score regularization for perceptual speech enhancement,

    T. Rosenbaum, I. Cohen, E. Winebrand, and O. Gabso, “Differ- entiable mean opinion score regularization for perceptual speech enhancement,” Pattern Recognition Letters, vol. 166, 2023

    2023

  37. [46]

    Streamvc: Real-time low-latency voice conver- sion,

    Y . Yang, Y . Kartynnik, Y . Li, J. Tang, X. Li, G. Sung, and M. Grundmann, “Streamvc: Real-time low-latency voice conver- sion,” in ICASSP 2024, 2024, pp. 11 016–11 020

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.