Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Miipher-2: A Universal Speech Restoration Model for Million-Hour Scale Data Restoration

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Miipher-2 claims a frozen, 300-plus-language speech encoder plus 20M adapter parameters can restore degraded audio in known and unseen languages without transcripts, fast enough to clean a million-hour corpus in about three days on 100…

desk verdict Miipher-2 is a practical efficiency advance in speech restoration, but the universal-language claim is undercut by consistently rising WER in every reported language, including the five unknown-locale tests. read the letter →

arxiv 2505.04457 v4 pith:7N46RRKU submitted 2025-05-07 cs.SD cs.CLeess.AS

Shigeki Karita , Yuma Koizumi , Heiga Zen , Haruko Ishikawa , Robin Scheibler , Michiel Bacchiani This is my paper · ORCID
classification cs.SDcs.CLeess.AS
keywords speechrestorationenhancementself-supervisedlearningUniversalModelparalleladaptersneuralvocoderdatacleaning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Miipher-2 is a speech restoration system aimed not at fixing one recording but at cleaning entire million-hour speech corpora before they are used to train generative models. The paper argues that a frozen, 300-plus-language pre-trained speech encoder can replace the text and speaker conditioning that earlier restoration required: small trainable adapters predict clean encoder features from noisy audio, and a neural vocoder turns those features back into waveforms. On English, Miipher-2 matches the quality of the text-conditioned Miipher-1 while raising predicted MOS and speaker similarity, and it reports similar gains on non-English and unseen low-resource languages. Its efficiency claim is concrete: a real-time factor of 0.0078 on small accelerators, meaning roughly three days of compute on 100 chips to restore a million hours of speech. If these results hold, audio data cleaning can scale to the sizes that text and image filtering already reach.

What carries the argument

The load-bearing object is the frozen USM encoder used as a fixed feature extractor: Miipher-2 takes its 13th-layer hidden features as the clean acoustic target, so the system never needs transcripts or speaker IDs. Around this sits a parallel-adapter (PA) feature cleaner, composed of small feed-forward additions appended to each frozen layer, which predicts clean features from noisy input in linear time, and a WaveFit vocoder made memory-efficient by replacing transposed-convolution upsampling with repetition and by simplifying the FiLM conditioning in the U-Net. Together these components let the model run at RTF 0.0078 in batches of eight 30-second clips on an 8 GB accelerator, which is the concrete mechanism behind the million-hour-in-three-days claim.

What would settle it

Choose ten low-resource languages outside the 44 training locales, restore a noisy test set with Miipher-2, and have fluent transcribers measure word error against the clean reference. If restored audio yields higher transcription error than the original degraded audio, or if the restored audio is not preferred over the degraded input in a blind listening test for any of those languages, then the universal-restoration claim is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that universal speech restoration can be built without any explicit conditioning by anchoring the system to a frozen Universal Speech Model (USM) pre-trained on over 300 languages. A parallel-adapter feature cleaner of only 20M trainable parameters predicts the clean 13th-layer USM features from a noisy waveform, and a memory-optimized WaveFit vocoder synthesizes the 24 kHz waveform. Trained on 3,000 hours of studio recordings across 44 languages with synthetic noise, reverberation, and codec degradation, Miipher-2 restores English speech at quality comparable to the text- and speaker-conditioned Miipher-1, improves predicted MOS and speaker similarity on known and unknown languages, and leaves word error rate close to the input's. The same recipe trained only on public data cleaned by Miipher-2 performs nearly as well as the studio-trained model, indicating that cleaned corpora can substitute for studio recordings in training downstream generative speech systems.

Load-bearing premise

The claim depends on the premise that the internal features of a pre-trained speech model generalize across languages, so the small trainable part, which was only trained on 44 languages, can also restore speech in languages it never encountered.

Editorial extensions

If this is right

  • Web-scraped speech at the scale used to train large audio-language models can be automatically restored to near-studio quality without transcripts or speaker IDs, so cleaning does not require annotation pipelines for every language.
  • A one-hundred-chip, three-day compute budget is enough to restore one million hours of speech, making speech restoration a practical front-end step rather than a research-only tool.
  • Because adapters, not the full encoder, are trained, extending restoration to new degradation types or languages requires only small additional trainable parameters and works with a frozen foundation model.
  • Training the same architecture on public data previously cleaned by the model gives nearly equivalent quality, implying restoration-cleaned corpora can be used as training data for further generative speech models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer the same 'frozen encoder plus tiny adapter plus vocoder' recipe should transfer to other restoration targets, such as music or environmental audio, whenever a large pre-trained self-supervised encoder exists for that domain; the paper only tests speech, but the mechanism is not speech-specific.
  • The paper's comparison of training losses suggests that contrastive self-supervised features, such as w2v-BERT, are worse suited to restoration than BEST-RQ-style masked-prediction features; a direct head-to-head using the same adapter and vocoder would test this design principle beyond USM.
  • The five unseen-language test locales all show WER rising after restoration, which the paper attributes to the ASR rather than the restorer. The cleanest way to decide is to retrain a per-language ASR on restored speech only; if WER still rises, the universal claim needs qualification.
  • Since code and checkpoints are withheld, the practical universality claim will live or die on open re-implementations using public encoders and vocoders; the paper itself notes that such reproduction is feasible.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper presents Miipher-2, a speech restoration model that combines a frozen Universal Speech Model (USM) feature extractor, parallel adapters as a parameter-efficient feature cleaner, and a memory-optimized WaveFit vocoder. It is designed for conditioning-free, multilingual restoration of large noisy speech datasets. The authors evaluate on LibriTTS, MLS, and FLEURS, report DNSMOS/SQuId/WER/SPK plus human MOS/SxS for English, and measure an inference RTF of 0.0078 on TPU v4i and memory reductions relative to Miipher-USM. They conclude that Miipher-2 matches or exceeds prior restoration quality, generalizes to unknown languages, and enables million-hour cleaning.

Significance. If the performance claims held, this would be a practically useful contribution: the parameter-efficient cleaner and memory optimizations are concrete, and the reported speedups are substantial. The paper contains useful design details (frozen USM, parallel adapters, pre-upsampler, FiLM simplification) and includes human MOS with confidence intervals, a public-data distillation experiment, and both known- and unknown-language evaluations. The main concerns are that content preservation is not established (WER rises after restoration in the paper's own tables), the zero-shot language generalization claim rests on only five FLEURS locales, and the objective evaluation loop shares representation family with the restored features. These issues are fixable with additional evidence or careful claim revision, so the work warrants revision rather than rejection.

major comments (4)
  1. [Section 3.6.1, Table 5] The unknown-language result is load-bearing for the 'universal' claim, but WER increases after Miipher-2 restoration in all five FLEURS locales (ca 5.01→5.46, ru 5.25→5.52, ur 21.0→22.1, sw 33.5→35.2, mi 38.4→40.7). DNSMOS and SQuId are non-intrusive quality predictors rather than intelligibility measures, so they do not establish that content is preserved. The attribution of the WER increase to 'low performance of the ASR model itself' is not supported, since the degradation appears in low-WER locales (Catalan, Russian) as well as high-WER ones. The paper should report confidence intervals or significance tests for these deltas, add a content-preservation metric (e.g., human transcription or CER), and either temper the universal-restoration claim or provide additional evidence.
  2. [Section 3.4, Tables 2 and 4] The claim of 'superior or comparable' WER performance is relative to Miipher-1, but relative to the original noisy signal Miipher-2 consistently increases WER on LibriTTS (0.132→0.149) and on every MLS locale (e.g., fr 15.6→19.4, pl 4.90→5.74). Since WER is the only objective content-preservation metric in these tables, the paper does not currently support the conclusion that Miipher-2 cleans data without introducing transcription-affecting distortions. The authors should either demonstrate that the WER increase is not statistically significant or is perceptually immaterial, or present a separate intelligibility test.
  3. [Section 3.4 and Section 2.1] The WER evaluator is a USM-based ASR with CTC, i.e., it consumes features from the same representation family that Miipher-2 is explicitly trained to predict (the 13th USM layer). This makes the objective evaluation loop partially self-referential and weakens the claim that WER behavior reflects general content preservation. I recommend an independent evaluation with a different ASR family (e.g., Whisper or a non-USM ASR), or human transcription listening tests, before relying on WER to support the 'minimal impact' conclusion.
  4. [Section 3.6, Tables 5 and 6] The 'universal' claim is tested on only five FLEURS locales that are not in the training languages, but the paper does not describe how these locales were selected or whether they are representative of the 300 languages in USM. A universal claim based on five locales should be explicitly scoped; otherwise the conclusion in Section 4 that the model has 'universal restoration capability' is stronger than the evidence.
minor comments (4)
  1. [Throughout] There are several typos and formatting issues: 'fintuned' in the Fig. 1 caption, 'porposed' in the Fig. 2 caption, 'dthe' in Section 3.4, 'V oiceFixer' in reference [4], duplicate 'languages. languages as well.' in Section 3.6.1, and 'it’s training data' in Section 3.6.2.
  2. [Table 3] The numeric formatting in Table 3 contains unintended spaces ('1 .208', '0 .044'); this should be corrected for readability.
  3. [Tables 4-6] The multilingual evaluations report only point values without confidence intervals or significance tests; adding these would strengthen the comparison, especially for the WER deltas discussed above.
  4. [Section 3.1] The comparison with TF-GridNet is described as only a reference because the training data differ; this caveat is appropriate, but it would be helpful to state explicitly that the URGENT2025 baseline may not be directly comparable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Miipher-2's core claims are supported by held-out evaluations and an externally grounded training setup, not by construction.

full rationale

Miipher-2's derivation chain is self-contained. The feature cleaner is trained to minimize L1/L2/spectral-convergence losses between PA-predicted and actual clean 13th-layer USM features on 3,195 hours of studio-quality recordings with simulated degradations (SNR 5–30 dB, stochastic RIR, codec artifacts); the WaveFit vocoder is pretrained on clean USM features and then fine-tuned on predicted features. Reported evaluations use held-out LibriTTS test-other, MLS known-locale, and FLEURS unknown-locale sets, with metrics (DNSMOS, SQuId, human MOS/SxS, speaker similarity, WER) that are not identical to the training objective. No fitted parameter is renamed as a prediction: layer-13 selection and architecture choices are presented as design decisions with preliminary experiments, not as fitted results. The Miipher-2-P experiment is explicitly a dataset-distillation feasibility check rather than the load-bearing evidence for studio-quality restoration, and the paper does not claim self-consistency alone proves equivalence to studio recordings. The WER ASR does share the USM front-end, which is a possible independence limitation for that metric, but WER is not equal to the feature-matching objective by construction, and the paper's unknown-language inference is a logical weakness rather than a circular reduction. Self-citations to Miipher [6] for the loss function and augmentation settings are methodological and not load-bearing; the central claims are independently supported by held-out benchmark results. Score 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper is an empirical system result rather than a derivation, so the ledger lists the design choices and transfer assumptions the evaluation depends on. No new physical entities are introduced; the model itself is a trained artifact.

free parameters (4)
  • USM intermediate layer index = 13
    Selected via preliminary experiments (Section 2.1) to balance linguistic content and fine-grained acoustic detail; no ablation is shown, so the choice is effectively tuned by hand.
  • Parallel adapter hidden dimension = 1024
    Set in Section 3.2 without reported ablations; the feature cleaner capacity depends on this choice.
  • Feature cleaner loss weights (L1, L2, spectral convergence) = not reported
    Section 2.2 says the loss is a sum of L1, L2, and spectral convergence but does not give the weights; the training target is defined up to this unspecified weighting.
  • Training step counts = 800k PA, 200k WaveFit pretrain, 675k WaveFit fine-tune
    Reported in Section 3.2; early stopping or model selection criteria are not described, and the final model is defined by these step counts.
assumptions (6)
  • domain assumption USM 13th-layer features are a sufficient and invertible target representation for clean speech across languages.
    The feature cleaner (Section 2.2) and vocoder training assume these features preserve all content needed for downstream ASR/TTS and can be inverted to waveform by WaveFit.
  • domain assumption Parallel adapters trained on 44 languages with simulated degradation transfer to unseen languages and unseen noise conditions.
    Section 3.6 tests only five unknown locales; the universal claim assumes transfer from the 44-language training set to the 300-plus languages of USM.
  • domain assumption BEST-RQ-style random-quantizer SSL training retains fine-grained acoustic detail needed for speaker similarity and naturalness without text or speaker conditioning.
    Section 2.1 argues this by analogy to HuBERT and WavLM rather than by a controlled experiment.
  • domain assumption Simulated degradations (SNR 5-30 dB, stochastic image-source RIRs, codec artifacts) match the statistics of real web-scraped speech.
    Section 3.2 describes synthetic pairing of clean studio speech with added noise; match to real million-hour web audio is untested.
  • domain assumption A USM-based CTC ASR with language ID embedding provides valid word-error-rate measurements for restored audio.
    Section 3.4 uses a single multilingual USM-based ASR; low-resource WER degradations are attributed to the ASR, so the interpretation depends on that ASR's reliability.
  • standard math The image-source method produces room impulse responses representative of real rooms.
    Used in Section 3.2 to generate stochastic RIRs; a standard acoustics method with known limitations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Miipher-2: A Universal Speech Restoration Model for Million-Hour Scale Data Restoration." pith.science (2026). https://pith.science/paper/7N46RRKU

@misc{pith2026250504457,
  author       = {Pith},
  title        = {Pith review of: Miipher-2: A Universal Speech Restoration Model for Million-Hour Scale Data Restoration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7N46RRKU}},
  note         = {Machine review of arXiv:2505.04457}
}
read the original abstract

Training data cleaning is a new application for generative model-based speech restoration (SR). This paper introduces Miipher-2, an SR model designed for million-hour scale data, for training data cleaning for large-scale generative models like large language models. Key challenges addressed include generalization to unseen languages, operation without explicit conditioning (e.g., text, speaker ID), and computational efficiency. Miipher-2 utilizes a frozen, pre-trained Universal Speech Model (USM), supporting over 300 languages, as a robust, conditioning-free feature extractor. To optimize efficiency and minimize memory, Miipher-2 incorporates parallel adapters for predicting clean USM features from noisy inputs and employs the WaveFit neural vocoder for waveform synthesis. These components were trained on 3,000 hours of multi-lingual, studio-quality recordings with augmented degradations, while USM parameters remained fixed. Experimental results demonstrate Miipher-2's superior or comparable performance to conventional SR models in word-error-rate, speaker similarity, and both objective and subjective sound quality scores across all tested languages. Miipher-2 operates efficiently on consumer-grade accelerators, achieving a real-time factor of 0.0078, enabling the processing of a million-hour speech dataset in approximately three days using only 100 such accelerators.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ReverbMiipher: Generative Speech Restoration meets Reverberation Characteristics Controllability

    cs.SD 2025-05 conditional novelty 6.0 of 10

    ReverbMiipher, an extension of Miipher-2 with a dedicated reverb encoder, preserves and controls reverberation during generative speech restoration.

Reference graph

Works this paper leans on

55 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Parametric resynthesis with neural vocoders,

    S. Maiti and M. I. Mandel, “Parametric resynthesis with neural vocoders,” in WASPAA, 2019

    2019

  2. [2]

    SelfRemaster: Self-supervised speech restoration with analysis-by-synthesis approach using channel modeling,

    T. Saeki, S. Takamichi, et al. , “SelfRemaster: Self-supervised speech restoration with analysis-by-synthesis approach using channel modeling,” in Interspeech, 2022

    2022

  3. [3]

    HiFi-GAN-2: Studio-quality speech enhancement via generative adversarial networks conditioned on acoustic features,

    J. Su, Z. Jin, and A. Finkelstein, “HiFi-GAN-2: Studio-quality speech enhancement via generative adversarial networks conditioned on acoustic features,” in WASPAA, 2021

    2021

  4. [4]

    V oiceFixer: A unified framework for high-fidelity speech restoration,

    H. Liu, X. Liu, et al. , “V oiceFixer: A unified framework for high-fidelity speech restoration,” in Interspeech, 2022

    2022

  5. [5]

    Universal speech enhancement with score- based diffusion,

    J. Serr `a, S. Pascual, et al. , “Universal speech enhancement with score- based diffusion,” arXiv:2206.03065, 2022

    arXiv 2022

  6. [6]

    Miipher: A robust speech restoration model integrating self-supervised speech and text representations,

    Y . Koizumi, H. Zen, et al. , “Miipher: A robust speech restoration model integrating self-supervised speech and text representations,” in WASPAA, 2023, pp. 1–5

    work page 2023

  7. [7]

    Speech enhancement and dereverberation with diffusion-based generative models,

    J. Richter, S. Welker, et al. , “Speech enhancement and dereverberation with diffusion-based generative models,” IEEE/ACM TASLP, 2023

    work page 2023

  8. [8]

    Diffusion models for audio restoration: A review,

    J.-M. Lemercier, J. Richter, et al., “Diffusion models for audio restoration: A review,” IEEE Signal Process Mag. , 2024

    2024

Show all 55 references
  1. [9]

    Universal score-based speech enhancement with high content preservation,

    R. Scheibler, Y . Fujita,et al., “Universal score-based speech enhancement with high content preservation,” in Interspeech, 2024

    2024

  2. [10]

    LLaSE-G1: Incentivizing generalization capability for llama-based speech enhancement,

    B. Kang, X. Zhu, et al. , “LLaSE-G1: Incentivizing generalization capability for llama-based speech enhancement,” 2025. [Online]. Available: https://arxiv.org/abs/2503.00493

    2025 arXiv

  3. [11]

    Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens,

    H. Yang, J. Su, et al. , “Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens,” in Interspeech, 2024

    2024

  4. [12]

    Joint semantic knowledge distillation and masked acoustic modeling for full-band speech restoration with improved intelligibility,

    X. Liu, X. Li, et al. , “Joint semantic knowledge distillation and masked acoustic modeling for full-band speech restoration with improved intelligibility,” in ICASSP, 2025

    2025

  5. [13]

    DiTSE: High-fidelity generative speech enhancement via latent diffusion transformers,

    H. R. Guimar ˜aes, J. Su, et al. , “DiTSE: High-fidelity generative speech enhancement via latent diffusion transformers,” 2025. [Online]. Available: https://arxiv.org/abs/2504.09381

    2025

  6. [14]

    LibriTTS-R: A restored multi-speaker text- to-speech corpus,

    Y . Koizumi, H. Zen, et al. , “LibriTTS-R: A restored multi-speaker text- to-speech corpus,” in Interspeech 2023 , 2023, pp. 5496–5500

    2023

  7. [15]

    FLEURS-R: A restored multilingual speech corpus for generation tasks,

    M. Ma, Y . Koizumi, et al. , “FLEURS-R: A restored multilingual speech corpus for generation tasks,” in Interspeech 2024 , 2024, pp. 1835–1839

    2024

  8. [16]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    G. Team, “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” 2024. [Online]. Available: https: //arxiv.org/abs/2403.05530

    2024 arXiv

  9. [17]

    GPT-4 technical report,

    OpenAI, “GPT-4 technical report,” 2024. [Online]. Available: https: //arxiv.org/abs/2303.08774

    2024 arXiv

  10. [18]

    Moshi: a speech-text foundation model for real-time dialogue,

    A. D ´efossez, L. Mazar ´e, et al. , “Moshi: a speech-text foundation model for real-time dialogue,” 2024. [Online]. Available: https: //arxiv.org/abs/2410.00037

    2024 arXiv

  11. [19]

    Google USM: Scaling automatic speech recognition beyond 100 languages,

    Y . Zhang, W. Han, et al. , “Google USM: Scaling automatic speech recognition beyond 100 languages,” 2023. [Online]. Available: https://arxiv.org/abs/2303.01037

    2023 arXiv

  12. [20]

    Towards a unified view of parameter- efficient transfer learning,

    J. He, C. Zhou, et al. , “Towards a unified view of parameter- efficient transfer learning,” in ICLR, 2022. [Online]. Available: https://openreview.net/forum?id=0RDcd5Axok

    2022

  13. [21]

    WaveFit: An iterative and non- autoregressive neural vocoder based on fixed-point iteration,

    Y . Koizumi, K. Yatabe, et al. , “WaveFit: An iterative and non- autoregressive neural vocoder based on fixed-point iteration,” in SLT, 2023

    2023

  14. [22]

    Ten lessons from three generations shaped google’s tpuv4i : Industrial product,

    N. P. Jouppi, D. Hyun Yoon, et al. , “Ten lessons from three generations shaped google’s tpuv4i : Industrial product,” in 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) , 2021, pp. 1–14

    2021

  15. [23]

    Self-supervised learning with random- projection quantizer for speech recognition,

    C.-C. Chiu, J. Qin, et al. , “Self-supervised learning with random- projection quantizer for speech recognition,” in ICML, 2022

    2022

  16. [24]

    wav2vec 2.0: A framework for self- supervised learning of speech representations,

    A. Baevski, H. Zhou, et al. , “wav2vec 2.0: A framework for self- supervised learning of speech representations,” in NeurIPS, 2020

    2020

  17. [25]

    w2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre- training,

    Y .-A. Chung, Y . Zhang, et al. , “w2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre- training,” in ASRU, 2021

    2021

  18. [26]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, et al. , “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” TASLP ., vol. 29, p. 3451–3460, Oct. 2021. [Online]. Available: https://doi.org/10.1109/TASLP.2021.3122291

    2021

  19. [27]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, et al. , “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” 2021. [Online]. Available: https://arxiv.org/abs/2110.13900

    2021

  20. [28]

    V oice conversion with just nearest neighbors,

    M. Baas, B. van Niekerk, and H. Kamper, “V oice conversion with just nearest neighbors,” in Interspeech, 2023

    2023

  21. [29]

    Vec-tok speech: speech vectorization and tokenization for neural speech generation,

    X. Zhu, Y . Lv, et al. , “Vec-tok speech: speech vectorization and tokenization for neural speech generation,” 2023. [Online]. Available: https://arxiv.org/abs/2310.07246

    2023 arXiv

  22. [30]

    DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancement,

    Y . Koizumi, S. Karita, et al. , “DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancement,” in WASPAA, 2021

    2021

  23. [31]

    Fast spectrogram inversion using multi-head convolutional neural networks,

    S. ¨O. Arık, H. Jun, and G. Diamos, “Fast spectrogram inversion using multi-head convolutional neural networks,” IEEE Signal Processing Letters, vol. 26, no. 1, pp. 94–98, 2019

    2019

  24. [32]

    FiLM: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, et al. , “FiLM: Visual reasoning with a general conditioning layer,” in AAAI, 2018

    2018

  25. [33]

    WaveGrad: Estimating gradients for waveform generation,

    N. Chen, Y . Zhang,et al., “WaveGrad: Estimating gradients for waveform generation,” in ICLR, 2021

    2021

  26. [34]

    URGENT challenge 2025 baseline,

    K. Saijo, W. Zhang, et al. , “URGENT challenge 2025 baseline,” 2025. [Online]. Available: https://huggingface.co/kohei0209/tfgridnet urgent25

    2025

  27. [35]

    PromptTTS++: Controlling speaker identity in prompt-based text-to-speech using natural language descrip- tions,

    R. Shimizu, R. Yamamoto, et al. , “PromptTTS++: Controlling speaker identity in prompt-based text-to-speech using natural language descrip- tions,” in ICASSP, 2024

    2024

  28. [36]

    Xtts: a massively multilingual zero-shot text-to-speech model,

    E. Casanova, K. Davis, et al. , “Xtts: a massively multilingual zero-shot text-to-speech model,” in Interspeech, 2024

    2024

  29. [37]

    Natural language guidance of high-fidelity text-to-speech with synthetic annotations,

    D. Lyth and S. King, “Natural language guidance of high-fidelity text-to-speech with synthetic annotations,” 2024. [Online]. Available: https://arxiv.org/abs/2402.01912

    2024 arXiv

  30. [38]

    CoVoST: A diverse multilingual speech-to-text translation corpus,

    C. Wang, J. Pino, et al. , “CoVoST: A diverse multilingual speech-to-text translation corpus,” in LREC. Marseille, France: European Language Resources Association, May 2020, pp. 4197–4203. [Online]. Available: https://www.aclweb.org/anthology/2020.lrec-1.517

    2020

  31. [39]

    CVSS corpus and massively multilingual speech-to-speech translation,

    Y . Jia, M. Tadmor Ramanovich, et al. , “CVSS corpus and massively multilingual speech-to-speech translation,” in LREC, 2022, pp. 6691– 6703

    2022

  32. [40]

    Mls: A large-scale multilingual dataset for speech research,

    V . Pratap, Q. Xu, et al. , “Mls: A large-scale multilingual dataset for speech research,” in Interspeech 2020 , 2020, pp. 2757–2761

    2020

  33. [41]

    Fleurs: Few-shot learning evaluation of universal representations of speech,

    A. Conneau, M. Ma, et al. , “Fleurs: Few-shot learning evaluation of universal representations of speech,” in 2022 IEEE Spoken Language Technology Workshop (SLT), 2023, pp. 798–805

    2022

  34. [42]

    Image method for efficiently simulating small-room acoustics,

    J. B. Allen and D. A. Berkley, “Image method for efficiently simulating small-room acoustics,” J. Acoust. Soc. Am. , 1979

    1979

  35. [43]

    HiFi-GAN: High-fidelity denoising and dereverberation based on speech deep features in adversarial networks,

    J. Su, Z. Jin, and A. Finkelstein, “HiFi-GAN: High-fidelity denoising and dereverberation based on speech deep features in adversarial networks,” in Interspeech, 2020

    2020

  36. [44]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, et al. , “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in ICML, 2006, p. 369–376. [Online]. Available: https://doi.org/10.1145/ 1143844.1143891

    2006

  37. [45]

    Transfer learning from speaker verification to multispeaker text-to-speech synthesis,

    Y . Jia, Y . Zhang,et al. , “Transfer learning from speaker verification to multispeaker text-to-speech synthesis,” in NeurIPS, 2018

    2018

  38. [46]

    Sample efficient adaptive text-to-speech,

    Y . Chen, Y . Assael,et al. , “Sample efficient adaptive text-to-speech,” in ICLR, 2019

    2019

  39. [47]

    DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors,

    C. K. Reddy, V . Gopal, and R. Cutler, “DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors,” in ICASSP. IEEE, 2021, pp. 6493–6497

    2021

  40. [48]

    SQuId: Measuring speech naturalness in many languages,

    T. Sellam, A. Bapna, et al. , “SQuId: Measuring speech naturalness in many languages,” arXiv:2210.06324, 2022

    2022 arXiv

  41. [49]

    Tf-gridnet: Making time-frequency domain models great again for monaural speaker separation,

    Z.-Q. Wang, S. Cornell, et al. , “Tf-gridnet: Making time-frequency domain models great again for monaural speaker separation,” in ICASSP, 2023, pp. 1–5

    2023

  42. [50]

    The design for the Wall Street Journal-based CSR corpus,

    D. B. Paul and J. M. Baker, “The design for the Wall Street Journal-based CSR corpus,” in Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992 , 1992. [Online]. Available: https://aclanthology.org/H92-1073/

    1992

  43. [51]

    Common V oice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, et al., “Common V oice: A massively-multilingual speech corpus,” in LREC, 2020

    2020

  44. [52]

    mHuBERT-147: A Compact Multilingual HuBERT Model,

    M. Z. Boito, V . Iyer, et al. , “mHuBERT-147: A Compact Multilingual HuBERT Model,” in Interspeech 2024 , 2024

    2024

  45. [53]

    Towards robust speech representation learning for thousands of languages,

    W. Chen, W. Zhang, et al. , “Towards robust speech representation learning for thousands of languages,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, Eds. Miami, Florida, USA: Association...

    2024

  46. [54]

    Nakata, https://github.com/Wataru-Nakata/miipher

    W. Nakata, https://github.com/Wataru-Nakata/miipher

  47. [55]

    Ikemiya, https://github.com/yukara-ikemiya/wavefit-pytorch

    Y . Ikemiya, https://github.com/yukara-ikemiya/wavefit-pytorch

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.