Pith. sign in

REVIEW 2 major objections 4 minor 42 references

Voice-ENHANCE: Speech Restoration using a Diffusion-based Voice Conversion Framework

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Voice-ENHANCE restores noisy speech by pairing speaker-agnostic generative cleanup with speaker-guided diffusion voice conversion, matching state-of-the-art MOS scores at a fraction of the largest competitor's size.

desk verdict A sensible two-stage restoration pipeline with promising objective scores, but the evaluation never specifies how the clean speaker enrollment is built, leaving a load-bearing hole in the SOTA comparison. read the letter →

arxiv 2505.15254 v1 pith:KCTBKPHM submitted 2025-05-21 cs.SD eess.AS

Kyungguen Byun , Jason Filos , Erik Visser , Sunkuk Moon This is my paper · ORCID
classification cs.SDeess.AS
keywords noisesuppressionspeechrestorationvoiceconversiondiffusionmodelsspeakerembeddingsself-supervisedlearninggenerativeenhancement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a voice conversion model, normally used to change who is speaking, can instead restore a noisy or distorted recording to near-studio quality when the target voice is the original speaker. The proposed system, Voice-ENHANCE, first passes degraded audio through a speaker-agnostic generative speech restoration (GSR) front end that suppresses noise and repairs damage, then through a diffusion-based voice conversion stage conditioned on a clean speaker embedding extracted from a short enrollment clip. On the VCTK-DEMAND and UNIVERSE benchmarks, this two-stage cascade matches or beats recent state-of-the-art restoration systems on non-intrusive MOS metrics while using 209 million parameters, less than half the size of the largest competitor. If correct, the result matters because it shows speaker identity guidance can substitute for large transcript-based or text-dependent models in high-quality speech restoration.

What carries the argument

The load-bearing mechanism is the two-stage cascade itself. The front end is GSR, a VoiceFixer-derived ResU-Net that predicts additive mel-spectrogram completions (element-wise addition rather than masking) and synthesizes waveforms with HiFi-GAN, trained with GAN, feature-matching, and mel losses. The back end is a Diff-VC-inspired diffusion model whose content encoder converts discrete HuBERT+VQ embeddings (cluster size 2000) through a transformer into a coarse spectrogram, and whose U-Net diffusion decoder is conditioned on that content and on a speaker embedding from ECAPA-TDNN; training uses an encoder L1 loss and a diffusion score-matching loss with classifier-free guidance at probability 0.1. A pre-trained HiFi-GAN generates the final waveform. The cascade works because the GSR stage makes the input clean enough for the VC stage, and the VC stage then uses speaker identity from a clean enrollment to regenerate residual speech components damaged by the first stage.

What would settle it

Run the proposed GSR+VC system on the UNIVERSE test set with three altered conditions for the speaker embedding: enrollment from a different speaker, enrollment corrupted with additive noise, and a randomly sampled embedding; if the reported NISQA (4.21) and DNSMOS (3.26) scores do not drop substantially, then the speaker guidance is not the mechanism carrying the second-stage improvement.

Watch

Extended reading notes

Core claim

The central claim is that a diffusion-based voice converter, fed with self-supervised content features and a clean speaker embedding, acts as an effective second-stage restorer after speaker-agnostic generative cleanup. The authors show that GSR alone outperforms its VoiceFixer baseline on both test sets, and that adding the VC stage further lifts all four evaluated MOS scores on both datasets. On the UNIVERSE set the full GSR+VC system beats the diffusion-based UNIVERSE model on three of four metrics and beats FINALLY on two of four, while remaining competitive or superior in model size. The ablation attributes a substantial part of the gain under severe distortion to replacing Mel-spectrogram content with HuBERT+VQ discrete units, and the authors argue the design avoids transcript data and large text-predictive SSL models used by similar approaches.

Load-bearing premise

The system requires a short, clean enrollment recording of the target speaker at inference time; if no clean speaker embedding can be extracted, the voice-conversion stage loses its guiding signal and the claimed quality gain collapses.

Editorial extensions

If this is right

  • GSR+VC outperforms the VoiceFixer baseline on both VCTK-DEMAND and UNIVERSE across all four non-intrusive metrics (NISQA, UTMOS, WV-MOS, DNSMOS).
  • On the UNIVERSE test set, the full system beats UNIVERSE on three of four metrics and FINALLY on two of four, while FINALLY is more than twice the size (454M vs 209M parameters).
  • The VC stage consistently improves over GSR alone on both datasets, and replacing Mel-spectrogram content with HuBERT+VQ features yields the largest gains under severe distortions.
  • Because the content path uses HuBERT+VQ and a speaker embedding rather than transcripts or text-based SSL models, the method avoids the data and compute overhead of approaches like Miipher.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical boundary follows that the paper does not discuss: the clean-enrollment requirement means the method is best suited to known-speaker scenarios such as personal assistants, hearing aids, or archive restoration, and it will not work out-of-the-box on completely unseen speakers without a supplementary speaker-embedding estimator.
  • An immediate testable extension is a sensitivity sweep over enrollment duration and noise level to find the minimum clean enrollment that preserves the reported gains; the paper's assumption that short uncorrelated clean segments are obtained beforehand is never stress-tested.
  • The ablation suggests the HuBERT+VQ path is the load-bearing innovation for severe distortions; swapping HuBERT for a different self-supervised speech feature would clarify whether the benefit is generic to SSL content features or specific to HuBERT's discrete units.
  • The same GSR+VC cascade could be redirected to cross-speaker conversion, turning the restorer into a voice style-transfer system without architectural change, since the only difference would be the target embedding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper proposes Voice-ENHANCE, a two-stage speech-restoration pipeline. The first stage, GSR, is a speaker-agnostic restoration network derived from VoiceFixer (ResU-Net analysis with HiFi-GAN synthesis) trained on noisy speech data with a GAN-based objective. The second stage is a diffusion-based voice-conversion-inspired model based on Diff-VC: HuBERT features are vector-quantized, passed through a transformer content encoder, and then a diffusion U-Net generates a mel-spectrogram conditioned on this content embedding and on an ECAPA-TDNN speaker embedding obtained from a clean enrollment utterance of the target speaker. Experiments on VCTK-DEMAND and UNIVERSE report NISQA, UTMOS, WV-MOS, and DNSMOS scores, comparing with VoiceFixer, UNIVERSE, and FINALLY, and include an ablation of GSR, VC(Mel), VC(SSL), and GSR+VC. The central claim is that the combined system achieves objective speech-quality scores comparable to state-of-the-art methods while using about 209M parameters.

Significance. The proposed combination is practically interesting: adding a diffusion-based voice-conversion stage after a speaker-agnostic restoration front end is a sensible way to exploit clean enrollment information, and the ablation study clearly shows the incremental contribution of the VC stage. The use of two public benchmarks and four non-intrusive quality metrics is a strength, as is the comparison with recent strong systems. However, the evaluation currently does not establish that the comparison is apples-to-apples, because the clean-enrollment protocol used for the speaker embedding is not specified. The central quantitative claim is therefore conditional on an unresolved evaluation detail, and the qualitative claim of 'studio-level quality' goes beyond what non-intrusive MOS predictors can support without listening tests.

major comments (2)
  1. [Section 4.1, Tables 1-3] The central comparative claim is confounded by an unspecified clean-enrollment protocol. Section 1 states that 'short, uncorrelated segments of clean speech are obtained beforehand,' and Section 3.2 conditions the diffusion model on an ECAPA-TDNN speaker embedding extracted from clean speech. Both VCTK-DEMAND and UNIVERSE contain clean reference utterances from the test speakers (the Ground Truth rows in Tables 1 and 2), and Section 4.1 describes the UNIVERSE validation clips as formed from clean utterances sampled from VCTK and Harvard sentences. If the speaker embedding used at inference is derived from a clean reference of the test utterance, or even from another clean utterance of the same test speaker, then GSR+VC receives oracle speaker-identity information that UNIVERSE and FINALLY do not receive. Table 3 shows that the entire advantage of GSR+VC over GSR (NISQA 3.84 to 4.24 on VCTK-DEMAND and 3.64 to 4.21 on UNIVERSE) comes from the VC stage, so this is exactly the information that could explain the comparative result. The paper must specify the enrollment protocol and supplement the comparison with at least one of the following: speaker embeddings extracted from the degraded input itself, enrollment from disjoint clean utterances that are not the test references, or an explicit matched-condition comparison showing results without clean enrollment.
  2. [Section 4.2, Tables 1-2] All reported scores are outputs of non-intrusive quality predictors (NISQA, UTMOS, WV-MOS, DNSMOS), and the paper reports no confidence intervals, standard deviations, or significance tests. Several comparative claims rest on differences as small as 0.01-0.03 MOS points (e.g., DNSMOS 3.26 vs. 3.23 and NISQA 4.21 vs. 4.20 on UNIVERSE), which are likely within the utterance-level or predictor variability. In addition, the abstract and Section 5 describe the output as 'studio-level quality,' a claim that cannot be supported by non-intrusive MOS predictors alone without listening tests. I request that the authors report uncertainty estimates (e.g., bootstrap confidence intervals over utterances), perform statistical tests on the key comparisons, or substantially temper the qualitative claims.
minor comments (4)
  1. [Section 4.3, Table 1] The text says GSR on its own outperforms VoiceFixer on VCTK-DEMAND, but in Table 1 the WV-MOS score for GSR is 4.12 versus 4.18 for VoiceFixer; the claim should be limited to NISQA, UTMOS, and DNSMOS.
  2. [Section 4.3, Table 1] The statement that the proposed method 'outperforms the baseline VoiceFixer and the clean speech reference' on VCTK-DEMAND is not supported by the WV-MOS row, where the proposed method scores 4.34 and the ground truth scores 4.50.
  3. [Section 4.1, Table 4] There is a numerical inconsistency in the GSR training-data size: Section 4.1 says each epoch contains 175 hours of noisy speech, while Table 4 reports 1750 hours of private data for GSR. The correct value should be stated.
  4. [Section 3.2, Figure 2] The description of the 'training and content encoder adaptation framework' in Figure 2 is not fully expanded in the text; a short paragraph describing what is frozen, what is fine-tuned, and what is trained from scratch would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: evaluation is against held-out public datasets and no fitted quantity is relabeled as a prediction.

full rationale

Walking the paper's derivation chain: GSR is presented as an adaptation of VoiceFixer with a modified residual spectrogram objective, and the VC stage is a Diff-VC variant using HuBERT+VQ content features, an ECAPA-TDNN speaker embedding, and a diffusion loss. The training data are proprietary speech enhancement data for GSR and LibriTTS for the VC module, while the evaluation uses the public VCTK-DEMAND and UNIVERSE validation sets with external non-intrusive metrics (NISQA, UTMOS, WV-MOS, DNSMOS). Nothing in Eqs. (1)-(3) defines the reported metric scores in terms of the model's inputs, and no parameter is fitted to those test metrics. The GSR+VC gain over GSR alone is presented as an empirical ablation result, not as an identity forced by construction. The assumptions about clean enrollment speech and classifier-free guidance are architectural and deployment assumptions, not circular definitions of the output quality. The paper also does not rely on load-bearing self-citations: VoiceFixer and Diff-VC are external prior works credited as baselines or inspiration, not invoked as author-derived uniqueness theorems. The underspecified clean-enrollment protocol for the two test sets is a legitimate experimental-design and evaluation-validity concern, but it is not a circularity: even if the enrollment were oracle-like, the claimed result would be confounded rather than equivalent to its input by definition. Accordingly, no specific circular step can be quoted, and the correct finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters (loss weight, guidance scales, steps, cluster size), the availability of clean enrollment speech at inference, and the validity of non-intrusive metrics as perceptual proxies. No new physical entities are introduced; pre-trained models (HuBERT, ECAPA-TDNN, HiFi-GAN) are taken as given.

free parameters (5)
  • alpha (encoder loss weight) = 0.5
    Weight in L_total = L_d + alpha * L_enc; chosen to balance content encoder and diffusion losses.
  • classifier-free guidance scale (speaker) = 0.25
    Inference-time scaling in classifier-free guidance; reported in Section 4.1.
  • classifier-free guidance scale (content) = 1.0
    Inference-time scaling in classifier-free guidance; reported in Section 4.1.
  • diffusion steps = 30
    Number of reverse diffusion steps at inference; affects quality and compute.
  • VQ cluster size = 2000
    Number of HuBERT discrete units; chosen for content representation granularity.
assumptions (5)
  • standard math The diffusion reverse process trained with score matching can generate high-fidelity speech from the forward-noised input.
    Background in Section 2 cites score-based generative modeling; the VC stage relies on this.
  • domain assumption Clean speaker embeddings are available at inference from short uncorrelated enrollment segments.
    Stated in Section 1: 'We assume short, uncorrelated segments of clean speech are obtained beforehand.' This restricts deployment.
  • domain assumption The non-intrusive metrics NISQA, UTMOS, WV-MOS, and DNSMOS are valid proxies for perceptual quality.
    All evaluation uses these predictors; no listening test is reported (Section 4.2).
  • domain assumption Pre-trained HuBERT and ECAPA-TDNN models provide sufficient content and speaker information for restoration.
    Used as feature extractors in the VC stage; their outputs are treated as ground-truth conditioners.
  • ad hoc to paper A two-stage decomposition (restoration followed by voice conversion) does not compound errors and improves final quality.
    This is the core design hypothesis validated by the ablation in Table 3, but not proven beyond these datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Voice-ENHANCE: Speech Restoration using a Diffusion-based Voice Conversion Framework." pith.science (2026). https://pith.science/paper/KCTBKPHM

@misc{pith2026250515254,
  author       = {Pith},
  title        = {Pith review of: Voice-ENHANCE: Speech Restoration using a Diffusion-based Voice Conversion Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KCTBKPHM}},
  note         = {Machine review of arXiv:2505.15254}
}
read the original abstract

We propose a speech enhancement system that combines speaker-agnostic speech restoration with voice conversion (VC) to obtain a studio-level quality speech signal. While voice conversion models are typically used to change speaker characteristics, they can also serve as a means of speech restoration when the target speaker is the same as the source speaker. However, since VC models are vulnerable to noisy conditions, we have included a generative speech restoration (GSR) model at the front end of our proposed system. The GSR model performs noise suppression and restores speech damage incurred during that process without knowledge about the target speaker. The VC stage then uses guidance from clean speaker embeddings to further restore the output speech. By employing this two-stage approach, we have achieved speech quality objective metric scores comparable to state-of-the-art (SOTA) methods across multiple datasets.

Figures

Figures reproduced from arXiv: 2505.15254 by the authors.

Figure 1
Figure 1. Overview of Voice-Enhance framework, noisy mixture is given to speaker-agnostic Generative speech restoration (GSR) module, then VC-inspired generative model generates high quality output guided by speaker embedding from the clean speech enroll￾ment. x via time-domain methods, or mask-based frequency-domain methods. The goal, however in speech restoration is to find a mapping function g : R T → R T that transforms t… view at source ↗
Figure 2
Figure 2. Training and content encoder adaptation framework for voice conversion-based speech restoration model. Next, using a U-Net decoder in the diffusion process, which takes the coarse spectrogram and speaker embedding, we obtain a restored spectrogram that closely matches the target. For the speaker encoder, we use the pre-trained ECAPA-TDNN [22] model, which provides robust and discriminative speaker rep￾resentations. … view at source ↗
Figure 3
Figure 3. Spectrogram comparison: (a) input mixture with noise and packet loss, (b) GSR output, and (c) final output from the proposed GSR+VC model. We conducted experiments to verify the performance of the proposed method and compared it to three other state-of-the￾art (SOTA) models: VoiceFixer (used as a baseline for GSR), FINALLY, and UNIVERSE (used for comparison with our fi￾nal proposed system). We used two validation da… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 22 canonical work pages

  1. [1]

    Introduction Speech enhancement (SE), also known as noise suppression (NS), is crucial in real-world applications such as telecommuni- cations, voice assistants, and hearing aids. One notable limita- tion of SE systems is their performance in challenging signal-to- noise ratio (SNR) scenarios, where the enhanced speech often ends up being muffled or chopp...

    work page

  2. [2]

    Voice-ENHANCE: Speech Restoration using a Diffusion-based Voice Conversion Framework

    Background Speech restoration is a challenging task that aims to recover speech components from various degradation factors. We can define the degraded speech signalxasx=f(s) +n∈R T , wherefrepresents various degradation factors, andsandnare the original speech and noise signals, respectively. It is common in speech enhancement to predict the speech direc...

    work page Pith review arXiv 2025

  3. [3]

    Proposed method An overview of our proposed system is shown in Fig. 1. The noisy speech is first processed by the speaker-agnostic GSR model. In this work, we considerzas the Mel-spectrogram ofxas our intermediate representation and use HiFi-GAN [21] as the generator. Then, a VC model generates restored speech by conditioning the diffusion process with a ...

    work page 2000

  4. [4]

    Evaluation 4.1. Database and Training Setting GSR is trained on a proprietary speech enhancement dataset for 100 epochs with a batch size of 32, the AdamW [24] optimizer and an initial learning rate of 2e-4, which decays exponentially withγ= 0.999. Each epoch contains 175 hours of noisy speech in the SNR range [-15, 40] dB. We used similar aug- mentations...

    work page

  5. [5]

    Our method leverages a two- stage approach, incorporating a generative speech restoration (GSR) model at the front end, followed by a VC module

    Conclusion In this paper, we proposed an integrated framework that com- bines speech restoration and diffusion-based voice conversion (VC) to enhance speech quality. Our method leverages a two- stage approach, incorporating a generative speech restoration (GSR) model at the front end, followed by a VC module. This design addresses the vulnerability of VC ...

    work page

  6. [6]

    and exhibiting a better DNSMOS score on VCTK-DEMAND (Table 1) whilst being less than half the size. Finally, it is noted that despite the similarities between our proposed method and Miipher, conducting comparative experiments was challenging due to the unavailability of Miipher’s data and model. Fig. 3 illustrates an input example and stage-wise outputs ...

    work page

  7. [7]

    Generative adver- sarial networks,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adver- sarial networks,”Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020

    2020

  8. [8]

    Supervised speech separation based on deep learning: An overview,

    D. Wang and J. Chen, “Supervised speech separation based on deep learning: An overview,”IEEE/ACM transactions on audio, speech, and language processing, vol. 26, no. 10, pp. 1702–1726, 2018

    2018

Show all 42 references
  1. [9]

    V oicefixer: A unified framework for high-fidelity speech restoration,

    H. Liu, X. Liu, Q. Kong, Q. Tian, Y . Zhao, D. Wang, C. Huang, and Y . Wang, “V oicefixer: A unified framework for high-fidelity speech restoration,”arXiv preprint arXiv:2204.05841, 2022

    2022 arXiv

  2. [10]

    Restoring de- graded speech via a modified diffusion model,

    J. Zhang, S. Jayasuriya, and V . Berisha, “Restoring de- graded speech via a modified diffusion model,”arXiv preprint arXiv:2104.11347, 2021

    2021 arXiv

  3. [11]

    Universal speech enhancement with score-based diffusion,

    J. Serr `a, S. Pascual, J. Pons, R. O. Araz, and D. Scaini, “Universal speech enhancement with score-based diffusion,”arXiv preprint arXiv:2206.03065, 2022

    2022 arXiv

  4. [12]

    Finally: fast and universal speech enhancement with studio-like quality,

    N. Babaev, K. Tamogashev, A. Saginbaev, I. Shchekotov, H. Bae, H. Sung, W. Lee, H.-Y . Cho, and P. Andreev, “Finally: fast and universal speech enhancement with studio-like quality,”arXiv preprint arXiv:2410.05920, 2024

    2024 arXiv

  5. [13]

    Cmgan: Conformer-based metric-gan for monaural speech enhancement,

    S. Abdulatif, R. Cao, and B. Yang, “Cmgan: Conformer-based metric-gan for monaural speech enhancement,”arXiv preprint arXiv:2209.11112, 2022

    2022 arXiv

  6. [14]

    Two- stage single-channel audio source separation using deep neural networks,

    E. M. Grais, G. Roma, A. J. Simpson, and M. D. Plumbley, “Two- stage single-channel audio source separation using deep neural networks,”IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, vol. 25, no. 9, pp. 1773–1783, 2017

    2017

  7. [15]

    Score-based generative modeling through stochas- tic differential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochas- tic differential equations,”arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  8. [16]

    Vqmivc: Vector quantization and mutual information-based un- supervised speech representation disentanglement for one-shot voice conversion,

    D. Wang, L. Deng, Y . T. Yeung, X. Chen, X. Liu, and H. Meng, “Vqmivc: Vector quantization and mutual information-based un- supervised speech representation disentanglement for one-shot voice conversion,”arXiv preprint arXiv:2106.10132, 2021

    2021 arXiv

  9. [17]

    Freevc: Towards high-quality text-free one-shot voice conversion,

    J. Li, W. Tu, and L. Xiao, “Freevc: Towards high-quality text-free one-shot voice conversion,” 2023

    2023

  10. [18]

    Cyclegan- vc3: Examining and improving cyclegan-vcs for mel-spectrogram conversion,

    T. Kaneko, H. Kameoka, K. Tanaka, and N. Hojo, “Cyclegan- vc3: Examining and improving cyclegan-vcs for mel-spectrogram conversion,”arXiv preprint arXiv:2010.11672, 2020

    2010 arXiv

  11. [19]

    Diffusion-based voice conversion with fast maximum likelihood sampling scheme,

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, M. S. Kudinov, and J. Wei, “Diffusion-based voice conversion with fast maximum likelihood sampling scheme,” inICLR, 2022

    2022

  12. [20]

    Speech enhancement by lstm-based noise suppression followed by cnn-based speech restoration,

    M. Strake, B. Defraene, K. Fluyt, W. Tirry, and T. Fingscheidt, “Speech enhancement by lstm-based noise suppression followed by cnn-based speech restoration,”EURASIP Journal on Advances in Signal Processing, vol. 2020, pp. 1–26, 2020

    2020

  13. [21]

    Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,”NeurIPS, 2020

    2020

  14. [22]

    Convolutional neural net- works to enhance coded speech,

    Z. Zhao, H. Liu, and T. Fingscheidt, “Convolutional neural net- works to enhance coded speech,”IEEE/ACM Transactions on Au- dio, Speech, and Language Processing, vol. 27, no. 4, pp. 663– 678, 2018

    2018

  15. [23]

    Separated noise suppression and speech restoration: Lstm-based speech enhancement in two stages,

    M. Strake, B. Defraene, K. Fluyt, W. Tirry, and T. Fingscheidt, “Separated noise suppression and speech restoration: Lstm-based speech enhancement in two stages,” in2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WAS- PAA). IEEE, 2019, pp. 239–243

    2019

  16. [24]

    Audiolm: a language modeling approach to audio gener- ation,

    Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Sharifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchi et al., “Audiolm: a language modeling approach to audio gener- ation,”IEEE/ACM transactions on audio, speech, and language processing, vol. 31, pp. 25...

    2023

  17. [25]

    Grad-tts: A diffusion probabilistic model for text-to-speech,

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, and M. Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” in International Conference on Machine Learning. PMLR, 2021, pp. 8599–8608

    2021

  18. [26]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W. Hsu, B. Bolte, Y . H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”IEEE ACM Trans. Audio Speech Lang. Process., vol. 29, pp. 3451–3460, 2021

    2021

  19. [27]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,”arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  20. [28]

    Noisy speech database for training speech enhancement algorithms and tts models,

    C. Valentini-Botinhaoet al., “Noisy speech database for training speech enhancement algorithms and tts models,”University of Ed- inburgh. School of Informatics. Centre for Speech Technology Re- search (CSTR), 2017

    2017

  21. [29]

    ECAPA- TDNN: emphasized channel attention, propagation and aggrega- tion in TDNN based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “ECAPA- TDNN: emphasized channel attention, propagation and aggrega- tion in TDNN based speaker verification,” inInterspeech, 2020

    2020

  22. [30]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,”CoRR, vol. abs/2207.12598, 2022

    2022 arXiv

  23. [31]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,”arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  24. [32]

    Miipher: A ro- bust speech restoration model integrating self-supervised speech and text representations,

    Y . Koizumi, H. Zen, S. Karita, Y . Ding, K. Yatabe, N. Morioka, Y . Zhang, W. Han, A. Bapna, and M. Bacchiani, “Miipher: A ro- bust speech restoration model integrating self-supervised speech and text representations,” in2023 IEEE Workshop on Applications of Signal Processing...

    2023

  25. [33]

    Libritts: A corpus derived from librispeech for text- to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text- to-speech,” inInterspeech, 2019

    2019

  26. [34]

    Adam: A method for stochastic optimization,

    P. K. Diederik, “Adam: A method for stochastic optimization,” (No Title), 2014

    2014

  27. [35]

    GSR also considers more augmentations than V oiceFixer, e.g., codec artifacts and packet drops

    for V oiceFixer). GSR also considers more augmentations than V oiceFixer, e.g., codec artifacts and packet drops. The experimental results on the VCTK-DEMAND and UNIVERSE validation sets demonstrate the effectiveness of our proposed GSR+VC method. On the VCTK-DEMAND set, our m...

  28. [36]

    NISQA: A deep cnn-self-attention model for multidimensional speech quality pre- diction with crowdsourced datasets,

    G. Mittag, B. Naderi, A. Chehadi, and S. M¨oller, “NISQA: A deep cnn-self-attention model for multidimensional speech quality pre- diction with crowdsourced datasets,” inInterspeech, 2021

    2021

  29. [37]

    Utmos: Utokyo-sarulab system for voicemos challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “Utmos: Utokyo-sarulab system for voicemos challenge 2022,”arXiv preprint arXiv:2204.02152, 2022

    2022 arXiv

  30. [38]

    Hifi++: a unified framework for neural vocoding, bandwidth extension and speech enhancement,

    P. Andreev, A. Alanov, O. Ivanov, and D. Vetrov, “Hifi++: a unified framework for neural vocoding, bandwidth extension and speech enhancement,” 2022. [Online]. Available: https: //arxiv.org/abs/2203.13086

    2022 arXiv

  31. [39]

    Dnsmos p. 835: A non-intrusive perceptual objective speech quality metric to eval- uate noise suppressors,

    C. K. Reddy, V . Gopal, and R. Cutler, “Dnsmos p. 835: A non-intrusive perceptual objective speech quality metric to eval- uate noise suppressors,” inICASSP 2022-2022 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 886–890

    2022

  32. [40]

    Png bert: Aug- mented bert on phonemes and graphemes for neural tts,

    Y . Jia, H. Zen, J. Shen, Y . Zhang, and Y . Wu, “Png bert: Aug- mented bert on phonemes and graphemes for neural tts,”arXiv preprint arXiv:2103.15060, 2021

    2021 arXiv

  33. [41]

    W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training,

    Y .-A. Chung, Y . Zhang, W. Han, C.-C. Chiu, J. Qin, R. Pang, and Y . Wu, “W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2021, p...

    2021

  34. [42]

    Tfgan: Time and frequency domain based generative adversar- ial network for high-fidelity speech synthesis,

    Q. Tian, Y . Chen, Z. Zhang, H. Lu, L. Chen, L. Xie, and S. Liu, “Tfgan: Time and frequency domain based generative adversar- ial network for high-fidelity speech synthesis,”arXiv preprint arXiv:2011.12206, 2020

    2011 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.