Pith. sign in

REVIEW 3 major objections 6 minor 73 references

ESTVocoder: An Excitation-Spectral-Transformed Neural Vocoder Conditioned on Mel Spectrogram

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ESTVocoder claims that a full-harmonic, F0-derived excitation, transformed into speech spectra by a ConvNeXt v2 neural filter, yields a vocoder that matches or beats HiFi-GAN, SiFi-GAN, and Vocos while training markedly faster.

desk verdict A solid architecture and fair comparison, but 'outperforms' and the excitation-prior convergence claim both exceed the evidence; worth refereeing with revisions. read the letter →

arxiv 2411.11258 v1 pith:NCI7F63P submitted 2024-11-18 cs.SD eess.AS

Xiao-Hang Jiang , Hui-Peng Du , Yang Ai , Ye-Xin Lu , Zhen-Hua Ling This is my paper · ORCID
classification cs.SDeess.AS
keywords ESTVocoderneuralvocodersource-filtertheoryexcitationsignalspectraltransformationConvNeXtv2adversarialtrainingtext-to-speech
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ESTVocoder tries to show that a neural vocoder works better when the network is given a spectral prior drawn from the source-filter idea: construct an excitation from the fundamental frequency, transform that excitation's amplitude and phase spectra into speech spectra, then rebuild the waveform. The paper's central assertion is that this excitation-spectral transformation, trained adversarially, delivers synthesized speech quality comparable to or better than leading waveform- and spectral-domain vocoders, and that the excitation measurably accelerates training convergence. A sympathetic reader would care because the claim, if true, points to a cheap way to lower the modeling burden in neural vocoders: provide the network with physically meaningful spectral structure instead of making it learn everything from the mel spectrogram alone.

What carries the argument

The central object is the excitation-spectral-transformed neural filter: an untrainable excitation producer that turns an F0 contour into a full-harmonic sine-plus-noise waveform, followed by an STFT that yields excitation amplitude and phase spectra, and a trainable ConvNeXt v2 network that maps those spectra, conditioned on the mel spectrogram, to predicted speech amplitude and phase spectra. The excitation's full-harmonic phase accumulation, $\sum_{h=1}^t 2\pi k f_h / S_r$, is what gives the filter a phase prior; the ConvNeXt v2 blocks with global response normalization perform the spectral refinement. This machinery is doing the work of reducing modeling difficulty: the filter starts from a spectrum that already resembles speech, rather than from Gaussian noise or mel features alone.

What would settle it

Train ESTVocoder with the excitation's phase spectrum replaced by uniformly random phases in $[0, 2\pi)$ while keeping the full-harmonic amplitude spectrum unchanged, and compare mel-loss training curves and PESQ/ViSQOL against the original ESTVocoder; if quality and convergence are unchanged, the claim that the F0-derived phase prior accelerates learning is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that an excitation built from F0, containing all harmonics for voiced frames and Gaussian noise for unvoiced frames, carries enough spectral prior that a neural filter can transform its amplitude and phase spectra into accurate speech amplitude and phase spectra. The excitation phase is synthesized by accumulating pitch-rate phase increments $2\pi k f_h / S_r$ across samples, giving a coherent harmonic phase profile. Conditioned on the mel spectrogram, a ConvNeXt v2 backbone refines these spectra, and inverse STFT reconstructs the waveform; adversarial training with multi-period and multi-resolution discriminators keeps the output faithful. The authors report that ESTVocoder achieves the best mel cepstrum distortion, F0 error, V/UV error, PESQ, and ViSQOL among the compared vocoders in analysis-synthesis, and MOS statistically comparable to HiFi-GAN and Vocos in TTS, with CPU generation more than ten times faster than the waveform-prediction baselines and training roughly 40 percent faster per epoch than Vocos.

Load-bearing premise

The load-bearing premise is that the synthetic excitation's phase spectrum, built by accumulating F0 over time, is close enough to natural speech phase to give the network a useful head start; if that phase pattern is not meaningfully speech-like, the claimed reduction in modeling difficulty and the convergence acceleration would lack support.

Editorial extensions

If this is right

  • If ESTVocoder's central claim is correct, a full-harmonic excitation gives a neural vocoder a head start: training reaches good mel-spectrogram loss in fewer iterations, lowering the compute cost per deployed voice.
  • Spectral-domain source-filter synthesis can combine pitch controllability with fast CPU inference, since the network operates at frame rate on spectra rather than predicting each waveform sample, and the F0-to-excitation path keeps fundamental frequency accuracy explicit.
  • The ablation evidence suggests the excitation prior, not just the ConvNeXt v2 backbone, is responsible for the quality gain: replacing the excitation with the mel-only path degrades objective metrics and intermediate spectral fidelity.
  • The TTS MOS results imply that ESTVocoder does not amplify acoustic-model errors beyond what standard vocoders do, so it can be dropped into existing text-to-speech pipelines as a compatible, faster-training waveform backend.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is testing how much of the benefit comes specifically from the phase prior: ablating excitation phase by replacing it with random phases while keeping the harmonic amplitude would isolate whether the claimed convergence acceleration is due to phase structure or merely to having a harmonic amplitude scaffold.
  • The phase-accumulation excitation should transfer well to expressive or singing voices, where harmonic structure is strong and F0 variation is fast; the paper's experiments only cover a single English speaker, so multi-speaker and singing tests would show whether the prior generalizes.
  • The same excitation-spectral transform could be reused as a conditioning pathway in speech enhancement or neural audio codecs, since both tasks need to reconstruct phase-rich spectra from degraded or compressed inputs, and the paper names speech enhancement as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. ESTVocoder is a source-filter neural vocoder that constructs a full-harmonic excitation waveform from F0, transforms its STFT amplitude and phase spectra with a ConvNeXt v2-based neural filter conditioned on a mel spectrogram, reconstructs the waveform via ISTFT, and trains adversarially with MPD and MRD discriminators plus a mel-spectrogram loss. On the LJSpeech analysis-synthesis experiments, the paper reports the best MCD, F0-RMSE, V/UV error, PESQ, and ViSQOL among HiFi-GAN, SiFi-GAN, and Vocos, while its LAS-RMSE is worse than SiFi-GAN. In the TTS MOS test, differences between ESTVocoder and HiFi-GAN or Vocos are not statistically significant. The authors further argue, based on a comparison with Vocos-CNXv2 and on training-loss curves, that the excitation provides spectral prior information that accelerates convergence.

Significance. If the claims hold, ESTVocoder is a useful contribution to spectral-domain source-filter vocoding: it combines an F0-derived full-harmonic excitation with ConvNeXt v2 spectral transformation, and it reports meaningful engineering comparisons on generation and training speed. The paper has concrete strengths: the baselines are trained on the same data, the full-harmonic excitation is clearly specified, audio samples are provided, and the TTS MOS comparison is accompanied by significance tests. However, the central mechanistic claim that the excitation accelerates convergence through spectral prior information is supported only by a confounded comparison, and the objective 'outperforms' claim lacks inferential statistics. The contribution is incremental but likely of interest to the speech synthesis community.

major comments (3)
  1. [§3.3, 'Analysis of the role of the excitation producer'] The comparison between ESTVocoder and Vocos-CNXv2 is not a valid ablation of the excitation producer. The two models differ not only in the presence of the excitation spectra but also in the conditioning pathway (concatenated excitation spectra plus mel versus mel alone), in the initial projection layers, and in the input dimensionality of the network. As a result, the Table 4 improvements and the Fig. 6 convergence difference cannot be attributed specifically to the excitation prior. Please provide a controlled ablation in which the excitation branch is removed or zeroed while the input interface and projection remain identical, and report results over multiple seeds. The phase and amplitude components of the excitation should also be varied independently.
  2. [Table 1 and §3.2] All objective metrics in Table 1 are point estimates without standard errors, confidence intervals, or significance tests, and several differences are small (e.g., PESQ 3.70 vs. 3.51; ViSQOL 4.895 vs. 4.863). The paper's 'outperforms' claim therefore rests on non-inferential comparisons. This is particularly important because ESTVocoder is substantially worse than SiFi-GAN on LAS-RMSE (36.51 dB vs. 29.39 dB); the argument that MCD is more perceptually relevant does not remove the need to report uncertainty or to test whether the claimed advantages are reliable. Please add paired significance tests or bootstrap intervals and interpret the LAS-RMSE result directly.
  3. [§2.2 and §3.3] The excitation of Eq. (5) simultaneously provides a full harmonic amplitude comb and a cumulative-phase harmonic phase, but the paper never tests these components separately or evaluates whether the synthetic phase is actually close to the natural phase. The convergence-acceleration claim is specifically attributed to 'speech spectral prior information', yet the evidence only shows that a different architecture with an extra input channel converges faster. Without an ablation isolating the phase prior (e.g., using random, zero, or noisy phase while keeping the harmonic amplitude comb), the phase-prior interpretation remains unsupported.
minor comments (6)
  1. [§2.2, Eq. (5)] The definition K = floor(Sr/2 / min(f)) is ambiguous because min(f) is undefined when some frames are unvoiced (f_t = 0); please specify that the minimum is taken over voiced F0 values or define a floor for unvoiced frames.
  2. [§3.1, 'Evaluation metircs'] There is a typo 'Evaluation metircs'; additionally, Table 4 labels F0-RMSE as dB while Table 1 uses cent, and the unit row in Table 4 is inconsistent. Please unify the units across tables.
  3. [§3.3] The models Vocos-CNXv2 and ESTVocoder-CNXv1 are introduced only in the analysis section; they should be defined in the experimental setup, and the phrase 'approximately regarded as the ablated version' should be replaced by a precise statement of which components differ.
  4. [Fig. 6] The training curves show a single run without error bars or multiple seeds; please state the number of runs and whether the plotted curves are representative.
  5. [§3.2, TTS evaluation] Reporting only average MOS and p > 0.05 is insufficient for a meaningful 'comparable' claim; please report the number of ratings per system and confidence intervals or effect sizes.
  6. [Introduction] There is a typo in 'acoustic fratures' in the second paragraph; it should read 'acoustic features'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the ESTVocoder claims are supported by held-out comparisons against external baselines, and the only same-author citation is a downstream F0-prediction tool, not a load-bearing premise.

full rationale

ESTVocoder is an empirical supervised-learning paper. The generator is trained on a training split and evaluated on a held-out test split against externally implemented baselines (HiFi-GAN, SiFi-GAN, Vocos). No reported result is algebraically identical to a fitted parameter or to the input conditioning by construction: Eqs. (1)-(4) define the architecture, but the evaluated metrics (MCD, PESQ, ViSQOL, F0-RMSE) are measured on synthesized waveforms and are not forced by those equations. The F0 predictor from the authors' prior work [37] is used only as a component in the TTS pipeline, so it is a minor self-citation that is not load-bearing for the central derivation. The main interpretational weakness is in Section 3.3, where Vocos-CNXv2 is described as 'approximately' the ablated version of ESTVocoder without the excitation producer; this comparison is confounded by differences in input channels, conditioning, and front-end projection, so the causal attribution of faster convergence to the excitation prior is not rigorously isolated. That is a correctness/validity concern about an empirical ablation, not a circularity of the kind where a derivation reduces to its own input. No circular step can be exhibited from the paper's equations or claims.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The model introduces no new physical entities. It depends on un-reported hyperparameters and domain assumptions about F0-based excitation. The core components (ConvNeXt v2, MPD, MRD, ISTFT, mel conditioning) are from prior literature.

free parameters (4)
  • alpha (excitation amplitude scale)
    Hyperparameter in Eq. (5) controlling the amplitude of harmonic components in voiced excitation; value not reported.
  • sigma (noise standard deviation)
    Hyperparameter in Eq. (5) controlling unvoiced noise amplitude; value not reported.
  • lambda_M (mel loss weight)
    Weight for mel-spectrogram MAE loss in Eq. (9); not reported.
  • lambda_MRD (MRD loss weight)
    Weight for multi-resolution discriminator losses in Eq. (9); not reported.
assumptions (3)
  • domain assumption F0 can be extracted reliably and serves as the sole source of excitation structure.
    The excitation producer (Eq. 5) depends entirely on the F0 contour; errors in F0 propagate to the excitation and to synthesized speech.
  • domain assumption A synthetic harmonic excitation with cumulative phases contains a useful spectral prior for natural speech.
    The claimed benefit of the excitation (Section 2.2) is that its spectra already resemble speech spectra; if false, the mechanism fails.
  • domain assumption ConvNeXt v2 blocks, validated on image tasks, transfer their modeling advantage to speech spectra.
    Justified in Section 3.3 by ablation, but the transfer is not a mathematical guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ESTVocoder: An Excitation-Spectral-Transformed Neural Vocoder Conditioned on Mel Spectrogram." pith.science (2026). https://pith.science/paper/NCI7F63P

@misc{pith2026241111258,
  author       = {Pith},
  title        = {Pith review of: ESTVocoder: An Excitation-Spectral-Transformed Neural Vocoder Conditioned on Mel Spectrogram},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NCI7F63P}},
  note         = {Machine review of arXiv:2411.11258}
}
read the original abstract

This paper proposes ESTVocoder, a novel excitation-spectral-transformed neural vocoder within the framework of source-filter theory. The ESTVocoder transforms the amplitude and phase spectra of the excitation into the corresponding speech amplitude and phase spectra using a neural filter whose backbone is ConvNeXt v2 blocks. Finally, the speech waveform is reconstructed through the inverse short-time Fourier transform (ISTFT). The excitation is constructed based on the F0: for voiced segments, it contains full harmonic information, while for unvoiced segments, it is represented by noise. The excitation provides the filter with prior knowledge of the amplitude and phase patterns, expecting to reduce the modeling difficulty compared to conventional neural vocoders. To ensure the fidelity of the synthesized speech, an adversarial training strategy is applied to ESTVocoder with multi-scale and multi-resolution discriminators. Analysis-synthesis and text-to-speech experiments both confirm that our proposed ESTVocoder outperforms or is comparable to other baseline neural vocoders, e.g., HiFi-GAN, SiFi-GAN, and Vocos, in terms of synthesized speech quality, with a reasonable model complexity and generation speed. Additional analysis experiments also demonstrate that the introduced excitation effectively accelerates the model's convergence process, thanks to the speech spectral prior information contained in the excitation.

Figures

Figures reproduced from arXiv: 2411.11258 by the authors.

Figure 1
Figure 1. Overview of the proposed ESTVocoder. Here, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Visualization of the amplitude spectrum of the excitation waveform and [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The specific compositional structure of the ConvNeXt v2 block. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The structure of the F0 predictor. Here, [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualized amplitude spectra of natural speech and speeches generated by [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Training curves of mel-spectrogram loss for ESTVocoder and Vocos [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 59 canonical work pages

  1. [1]

    Apcodec: A neural audio codec with parallel amplitude and phase spectrum encoding and decoding

    YangAi,Xiao-HangJiang,Ye-XinLu,Hui-PengDu,andZhen-HuaLing. Apcodec: A neural audio codec with parallel amplitude and phase spectrum encoding and decoding. arXiv preprint arXiv:2402.10533 , 2024

    arXiv 2024

  2. [2]

    Yang Ai and Zhen-Hua Ling. A neural vocoder with hierarchical generation of am- plitude and phase spectra for statistical parametric speech synthesis.IEEE/ACM Transactions on Audio, Speech, and Language Processing , 28:839–851, 2020

    work page 2020

  3. [3]

    APNet: An all-frame-level neural vocoder incorpo- rating direct prediction of amplitude and phase spectra.IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2023

    Yang Ai and Zhen-Hua Ling. APNet: An all-frame-level neural vocoder incorpo- rating direct prediction of amplitude and phase spectra.IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2023

    work page 2023

  4. [4]

    Neural speech phase prediction based on parallel estimation architecture and anti-wrapping losses

    Yang Ai and Zhen-Hua Ling. Neural speech phase prediction based on parallel estimation architecture and anti-wrapping losses. In Proc. ICASSP, pages 1–5, 2023

    work page 2023

  5. [5]

    Long-frame-shift neural speech phase prediction with spectral continuity enhancement and interpolation error compen- sation

    Yang Ai, Ye-Xin Lu, and Zhen-Hua Ling. Long-frame-shift neural speech phase prediction with spectral continuity enhancement and interpolation error compen- sation. IEEE Signal Processing Letters , 2023

    work page 2023

  6. [6]

    vq-wav2vec: Self-supervised learning of discrete speech representations.arXiv preprint arXiv:1910.05453, 2019

    Alexei Baevski, Steffen Schneider, and Michael Auli. vq-wav2vec: Self-supervised learning of discrete speech representations.arXiv preprint arXiv:1910.05453, 2019

    arXiv 1910

  7. [7]

    Audiolm: a language modeling approach to audio generation

    Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco ESTVocoder 15 Tagliasacchi, et al. Audiolm: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2023

    work page 2023

  8. [8]

    Iso/mpeg-1 audio: A generic standard for coding of high-quality digital audio.Journal of the Audio Engineering Society , 42(10):780–792, 1994

    Karlheinz Brandenburg and Gerhard Stoll. Iso/mpeg-1 audio: A generic standard for coding of high-quality digital audio.Journal of the Audio Engineering Society , 42(10):780–792, 1994

    work page 1994

Show all 73 references
  1. [9]

    Crowdsourcing preference tests, and how to detect cheating

    Sabine Buchholz and Javier Latorre. Crowdsourcing preference tests, and how to detect cheating. In Twelfth Annual Conference of the International Speech Communication Association, 2011

    2011

  2. [10]

    ViSQOL v3: An open source production ready objective speech and audio metric

    Michael Chinen, Felicia SC Lim, Jan Skoglund, Nikita Gureev, Feargus O’Gorman, and Andrew Hines. ViSQOL v3: An open source production ready objective speech and audio metric. InProc. QoMEX, pages 1–6, 2020

    2020

  3. [11]

    High fidelity neural audio compression

    Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression. Transactions on Machine Learning Research , 2023

    2023

  4. [12]

    Overview of the evs codec architecture

    Martin Dietz, Markus Multrus, Vaclav Eksler, Vladimir Malenovsky, Erik Norvell, Harald Pobloth, Lei Miao, Zhe Wang, Lasse Laaksonen, Adriana Vasilache, et al. Overview of the evs codec architecture. In2015 IEEE International Conference on Acoustics, Speech and Signal Processin...

    2015

  5. [13]

    Adversarial audio synthesis

    Chris Donahue, Julian McAuley, and Miller Puckette. Adversarial audio synthesis. In Proc. ICLR, 2018

    2018

  6. [14]

    Vqtts: High-fidelity text- to-speech synthesis with self-supervised vq acoustic feature

    Chenpeng Du, Yiwei Guo, Xie Chen, and Kai Yu. Vqtts: High-fidelity text- to-speech synthesis with self-supervised vq acoustic feature. arXiv preprint arXiv:2204.00768, 2022

    2022 arXiv

  7. [15]

    Apnet2: High-quality and high-efficiency neural vocoder with direct prediction of amplitude and phase spectra

    Hui-Peng Du, Ye-Xin Lu, Yang Ai, and Zhen-Hua Ling. Apnet2: High-quality and high-efficiency neural vocoder with direct prediction of amplitude and phase spectra. In National Conference on Man-Machine Speech Communication , pages 66–80. Springer, 2023

    2023

  8. [16]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proc. NeurIPS, volume 27, 2014

    2014

  9. [17]

    Long short-term memory

    Alex Graves and Alex Graves. Long short-term memory. Supervised sequence labelling with recurrent neural networks , pages 37–45, 2012

    2012

  10. [18]

    A spectral energy distance for parallel speech synthesis

    Alexey Gritsenko, Tim Salimans, Rianne van den Berg, Jasper Snoek, and Nal Kalchbrenner. A spectral energy distance for parallel speech synthesis. InProc. NeurIPS, volume 33, pages 13062–13072, 2020

    2020

  11. [19]

    A multi-stage multi-codebook vq-vae approach to high-performance neural tts.arXiv preprint arXiv:2209.10887, 2022

    Haohan Guo, Fenglong Xie, Frank K Soong, Xixin Wu, and Helen Meng. A multi-stage multi-codebook vq-vae approach to high-performance neural tts.arXiv preprint arXiv:2209.10887, 2022

    2022 arXiv

  12. [20]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  13. [21]

    Gaussian error linear units (GELUs)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (GELUs). InProc. ICML, volume 70, pages 3441–3450, 2017

    2017

  14. [22]

    Hubert: Self-supervised speech repre- sentation learning by masked prediction of hidden units.IEEE/ACM Transactions on Audio, Speech, and Language Processing , 29:3451–3460, 2021

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech repre- sentation learning by masked prediction of hidden units.IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2...

    2021

  15. [23]

    Repcodec: A speech representation codec for speech tokenization.arXiv preprint arXiv:2309.00169 , 2023

    Zhichao Huang, Chutong Meng, and Tom Ko. Repcodec: A speech representation codec for speech tokenization.arXiv preprint arXiv:2309.00169 , 2023

    2023 arXiv

  16. [24]

    The LJ speech dataset

    Keith Ito and Linda Johnson. The LJ speech dataset. https://keithito.com/ LJ-Speech-Dataset, 2017. 16 X.-H. Jiang et al

    2017

  17. [25]

    Univnet:Aneuralvocoder with multi-resolution spectrogram discriminators for high-fidelity waveform gener- ation

    WonsungJang,DongwookLim,JiyoungYoon,andetal. Univnet:Aneuralvocoder with multi-resolution spectrogram discriminators for high-fidelity waveform gener- ation. In Proc. Interspeech, 2021

    2021

  18. [26]

    GlotNet—a raw waveform model for the glottal excitation in statistical parametric speech synthesis

    Lauri Juvela, Bajibabu Bollepalli, Vassilis Tsiaras, and Paavo Alku. GlotNet—a raw waveform model for the glottal excitation in statistical parametric speech synthesis. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 27(6):1019–1030, 2019

    2019

  19. [27]

    iSTFTNet: Fast and lightweight mel-spectrogram vocoder incorporating inverse short-time Fourier transform

    Takuhiro Kaneko, Kou Tanaka, Hirokazu Kameoka, and Shogo Seki. iSTFTNet: Fast and lightweight mel-spectrogram vocoder incorporating inverse short-time Fourier transform. InProc. ICASSP, pages 6207–6211, 2022

    2022

  20. [28]

    Hideki Kawahara, Ikuyo Masuda-Katsuse, and Alain De Cheveigne. Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based f0 extraction: Possible role of a repetitive structure in sounds. Speech communication, 27(3...

    1999

  21. [29]

    HiFi-GAN: Generative adver- sarial networks for efficient and high fidelity speech synthesis

    Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. HiFi-GAN: Generative adver- sarial networks for efficient and high fidelity speech synthesis. Proc. NeurIPS, 33:17022–17033, 2020

    2020

  22. [30]

    Peter Kroon, E Deprettere, and R Sluyter. Regular-pulse excitation–a novel ap- proach to effective and efficient multipulse coding of speech.IEEE transactions on acoustics, speech, and signal processing , 34(5):1054–1063, 1986

    1986

  23. [31]

    MelGAN: generative adversarial networks for conditional waveform synthesis

    Kundan Kumar, Rithesh Kumar, Thibault de Boissiere, Lucas Gestin, Wei Zhen Teoh, Jose Sotelo, Alexandre de Brebisson, Yoshua Bengio, and Aaron Courville. MelGAN: generative adversarial networks for conditional waveform synthesis. In Proc. NeurIPS, pages 14910–14921, 2019

    2019

  24. [32]

    PHASEAUG: A differentiable augmentation for speech synthesis to simulate one-to-many mapping

    Junhyeok Lee, Seungu Han, Hyunjae Cho, and Wonbin Jung. PHASEAUG: A differentiable augmentation for speech synthesis to simulate one-to-many mapping. In Proc. ICASSP, pages 1–5, 2023

    2023

  25. [33]

    Neural vocoder is all you need for speech super-resolution, 2022

    Haohe Liu, Woosung Choi, Xubo Liu, Qiuqiang Kong, Qiao Tian, and DeLiang Wang. Neural vocoder is all you need for speech super-resolution, 2022

    2022

  26. [34]

    Diffgan-tts: High-fidelity and efficient text- to-speech with denoising diffusion gans.arXiv preprint arXiv:2201.11972 , 2022

    Songxiang Liu, Dan Su, and Dong Yu. Diffgan-tts: High-fidelity and efficient text- to-speech with denoising diffusion gans.arXiv preprint arXiv:2201.11972 , 2022

    2022 arXiv

  27. [35]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. InProc. CVPR, pages 11976–11986, 2022

    2022

  28. [36]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InProc. ICLR, 2018

    2018

  29. [37]

    Source-filter-based generative adversarial neural vocoder for high fidelity speech synthesis

    Ye-Xin Lu, Yang Ai, and Zhen-Hua Ling. Source-filter-based generative adversarial neural vocoder for high fidelity speech synthesis. InNational Conference on Man- Machine Speech Communication, pages 68–80. Springer, 2022

    2022

  30. [38]

    Mp-senet: A speech enhancement model with parallel denoising of magnitude and phase spectra

    Ye-Xin Lu, Yang Ai, and Zhen-Hua Ling. Mp-senet: A speech enhancement model with parallel denoising of magnitude and phase spectra. arXiv preprint arXiv:2305.13686, 2023

    2023 arXiv

  31. [39]

    Rectifier nonlinearities improve neural network acoustic models

    Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. Rectifier nonlinearities improve neural network acoustic models. InProc. ICML, volume 30, page 3, 2013

    2013

  32. [40]

    SampleRNN: An unconditional end-to-end neural audio generation model

    Soroush Mehri, Kundan Kumar, Ishaan Gulrajani, Rithesh Kumar, Shubham Jain, Jose Sotelo, Aaron Courville, and Yoshua Bengio. SampleRNN: An unconditional end-to-end neural audio generation model. InProc. ICLR, 2016

    2016

  33. [41]

    Finite scalar quantization: Vq-vae made simple

    Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. InThe Twelfth International Conference on Learning Representations, 2023. ESTVocoder 17

    2023

  34. [42]

    WORLD: A vocoder-based high-quality speech synthesis system for real-time applications.IEICE Transac- tions on Information and Systems , 99(7):1877–1884, 2016

    Masanori Morise, Fumiya Yokomori, and Kenji Ozawa. WORLD: A vocoder-based high-quality speech synthesis system for real-time applications.IEICE Transac- tions on Information and Systems , 99(7):1877–1884, 2016

    2016

  35. [43]

    Expediting tts synthesis with adversarial vocoding

    Paarth Neekhara, Chris Donahue, Miller Puckette, Shlomo Dubnov, and Julian McAuley. Expediting tts synthesis with adversarial vocoding. InProc. Interspeech, volume 2019, pages 186–190, 2019

    2019

  36. [44]

    Ex- presso: A benchmark and analysis of discrete expressive speech resynthesis.arXiv preprint arXiv:2308.05725, 2023

    Tu Anh Nguyen, Wei-Ning Hsu, Antony d’Avirro, Bowen Shi, Itai Gat, Maryam Fazel-Zarani, Tal Remez, Jade Copet, Gabriel Synnaeve, Michael Hassid, et al. Ex- presso: A benchmark and analysis of discrete expressive speech resynthesis.arXiv preprint arXiv:2308.05725, 2023

    2023 arXiv

  37. [45]

    Parallel WaveNet: Fast high-fidelity speech synthesis

    Aaron Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George Driessche, Edward Lockhart, Luis Cobo, Florian Stimberg, et al. Parallel WaveNet: Fast high-fidelity speech synthesis. InProc. ICML, pages 3918–3926, 2018

    2018

  38. [46]

    WaveNet: A generative model for raw audio

    Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. WaveNet: A generative model for raw audio. InProc. SSW, pages 125–125, 2016

    2016

  39. [47]

    Linear predictive coding

    Douglas O’Shaughnessy. Linear predictive coding. IEEE potentials, 7(1):29–32, 1988

    1988

  40. [48]

    Generativeadversarialnetwork-basedapproachtosignal reconstruction from magnitude spectrogram

    Keisuke Oyamada, Hirokazu Kameoka, Takuhiro Kaneko, Kou Tanaka, Nobukatsu Hojo,andHiroyasuAndo. Generativeadversarialnetwork-basedapproachtosignal reconstruction from magnitude spectrogram. InProc. EUSIPCO, pages 2514–2518, 2018

    2018

  41. [49]

    Wave-gan: a deep learning approach for the prediction of nonlinear regular wave loads and run-up on a fixed cylinder.Coastal Engineering, 167:103902, 2021

    Blanca Pena and Luofeng Huang. Wave-gan: a deep learning approach for the prediction of nonlinear regular wave loads and run-up on a fixed cylinder.Coastal Engineering, 167:103902, 2021

    2021

  42. [50]

    ClariNet: Parallel wave generation in end-to-end text-to-speech

    Wei Ping, Kainan Peng, and Jitong Chen. ClariNet: Parallel wave generation in end-to-end text-to-speech. In Proc. ICLR, 2018

    2018

  43. [51]

    Waveflow: A compact flow- based model for raw audio

    Wei Ping, Kainan Peng, Kexin Zhao, and Zhao Song. Waveflow: A compact flow- based model for raw audio. InProc. ICML, pages 7706–7716, 2020

    2020

  44. [52]

    Waveglow: A flow-based gen- erative network for speech synthesis

    Ryan Prenger, Rafael Valle, and Bryan Catanzaro. Waveglow: A flow-based gen- erative network for speech synthesis. InProc. ICASSP, pages 3617–3621, 2019

    2019

  45. [53]

    ITUT Recommendation. P. 862.2: Wideband extension to recommendation P. 862 for the assessment of wideband telephone networks and speech codecs.Interna- tional Telecommunication Union, 2007

    2007

  46. [54]

    Fastspeech 2: Fast and high-quality end-to-end text to speech

    Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. Fastspeech 2: Fast and high-quality end-to-end text to speech. InProc. ICLR, 2020

    2020

  47. [55]

    Fewer-token neural speech codec with time-invariant codes

    Yong Ren, Tao Wang, Jiangyan Yi, Le Xu, Jianhua Tao, Chu Yuan Zhang, and Junzuo Zhou. Fewer-token neural speech codec with time-invariant codes. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), pages 12737–12741. IEEE, 2024

    2024

  48. [56]

    Utmos: Utokyo-sarulab system for voicemos challenge 2022

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. Utmos: Utokyo-sarulab system for voicemos challenge 2022. Interspeech 2022, 2022

    2022

  49. [57]

    A toll quality 8 kb/s speech codec for the personal communications system (pcs).IEEE Transactions on Vehicular Technology, 43(3):808–816, 1994

    Redwan Salami, Claude Laflamme, J-P Adoul, and Dominique Massaloux. A toll quality 8 kb/s speech codec for the personal communications system (pcs).IEEE Transactions on Vehicular Technology, 43(3):808–816, 1994

    1994

  50. [58]

    Jonathan Shen, Ruoming Pang, Ron J Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, et al. 18 X.-H. Jiang et al. Natural TTS synthesis by conditioning WaveNet on mel spectrogram predictions. In Proc. ICASSP, pages 4779–4...

    2018

  51. [59]

    Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis.arXiv preprint arXiv:2306.00814, 2023

    Hubert Siuzdak. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis.arXiv preprint arXiv:2306.00814, 2023

    2023 arXiv

  52. [60]

    Linear predictive coding systems

    Thomas Tremain. Linear predictive coding systems. InICASSP’76. IEEE Inter- national Conference on Acoustics, Speech, and Signal Processing , volume 1, pages 474–478. IEEE, 1976

    1976

  53. [61]

    High- quality, low-delay music coding in the opus codec

    Jean-Marc Valin, Gregory Maxwell, Timothy B Terriberry, and Koen Vos. High- quality, low-delay music coding in the opus codec. InAudio Engineering Society Convention 135. Audio Engineering Society, 2013

    2013

  54. [62]

    LPCNet: Improving neural speech synthesis through linear prediction

    Jean-Marc Valin and Jan Skoglund. LPCNet: Improving neural speech synthesis through linear prediction. InProc. ICASSP, pages 5891–5895, 2019

    2019

  55. [63]

    A review of vector quantization techniques

    A Vasuki and PT Vanathi. A review of vector quantization techniques. IEEE Potentials, 25(4):39–47, 2006

    2006

  56. [64]

    Neural codec lan- guage models are zero-shot text to speech synthesizers.(2023)

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. Neural codec lan- guage models are zero-shot text to speech synthesizers.(2023). arXiv preprint arXiv:2301.02111, 2023

    2023 arXiv

  57. [65]

    Neural source-filter-based wave- form model for statistical parametric speech synthesis

    Xin Wang, Shinji Takaki, and Junichi Yamagishi. Neural source-filter-based wave- form model for statistical parametric speech synthesis. InProc. ICASSP, pages 5916–5920, 2019

    2019

  58. [66]

    Tacotron: Towards end-to-end speech synthesis.Interspeech 2017, 2017

    Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, et al. Tacotron: Towards end-to-end speech synthesis.Interspeech 2017, 2017

    2017

  59. [67]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders

    Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In Proc. CVPR, pages 16133–16142, 2023

    2023

  60. [68]

    Audiodec: An open-source streaming high-fidelity neural audio codec

    Yi-Chiao Wu, Israel D Gebru, Dejan Marković, and Alexander Richard. Audiodec: An open-source streaming high-fidelity neural audio codec. In ICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

  61. [69]

    Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92)

    Junichi Yamagishi, Corinna Veaux, Keiichiro Oura MacDonald, et al. Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92). University of Edinburgh. The Centre for Speech Technology Research (CSTR), 2019

    2019

  62. [70]

    Hifi-codec: Group-residual vector quantization for high fidelity audio codec

    Dongchao Yang, Songxiang Liu, Rongjie Huang, Jinchuan Tian, Chao Weng, and Yuexian Zou. Hifi-codec: Group-residual vector quantization for high fidelity audio codec. arXiv preprint arXiv:2305.02765 , 2023

    2023 arXiv

  63. [71]

    Source-filter hifi-gan: Fast and pitch controllable high-fidelity neural vocoder

    Reo Yoneyama, Yi-Chiao Wu, and Tomoki Toda. Source-filter hifi-gan: Fast and pitch controllable high-fidelity neural vocoder. InICASSP 2023-2023 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1–5. IEEE, 2023

    2023

  64. [72]

    Soundstream: An end-to-end neural audio codec.IEEE/ACM Trans- actions on Audio, Speech, and Language Processing , 30:495–507, 2021

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. Soundstream: An end-to-end neural audio codec.IEEE/ACM Trans- actions on Audio, Speech, and Language Processing , 30:495–507, 2021

    2021

  65. [73]

    Speechtok- enizer: Unified speech tokenizer for speech large language models.arXiv preprint arXiv:2308.16692, 2023

    Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. Speechtok- enizer: Unified speech tokenizer for speech large language models.arXiv preprint arXiv:2308.16692, 2023

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.