Pith. sign in

REVIEW 3 major objections 6 minor 51 references

QHARMA-GAN: Quasi-Harmonic Neural Vocoder based on Autoregressive Moving Average Model

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A neural vocoder that encodes speech into ARMA filter responses can resynthesize and pitch-shift speech at any frequency with a smaller network.

desk verdict A solid incremental hybrid vocoder whose core 'any frequency' claim needs a stability check and whose abstract overstates speed and quality. read the letter →

arxiv 2507.01611 v1 pith:ZHYVHZ5U submitted 2025-07-02 eess.AS cs.SDeess.SP

Shaowen Chen , Tomoki Toda This is my paper
classification eess.AScs.SDeess.SP
keywords neuralvocoderquasi-harmonicmodelARMAspeechmodificationpitchscalingsource-filtermodelingGANresonance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes QHARMA-GAN, a vocoder that replaces black-box waveform generation with an interpretable two-stage pipeline: a neural network predicts per-frame autoregressive moving average (ARMA) filter coefficients from a mel-spectrogram, and a classical synthesis stage evaluates the filter's frequency response at each quasi-harmonic to obtain its amplitude and phase. The authors argue this yields accurate estimates of quasi-harmonic amplitudes and phases at any frequency, so speech can be resynthesized and then modified by pitch shifting or time stretching while keeping the spectral envelope fixed, and the network can be smaller and faster than end-to-end neural vocoders. Experimental comparisons report competitive or better subjective quality, stable $f_0$ extrapolation, faster inference, and improved generalization to out-of-distribution voices. If true, the paper's significance is showing that the controllability of classical source-filter vocoders and the robustness of learned vocoders can coexist in one architecture.

What carries the argument

The load-bearing object is the ARMA frequency response, $H(t_l, \omega) = G_l \, (1 + \sum_{q=1}^{Q} b_q e^{-i\omega q}) / (1 + \sum_{p=1}^{P} a_p e^{-i\omega p})$, split into a cascade of $r$ mini-ARMA stages to enlarge the achievable phase-delay range. It acts as a resonance filter: the amplitude of each quasi-harmonic is the filter magnitude evaluated at that harmonic's frequency, and the phase is the filter's phase delay added to the excitation phase. Because the response is an analytic function of $\omega$, the same coefficients can be evaluated at modified or extrapolated frequencies without re-estimation, which is the mechanism that permits pitch modification while preserving the spectral envelope.

What would settle it

Train QHARMA-GAN on one corpus, pitch-shift held-out speech by a factor of four (two octaves up), and check whether the detected $f_0$ follows the target and whether the magnitude response of the estimated ARMA filters at those high frequencies matches the measured spectral envelope of the source; if the response diverges or the filters become unstable at extrapolated frequencies, the claim of accurate amplitudes and phases at any frequency fails.

Watch

Extended reading notes

Core claim

The central claim is that speech can be encoded as the frequency response of a time-varying ARMA filter, so a single learned filter carries the resonance characteristics and yields both the amplitude and the phase of every quasi-harmonic component at any chosen frequency. The DNN part predicts ARMA coefficients, organized as a cascade of $r$ mini-ARMA stages, per frame from the mel-spectrogram; the synthesis part computes $\hat{A}_k(t_l) = |G_l| \prod_{j=1}^{r} |\tilde{H}_j(t_l, \omega_k)|$ and the phase delay $\angle H(t_l, \omega_k) = \sum_{j=1}^{r} \angle \tilde{H}_j(t_l, \omega_k)$, adds the excitation phase, interpolates amplitudes and phases, and sums sinewaves to produce the waveform. Because amplitude and phase are obtained by evaluating an analytic filter response rather than by learning them directly, changing $f_0$ means evaluating the same response at scaled harmonic frequencies, and time stretching means interpolating the frame parameters at a modified rate. The authors state that this yields accurate amplitudes and phases at arbitrary frequencies, faster synthesis, smaller network size, stable $f_0$ extrapolation, and preserved spectral envelope shape under modification.

Load-bearing premise

The argument rests on the assumption that the ARMA coefficients predicted from a mel-spectrogram remain accurate and stable when the filter response is evaluated at frequencies far outside the $f_0$ range seen in training, even though the estimated filter poles are not constrained to be stable and no generalization guarantee is given.

Editorial extensions

If this is right

  • Pitch can be shifted by octaves without retraining, because amplitudes and phases of the shifted harmonics are computed from the ARMA response at the new frequencies rather than predicted by the network.
  • Time stretching preserves the spectral envelope by interpolating the frame-wise ARMA parameters and phases at modified frame times.
  • The network size and inference cost can be reduced while keeping quality, because the DNN outputs low-rate filter coefficients instead of waveforms.
  • Voiced and unvoiced speech are modified separately under a voice/unvoiced mask, so pitch raising does not degrade the stochastic unvoiced parts.
  • The same framework generalizes to out-of-distribution voices, including soprano singing with $f_0$ outside the training range, due to analytic response evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the ARMA parameterization remains stable outside the training range, a natural extension is direct formant editing: modifying the $a_p$ and $b_q$ coefficients in selected frequency bands would let a user reshape the spectral envelope independently of pitch and duration.
  • Because the synthesis stage is differentiable and parameter-light, one could supervise the ARMA coefficients against ground-truth filter responses computed from clean speech, potentially reducing training data needs further than the few-shot results already show.
  • The any-frequency property suggests the architecture could also serve voice conversion by replacing only the ARMA envelope while keeping the source excitation, a use case the paper does not explicitly evaluate.
  • A testable extension is to ablate the cascade factor $r$: the paper motivates the cascade by the phase-delay range in Eqs. (20)-(22), but does not measure whether a single high-order ARMA with an enlarged phase range would match its quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes QHARMA-GAN, a hybrid neural vocoder that combines a quasi-harmonic model (QHM) with ARMA resonance modeling. A DNN predicts framewise ARMA coefficients from a mel-spectrogram, and a signal-processing synthesis module evaluates the ARMA frequency response at harmonic frequencies to obtain the amplitude and phase of each quasi-harmonic, then reconstructs speech by summing sinusoids with interpolated amplitude and phase. The same framework is used for pitch scaling and time stretching by evaluating the ARMA response at modified frequencies. Experiments on VCTK and JVS compare QHARMA-GAN with HiFi-GAN, Vocos, hn-NSF, WORLD, QHM, and the authors' previous QHM-GAN in terms of objective and subjective quality, f0 extrapolation, generation speed, out-of-distribution singing voices, and few-shot learning. The central claims are that ARMA encoding yields accurate estimates of quasi-harmonic amplitudes and phases 'at any frequency,' enabling high-quality resynthesis and flexible modification with reduced network size and time.

Significance. If the central claims are validated, QHARMA-GAN offers a compact and interpretable vocoder with explicit source-filter structure, good pitch controllability, and modest inference cost, which could be useful for TTS and voice-conversion backends. The paper's strengths include a physically motivated model, a wide range of experimental comparisons on two corpora with multiple baselines, evaluation of f0 extrapolation and OOD singing voices, and a public demo site with audio samples. However, the headline 'any frequency' generalization claim is currently supported only through aggregate subjective and objective scores at the utterance level, not by direct measurement of ARMA frequency-response accuracy at extrapolated harmonics. Some abstract-level claims of outperforming baselines are also stronger than the paper's own tables indicate.

major comments (3)
  1. [§IV-B, Eqs. (22)–(23), Algorithm 3] The central 'any frequency' claim requires that the DNN-predicted ARMA filters remain stable and accurate when H(t_l, ω) is evaluated at arbitrary ω_k, especially at f0 values outside the training range. The paper places no stability constraint on the predicted ARMA coefficients: the output activations are described as leaky ReLU or Tanh (Fig. 2 and its caption), and there is no pole-radius check or stabilization step. An unstable mini-filter (pole radius > 1) can make |H| diverge or peak spuriously at some frequencies, which would directly corrupt modified speech. The experimental evidence in Tables II, IV, and VI consists of aggregate MOS/UTMOS, V/UV rates, and f0 RMSE; these do not directly verify that the ARMA response evaluated at shifted harmonics matches the actual harmonic amplitudes and phases of the modified speech. Please add a stability constraint or a post-hoc stabilization procedure, and report a direct analysis of ARMA response accuracy at extrapolated frequencies (e.g., comparing evaluated H against measured harmonic parameters on modified utterances, or at least reporting the pole-radius distribution). Without this, the abstract's claim of accurate estimation 'at any frequency' is not established.
  2. [Abstract and §V-B, Tables I, III, IV, V] The abstract claims that the proposed method outperforms other methods 'in terms of generation speed, synthesis quality, and modification flexibility.' This is not fully supported by the paper's own results. Table V shows QHARMA-GAN RTF 0.187, which is slower than HiFi-GAN (0.153) and much slower than Vocos (0.040); only QHARMA-GAN-small is faster than HiFi-GAN (0.084), and it remains slower than Vocos. In Table I, QHARMA-GAN on VCTK has MCD 4.09 versus 3.61/3.62 for HiFi-GAN/Vocos and UTMOS 3.76 versus 3.91/3.89, i.e., it is worse on these objective metrics; Table III on VCTK shows PESQ 2.72 versus 3.14 for HiFi-GAN and 3.45 for QHM. In the modification experiments, Table IV shows WORLD achieves higher MOS than QHARMA-GAN for pitch raising on VCTK (ρ=2^0.5: 3.78 vs 3.66; ρ=2^1: 2.82 vs 2.72). The paper should either temper the headline claim to reflect the specific metrics and conditions where QHARMA-GAN is best (e.g., subjective MOS on VCTK/JVS and pitch-lowering scenarios) or provide a statistically grounded aggregate comparison (e.g., significance tests, mean rank across metrics, or a defined primary metric).
  3. [§IV-A, Eq. (20)] The derivation of the bound on cumulative frequency correction from phase delay is not mathematically correct as written. The equality sum_{l=1}^L [∠H(t_l)−∠H(t_{l−1})] = ∠H(t_L)−∠H(t_0) holds only when the phases are unwrapped along the sequence. If ∠H is taken as the wrapped principal value in [−π,π), the per-frame differences can include jumps of ±2π, and their sum is not generally equal to the difference of the two endpoint principal values. Consequently, the claimed bound [−1/Δt, 1/Δt] on the cumulative correction is not valid for wrapped phases; an unwrapped phase trajectory can change by far more than 2π over a long utterance. Since this bound is used to argue that phase delay provides limited frequency correction and to motivate the cascade trick in §IV-B, the exposition should be repaired by using unwrapped phase (with an explicit continuity assumption) or by clearly labeling the bound as a heuristic. The cascade structure itself is not invalidated, but the motivation as stated is flawed.
minor comments (6)
  1. [§III-A, Eq. (12)] The standard deviation σ of the Gaussian window in the pseudo-STFT is mentioned but never defined numerically; please specify its value and how it is chosen.
  2. [§V-A and §V-C] Training details are missing: batch size, learning rate, number of epochs, loss weights λ_fm and λ_mel, discriminator configurations, and model parameter counts. Please add these for reproducibility and to support the claim of 'network size decrease.'
  3. [Algorithm 1 and Fig. 1] Algorithm 1 Step 1 mentions computing S_x(t,ω), but this quantity is not used later in the algorithm; the relation between the pseudo-STFT input and the DNN output should be clarified in the pseudocode.
  4. [§IV-A] There is a typo 'hiqh-quality' in the sentence after Eq. (18); please correct it.
  5. [§IV-C, Eq. (27)] The time-scale factor in Eq. (27) is written as β_l, while Eq. (24) uses β_i; please make the notation consistent.
  6. [Fig. 2 caption] The caption says 'Activation of output layer' but does not specify which activation is used for the AR, MA, and gain outputs; please state the activation choices (leaky ReLU vs Tanh, real vs complex) in the caption or nearby text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ARMA-based synthesis and modification pipeline is trained and evaluated against external data and baselines; the only self-citations document the predecessor framework and are not load-bearing.

full rationale

The central claim—that DNN-estimated ARMA coefficients yield accurate quasi-harmonic amplitudes and phases at arbitrary frequencies—is not equivalent to the inputs by construction. The DNN is trained end-to-end on real speech with adversarial, feature-matching, and mel-spectrogram losses (Eq. (8), Section III-A-3), and the synthesis/modification equations (9), (15)-(17), (22)-(26) define a transformation from predicted coefficients to waveforms. Accuracy is then evaluated on held-out VCTK, JVS, LJspeech, and OpenSinger data against WORLD, QHM, HiFi-GAN, Vocos, and hn-NSF (Tables I-VII). No fitted parameter is renamed as a prediction: the ARMA coefficients themselves are the predicted outputs, and evaluating H(t_l, ω) at arbitrary ω is an empirical generalization claim, not an identity. The phase-delay frequency-correction bound in Eq. (20) follows from telescoping a finite sum over ∠H ∈ [−π, π] and is a mathematical derivation, not a fitted target. The cascaded mini-ARMA construction (Eq. (22)) is an explicit design choice to enlarge the phase-delay range, not a result imported from prior work. Self-citations to QHM-GAN [31] and BP-QHM [32] describe the predecessor framework and are used transparently ("This contribution is identical to that in our previous work [31]"); they do not establish the ARMA extension or forbid alternatives. The paper's own stated limitations—V/UV detection errors degrading modified unvoiced quality and the need for an external f0 predictor—are correctness and generalization concerns, not circularity. No step in the derivation reduces to its own inputs by construction, so the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard QHM and ARMA speech-modeling assumptions plus an unstated stability and generalization premise for DNN-predicted ARMA coefficients. The ARMA orders and cascade size are hand-chosen hyperparameters that could change results.

free parameters (5)
  • AR order P = 128
    Set to 128 for all experiments (Section IV-B, Fig. 3); no ablation justifies this value, but it controls the spectral detail of the resonance model.
  • MA order Q = 128
    Set to 128, same as P; chosen by hand, not fitted to data.
  • Cascade parts r = 8
    Introduced in Eq. (22) to widen the phase-delay frequency-correction range; value 8 used in experiments without a tuning study.
  • Frame shift Delta t = Not specified
    Frame shift affects interpolation and phase accumulation in Eqs. (10)-(11) and (27); the text does not report the value used.
  • MRF modules and dilation rates = N=4 with dilations 1,3,5 (QHARMA-GAN); N=3 with dilation 1 (small)
    Generator architecture inherited from HiFi-GAN and QHM-GAN; no dedicated ablation is given for QHARMA-GAN.
assumptions (5)
  • domain assumption Speech can be modeled frame-wise as a sum of quasi-harmonics: x(t) = sum_k (a_k + t b_k) e^{j2 pi f_k t} w(t).
    Core QHM assumption imported from [12] and used in Section III synthesis; it posits that voiced and unvoiced speech are representable by sparse harmonic components with linear amplitude correction.
  • domain assumption The excitation signal is a sum of unit-amplitude harmonics u_k(t) = e^{j2 pi f_k t}.
    Eq. (14) defines the glottal source for ARMA analysis; real excitation is not strictly unit-amplitude harmonic, but this is a standard modeling simplification.
  • domain assumption The ARMA transfer function H(t_l, omega) = X(t_l, omega) / U(t_l, omega) represents the vocal tract resonance including phase delay.
    Eq. (15) treats the ratio of Fourier transforms as a well-defined filter response even though U is a sparse harmonic sum; the ratio is conceptual rather than computed.
  • ad hoc to paper Neural-network-predicted ARMA coefficients are accurate and stable for unseen f0.
    No stability constraint is imposed on a_p and b_q; the f0 extrapolation experiments rely on this unstated assumption.
  • domain assumption Modifying f0 while keeping the ARMA envelope and V/UV mask unchanged preserves the spectral envelope.
    Section IV-C, Eqs. (25)-(26) assume a shape-invariant modification: the filter magnitude is evaluated at shifted frequencies but the filter itself is unchanged.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QHARMA-GAN: Quasi-Harmonic Neural Vocoder based on Autoregressive Moving Average Model." pith.science (2026). https://pith.science/paper/ZHYVHZ5U

@misc{pith2026250701611,
  author       = {Pith},
  title        = {Pith review of: QHARMA-GAN: Quasi-Harmonic Neural Vocoder based on Autoregressive Moving Average Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZHYVHZ5U}},
  note         = {Machine review of arXiv:2507.01611}
}
read the original abstract

Vocoders, encoding speech signals into acoustic features and allowing for speech signal reconstruction from them, have been studied for decades. Recently, the rise of deep learning has particularly driven the development of neural vocoders to generate high-quality speech signals. On the other hand, the existing end-to-end neural vocoders suffer from a black-box nature that blinds the speech production mechanism and the intrinsic structure of speech, resulting in the ambiguity of separately modeling source excitation and resonance characteristics and the loss of flexibly synthesizing or modifying speech with high quality. Moreover, their sequence-wise waveform generation usually requires complicated networks, leading to substantial time consumption. In this work, inspired by the quasi-harmonic model (QHM) that represents speech as sparse components, we combine the neural network and QHM synthesis process to propose a novel framework for the neural vocoder. Accordingly, speech signals can be encoded into autoregressive moving average (ARMA) functions to model the resonance characteristics, yielding accurate estimates of the amplitudes and phases of quasi-harmonics at any frequency. Subsequently, the speech can be resynthesized and arbitrarily modified in terms of pitch shifting and time stretching with high quality, whereas the time consumption and network size decrease. The experiments indicate that the proposed method leverages the strengths of QHM, the ARMA model, and neural networks, leading to the outperformance of our methods over other methods in terms of generation speed, synthesis quality, and modification flexibility.

Figures

Figures reproduced from arXiv: 2507.01611 by the authors.

Figure 1
Figure 1. Structure of QHM-GAN generator, which takes mel-spectrogram as inputs and outputs sparse framewise amplitude and [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Structure of QHARMA-GAN generator, which takes mel-spectrogram as input and outputs ARMA coefficients. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. (a) Ground truth of utterance sample. Magnitude [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: (a) The waveform of ground truth in Fig. 3 at [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The spectrograms of the speech generated by (a) Vocos, [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: The spectrograms of the soprano voice generated by (a) [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 49 canonical work pages

  1. [1]

    Fast and reliable F0 estimation method based on the period extraction of vocal fold vibration of singing voice and speech,

    M. Morise, H. Kawahara, and H. Katayose, “Fast and reliable F0 estimation method based on the period extraction of vocal fold vibration of singing voice and speech,” inAudio Engineering Society Conference: 35th International Conference: Audio for Games, 2009

    work page 2009

  2. [2]

    Cheaptrick, a spectral envelope estimator for high-quality speech synthesis,

    M. Morise, “Cheaptrick, a spectral envelope estimator for high-quality speech synthesis,”Speech Communication, vol. 67, pp. 1–7, 2015

    work page 2015

  3. [3]

    Error evaluation of an F0-adaptive spectral envelope es- timator in robustness against the additive noise and f0 error,

    M. Morise, “Error evaluation of an F0-adaptive spectral envelope es- timator in robustness against the additive noise and f0 error,”IEICE Transactions on Information and Systems, vol. 98, no. 7, pp. 1405–1408, 2015

    work page 2015

  4. [4]

    D4C, a band-aperiodicity estimator for high-quality speech synthesis,

    M. Morise, “D4C, a band-aperiodicity estimator for high-quality speech synthesis,”Speech Communication, vol. 84, pp. 57–65, 2016

    work page 2016

  5. [5]

    Synchrosqueezed wavelet trans- forms: An empirical mode decomposition-like tool,

    I. Daubechies, J. Lu, and H.-T. Wu, “Synchrosqueezed wavelet trans- forms: An empirical mode decomposition-like tool,”Applied and Com- putational Harmonic Analysis, vol. 30, no. 2, pp. 243–261, 2011. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 16

    work page 2011

  6. [6]

    The synchrosqueez- ing algorithm for time-varying spectral analysis: Robustness properties and new paleoclimate applications,

    G. Thakur, E. Brevdo, N. S. Fu ˇckar, and H.-T. Wu, “The synchrosqueez- ing algorithm for time-varying spectral analysis: Robustness properties and new paleoclimate applications,”Signal Processing, vol. 93, no. 5, pp. 1079–1094, 2013

    work page 2013

  7. [7]

    Second-order synchrosqueezing transform or invertible reassignment? Towards ideal time-frequency representations,

    T. Oberlin, S. Meignen, and V . Perrier, “Second-order synchrosqueezing transform or invertible reassignment? Towards ideal time-frequency representations,”IEEE Transactions on Signal Processing, vol. 63, no. 5, pp. 1335–1344, 2015

    work page 2015

  8. [8]

    High-order synchrosqueezing transform for multicomponent signals analysis-with an application to gravitational- wave signal,

    D.-H. Pham and S. Meignen, “High-order synchrosqueezing transform for multicomponent signals analysis-with an application to gravitational- wave signal,”IEEE Transactions on Signal Processing, vol. 65, no. 12, pp. 3168–3178, 2017

    work page 2017

Show all 51 references
  1. [9]

    Matching de- modulation transform and synchrosqueezing in time-frequency analysis,

    S. Wang, X. Chen, G. Cai, B. Chen, X. Li, and Z. He, “Matching de- modulation transform and synchrosqueezing in time-frequency analysis,” IEEE Transactions on Signal Processing, vol. 62, no. 1, pp. 69–84, 2014

    2014

  2. [10]

    Instantaneous frequency band and synchrosqueezing in time-frequency analysis,

    S. Chen, S. Wang, B. An, R. Yan, and X. Chen, “Instantaneous frequency band and synchrosqueezing in time-frequency analysis,”IEEE Transactions on Signal Processing, vol. 71, pp. 539–554, 2023

    2023

  3. [11]

    Synchroextracting transform,

    G. Yu, M. Yu, and C. Xu, “Synchroextracting transform,”IEEE Trans- actions on Industrial Electronics, vol. 64, no. 10, pp. 8042–8054, 2017

    2017

  4. [12]

    On the properties of a time- varying quasi-harmonic model of speech,

    Y . Pantazis, O. Rosec, and Y . Stylianou, “On the properties of a time- varying quasi-harmonic model of speech,” inProc. Interspeech, 2008

    2008

  5. [13]

    Adaptive AM-FM signal decomposition with application to speech analysis,

    Y . Pantazis, O. Rosec, and Y . Stylianou, “Adaptive AM-FM signal decomposition with application to speech analysis,”IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 2, pp. 290– 300, 2010

    2010

  6. [14]

    An extension of the adaptive quasi-harmonic model,

    G. P. Kafentzis, Y . Pantazis, O. Rosec, and Y . Stylianou, “An extension of the adaptive quasi-harmonic model,” inProc. IEEE ICASSP, 2012, pp. 4605–4608

    2012

  7. [15]

    Parallel wavenet: Fast high-fidelity speech synthesis,

    A. Oord, Y . Li, I. Babuschkin, K. Simonyan, O. Vinyals, K. Kavukcuoglu, G. Driessche, E. Lockhart, L. Cobo, F. Stimberget al., “Parallel wavenet: Fast high-fidelity speech synthesis,” inProc. ICML, 2018, pp. 3918–3926

    2018

  8. [16]

    Waveglow: A flow-based generative network for speech synthesis,

    R. Prenger, R. Valle, and B. Catanzaro, “Waveglow: A flow-based generative network for speech synthesis,” inProc. IEEE ICASSP, 2019, pp. 3617–3621

    2019

  9. [17]

    Efficient neural audio synthesis,

    N. Kalchbrenner, E. Elsen, K. Simonyan, S. Noury, N. Casagrande, E. Lockhart, F. Stimberg, A. Oord, S. Dieleman, and K. Kavukcuoglu, “Efficient neural audio synthesis,” inProc. ICML, 2018, pp. 2410–2419

    2018

  10. [18]

    Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,

    R. Yamamoto, E. Song, and J.-M. Kim, “Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,” inProc. IEEE ICASSP, 2020, pp. 6199–6203

    2020

  11. [19]

    MelGAN: Generative adversarial networks for conditional waveform synthesis,

    K. Kumar, R. Kumar, T. De Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. De Brebisson, Y . Bengio, and A. C. Courville, “MelGAN: Generative adversarial networks for conditional waveform synthesis,”Advances in neural information processing systems, vol. 32, 2019

    2019

  12. [20]

    StylemelGAN: An efficient high- fidelity adversarial vocoder with temporal adaptive normalization,

    A. Mustafa, N. Pia, and G. Fuchs, “StylemelGAN: An efficient high- fidelity adversarial vocoder with temporal adaptive normalization,” in Proc. IEEE ICASSP, 2021, pp. 6034–6038

    2021

  13. [21]

    HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,”Advances in Neural Information Processing Systems, vol. 33, pp. 17 022–17 033, 2020

    2020

  14. [22]

    Speech transformations based on a sinusoidal representation,

    T. Quatieri and R. McAulay, “Speech transformations based on a sinusoidal representation,”IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 34, no. 6, pp. 1449–1464, 1986

    1986

  15. [23]

    Time-scale modification of speech signals,

    B. Ninness and S. J. Henriksen, “Time-scale modification of speech signals,”IEEE Transactions on Signal Processing, vol. 56, no. 4, pp. 1479–1488, 2008

    2008

  16. [24]

    Shape invariant time-scale and pitch modification of speech,

    T. F. Quatieri and R. J. McAulay, “Shape invariant time-scale and pitch modification of speech,”IEEE Transactions on Signal Processing, vol. 40, no. 3, pp. 497–510, 1992

    1992

  17. [25]

    LPCNet: Improving neural speech synthe- sis through linear prediction,

    J.-M. Valin and J. Skoglund, “LPCNet: Improving neural speech synthe- sis through linear prediction,” inProc. IEEE ICASSP, 2019, pp. 5891– 5895

    2019

  18. [26]

    Discrete all-pole modeling,

    A. El-Jaroudi and J. Makhoul, “Discrete all-pole modeling,”IEEE Transactions on signal processing, vol. 39, no. 2, pp. 411–423, 1991

    1991

  19. [27]

    Source-filter HiFi-GAN: Fast and pitch controllable high-fidelity neural vocoder,

    R. Yoneyama, Y .-C. Wu, and T. Toda, “Source-filter HiFi-GAN: Fast and pitch controllable high-fidelity neural vocoder,” inProc. IEEE ICASSP, 2023, pp. 1–5

    2023

  20. [28]

    Unified source-filter GAN with harmonic-plus-noise source excitation generation,

    R. Yoneyama, Y .-C. Wu, and T. Toda, “Unified source-filter GAN with harmonic-plus-noise source excitation generation,” inProc. Interspeech, 2022, pp. 848—-852

    2022

  21. [29]

    Quasi- periodic wavenet: An autoregressive raw waveform generative model with pitch-dependent dilated convolution neural network,

    Y .-C. Wu, T. Hayashi, P. L. Tobing, K. Kobayashi, and T. Toda, “Quasi- periodic wavenet: An autoregressive raw waveform generative model with pitch-dependent dilated convolution neural network,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1134...

    2021

  22. [30]

    Quasi- periodic parallel wavegan: A non-autoregressive raw waveform gener- ative model with pitch-dependent dilated convolution neural network,

    Y .-C. Wu, T. Hayashi, T. Okamoto, H. Kawai, and T. Toda, “Quasi- periodic parallel wavegan: A non-autoregressive raw waveform gener- ative model with pitch-dependent dilated convolution neural network,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29,...

    2021

  23. [31]

    QHM-GAN: Neural vocoder based on quasi- harmonic modeling,

    S. Chen and T. Toda, “QHM-GAN: Neural vocoder based on quasi- harmonic modeling,” inProc. Interspeech, 2024, pp. 3889—-3893

    2024

  24. [32]

    Sequence-wise speech waveform model- ing via backpropagation optimization of quasi-harmonic parameters,

    S. Chen and T. Toda, “Sequence-wise speech waveform model- ing via backpropagation optimization of quasi-harmonic parameters,” IEEE/ACM Transactions on Audio, Speech and Language Processing, pp. 1–14, 2024

    2024

  25. [33]

    Wavehax: Aliasing-Free Neural Waveform Synthesis Based on 2D Convolution and Harmonic Prior for Reliable Complex Spectrogram Estimation,

    R. Yoneyama, A. Miyashita, R. Yamamoto, and T. Toda, “Wavehax: Aliasing-Free Neural Waveform Synthesis Based on 2D Convolution and Harmonic Prior for Reliable Complex Spectrogram Estimation,”arXiv preprint arXiv:2411.06807, 2024

    2024

  26. [34]

    Harmonic-net: Fundamental frequency and speech rate controllable fast neural vocoder,

    K. Matsubara, T. Okamoto, R. Takashima, T. Takiguchi, T. Toda, and H. Kawai, “Harmonic-net: Fundamental frequency and speech rate controllable fast neural vocoder,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1902–1915, 2023

    1902

  27. [35]

    Initial investigation of fundamental frequency controllable HiFi-GAN conditioned on mel- spectrogram,

    S. Shimizu, T. Okamoto, R. Takashima, et al., “Initial investigation of fundamental frequency controllable HiFi-GAN conditioned on mel- spectrogram,” inProc. Acoustical Society of Japan, 2022, pp. 1137—- 1140

    2022

  28. [36]

    Hamonic-Net+: Fundamental frequency controllable fast neural vocoder with harmonic wave input and Layerwise-Quasi Periodic CNNs,

    K. Matsubara, T. Okamoto, R. Takashima, et al., “Hamonic-Net+: Fundamental frequency controllable fast neural vocoder with harmonic wave input and Layerwise-Quasi Periodic CNNs,” inProc. Acoustical Society of Japan, 2022, pp. 1133—-1136

    2022

  29. [37]

    UnivNet: A neural vocoder with multi-resolution spectrogram discriminators for high-fidelity waveform generation,

    W. Jang, D. Lim, J.Yoon, et al., “UnivNet: A neural vocoder with multi-resolution spectrogram discriminators for high-fidelity waveform generation,” inProc. Interspeech, 2021, pp. 2207—-2211

    2021

  30. [38]

    Waveform preserving time stretching and pitch shifting for sinusoidal models of sound,

    R. Di Federico, “Waveform preserving time stretching and pitch shifting for sinusoidal models of sound,” inProceedings of the COST-G6 Digital Audio Effects Workshop, 1998, pp. 44–48

    1998

  31. [39]

    Adaptive sinusoidal models for speech with applications in speech modifications and audio analysis,

    G. Kafentzis, “Adaptive sinusoidal models for speech with applications in speech modifications and audio analysis,” Ph.D. dissertation, Univer- sité de Rennes; Panepist ¯ıímio Kr¯ıít¯ıs, 2014

    2014

  32. [40]

    Harmonic plus noise models for speech, combined with statistical methods, for speech and speaker modification,

    Y . Stylianou, “Harmonic plus noise models for speech, combined with statistical methods, for speech and speaker modification,”Ph.D. dissertation, E.N.S.T- Paris, 1996

    1996

  33. [41]

    Improving the modeling of the noise part in the harmonic plus noise model of speech,

    Y . Pantazis and Y . Stylianou, “Improving the modeling of the noise part in the harmonic plus noise model of speech,” inProc. IEEE ICASSP, 2008, pp. 4609–4612

    2008

  34. [42]

    CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),

    J. Yamagishi, C. Veaux, K. MacDonaldet al., “CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),”University of Edinburgh. The Centre for Speech Technology Research (CSTR), pp. 271–350, 2019

    2019

  35. [43]

    JVS corpus: free Japanese multi-speaker voice corpus,

    S. Takamichi, K. Mitsui, Y . Saito, T. Koriyama, N. Tanji, and H. Saruwatari, “JVS corpus: free Japanese multi-speaker voice corpus,” arXiv preprint arXiv:1908.06248, 2019

    1908 arXiv

  36. [44]

    V ocos: Closing the gap between time-domain and Fourier- based neural vocoders for high-quality audio synthesis,

    H. Siuzdak, “V ocos: Closing the gap between time-domain and Fourier- based neural vocoders for high-quality audio synthesis,” inProc. ICLR, 2024

    2024

  37. [45]

    Neural source-filter waveform models for statistical parametric speech synthesis,

    X. Wang, S. Takaki, and J. Yamagishi, “Neural source-filter waveform models for statistical parametric speech synthesis,”IEEE/ACM Transac- tions on Audio, Speech, and Language Processing, vol. 28, pp. 402–415, 2019

    2019

  38. [46]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” inProc. IEEE/CVF CVPR, 2022, pp. 11 966– 11 976

    2022

  39. [47]

    Harvest: A high-performance fundamental frequency estimator from speech signals

    M. Moriseet al., “Harvest: A high-performance fundamental frequency estimator from speech signals.” inProc. Interspeech, 2017, pp. 2321– 2325

    2017

  40. [48]

    UTMOS: Utokyo-sarulab system for voicemos challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: Utokyo-sarulab system for voicemos challenge 2022,” inProc. Interspeech, 2022, pp. 4521–4525

    2022

  41. [49]

    Iterative estimation of sinu- soidal signal parameters,

    Y . Pantazis, O. Rosec, and Y . Stylianou, “Iterative estimation of sinu- soidal signal parameters,”IEEE Signal Processing Letters, vol. 17, no. 5, pp. 461–464, 2010

    2010

  42. [50]

    Multi-singer: Fast multi-singer singing voice vocoder with A large-scale corpus,

    R. Huang, F. Chen, Y . Ren, J. Liu, C. Cui, and Z. Zhao, “Multi-singer: Fast multi-singer singing voice vocoder with A large-scale corpus,” in ACM Multimedia, 2021, pp. 3945–3954

    2021

  43. [51]

    The LJ Speech dataset,

    K. Ito and L. Johnson, “The LJ Speech dataset,” https://keithito.com/ LJ-Speech-Dataset/, 2017

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.