REVIEW 4 major objections 5 minor 18 references
A Neural Denoising Vocoder for Clean Waveform Generation from Noisy Mel-Spectrogram based on Amplitude and Phase Predictions
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A two-stage denoising vocoder generates clean speech directly from noisy mel-spectrograms.
desk verdict A useful mel-only denoising vocoder with a defensible two-stage design; the SOTA claim needs a bigger comparison set and a stated training recipe. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the two-stage cascade in which noisy amplitude and phase spectra serve as an explicit intermediate representation. The first stage, a spectrum predictor adapted from the APNet vocoder, uses separate ResNet-based amplitude and phase predictors to convert the noisy mel-spectrogram into full noisy spectra; the phase predictor matters most because phase information is entirely absent from the mel input. The second stage is the MP-SENet enhancement module, a conformer-based model with separate amplitude-mask and phase decoders that turns the noisy spectra into clean ones, followed by iSTFT to synthesize the waveform. Training the spectrum predictor uses a multi-resolution discriminator and linear anti-wrapping phase losses, while the enhancement module follows its original time-domain, amplitude, complex-spectrum, and anti-wrapping losses with adversarial training.
What would settle it
Ablate the spectrum predictor by feeding the noisy mel-spectrogram straight into the enhancement module and measuring PESQ and STOI on VoiceBank+DEMAND. If the simplified system matches the full pipeline, the claim that noisy spectra must be predicted before enhancement is not supported.
Extended reading notes
Core claim
The central claim is that clean waveforms can be recovered from a noisy mel-spectrogram by explicitly predicting the noisy amplitude and phase spectra first and then denoising them, rather than asking a single network to jump directly from mel input to clean speech. The authors argue that direct clean-phase recovery from the mel-spectrogram is especially fragile at low SNR because the noisy phase is the clean phase plus an angle term that grows as the noise-to-signal ratio rises. Their two-stage design uses the predicted noisy spectra as a bridge: the spectrum predictor supplies what the mel-spectrogram is missing, and the enhancement module, built on a parallel magnitude-and-phase denoising model, removes the residual noise. Experiments on VoiceBank+DEMAND show the full system reaching a PESQ of 2.88 and STOI of 0.94, surpassing the two vocoder baselines and landing between MetricGAN and MetricGAN+ on the composite quality scores.
Load-bearing premise
The whole pipeline leans on a pre-trained enhancement checkpoint, built for one audio feature configuration, working correctly on the different configuration produced by the spectrum predictor, even though the two stages were never jointly trained or explicitly aligned.
Editorial extensions
If this is right
- A vocoder that accepts only mel-spectrograms can perform denoising, so noisy speech from a target speaker can be converted to clean synthesis without requiring a separate waveform-enhancement front end.
- Predicting noisy spectra as an intermediate beats trying to predict clean spectra directly, especially for phase recovery in low-SNR conditions.
- Off-the-shelf speech-enhancement models can be recycled as the second stage of a denoising vocoder, suggesting a modular route to other vocoder–enhancer combinations.
- The system reaches the quality level of amplitude-input speech-enhancement models despite operating from a lossy mel-spectrogram, setting a practical baseline for mel-only denoising.
Reading between the lines
- Because the enhancement module is used as a frozen checkpoint with different STFT settings from the spectrum predictor, the reported gains may partly reflect how well that checkpoint generalizes across feature configurations; a version fine-tuned end-to-end on the bridge features would separate this transfer effect from the architecture's benefit.
- The same two-stage bridge could be tested with other lossy acoustic inputs, such as speaker embeddings or noise-robust features, or with other enhancement backbones, to see whether the noisy-spectrum bridge is the general principle.
- The phase-only angle equation suggests a direct extension: a predictor that estimates the noise-to-signal ratio per time-frequency bin could let the system condition its denoising on expected phase deviation, potentially improving very-low-SNR cases.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a neural denoising vocoder that maps a noisy mel-spectrogram to a clean waveform in two stages. A spectrum predictor (an APNet-derived module with separate amplitude and phase predictors) first estimates the noisy amplitude and phase spectra from the noisy mel-spectrogram; an enhancement module (MP-SENet) then denoises these spectra; the clean waveform is reconstructed by iSTFT. Experiments on VoiceBank+DEMAND compare against HiFi-GAN and Vocos as vocoder baselines and against MetricGAN/MetricGAN+ as speech-enhancement baselines, with additional ablations that replace the spectrum predictor with the original APNet and that match the STFT configuration of MP-SENet. The paper claims state-of-the-art performance among neural vocoders and performance comparable to several speech-enhancement methods.
Significance. If the results hold, the paper demonstrates that a mel-only denoising vocoder is practical and that a two-stage spectral bridge (noisy amplitude/phase prediction followed by spectral enhancement) is an effective design. The work builds on existing components (APNet and MP-SENet) and is clearly motivated by real noisy TTS scenarios. The paper includes a useful ablation: Table 3 shows that deliberately matching the STFT configuration of MP-SENet degrades PESQ from 2.88 to 2.26, which is an empirical defense of the mismatched-configuration design. Equation (1) is a true identity and is used only as motivation, so there is no circularity in the evaluation. The main weaknesses are in the evaluation protocol: the SOTA claim rests on only two vocoder baselines, no significance testing accompanies the metric comparisons, and the training/fine-tuning status of MP-SENet is ambiguous.
major comments (4)
- [Section 3.2, Table 1] The central claim of state-of-the-art performance among neural vocoders is supported only by comparison with HiFi-GAN and Vocos, both of which are standard clean-speech vocoders that the paper modifies by training them on noisy mel-spectrograms. The manuscript itself identifies DNR-HiNet [7] as the closest prior denoising vocoder, but DNR-HiNet is never evaluated. A SOTA claim requires either an evaluation against the most relevant existing denoising vocoder or an explicit and justified statement of why that comparison is excluded. As written, the claim overreaches the evidence.
- [Section 3.1, Implementation] The manuscript does not specify whether the open-source MP-SENet checkpoint is kept frozen, fine-tuned, or jointly trained with the spectrum predictor during the 700k training steps. The text says both 'we use the open source checkpoint of MP-SENet' and 'Training of all models employed AdamW ... for 700k steps,' which is ambiguous. This is load-bearing because the enhancement module receives spectra computed with a 1024-point FFT that differ from the 400-point FFT configuration used when the MP-SENet checkpoint was trained, and the behavior of the cascade depends critically on whether the enhancement module is adapted to this input distribution. Please state explicitly which parameters are updated in each training stage.
- [Section 3.2, Tables 1-3] All objective metrics are reported as point estimates on a single test set, without confidence intervals, significance tests, or multiple training runs. The word 'significantly' in Section 3.2 is therefore not statistically supported. While the gaps over HiFi-GAN and Vocos appear large, the comparison with MetricGAN+ in Table 2 is close (e.g., PESQ 2.88 vs 3.15, CBAK 3.06 vs 3.16), and the claim of 'comparable' performance cannot be assessed without error bars. The authors should report variance or significance, or at minimum temper the language of 'significantly.'
- [Section 3.3, Table 2] The speech-enhancement comparisons are limited to MetricGAN (2019) and MetricGAN+ (2021), which are not representative of current state-of-the-art SE systems. The abstract's claim of 'comparable performance with the serveral advanced speech enhancement methods' is therefore weakened, since the selected baselines are more than three years old and are amplitude-only methods. Either update the SE baseline set to more recent methods or narrow the claim to 'comparable to the evaluated SE methods.'
minor comments (5)
- [Abstract, Section 2.1, Section 3.1] There are several typos: 'serveral' in the abstract, 'spectrum predictior' in Section 2.1, 'metircs' in Section 3.1, 'T able' in table captions, 'SE filed' in Section 2.3, and 'MeticGAN+' in Section 3.3.
- [Section 3.1, Implementation] Training details are incomplete: the manuscript gives the learning rate and number of steps but not the learning-rate schedule, batch size, number of GPUs, or whether the spectrum predictor and enhancement module are trained separately or jointly. These details are needed for reproducibility.
- [Section 3.4, Table 3] The STFT-mismatch ablation is commendable and directly addresses a likely reviewer concern, but it is also a single-run point estimate. Reporting variance or a small number of runs would make the conclusion that the mismatched configuration is preferable more robust.
- [Section 3.3, Table 2] Table 2 omits SSNR and STOI because the original MetricGAN papers did not report them, but these metrics are standard on VoiceBank+DEMAND. If released checkpoints are available, computing them for MetricGAN/MetricGAN+ would provide a more complete comparison; otherwise the omission should be noted in the table caption.
- [Section 2.3] The enhancement module is described as using MP-SENet with Conformer blocks, amplitude mask decoder, and phase decoder. A few sentences on how the phase decoder is trained (e.g., the anti-wrapping loss) would help readers who do not know MP-SENet.
Circularity Check
No significant circularity: the central claim rests on held-out evaluation against external baselines, not on the paper's own construction.
full rationale
The paper's derivation chain is not circular. Eq. (1) is a standard trigonometric identity used only to motivate the need for phase prediction in low-SNR conditions; it is not used to derive the claimed results. The proposed system is a cascade of a spectrum predictor (modified from APNet) and an enhancement module (MP-SENet), and its performance is measured on the held-out VoiceBank+DEMAND test set against external baselines (HiFi-GAN, Vocos, MetricGAN, MetricGAN+). The self-citations to APNet, APNet2, and MP-SENet are dependencies on prior systems, but those systems are externally published artifacts with their own training and evaluations; invoking them does not make the benchmark numbers equivalent to the paper's inputs. The STFT configuration mismatch between the two stages is explicitly investigated in Table 3, where the paper reports that matching MP-SENet's STFT settings degrades performance (PESQ drops from 2.88 to 2.26), so the design choice is empirically defended rather than assumed. Concerns about the limited number of vocoder baselines, missing significance tests, and ambiguity about whether MP-SENet is frozen or fine-tuned are legitimate scientific-quality concerns, but they are not circularity: the predictions are not fitted to the test set, and no claim reduces by construction to a fitted parameter or to an unverified self-citation.
Assumptions & free parameters
free parameters (3)
- Initial learning rate =
0.0002
- Training duration =
700k steps
- STFT configurations =
1024/320/80 for predictor; 400/400/100 for enhancer
assumptions (4)
- domain assumption The 80-dimensional mel-spectrogram retains enough spectral detail to recover usable noisy full-band amplitude and phase spectra.
- standard math The additive noise phase relation in Eq. (1) is the standard identity for complex-valued speech plus noise.
- domain assumption MP-SENet's pretrained checkpoint transfers to the spectral representations produced by the spectrum predictor.
- domain assumption Objective metrics (PESQ, CSIG, CBAK, COVL, SSNR, STOI) are sufficient proxies for perceptual speech quality.
Cite this review
Pith. "Pith review of A Neural Denoising Vocoder for Clean Waveform Generation from Noisy Mel-Spectrogram based on Amplitude and Phase Predictions." pith.science (2026). https://pith.science/paper/AXC4JHP3
@misc{pith2026241112268,
author = {Pith},
title = {Pith review of: A Neural Denoising Vocoder for Clean Waveform Generation from Noisy Mel-Spectrogram based on Amplitude and Phase Predictions},
year = {2026},
howpublished = {\url{https://pith.science/paper/AXC4JHP3}},
note = {Machine review of arXiv:2411.12268}
}
read the original abstract
This paper proposes a novel neural denoising vocoder that can generate clean speech waveforms from noisy mel-spectrograms. The proposed neural denoising vocoder consists of two components, i.e., a spectrum predictor and a enhancement module. The spectrum predictor first predicts the noisy amplitude and phase spectra from the input noisy mel-spectrogram, and subsequently the enhancement module recovers the clean amplitude and phase spectrum from noisy ones. Finally, clean speech waveforms are reconstructed through inverse short-time Fourier transform (iSTFT). All operations are performed at the frame-level spectral domain, with the APNet vocoder and MP-SENet speech enhancement model used as the backbones for the two components, respectively. Experimental results demonstrate that our proposed neural denoising vocoder achieves state-of-the-art performance compared to existing neural vocoders on the VoiceBank+DEMAND dataset. Additionally, despite the lack of phase information and partial amplitude information in the input mel-spectrogram, the proposed neural denoising vocoder still achieves comparable performance with the serveral advanced speech enhancement methods.
Figures
Reference graph
Works this paper leans on
-
[7]
IEEE/ACM Transac- tions on Audio, Speech, and Language Processing30, 2036–2048 (2022)
Ai, Y., Ling, Z.H., Wu, W.L., Li, A.: Denoising-and-dereverberation hierarchical neural vocoder for statistical parametric speech synthesis. IEEE/ACM Transac- tions on Audio, Speech, and Language Processing30, 2036–2048 (2022)
work page 2022
-
[1]
Kong, J., Kim, J., Bae, J.: HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. Proc. NeurIPS33, 17022–17033 (2020)
work page 2020
-
[2]
arXiv preprint arXiv:2306.00814 (2023)
Siuzdak, H.: Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. arXiv preprint arXiv:2306.00814 (2023)
arXiv 2023
-
[3]
IEEE/ACM Transactions on Audio, Speech, and Language Processing 26(8), 1420–1433 (2018)
Valentini-Botinhao, C., Yamagishi, J.: Speech enhancement of noisy and reverber- ant speech for text-to-speech. IEEE/ACM Transactions on Audio, Speech, and Language Processing 26(8), 1420–1433 (2018)
work page 2018
- [4]
- [5]
- [6]
-
[8]
IEEE/ACM Transactions on Audio, Speech, and Language Processing (2023)
Ai, Y., Ling, Z.H.: APNet: An all-frame-level neural vocoder incorporating direct prediction of amplitude and phase spectra. IEEE/ACM Transactions on Audio, Speech, and Language Processing (2023)
work page 2023
Show all 18 references
-
[9]
In: Proc
Lu,Y.X.,Ai,Y.,Ling,Z.H.:MP-SENet:Aspeechenhancementmodelwithparallel denoising of magnitude and phase spectra. In: Proc. Interspeech. pp. 3834–3838 (2023)
2023
-
[10]
In: Proc
Du, H.P., Lu, Y.X., Ai, Y., Ling, Z.H.: APNet2: High-quality and high-efficiency neural vocoder with direct prediction of amplitude and phase spectra. In: Proc. NCMMSC. pp. 66–80 (2023)
2023
-
[11]
In: Proc
Ai, Y., Ling, Z.H.: Neural speech phase prediction based on parallel estimation architecture and anti-wrapping losses. In: Proc. ICASSP. pp. 1–5 (2023)
2023
-
[12]
In: Proc
Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I.S., Xie, S.: ConvNeXt v2: Co-designing and scaling convnets with masked autoencoders. In: Proc. CVPR. pp. 16133–16142 (2023)
2023
-
[13]
In: Proc
Fu, S.W., Liao, C.F., Tsao, Y., Lin, S.D.: MetricGAN: Generative Adversarial Networks Based Black-Box Metric Scores Optimization for Speech Enhancement. In: Proc. ICML. pp. 2031–2041 (2019)
2019
-
[14]
In: Proc
Valentini-Botinhao, C., Wang, X., Takaki, S., Yamagishi, J.: Investigating rnn- based speech enhancement methods for noise-robust text-to-speech. In: Proc. SSW. pp. 146–152 (2016)
2016
-
[15]
In: Proc
Veaux, C., Yamagishi, J., King, S.: The voice bank corpus: Design, collection and data analysis of a large regional accent speech database. In: Proc. O- COCOSDA/CASLRE. pp. 1–4 (2013)
2013
-
[16]
In: Proc
Thiemann, J., Ito, N., Vincent, E.: The diverse environments multi-channel acous- tic noise database (DEMAND): A database of multichannel environmental noise recordings. In: Proc. ICA. vol. 19, p. 035081 (2013)
2013
-
[17]
In: Proc
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: Proc. ICLR (2018)
2018
-
[18]
In: Proc
Fu, S.W., Yu, C., Hsieh, T.A., Plantinga, P., Ravanelli, M., Lu, X., Tsao, Y.: MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement. In: Proc. Interspeech. pp. 201–205 (2021)
2021
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.