Pith. sign in

REVIEW 4 major objections 5 minor 31 references

MeloCodec: Harnessing Melodic Priors for High-Fidelity Singing Voice Representation

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MeloCodec claims that a singing-voice codec keeps pitch accurate at very low bitrates by quantizing a chromagram-based melodic prior into tokens before fusing it with acoustic features, and that the same design enables controllable pitch…

desk verdict Clean codec design with real ablations, but the pitch gains are measured with an oracle melody at inference; the missing downstream test matters. read the letter →

arxiv 2608.03021 v1 pith:OVVGBTUG submitted 2026-08-04 cs.SD

Yizhong Geng , Wenxin Fu , Kecan Mao , Qifei Li , Yingming Gao , Ruimin Wang , Chunfeng Wang , Hao Li
show 2 more authors
Ya Li Wei Chen
This is my paper · ORCID
classification cs.SD
keywords neuralaudiocodecsingingvoicerepresentationmelodicpriorschromagramresidualvectorquantizationpitchconsistencydisentangledLLM-basedgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural audio codecs provide the discrete tokens that language models use to generate speech and singing, yet most current codecs optimise for semantic content and lose fine pitch information when bandwidth is tight. This paper argues that singing needs an explicit acoustic prior, and that the right prior is melody: MeloCodec extracts a chromagram (a 12-bin pitch-class map), trains a small codec to quantize it, and only then fuses the quantized melody tokens with the acoustic stream. The central claim is that this 'Tokenize-then-Fuse' ordering, trained in two stages, stops the decoder from ignoring the prior and keeps the codebook alive. In the reported measurements it cuts F0 error from 4.12 to 1.64 RMSE at 1.5 kbps against DAC and supports pitch shifts of plus or minus two or four semitones with little timbre loss. If the claim holds, LLM-based singing synthesis gets a token interface that preserves pitch and exposes a controllable pitch knob.

What carries the argument

The carrier of the argument is the 'Tokenize-then-Fuse' paradigm. A chromagram $C$ (a 12-bin pitch-class projection of the STFT) is encoded and quantized by a dedicated melody codec into tokens $M^q$; only these discrete tokens are projected into the acoustic stream and fused by concatenation, followed by a second RVQ that produces unified tokens $U^q$. The pre-trained melody decoder stays attached via an auxiliary chroma-reconstruction loss, and a two-stage schedule first locks in the melody codebook and then fine-tunes the melody decoder while the encoder and quantizer stay frozen. This information bottleneck, plus the auxiliary reconstruction gradient, is what prevents shortcut learning and keeps the codebook from collapsing.

What would settle it

Run a language-model singing pipeline that must predict MeloCodec's unified tokens from text without seeing the target chromagram, synthesize, and measure F0-RMSE; if error returns to the 4.1 range of DAC at 1.5 kbps, the reported pitch gains are artifacts of oracle melody access.

Watch

Extended reading notes

Core claim

The paper's central claim is that embedding an explicit melodic prior into a neural codec's discrete latent space, rather than conditioning the decoder on continuous features, yields a singing-voice representation that stays pitch-accurate under strong compression and supports pitch manipulation. MeloCodec pre-trains a melody codec on chromagrams, freezes its encoder and residual vector quantizer, and then fuses the quantized melody tokens with acoustic features through a second RVQ, while an auxiliary head predicts the melody tokens from the unified code. In the authors' measurements this reaches F0-RMSE 0.96 at 6.0 kbps and 1.64 at 1.5 kbps, with pitch shifts of plus or minus two and four semitones tracking the target contour at 0.72 and 0.70 correlation while keeping speaker similarity above 0.9.

Load-bearing premise

The load-bearing premise is that the Opencpop reconstruction benchmark, where the melody prior comes straight from the ground-truth input, predicts how MeloCodec will behave as a tokenizer in language-model singing generation, where melody tokens have to be guessed rather than read off the target waveform.

Editorial extensions

If this is right

  • At 1.5 kbps, F0-RMSE drops from 4.12 for DAC to 1.64 for MeloCodec, and chroma similarity rises from 0.84 to 0.95, so melodic structure survives bandwidth compression that wrecks pitch in standard codecs.
  • Shifting the melodic prior by plus or minus two or four semitones tracks the target contour at 0.72 and 0.70 correlation while speaker similarity remains 0.93 and 0.91, giving a controllable pitch axis in the discrete latent space.
  • The two-stage schedule is necessary: direct fusion of continuous melody features collapses with F0-RMSE 5.95 and codebook utilization below five percent, whereas Tokenize-then-Fuse converges stably.
  • Because melody tokens act as priors, the reported bitrates reflect only the transmitted unified tokens, so the pitch-preservation gains carry no extra bandwidth cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the evaluation hands the model the ground-truth chromagram as the melodic prior; using the codec as a language-model tokenizer means the melody tokens must be predicted, so the transfer of these gains to generation is plausible but untested here.
  • Editorial inference: the same two-stage discrete-bottleneck recipe could be applied to other explicit acoustic priors, such as rhythm, dynamics, or timbre contours, to build a family of controllable singing and speech codecs; the paper only demonstrates melody.
  • Editorial inference: the residual F0-RMSE of 3.85 to 4.50 under pitch shift, despite high contour correlation, suggests the acoustic stream still anchors absolute pitch; a fully independent pitch axis might require an explicit orthogonality or parity constraint between the two streams.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. MeloCodec proposes a dual-stream neural audio codec that extracts a chromagram-based melodic prior from the input waveform, quantizes it into discrete tokens via RVQ in a pre-trained 'Tokenize' branch, and fuses those tokens with acoustic features in a 'Fuse' branch to form unified tokens U_q for reconstruction. A two-stage training strategy (melody codec pre-training, then joint training with a frozen melody encoder and fine-tuned decoder) is introduced to prevent shortcut learning and codebook collapse. The paper evaluates reconstruction quality on the Opencpop test set against EnCodec, DAC, X-Codec, and MuCodec, reports large improvements in F0-RMSE and Chroma-SIM at low bitrate, presents a MUSHRA listening test, and demonstrates pitch-shift controllability. The central claim is that the resulting unified discrete tokens provide a disentangled, melody-aware representation suitable for LLM-based singing voice generation.

Significance. If the reported gains hold in the intended deployment setting, the contribution is significant: it directly addresses a known weakness of semantic-only codecs for pitch-sensitive tasks and offers a concrete mechanism for embedding explicit acoustic priors into a discrete token interface. The ablation study in Table II is a genuine strength: comparing Acoustic-only, +Auxiliary Loss, Direct Fusion, and the full method isolates the discrete-fusion contribution and supports the claim that the information bottleneck, not merely the chroma supervision, is responsible for the stability and pitch improvements. The reported effect sizes at low bitrate (F0-RMSE 1.64 vs 4.12 for DAC) are large. However, the evaluation is carried out in an oracle-prior reconstruction regime, and several reported metrics overlap with the training objective; these issues need to be addressed before the central claim can be considered established. The architectural idea is well-motivated and the paper is clearly written, so the work merits further review after a major revision.

major comments (4)
  1. [Section III-A, Table I] The evaluation protocol computes the melodic prior from the ground-truth target waveform (C = Φ(x)) and feeds it to the model at inference. The motivation in Sections I and IV is that MeloCodec produces unified tokens for LLM-based singing generation, where at inference there is no target waveform and the melody tokens must be produced by the language model. All headline results—reconstruction F0-RMSE, Chroma-SIM, and the pitch-shift controllability in Table III—are therefore measured in an oracle-prior regime. The paper should add either a downstream generation experiment (e.g., a small LLM trained on U_q that predicts tokens from text or a prompt and then reconstructs audio) or an explicit robustness study that perturbs M_q with the types of errors a language model would make. Without this, the central claim that MeloCodec improves pitch consistency for the intended LLM-generation use case is not established.
  2. [Section III-B, Eq. (6)] Chroma-SIM is reported as evidence of melodic integrity, but the model is explicitly trained to minimize ||C − D_mel(φ(U_q))||² in Eq. (6), where C is the same chromagram used for the metric. The auxiliary loss therefore guarantees that U_q contains enough chroma information to reconstruct C when decoded by D_mel; a high Chroma-SIM partly reflects the training objective rather than an independent property of the learned representation. The F0-RMSE numbers are less vulnerable to this concern because they come from a separate F0 estimator, but the Chroma-SIM columns in Tables I and II should be discussed with this overlap acknowledged, or replaced by a downstream metric that is not directly optimized.
  3. [Tables I–III] No error bars, confidence intervals, or significance tests are reported for any objective metric, and the MUSHRA test uses 15 experts on 30 samples with no inferential statistic other than plotted confidence intervals. Several reported advantages at high bitrate are small (e.g., ViSQOL 4.08 vs 3.97, SPK-SIM 0.99 vs 0.98), and without a measure of run-to-run or subject variability it is not possible to determine which differences are reliable. The authors should report means and standard deviations over multiple training runs or bootstrap resampling, and apply a paired significance test or report effect sizes for the headline comparisons.
  4. [Table III] The controllability experiment shifts the extracted ground-truth melodic prior while keeping acoustic features fixed, which again assumes oracle access to the melody. Moreover, the target F0-RMSE values of 3.85 and 4.50 in the shifted conditions are large in absolute terms, and the 'Degrade %' column is not defined in the text or caption. The claim of 'minimal timbre degradation' rests on a single speaker-similarity metric. The paper should define the degradation measure, report confidence intervals for the shift condition, and justify that the observed RMSE increase is acceptable for the intended interactive pitch-control use case.
minor comments (5)
  1. [Table I] MuCodec is evaluated at a different bitrate (1.33 kbps vs 1.5 kbps) and token rate (25 Hz vs 75 Hz), so the comparison is not matched; this limitation should be stated explicitly in the text.
  2. [Title and Section II-B] There is a typo in the title ('V oice' should be 'Voice'), and Eq. (3) omits the standard VQ commitment-loss weighting and any codebook-reset details, which are relevant to the codebook-collapse claims.
  3. [Section II-D] The decision to unfreeze D_mel in Stage 2 is described only as an empirical observation; a brief sensitivity analysis or a training-curve comparison of frozen vs fine-tuned decoders would make the claim more convincing.
  4. [References] Several references appear tangential to the main argument (e.g., [17] on audio-driven portrait animation and [23] on a multi-agent framework), which distracts from the focused narrative.
  5. [Figure 3] The F0 contour plot lacks axis labels and a clear legend for the shaded deviation region; adding these would improve interpretability.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor Chroma-SIM objective echo; central pitch-consistency claim has independent F0-RMSE support.

  1. other [Eq. (6) in Section II-C; Chroma-SIM metric in Section III-A and Table I]
    "To enforce melodic integrity in Uq, an auxiliary projection head φ(·) predicts cMq = φ(Uq), which reconstructs the chromagram through the melody decoder. The auxiliary supervision loss is defined as: Laux = ∥C− Dmel(cMq)∥2 2 (6)"

    Laux explicitly trains the unified tokens Uq to reconstruct the input chromagram C. The evaluation then uses Chroma-SIM (Section III-A: 'pitch accuracy via F0-RMSE and Chroma-SIM'; Table I) as evidence of pitch consistency. Since Chroma-SIM measures chroma similarity between original and reconstruction, a portion of the reported gain on this metric is the training objective itself rather than an emergent property. However, F0-RMSE is not directly optimized and the ablations show analogous improvements, so the central claim is not fully forced by construction.

full rationale

MeloCodec's derivation is largely self-contained: the melodic prior C = Φ(x) is an explicit input, the Tokenize-then-Fuse objective (Eqs. 3, 6, 7) is a real training procedure, and the headline F0-RMSE improvements (1.64 vs 4.12 at 1.5 kbps) are measured on a quantity that does not appear in any loss. The Direct Fusion ablation empirically supports the claimed optimization instability, and the self-citation [22] is one of several references for gradient conflicts and is not load-bearing. The only mild circularity is that Chroma-SIM overlaps with the auxiliary chroma reconstruction loss Laux (Eq. 6), making part of the melody/pitch gain on that particular metric a training-objective echo. The independent F0-RMSE results and the ablation study keep the core conclusion intact. The oracle-melody evaluation protocol (melodic prior extracted from the target waveform during reconstruction and pitch-shift tests) is a genuine external-validity concern for LLM-based generation, but it is not circularity and does not raise the circularity score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

All central claims are empirical rather than derived. The paper's contribution is an architectural and training-schedule proposal; it relies on chromagram extraction as a fixed DSP front-end, on the assumption that quantizing melody features creates a useful bottleneck, and on a single evaluation protocol. No new physical entities are introduced. The list above records the hand-chosen hyperparameters and load-bearing domain assumptions.

free parameters (3)
  • Loss weights lambda_adv and lambda_aux = not reported
    Stage 2 objective (Eq. 7) depends on these weights; without reported values, sensitivity of the pitch/timbre tradeoff cannot be assessed.
  • STFT and chromagram parameters = not reported
    Chromagram extraction Phi in Eq. 1 is a fixed DSP front-end, but its window, hop, and FFT size determine temporal resolution and melody token rate; values are not given.
  • Codebook size and quantization levels = 1024; Nq=8 or 2
    Hand-chosen design choices that define the bitrate anchors and the capacity of the melodic bottleneck.
assumptions (4)
  • domain assumption The chromagram computed from the input waveform is a sufficient and stable melodic prior, retaining pitch-class structure while discarding envelope and timbre.
    This grounds the Tokenize Branch in Section II-B; if chroma lost the octave, timing, or dynamic information needed for singing, the prior would be misleading.
  • ad hoc to paper Discretizing melodic features with RVQ creates an information bottleneck that prevents shortcut learning and forces the decoder to attend to the melodic branch.
    This is the core design rationale asserted in Sections II-C and III-D; the ablation study is the only evidence, and no formal proof is given.
  • domain assumption Opencpop is a representative held-out test set for singing voice codec quality, and objective metrics without error bars are treated as meaningful.
    All performance claims in Tables I-III rest on this protocol; no cross-dataset validation is provided.
  • standard math Standard VQ-VAE machinery (stop-gradient, EMA codebook updates) converges without codebook collapse under the proposed two-stage schedule.
    The paper relies on standard RVQ training; Section II-D describes but does not prove this stability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MeloCodec: Harnessing Melodic Priors for High-Fidelity Singing Voice Representation." pith.science (2026). https://pith.science/paper/OVVGBTUG

@misc{pith2026260803021,
  author       = {Pith},
  title        = {Pith review of: MeloCodec: Harnessing Melodic Priors for High-Fidelity Singing Voice Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OVVGBTUG}},
  note         = {Machine review of arXiv:2608.03021}
}
read the original abstract

Neural audio codecs serve as fundamental tokenizers for LLM-based audio generation. While semantic priors are widely exploited to enhance linguistic intelligibility, the integration of explicit acoustic priors remains underexplored, limiting synthesis fidelity in frequency-sensitive domains. To address this gap, we introduce MeloCodec, a novel framework designed to effectively incorporate melodic priors, a critical form of acoustic information for singing. To address the optimization instability typically caused by the direct fusion of such explicit priors, we propose a Tokenize-then-Fuse paradigm that pre-trains a discrete melodic branch to lock in structures before feature fusion. To robustly realize this paradigm, we further propose a two-stage training strategy that prevents codebook collapse and ensures stable convergence. Experiments show that MeloCodec outperforms baselines in singing voice representation, improving pitch consistency and enabling controllable pitch manipulation with minimal timbre degradation.

Figures

Figures reproduced from arXiv: 2608.03021 by the authors.

Figure 1
Figure 1. The architecture of MeloCodec implementing the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Subjective MUSHRA evaluation results comparing MeloCodec [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visual comparison of reconstructed pitch contours (F0). The upper [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 13 canonical work pages

  1. [1]

    Soundstream: An end-to-end neural audio codec,

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi, “Soundstream: An end-to-end neural audio codec,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 495–507, 2021

    2021

  2. [2]

    High fidelity neural audio compression,

    Alexandre D ´efossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi, “High fidelity neural audio compression,”arXiv preprint arXiv:2210.13438, 2022

    arXiv 2022

  3. [3]

    Language models are few-shot learners,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877– 1901, 2020

    1901

  4. [4]

    Zero-shot text-to-image generation,

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever, “Zero-shot text-to-image generation,” inInternational conference on machine learning. Pmlr, 2021, pp. 8821–8831

    work page 2021

  5. [5]

    Neural codec language models are zero-shot text to speech synthesiz- ers,

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al., “Neural codec language models are zero-shot text to speech synthesiz- ers,”arXiv preprint arXiv:2301.02111, 2023

    arXiv 2023

  6. [6]

    Audiolm: a language modeling approach to audio generation,

    Zal ´an Borsos, Rapha ¨el Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al., “Audiolm: a language modeling approach to audio generation,”IEEE/ACM transactions on audio, speech, and language processing, vol. 31, pp. 2523–2533, 2023

    2023

  7. [7]

    Simple and controllable music generation,

    Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre D´efossez, “Simple and controllable music generation,”Advances in Neural Information Processing Systems, vol. 36, pp. 47704–47720, 2023

    work page 2023

  8. [8]

    Codec does matter: Exploring the semantic shortcoming of codec for audio language model,

    Zhen Ye, Peiwen Sun, Jiahe Lei, Hongzhan Lin, Xu Tan, Zheqi Dai, Qiuqiang Kong, Jianyi Chen, Jiahao Pan, Qifeng Liu, et al., “Codec does matter: Exploring the semantic shortcoming of codec for audio language model,” inProceedings of the AAAI Conference on Artificial Intelligence, 2025, vol. 39, pp. 25697–25705

    work page 2025

Show all 31 references
  1. [9]

    Speechtokenizer: Unified speech tokenizer for speech large language models,

    Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu, “Speechtokenizer: Unified speech tokenizer for speech large language models,”arXiv preprint arXiv:2308.16692, 2023

    2023 arXiv

  2. [10]

    Spark-tts: An efficient llm-based text-to-speech model with single- stream decoupled speech tokens,

    Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, et al., “Spark-tts: An efficient llm-based text-to-speech model with single- stream decoupled speech tokens,”arXiv preprint arXiv:2503.01710, 2025

    2025 arXiv

  3. [11]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models,

    Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, et al., “Cosyvoice 2: Scalable streaming speech synthesis with large language models,” arXiv preprint arXiv:2412.10117, 2024

    2024 arXiv

  4. [12]

    Quantize more, lose less: Autoregressive generation from residually quantized speech representations,

    Yichen Han, Xiaoyang Hao, Keming Chen, Weibo Xiong, Jun He, Ruonan Zhang, Junjie Cao, Yue Liu, Bowen Li, Dongrui Zhang, et al., “Quantize more, lose less: Autoregressive generation from residually quantized speech representations,”arXiv preprint arXiv:2507.12197, 2025

    2025 arXiv

  5. [13]

    Scaling under-resourced tts: A data-optimized framework with advanced acoustic modeling for thai,

    Yizhong Geng, Jizhuo Xu, Zeyu Liang, Jinghan Yang, Xiaoyi Shi, and Xiaoyu Shen, “Scaling under-resourced tts: A data-optimized framework with advanced acoustic modeling for thai,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume...

    2025

  6. [14]

    Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al., “Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,”arXiv preprint arXiv:2403.03100, 2024

    2024 arXiv

  7. [15]

    Vevo2: Bridging controllable speech and singing voice generation via unified prosody learning,

    Xueyao Zhang, Junan Zhang, Yuancheng Wang, Chaoren Wang, Yuanzhe Chen, Dongya Jia, Zhuo Chen, and Zhizheng Wu, “Vevo2: Bridging controllable speech and singing voice generation via unified prosody learning,”arXiv preprint arXiv:2508.16332, 2025

    2025

  8. [16]

    Accompanied singing voice synthesis with fully text-controlled melody,

    Ruiqi Li, Zhiqing Hong, Yongqi Wang, Lichao Zhang, Rongjie Huang, Siqi Zheng, and Zhou Zhao, “Accompanied singing voice synthesis with fully text-controlled melody,”arXiv preprint arXiv:2407.02049, 2024

    2024 arXiv

  9. [17]

    Moee: Mixture of emotion experts for audio-driven portrait animation,

    Huaize Liu, Wenzhang Sun, Donglin Di, Shibo Sun, Jiahui Yang, Changqing Zou, and Hujun Bao, “Moee: Mixture of emotion experts for audio-driven portrait animation,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 26222–26231

    2025

  10. [18]

    Comelsinger: Discrete token-based zero-shot singing synthesis with structured melody control and guidance,

    Junchuan Zhao, Wei Zeng, Tianle Lyu, and Ye Wang, “Comelsinger: Discrete token-based zero-shot singing synthesis with structured melody control and guidance,”arXiv preprint arXiv:2509.19883, 2025

    2025 arXiv

  11. [19]

    Neural audio codecs for prompt- driven universal source separation,

    Adhiraj Banerjee and Vipul Arora, “Neural audio codecs for prompt- driven universal source separation,”arXiv e-prints, pp. arXiv–2509, 2025

    2025

  12. [20]

    Vasab: The variable size adaptive in- formation bottleneck for disentanglement on speech and singing voice,

    Frederik Bous and Axel Roebel, “Vasab: The variable size adaptive in- formation bottleneck for disentanglement on speech and singing voice,” arXiv preprint arXiv:2310.03444, 2023

    2023 arXiv

  13. [21]

    Generative de-quantization for neural speech codec via latent diffusion,

    Haici Yang, Inseon Jang, and Minje Kim, “Generative de-quantization for neural speech codec via latent diffusion,” inICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 1251–1255

    2024

  14. [22]

    Hq-svc: Towards high-quality zero- shot singing voice conversion in low-resource scenarios,

    Bingsong Bai, Yizhong Geng, Fengping Wang, Cong Wang, Puyuan Guo, Yingming Gao, and Ya Li, “Hq-svc: Towards high-quality zero- shot singing voice conversion in low-resource scenarios,”arXiv preprint arXiv:2511.08496, 2025

    2025

  15. [23]

    Muse: A multi-agent framework for unconstrained story envisioning via closed-loop cognitive orchestration,

    Wenzhang Sun, Zhenyu Wang, Zhangchi Hu, Chunfeng Wang, Hao Li, and Wei Chen, “Muse: A multi-agent framework for unconstrained story envisioning via closed-loop cognitive orchestration,”arXiv preprint arXiv:2602.03028, 2026

    2026

  16. [24]

    Anyac- comp: Generalizable accompaniment generation via quantized melodic bottleneck,

    Junan Zhang, Yunjia Zhang, Xueyao Zhang, and Zhizheng Wu, “Anyac- comp: Generalizable accompaniment generation via quantized melodic bottleneck,”arXiv preprint arXiv:2509.14052, 2025

    2025

  17. [25]

    High-fidelity audio compression with improved rvqgan,

    Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar, “High-fidelity audio compression with improved rvqgan,”Advances in Neural Information Processing Systems, vol. 36, pp. 27980–27993, 2023

    2023

  18. [26]

    Mucodec: Ultra low-bitrate music codec,

    Yaoxun Xu, Hangting Chen, Jianwei Yu, Wei Tan, Rongzhi Gu, Shun Lei, Zhiwei Lin, and Zhiyong Wu, “Mucodec: Ultra low-bitrate music codec,”arXiv preprint arXiv:2409.13216, 2024

    2024 arXiv

  19. [27]

    Opencpop: A high-quality open source chinese popular song corpus for singing voice synthesis,

    Yu Wang, Xinsheng Wang, Pengcheng Zhu, Jie Wu, Hanzhao Li, Heyang Xue, Yongmao Zhang, Lei Xie, and Mengxiao Bi, “Opencpop: A high-quality open source chinese popular song corpus for singing voice synthesis,”arXiv preprint arXiv:2201.07429, 2022

    2022 arXiv

  20. [28]

    Visqol: an objective speech quality model,

    Andrew Hines, Jan Skoglund, Anil C Kokaram, and Naomi Harte, “Visqol: an objective speech quality model,”EURASIP Journal on Audio, Speech, and Music Processing, vol. 2015, no. 1, pp. 13, 2015

    2015

  21. [29]

    An algorithm for intelligibility prediction of time–frequency weighted noisy speech,

    Cees H Taal, Richard C Hendriks, Richard Heusdens, and Jesper Jensen, “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,”IEEE Transactions on audio, speech, and language processing, vol. 19, no. 7, pp. 2125–2136, 2011

    2011

  22. [30]

    Resnet 34,

    Brett Koonce, “Resnet 34,” inConvolutional neural networks with swift for tensorflow: image recognition and dataset categorization, pp. 51–61. Springer, 2021

    2021

  23. [31]

    Towards the next generation of web-based experiments: A case study assessing basic audio quality following the itu-r recommendation bs. 1534 (mushra),

    Michael Schoeffler, Fabian-Robert St ¨oter, Bernd Edler, and J ¨urgen Herre, “Towards the next generation of web-based experiments: A case study assessing basic audio quality following the itu-r recommendation bs. 1534 (mushra),” in1st Web Audio Conference, 2015, pp. 1–6

    2015

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.