Pith. sign in

REVIEW 3 major objections 6 minor 111 references

Jointly generating SSL, mel and waveform vector fields inside one DiT beats alignment for low-rate waveform diffusion.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 05:09 UTC pith:3SJK7SOQ

load-bearing objection Solid systems paper: hierarchical multi-prompt generation + velocity-space GFM deliver real gains at extreme compression rates; the disentanglement story is oversold relative to the ablations. the 3 major comments →

arxiv 2607.09134 v1 pith:3SJK7SOQ submitted 2026-07-10 cs.SD cs.AIeess.ASeess.SP

ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models

Sang-Hoon Lee , Ha-Yeong Choi This is my paper
classification cs.SD cs.AIeess.ASeess.SP
keywords representation generationhierarchical multi-promptgeneralized flow matchingwaveform diffusionneural audio codeclatent diffusion TTSWave-DiTREPA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard representation alignment (REPA) speeds up diffusion Transformers for audio, yet the authors find that forcing intermediate DiT layers to match external features can entangle latents and starve generative capacity, especially at extreme compression. ReGen replaces alignment with hierarchical multi-prompt generation: the same network estimates vector fields for SSL semantics, mel acoustics and the raw waveform together, progressing layer by layer from abstract to concrete while using masked prompts as infilling targets. A second ingredient, generalized flow matching (GFM), adds a repulsive term between velocity predictions so trajectories do not collapse to a single average flow. The resulting single-stage Wave-DiT codecs (25 Hz FSQ at 400 bps, 12.5 Hz 32-dim VAE) reconstruct speech with markedly better intelligibility and speaker similarity than prior low-rate codecs. Feeding the VAE latents into an LDM that itself runs at 6.25 Hz yields ReGenVoice, a TTS system that reaches strong WER and SIM after one day of training on four GPUs and samples at real-time factor 0.08.

Core claim

Regularizing DiT intermediate states with REPA implicitly entangles latents and limits generative capacity under high compression; explicitly generating hierarchical multi-prompt vector fields (SSL, mel, waveform) inside one DiT, together with repulsive generalized flow matching, restores capacity and yields high-quality single-stage waveform reconstruction and efficient latent TTS from 12.5–6.25 Hz representations.

What carries the argument

ReGen: a hierarchical multi-prompt DiT that jointly estimates independent vector fields for SSL, mel-spectrogram and waveform under masked-infilling prompts, progressing from semantics to acoustics to waveform across layers; plus GFM, which subtracts a stop-gradient repulsive velocity term from the ordinary attractive CFM loss.

Load-bearing premise

That stacking hierarchical generation layers and masked multi-prompts truly disentangles semantic from acoustic capacity rather than simply adding extra supervised targets that still leave the same entanglement.

What would settle it

An ablation that keeps identical hierarchical multi-prompt losses and architecture but removes the layer-wise ordering (or the long skip connections) and measures whether speaker similarity and high-frequency energy collapse back to REPA-H levels on the same low-rate FSQ or VAE latents.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ReGen, a hierarchical multi-prompt framework that jointly estimates independent vector fields for SSL, mel-spectrogram, and waveform targets inside a single DiT, motivated by the claim that REPA-style intermediate regularization entangles latents and reduces generative capacity. It also introduces generalized flow matching (GFM), which adds an asymmetric repulsive term in velocity space to mitigate multi-modal collapse under CFM. The method is validated on single-stage waveform models (ReGenTokenizer at 25 Hz / 400 bps FSQ; ReGenVAE at 12.5 Hz / 32-dim) and on ReGenVoice, an LDM TTS system operating at 6.25 Hz. Empirically, ReGen improves reconstruction and prompted generation over REPA baselines and competitive low-bitrate codecs, and ReGenVoice reports strong WER/SIM with modest data and RTF 0.08 after one day of 4-GPU training.

Significance. If the empirical gains hold under broader scrutiny, ReGen is a practically useful recipe for single-stage waveform diffusion from highly compressed latents, and ReGenVoice is a competitive efficient LDM TTS design (6.25 Hz, small data, low RTF). The hierarchical multi-prompt generation idea and the velocity-space repulsive GFM objective are concrete, reusable contributions for audio diffusion. Strengths include a thorough ablation suite (Tables 4–5, 8–9), multi-setting validation (codec, VAE, TTS), standard metrics (WER, SIM, PESQ, UTMOS, MOS), efficiency numbers, and a stated plan to release code. The work is primarily empirical rather than theoretical; its value is in the architecture and training recipe, not a parameter-free derivation.

major comments (3)
  1. [§3, Table 4, Fig. 1] §3 and the abstract frame ReGen as restoring generative capacity by disentangling semantic/acoustic latents that REPA entangles. The main evidence is the Table 4 jump from REPA-H (SSL+Mel) to ReGen-H (SSL+Mel). Both inject the same targets into the same DiT; the architectural difference is hierarchical multi-prompt generation of independent vector fields plus long-skip connections (Fig. 1). Without a diagnostic of residual entanglement (e.g., mutual information between intermediate activations and SSL vs. waveform residuals, or a probe that freezes representation heads and measures remaining waveform quality), the gain is equally consistent with multi-task CFM supervision + GFM. Please either add such a diagnostic or reframe the claim as multi-target hierarchical generation rather than proven disentanglement.
  2. [§5, Table 4, Tables 1–3] Table 4 shows that adversarial post-training is still required for final quality (e.g., ReGenVAE PESQ 2.063→3.058, UTMOS 3.375→4.096). The abstract and contributions emphasize ReGen/GFM for efficient waveform diffusion, but the best reported numbers combine CFM pre-training with multi-discriminator GAN fine-tuning (§5). Please quantify how much of the SOTA-facing reconstruction/TTS quality is attributable to ReGen/GFM versus post-training, and state clearly which metrics are reported for the pure CFM model versus the 4-step GAN-accelerated model in Tables 1–3 and 6–7.
  3. [Tables 1–3, 6] Key objective tables (Tables 1–3, 6) report point estimates without error bars, multiple seeds, or statistical tests. Given small absolute gaps on WER/SIM versus strong baselines (e.g., Table 6 ReGenVoice 1.62/0.70 vs ZipVoice 1.70/0.69), confidence intervals or multi-seed means are needed to support “strong” / competitive claims, especially for the small-data (0.5k) setting.
minor comments (6)
  1. [Throughout] Throughout the manuscript, “VAE”, “Voice”, and related tokens appear broken as “V AE”, “V oice” (likely PDF extraction/typesetting). Please fix for the camera-ready version.
  2. [§4, Eqs. (4)–(6)] Eqs. (4)–(6): clarify whether GFM is applied only on unmasked positions (consistent with the ⊙M in Eq. (3)) and how the two noise trajectories share the same condition/prompt mask.
  3. [Table 5] Table 5: λ_neg = 0.01 is selected as default, but neighboring values are close; a short note on sensitivity across datasets (LibriTTS vs Emilia) would help reproducibility.
  4. [§7.1–7.2] §7.1: Emilia is MP3-encoded with high-frequency distortion; the text notes quality degradation, but Tables 1–2 still present Emilia-trained models as scalable. A brief discussion of when diversity helps vs. hurts high-frequency metrics would clarify the data-scaling claim.
  5. [Fig. 1] Fig. 1(c) is conceptual only; a small quantitative plot of velocity diversity (e.g., pairwise cosine of predicted fields with/without L_rep) would make GFM more concrete.
  6. [§2] Related work on multi-task / multi-resolution flow matching and hierarchical audio codecs could be tightened so the novelty of joint multi-prompt vector-field generation is sharper versus prior hierarchical tokenizers (SPEAR-TTS, X-Codec, etc.).

Circularity Check

0 steps flagged

Empirical ML paper with no derivation that reduces by construction; self-citations to authors' prior StreamFlow/PeriodWave are baselines/building blocks, not load-bearing uniqueness premises.

full rationale

The paper is an empirical methods contribution (hierarchical multi-prompt CFM + GFM inside a DiT for waveform codecs/VAEs/TTS). Its central claims are performance numbers on external benchmarks (LibriSpeech, LibriTTS, Seed-en) and ablations (Table 4: REPA-H vs ReGen-H; Table 5: λ_neg search). Equations (1)–(7) are standard CFM path definitions plus a repulsive term adapted from Gritsenko et al. 2020; none equal a fitted constant by construction. Hyper-parameters (λ_regen=0.1, λ_neg=0.01) are grid-searched and reported, not smuggled into a “prediction.” Self-citations (StreamFlow, PeriodWave-Turbo, HierSpeech) supply architecture baselines and training recipes; they do not supply a uniqueness theorem that forces the ReGen result. The skeptic’s concern that the REPA-to-ReGen jump may be multi-task supervision rather than true disentanglement is a validity/diagnostic gap, not circularity. Score 1 only for the presence of ordinary author self-citation that is not load-bearing.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 3 invented entities

The paper is an empirical ML systems contribution. Its claims rest on standard diffusion/flow-matching mathematics, a handful of hand-chosen loss weights and architectural depths, and the modeling assumption that jointly generating intermediate representations inside the same DiT disentangles capacity better than alignment. No new physical entities are postulated.

free parameters (6)
  • λ_regen = 0.1
    Weight on Mel and SSL GFM terms relative to waveform GFM; default 0.1, ablated in Table 5.
  • λ_neg = 0.01
    Strength of the repulsive term in GFM; default 0.01, searched in Table 5.
  • noise scales (wave, mel, SSL) = 0.25, 1, 0.25
    Per-representation σ_min / noise schedule scales chosen for training stability.
  • prompt/condition drop rates = 0.3 / 0.2
    Masked-infilling probabilities (0.3 / 0.2) used throughout pre-training.
  • DiT block depths and hidden size = [3,3,6,3,3] / 1024
    Wave-DiT [3,3,6,3,3] @ 1024 dim (or U-Net variants for LDM) chosen by authors.
  • λ_kl (VAE) = 1e-5 (pre), 1e-2 (post)
    Weak KL weight on the 12.5 Hz latent; raised for adversarial stage.
axioms (4)
  • domain assumption Conditional flow matching with linear paths (Eqs. 1–2) is a valid training objective for waveform and representation generation.
    Standard CFM assumption imported from prior flow-matching literature and used throughout §3–4.
  • ad hoc to paper A repulsive term in velocity space (inspired by GED) prevents multi-modal collapse without destabilizing high-dimensional waveform training.
    GFM construction in §4; supported only by the ablation in Table 5, not by a general theorem.
  • ad hoc to paper Hierarchical layering of DiT blocks (semantics → acoustics → waveform) plus long-skip connections sufficiently disentangles capacity.
    Core design claim of §3 and Figure 1; evidenced mainly by ReGen-H vs ReGen-U and REPA comparisons.
  • domain assumption MMS 7th-layer features and 256-bin mel-spectrograms are adequate semantic and acoustic targets for the multi-prompt objectives.
    Choice of supervision signals stated in §3 and Table 10.
invented entities (3)
  • ReGen (hierarchical multi-prompt representation generation) no independent evidence
    purpose: Jointly estimate vector fields for SSL, mel and waveform inside one DiT so that representations are generated rather than merely aligned.
    Central methodological proposal of the paper; independent evidence is the ablation and reconstruction tables, not an external falsifiable prediction.
  • Generalized Flow Matching (GFM) no independent evidence
    purpose: Attractive + asymmetric repulsive loss in velocity space to mitigate trajectory collapse.
    Reformulation of GED-style repulsion for CFM; evidence is internal ablation only.
  • ReGenVoice no independent evidence
    purpose: LDM TTS operating at 6.25 Hz on ReGenVAE latents.
    Application vehicle; performance numbers are the evidence.

pith-pipeline@v1.1.0-grok45 · 24372 in / 3354 out tokens · 33880 ms · 2026-07-13T05:09:29.910133+00:00 · methodology

0 comments
read the original abstract

Representation alignment (REPA) has been investigated to accelerate diffusion training, but we observe that regularizing intermediate representations in diffusion Transformers (DiT) may implicitly entangle latents and limit generative capacity. To address this issue, we propose ReGen, a hierarchical multi-prompt representation generation framework that jointly estimates multiple vector fields for both representations and data within a single diffusion model. We further introduce generalized flow matching (GFM) to improve the generalization of conditional flow matching (CFM). We validate ReGen on single-stage waveform diffusion models including neural audio codec and Wave-VAE. ReGen significantly improves waveform generation quality from highly compressed latent representations at 12.5 Hz. We also present ReGenVoice, a latent diffusion model (LDM)-based text-to-speech model that achieves strong speech intelligibility (WER) and speaker similarity (SIM) with a small dataset. Moreover, operating the LDM at 6.25 Hz with rich semantic and acoustic latent representation enables efficient training and sampling, requiring only 1 day of training on 4 GPUs and fast inference with an RTF of 0.08. Audio samples are available at https://regenvoice.github.io/demo/.

Figures

Figures reproduced from arXiv: 2607.09134 by Ha-Yeong Choi, Sang-Hoon Lee.

Figure 1
Figure 1. Figure 1: (a) A baseline that applies REPA to waveform generation, (b) the proposed ReGen architecture with hierarchical multi-prompting, and (c) a conceptual visualization of generalized flow matching (GFM) 3. ReGen While REPA can accelerate the early states of diffusion training, it can implicitly entangle latent representations in DiT, resulting in capacity mismatch that reduces generative capacity. This can resu… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

111 extracted references · 5 canonical work pages

  1. [1]

    Advances in Neural Information Processing Systems , volume=

    A spectral energy distance for parallel speech synthesis , author=. Advances in Neural Information Processing Systems , volume=

  2. [2]

    arXiv preprint arXiv:2508.16790 , year=

    TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling , author=. arXiv preprint arXiv:2508.16790 , year=

    Pith/arXiv arXiv

  3. [3]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Scalable diffusion models with transformers , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  4. [4]

    arXiv preprint arXiv:2304.07193 , year=

    Dinov2: Learning robust visual features without supervision , author=. arXiv preprint arXiv:2304.07193 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2410.00037 , year=

    Moshi: a speech-text foundation model for real-time dialogue , author=. arXiv preprint arXiv:2410.00037 , year=

    Pith/arXiv arXiv

  6. [6]

    arXiv preprint arXiv:2409.05377 , year=

    Bigcodec: Pushing the limits of low-bitrate neural speech codec , author=. arXiv preprint arXiv:2409.05377 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2502.04128 , year=

    Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis , author=. arXiv preprint arXiv:2502.04128 , year=

    Pith/arXiv arXiv

  8. [8]

    Journal of Machine Learning Research , volume=

    Scaling speech technology to 1,000+ languages , author=. Journal of Machine Learning Research , volume=

  9. [9]

    Advances in Neural Information Processing Systems , volume=

    High-fidelity audio compression with improved rvqgan , author=. Advances in Neural Information Processing Systems , volume=

  10. [10]

    The Thirteenth International Conference on Learning Representations , year=

    Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think , author=. The Thirteenth International Conference on Learning Representations , year=

  11. [11]

    The Thirteenth International Conference on Learning Representations , year=

    FlowDec: A flow-based full-band general audio codec with high perceptual quality , author=. The Thirteenth International Conference on Learning Representations , year=

  12. [12]

    International Conference on Learning Representations , volume=

    Rfwave: Multi-band rectified flow for audio waveform reconstruction , author=. International Conference on Learning Representations , volume=

  13. [13]

    The Thirteenth International Conference on Learning Representations , year=

    PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation , author=. The Thirteenth International Conference on Learning Representations , year=

  14. [14]

    arXiv preprint arXiv:1609.03499 , year=

    Wavenet: A generative model for raw audio , author=. arXiv preprint arXiv:1609.03499 , year=

    Pith/arXiv arXiv

  15. [15]

    International conference on machine learning , pages=

    Parallel wavenet: Fast high-fidelity speech synthesis , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  16. [16]

    International Conference on Learning Representations , year=

    ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech , author=. International Conference on Learning Representations , year=

  17. [17]

    ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Waveglow: A flow-based generative network for speech synthesis , author=. ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2019 , organization=

    2019

  18. [18]

    International Conference on Machine Learning , pages=

    FloWaveNet: A Generative Flow for Raw Audio , author=. International Conference on Machine Learning , pages=. 2019 , organization=

    2019

  19. [19]

    Advances in neural information processing systems , volume=

    Melgan: Generative adversarial networks for conditional waveform synthesis , author=. Advances in neural information processing systems , volume=

  20. [20]

    ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram , author=. ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2020 , organization=

    2020

  21. [21]

    arXiv preprint arXiv:1909.11646 , year=

    High fidelity speech synthesis with adversarial networks , author=. arXiv preprint arXiv:1909.11646 , year=

    Pith/arXiv arXiv 1909

  22. [22]

    Advances in neural information processing systems , volume=

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis , author=. Advances in neural information processing systems , volume=

  23. [23]

    Won Jang and Dan Lim and Jaesam Yoon and Bongwan Kim and Juntae Kim , title=. Proc. Interspeech 2021 , pages=. doi:10.21437/Interspeech.2021-1016 , issn=

    doi:10.21437/interspeech.2021-1016 2021

  24. [24]

    Lee, Sang-gil and Wei Ping and Boris Ginsburg and Bryan Catanzaro and Sungroh Yoon , booktitle=. Big. 2023 , url=

    2023

  25. [25]

    The Twelfth International Conference on Learning Representations , year=

    Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis , author=. The Twelfth International Conference on Learning Representations , year=

  26. [26]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=

    Soundstream: An end-to-end neural audio codec , author=. IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=. 2021 , publisher=

    2021

  27. [27]

    Transactions on Machine Learning Research , issn=

    High Fidelity Neural Audio Compression , author=. Transactions on Machine Learning Research , issn=. 2023 , url=

    2023

  28. [28]

    arXiv preprint arXiv:2301.02111 , year=

    Neural codec language models are zero-shot text to speech synthesizers , author=. arXiv preprint arXiv:2301.02111 , year=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2310.00704 , year=

    Uniaudio: An audio foundation model toward universal audio generation , author=. arXiv preprint arXiv:2310.00704 , year=

    Pith/arXiv arXiv

  30. [30]

    International Conference on Learning Representations , year=

    DiffWave: A Versatile Diffusion Model for Audio Synthesis , author=. International Conference on Learning Representations , year=

  31. [31]

    International Conference on Learning Representations , year=

    WaveGrad: Estimating Gradients for Waveform Generation , author=. International Conference on Learning Representations , year=

  32. [32]

    International Conference on Learning Representations , year=

    PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior , author=. International Conference on Learning Representations , year=

  33. [33]

    Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence,

    FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis , author =. Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence,. 2022 , month =. doi:10.24963/ijcai.2022/577 , url =

    work page doi:10.24963/ijcai.2022/577 2022

  34. [34]

    Yuma Koizumi and Heiga Zen and Kohei Yatabe and Nanxin Chen and Michiel Bacchiani , title=. Proc. Interspeech 2022 , pages=. doi:10.21437/Interspeech.2022-301 , issn=

    work page doi:10.21437/interspeech.2022-301 2022

  35. [35]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  36. [36]

    Advances in Neural Information Processing Systems , volume=

    Voicemixer: Adversarial voice style mixup , author=. Advances in Neural Information Processing Systems , volume=

  37. [37]

    Advances in Neural Information Processing Systems , volume=

    Neural analysis and synthesis: Reconstructing speech from self-supervised representations , author=. Advances in Neural Information Processing Systems , volume=

  38. [38]

    Advances in neural information processing systems , volume=

    Fastspeech: Fast, robust and controllable text to speech , author=. Advances in neural information processing systems , volume=

  39. [39]

    Advances in neural information processing systems , volume=

    Voicebox: Text-guided multilingual universal speech generation at scale , author=. Advances in neural information processing systems , volume=

  40. [40]

    Advances in Neural Information Processing Systems , volume=

    P-Flow: A Fast and Data-Efficient Zero-Shot TTS through Speech Prompting , author=. Advances in Neural Information Processing Systems , volume=

  41. [41]

    Advances in Neural Information Processing Systems , volume=

    Styletts 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models , author=. Advances in Neural Information Processing Systems , volume=

  42. [42]

    Advances in Neural Information Processing Systems , volume=

    Nanoflow: Scalable normalizing flows with sublinear parameter complexity , author=. Advances in Neural Information Processing Systems , volume=

  43. [43]

    Advances in Neural Information Processing Systems , volume=

    HierSpeech: Bridging the gap between text and speech by hierarchical variational inference using self-supervised representations for speech synthesis , author=. Advances in Neural Information Processing Systems , volume=

  44. [44]

    Advances in Neural Information Processing Systems , volume=

    Generspeech: Towards style transfer for generalizable out-of-domain text-to-speech , author=. Advances in Neural Information Processing Systems , volume=

  45. [45]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , year=

    Diffsound: Discrete diffusion model for text-to-sound generation , author=. IEEE/ACM Transactions on Audio, Speech, and Language Processing , year=

  46. [46]

    International Conference on Machine Learning , pages=

    Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  47. [47]

    2023 , editor =

    Liu, Haohe and Chen, Zehua and Yuan, Yi and Mei, Xinhao and Liu, Xubo and Mandic, Danilo and Wang, Wenwu and Plumbley, Mark D , booktitle =. 2023 , editor =

    2023

  48. [48]

    Forty-first International Conference on Machine Learning , year=

    NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models , author=. Forty-first International Conference on Machine Learning , year=

  49. [49]

    arXiv preprint arXiv:2210.02747 , year=

    Flow matching for generative modeling , author=. arXiv preprint arXiv:2210.02747 , year=

    Pith/arXiv arXiv

  50. [50]

    arXiv preprint arXiv:2302.00482 , volume=

    Conditional flow matching: Simulation-free dynamic optimal transport , author=. arXiv preprint arXiv:2302.00482 , volume=

    Pith/arXiv arXiv

  51. [51]

    Digital music research network one-day workshop (DMRN+ 15) , year=

    auraloss: Audio focused loss functions in PyTorch , author=. Digital music research network one-day workshop (DMRN+ 15) , year=

  52. [52]

    Chunked Autoregressive

    Max Morrison and Rithesh Kumar and Kundan Kumar and Prem Seetharaman and Aaron Courville and Yoshua Bengio , booktitle=. Chunked Autoregressive. 2022 , url=

    2022

  53. [53]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Convnext v2: Co-designing and scaling convnets with masked autoencoders , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  54. [54]

    Keith Ito and Linda Johnson , title =

  55. [55]

    Weiss and Ye Jia and Zhifeng Chen and Yonghui Wu , title=

    Heiga Zen and Viet Dang and Rob Clark and Yu Zhang and Ron J. Weiss and Ye Jia and Zhifeng Chen and Yonghui Wu , title=. Proc. Interspeech 2019 , pages=. doi:10.21437/Interspeech.2019-2441 , issn=

    doi:10.21437/interspeech.2019-2441 2019

  56. [56]

    Seungu Han and Junhyeok Lee , title=. Proc. Interspeech 2022 , pages=. doi:10.21437/Interspeech.2022-45 , issn=

    work page doi:10.21437/interspeech.2022-45 2022

  57. [57]

    2024 IEEE Spoken Language Technology Workshop (SLT) , pages=

    E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts , author=. 2024 IEEE Spoken Language Technology Workshop (SLT) , pages=. 2024 , organization=

    2024

  58. [58]

    arXiv preprint arXiv:2406.05551 , year=

    Autoregressive Diffusion Transformer for Text-to-Speech Synthesis , author=. arXiv preprint arXiv:2406.05551 , year=

    Pith/arXiv arXiv

  59. [59]

    Keon Lee and Dong Won Kim and Jaehyeon Kim and Seungjun Chung and Jaewoong Cho , booktitle=. Di. 2025 , url=

    2025

  60. [60]

    Won Jang and Dan Lim and Heayoung Park , title=. Proc. INTERSPEECH 2023 , pages=. doi:10.21437/Interspeech.2023-2379 , issn=

    work page doi:10.21437/interspeech.2023-2379 2023

  61. [61]

    2022 IEEE Spoken Language Technology Workshop (SLT) , pages=

    WaveFit: An iterative and non-autoregressive neural vocoder based on fixed-point iteration , author=. 2022 IEEE Spoken Language Technology Workshop (SLT) , pages=. 2023 , organization=

    2022

  62. [62]

    arXiv preprint arXiv:2204.09934 , year=

    Fastdiff: A fast conditional diffusion model for high-quality speech synthesis , author=. arXiv preprint arXiv:2204.09934 , year=

    Pith/arXiv arXiv

  63. [63]

    The Thirteenth International Conference on Learning Representations , year=

    WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling , author=. The Thirteenth International Conference on Learning Representations , year=

  64. [64]

    arXiv preprint arXiv:2408.17175 , year=

    Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model , author=. arXiv preprint arXiv:2408.17175 , year=

    Pith/arXiv arXiv

  65. [65]

    The Twelfth International Conference on Learning Representations , year=

    Speechtokenizer: Unified speech tokenizer for speech language models , author=. The Twelfth International Conference on Learning Representations , year=

  66. [66]

    arXiv preprint arXiv:2408.08019 , year=

    Accelerating High-Fidelity Waveform Generation via Adversarial Flow Matching Optimization , author=. arXiv preprint arXiv:2408.08019 , year=

    Pith/arXiv arXiv

  67. [67]

    arXiv preprint arXiv:2305.02765 , year=

    Hifi-codec: Group-residual vector quantization for high fidelity audio codec , author=. arXiv preprint arXiv:2305.02765 , year=

    Pith/arXiv arXiv

  68. [68]

    ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Fregrad: Lightweight and Fast Frequency-Aware Diffusion Vocoder , author=. ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2024 , organization=

    2024

  69. [69]

    The Twelfth International Conference on Learning Representations , year=

    Generative Pre-training for Speech with Flow Matching , author=. The Twelfth International Conference on Learning Representations , year=

  70. [70]

    ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Multi-scale sub-band constant-q transform discriminator for high-fidelity vocoder , author=. ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2024 , organization=

    2024

  71. [71]

    International conference on machine learning , pages=

    Robust speech recognition via large-scale weak supervision , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  72. [72]

    2024 , url=

    Anonymous , booktitle=. 2024 , url=

    2024

  73. [73]

    arXiv preprint arXiv:2410.06885 , year=

    F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching , author=. arXiv preprint arXiv:2410.06885 , year=

    Pith/arXiv arXiv

  74. [74]

    The Thirteenth International Conference on Learning Representations , year=

    Scaling Transformers for Low-Bitrate High-Quality Speech Coding , author=. The Thirteenth International Conference on Learning Representations , year=

  75. [75]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    All are worth words: A vit backbone for diffusion models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  76. [76]

    2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages=

    Librispeech: an asr corpus based on public domain audio books , author=. 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages=. 2015 , organization=

    2015

  77. [77]

    Advances in neural information processing systems , volume=

    Score-based generative modeling in latent space , author=. Advances in neural information processing systems , volume=

  78. [78]

    Advances in Neural Information Processing Systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in Neural Information Processing Systems , volume=

  79. [79]

    Advances in Neural Information Processing Systems , volume=

    Diffusion models beat gans on image synthesis , author=. Advances in Neural Information Processing Systems , volume=

  80. [80]

    International Conference on Learning Representations , year=

    Score-Based Generative Modeling through Stochastic Differential Equations , author=. International Conference on Learning Representations , year=

Showing first 80 references.