Pith. sign in

REVIEW 5 major objections 5 minor 44 references

InfiniteAudio: Infinite-Length Audio Generation with Consistency

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A fixed-size sliding window lets pretrained text-to-audio diffusion models generate audio of any length at constant memory, without retraining.

desk verdict A useful inference-time adaptation of FIFO-Diffusion to audio with a real memory win, but the 'infinite consistency' headline is extrapolated from 10–20 s tests and the evaluation protocol is too selection-biased to trust the quantitative claims. read the letter →

arxiv 2506.03020 v1 pith:A3BI7ILX submitted 2025-06-03 eess.AS

Chaeyoung Jung , Hojoon Ki , Ji-Hoon Kim , Junmo Kim , Joon Son Chung This is my paper · ORCID
classification eess.AS
keywords text-to-audiogenerationlong-formaudiodiffusionmodelsFIFOsamplingcurveddenoisingself-attentionlatentinference-only
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

InfiniteAudio claims that a pre-trained diffusion-based text-to-audio model can generate arbitrarily long audio without any additional training. The method keeps the model's input size fixed and slides a window through the latent noise, so memory use stays constant no matter how long the output is. It also introduces curved denoising, which spends sampling steps only on the time regions that matter most, cutting the step count below 150 while preserving quality. This matters because current text-to-audio models either run out of memory on long clips or produce inconsistent audio by concatenating short segments.

What carries the argument

The machinery is FIFO sampling with a buffer zone, adapted from video generation to audio mel-spectrogram latents, combined with curved denoising. FIFO sampling assigns a different diffusion timestep to each frame in a fixed-length latent window and advances the window by one frame per denoising step; the buffer zone consists of leading frames that are left unperturbed so the pre-trained model always sees timestep values it encountered during training. Curved denoising partitions the diffusion trajectory into initial, middle, and final regions, measures each region's average self-attention score, and either keeps or skips steps accordingly, so a model that attends mostly to early steps spends its sampling budget there.

What would settle it

Generate a 60-second clip with a pre-trained TTA model using InfiniteAudio while recording, at each popping step, the per-frame diffusion timestep actually fed to the model. If a plausible accounting of the training noise schedule shows that frames outside the buffer zone receive timestep values the model was never trained on (or if quality measurably degrades as audio length grows beyond the window), the central claim fails. A simpler observable falsifier: run the method on the same prompt for 20, 60, and 120 seconds and check whether CLAP score drops with length; the paper's own data only shows stability up to 60 seconds.

Watch

Extended reading notes

Core claim

The central claim is that a latent diffusion text-to-audio model can be driven as a first-in, first-out queue: at each inference step, the model denoises a fixed-length latent window in which different frames carry different noise levels, the cleanest frame is popped out as finished audio, and a new noisy frame is pushed in at the end. Because the window never grows, memory stays bounded regardless of output length. A buffer zone of unperturbed frames keeps the per-frame noise values inside the distribution seen during training, and a curved denoising schedule derived from self-attention maps focuses sampling effort on the steps each model actually cares about. On AudioLDM and VoiceLDM, the paper reports that this inference-only procedure matches or beats the original full-sampling baselines on CLAP, FD, KL, and human ratings, while enabling seamless 20-to-60-second generation.

Load-bearing premise

The buffer zone keeps the noise levels seen by the pre-trained model within the training distribution, so the denoiser is never asked to handle out-of-distribution mixed-noise inputs.

Editorial extensions

If this is right

  • Any existing latent-diffusion text-to-audio model can produce arbitrarily long audio by plugging in InfiniteAudio's inference schedule, with no retraining or fine-tuning.
  • Memory consumption for generation becomes independent of the generated audio length, removing the principal scaling bottleneck of diffusion TTA.
  • The curved denoising schedule reduces the number of sampling steps below 150 while achieving quality comparable to or better than the original full-step baseline and better than equally spaced or middle-focused schedules.
  • Long-form generations stay temporally consistent, with stable CLAP scores across lengths from 10 to 60 seconds, in contrast to concatenation-based baselines that show repetitive patterns and discontinuities.
  • The method works on at least two different TTA architectures (AudioLDM and VoiceLDM) even though their attention maps assign importance to opposite ends of the sampling trajectory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The attention-based step-selection criterion may transfer to other diffusion domains such as music generation or long video, since it only inspects self-attention maps that any U-Net already computes.
  • The buffer-zone trick is a general way to keep sliding-window diffusion in-distribution, and its length is likely a hyperparameter worth tuning separately for each base model.
  • Because the input window stays at roughly 3 seconds of audio, the method may cap the effective temporal context a prompt can control; prompts requiring structure over, say, 10 seconds should be tested separately.
  • A direct test of the core assumption would be to measure the denoiser's noise-prediction error on FIFO 'mixed-noise' inputs versus standard single-timestep inputs; if the error spikes, the buffer zone is not actually protecting the training distribution.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes InfiniteAudio, a training-free inference method for diffusion-based text-to-audio models. It adapts FIFO-Diffusion to the audio latent space by treating mel-spectrogram latent frames as a fixed-size queue, denoising each frame with a different diffusion timestep, popping finished frames, and appending new noise. A second component, called curved denoising, selects which diffusion steps to emphasize based on self-attention maps in the U-Net, reducing the number of sampling steps. Experiments on AudioLDM and VoiceLDM report CLAP, FD, KL, and subjective scores for 10-second generation, a memory-consumption comparison, and qualitative spectrograms for 20- and 60-second audio. The central claim is that the method generates arbitrarily long, temporally consistent audio with constant memory and comparable-or-better quality than the base models.

Significance. If the central claim held, InfiniteAudio would be a practical plug-in extension for long-duration text-to-audio generation, requiring no retraining and offering a clear memory advantage. The paper's strengths are that the FIFO mechanics are plausible, the constant-memory result in Fig. 3 is clear and convincing, and the attention-based motivation for timestep allocation is interesting. However, the evaluation is not commensurate with the headline claim: quantitative results stop at 20 seconds, the test set and the focus-region strategy are selected post hoc on the same data used for the reported scores, Table 3 uses an incompatible CLAP checkpoint, and several reported metrics contradict the abstract's 'comparable or superior across all metrics' statement. As it stands, the contribution is a promising inference heuristic with a demonstrated memory advantage, not a validated method for infinite-length consistent audio.

major comments (5)
  1. [Table 1 and Abstract/Conclusion] The claim of 'comparable or superior performance across all metrics' is not supported by Table 1. For AudioLDM, the Ours (Last focused) row is worse than the original on CLAP (0.4559 vs 0.4908) and on both subjective metrics (OVL 2.63 vs 3.03, REL 2.80 vs 3.06), improving only on FD and KL. For VoiceLDM, the Ours (Initial focused) row is worse than the original on CLAP (0.4107 vs 0.4199), FD (51.5047 vs 51.4019), KL (2.3498 vs 2.2749), and OVL (2.38 vs 2.53). Please restate the claim to reflect the metrics that actually improve, or provide a corrected evaluation.
  2. [§3.1.1 and §3.2.2 / Table 1] The evaluation design selects both the test subset and the focus-region strategy using the same data that produces the headline numbers. Excluding the bottom 20% of CLAP pairs based on AudioLDM/VoiceLDM predictions removes the hardest examples from every reported CLAP/FD/KL figure, and the 'Ours' rows in Table 1 are chosen after comparing four strategies on that same test set (Last focused for AudioLDM, Initial focused for VoiceLDM). This makes the reported 'superior' performance a selected maximum rather than a validated design choice. Please evaluate on the full Audiocaps test set, or on a fixed held-out split determined before any filtering, and specify the strategy-selection rule independently of the test-set outcome.
  3. [§2.2.2 and Table 3 / Fig. 5] The quantitative evaluation for the 'infinite-length' claim stops at 10–20 seconds (Table 3), with only a few 20/60-second spectrograms in Fig. 5, while the input window is reduced to about 3 seconds. Once a frame is popped, the model can never condition on it again, so a ~3-second window cannot represent an acoustic phrase or event longer than the window, and no experiment tracks error accumulation over many frame replacements. The headline property of arbitrarily long, temporally consistent generation is therefore an extrapolation from short-window results. Please add long-horizon tests—for example, 60–120-second generations with prompt elements separated by more than 3 seconds, per-segment CLAP/FD over successive windows, and repeated-run stability—before claiming infinite-length consistency.
  4. [Table 3 and footnote 4] The CLAP scores in Table 3 are computed with a different CLAP checkpoint than the ones in Tables 1–2, so the 'fixed 10 second' baseline is not directly comparable across tables. In particular, AudioLDM's fixed-10s CLAP is 0.4559 in Table 1 but 0.3207 in Table 3, an inconsistency that the reader cannot resolve from the numbers alone. Please recompute all CLAP scores with a single checkpoint, or clearly state that no cross-table comparison is intended.
  5. [Tables 1–3] No error bars, confidence intervals, or multiple-run results are reported anywhere in the quantitative evaluation. The differences invoked to support 'comparable or superior' are often small (e.g., CLAP 0.4908 vs 0.4559 for AudioLDM), and without variance estimates it is impossible to tell whether they are meaningful. Please include at least standard errors over dataset subsets or random seeds for all reported metrics.
minor comments (5)
  1. [§2.2.2] The term 'curved denoising' is never defined formally. Please specify the exact step-skipping rule, the skipping factor P, and the buffer-zone size used in the experiments, since these are free parameters of the method and are needed for reproducibility.
  2. [§2.1] The description of the 'initial region' as 'early stages of sampling, where diffusion timesteps are close to τ' is ambiguous because τ decreases during sampling. Please clarify whether 'initial' refers to the temporal position in the sequence or to the noise level.
  3. [Fig. 4 caption] The numeric CLAP values in the subfigure table are not explained in the caption. Please state which model and which prompt or evaluation set they correspond to.
  4. [§3.1.1] The subjective evaluation says '30 speech samples,' but the task is text-to-audio generation. Please clarify whether this should read 'audio samples' and describe how the 30 items were selected.
  5. [Throughout] The model name 'V oiceLDM' appears with a spurious space in several places; also, the phrase 'we first propose InfiniteAudio' in the contributions list is redundant and should be trimmed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: InfiniteAudio's claims are empirical and its attention-guided timestep schedule is not derived from the evaluation metrics.

full rationale

The paper's central claims are empirical: FIFO sampling plus curved denoising yields comparable metrics on AudioCaps, with no derivation chain in which an output is equivalent to an input by construction. The buffer-zone design is adopted from the published FIFO-Diffusion method, and the paper does not rely on an unverified self-citation to force its conclusions. The choice of focused timestep region for each model is justified in Section 2.2.2 by self-attention analysis, not by the evaluation tables; although the reader may worry that the 'Ours' label is assigned to whichever region scores best per model in Table 1, the paper presents the attention maps as the selection criterion, and no equation or fitted parameter is renamed as a prediction. Potential concerns about test-set filtering (excluding bottom 20% CLAP pairs), the use of a different CLAP checkpoint in Table 3, and extrapolation from 10-20 s to 'infinite' are validity or scope issues, not circularity. Under the rule that circularity requires a quoted reduction, no such reduction is exhibited.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the pre-trained models' internal attention maps and the buffer-zone assumption. No new entities are postulated. The free parameters are mostly unspecified, and the focus-region choice is fitted to the evaluation set.

free parameters (4)
  • Focus region selection (initial vs middle vs last) = AudioLDM: last; VoiceLDM: initial
    Chosen per model based on which strategy scores best on the 500-pair evaluation set (Table 1). This is a post hoc fit to the evaluation data.
  • Skipping factor P = not specified
    Controls how many timesteps are skipped in less critical regions; the value is never given in the paper, making the method underspecified.
  • Buffer zone size = not specified
    Number of initial frames kept unperturbed in FIFO sampling; not quantified, yet it is critical for the claimed consistency.
  • Input latent length = about 3 seconds
    The paper states that the input size is reduced to about 3 seconds, but the exact number of frames is not provided.
assumptions (3)
  • domain assumption Pre-trained TTA models (AudioLDM, VoiceLDM) behave as described in the paper
    The method relies on the fidelity and internal structure of the pre-trained LDMs; any failure in the base model propagates to the infinite generation.
  • domain assumption Self-attention maps in the U-Net decoder measure cross-frame influence relevant to generation quality
    Used to select critical timestep regions; the paper does not prove that attention magnitude corresponds to perceptual importance.
  • standard math Standard Gaussian noise and denoising loss as in Eq. (1) faithfully model the TTA process
    This is the standard diffusion formulation taken from prior work and is not in question.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InfiniteAudio: Infinite-Length Audio Generation with Consistency." pith.science (2026). https://pith.science/paper/A3BI7ILX

@misc{pith2026250603020,
  author       = {Pith},
  title        = {Pith review of: InfiniteAudio: Infinite-Length Audio Generation with Consistency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A3BI7ILX}},
  note         = {Machine review of arXiv:2506.03020}
}
read the original abstract

This paper presents InfiniteAudio, a simple yet effective strategy for generating infinite-length audio using diffusion-based text-to-audio methods. Current approaches face memory constraints because the output size increases with input length, making long duration generation challenging. A common workaround is to concatenate short audio segments, but this often leads to inconsistencies due to the lack of shared temporal context. To address this, InfiniteAudio integrates seamlessly into existing pipelines without additional training. It introduces two key techniques: FIFO sampling, a first-in, first-out inference strategy with fixed-size inputs, and curved denoising, which selectively prioritizes key diffusion steps for efficiency. Experiments show that InfiniteAudio achieves comparable or superior performance across all metrics. Audio samples are available on our project page.

Figures

Figures reproduced from arXiv: 2506.03020 by the authors.

Figure 1
Figure 1. Comparison of Existing Methods and InfiniteAudio. Traditional methods apply uniform diffusion timesteps across all input latents, whereas InfiniteAudio dynamically selects timesteps based on their importance. This adaptive approach enables the generation of theoretically infinite audio while maintaining a fixed input size, ensuring both efficiency and high-quality synthesis. text-to-audio generation baselines. 2. Me… view at source ↗
Figure 3
Figure 3. Memory consumption comparison between Audi￾oLDM [24] and our method 3. Experiment 3.1. Experimental Settings 3.1.1. Datasets and Baselines. To evaluate our method on TTA generation, we utilize 500 audio-text pairs from the 975 test files in the Audiocaps dataset [38], which is commonly used for assessing TTA mod￾els. Given that our method relies heavily on the performance of existing baselines, we exclude the bottom… view at source ↗
Figure 5
Figure 5. presents the mel spectrograms of audio generated by Au￾dioLDM [24] at different lengths using InfiniteAudio, compared to audio produced via the concatenation method. The results demonstrate that InfiniteAudio generates high-quality, long￾duration audio while preserving stable CLAP scores across varying lengths. Moreover, it maintains seamless consistency and natural coherence throughout the entire audio, whereas the… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 35 canonical work pages

  1. [1]

    InfiniteAudio: Infinite-Length Audio Generation with Consistency

    Introduction Diffusion models [1, 2] have gained significant attention for their ability to generate high-quality and diverse outputs, achieving state-of-the-art performance across various domains, including image [3, 4, 5, 6], video [7, 8, 9, 10, 11, 12, 13], and audio [14, 15, 16, 17, 18, 19, 20]. However, their high computational cost limits their prac...

    work page Pith review arXiv 2025

  2. [2]

    Method 2.1. Preliminaries We provide a comprehensive overview of existing TTA genera- tion models, which synthesize realistic audio from text prompts y by representing audio as a 2D mel-spectrogram, capturing both time and frequency dimensions. Most TTA models share a common architecture, consist- ing of audio faudio(·) and text encoders ftext(·), a LDM, ...

    work page

  3. [3]

    Experimental Settings 3.1.1

    Experiment 3.1. Experimental Settings 3.1.1. Datasets and Baselines. To evaluate our method on TTA generation, we utilize 500 audio-text pairs from the 975 test files in the Audiocaps dataset [38], which is commonly used for assessing TTA mod- els. Given that our method relies heavily on the performance of existing baselines, we exclude the bottom 20 perc...

    work page

  4. [4]

    By maintaining a fixed memory footprint, InfiniteAudio overcomes memory constraints in existing mod- els and integrates seamlessly with diffusion-based TTA ap- proaches

    Conclusion We introduce InfiniteAudio, a novel inference method for gen- erating infinitely long, consistent audio using pretrained text- to-audio models. By maintaining a fixed memory footprint, InfiniteAudio overcomes memory constraints in existing mod- els and integrates seamlessly with diffusion-based TTA ap- proaches. Despite relying solely on infere...

    work page

  5. [5]

    RS-2023-00212845, Multimodal Speech Processing for Human-Computer Interaction)

    Acknowledgements This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00212845, Multimodal Speech Processing for Human-Computer Interaction). 4https://github.com/LAION-AI/CLAP

    work page 2023

  6. [6]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” NeurIPS, 2020

    2020

  7. [7]

    Score-based generative modeling through stochas- tic differential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochas- tic differential equations,” in Proc. ICLR, 2021

    work page 2021

  8. [8]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” NeurIPS, 2021

    work page 2021

Show all 44 references
  1. [9]

    High-resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proc. CVPR, 2022

    2022

  2. [10]

    Photorealistic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al. , “Photorealistic text-to-image diffusion models with deep language understanding,” in NeurIPS, 2022

    2022

  3. [11]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models,

    A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. Mc- Grew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” arXiv:2112.10741, 2021

    2021 arXiv

  4. [12]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,”NeurIPS, 2022

    2022

  5. [13]

    Make-a-video: Text- to-video generation without text-video data,

    U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni et al. , “Make-a-video: Text- to-video generation without text-video data,” arXiv:2209.14792, 2022

    2022 arXiv

  6. [14]

    Modelscope text-to-video technical report,

    J. Wang, H. Yuan, D. Chen, Y . Zhang, X. Wang, and S. Zhang, “Modelscope text-to-video technical report,” arXiv:2308.06571, 2023

    2023 arXiv

  7. [15]

    Diffusion probabilistic modeling for video generation,

    R. Yang, P. Srivastava, and S. Mandt, “Diffusion probabilistic modeling for video generation,” Entropy, vol. 25, p. 1469, 2023

    2023

  8. [16]

    Lavie: High-quality video generation with cascaded latent diffusion models,

    Y . Wang, X. Chen, X. Ma, S. Zhou, Z. Huang, Y . Wang, C. Yang, Y . He, J. Yu, P. Yanget al., “Lavie: High-quality video generation with cascaded latent diffusion models,” International Journal of Computer Vision, vol. 133, pp. 3059–3078, 2024

    2024

  9. [17]

    Lumiere: A space-time diffusion model for video generation,

    O. Bar-Tal, H. Chefer, O. Tov, C. Herrmann, R. Paiss, S. Zada, A. Ephrat, J. Hur, G. Liu, A. Raj et al., “Lumiere: A space-time diffusion model for video generation,” in SIGGRAPH Asia 2024 Conference Papers, 2024

    2024

  10. [18]

    Videocrafter1: Open diffusion models for high-quality video generation,

    H. Chen, M. Xia, Y . He, Y . Zhang, X. Cun, S. Yang, J. Xing, Y . Liu, Q. Chen, X. Wanget al., “Videocrafter1: Open diffusion models for high-quality video generation,” arXiv:2310.19512, 2023

    2023 arXiv

  11. [19]

    Grad-tts: A diffusion probabilistic model for text-to-speech,

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, and M. Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” in Proc. ICML, 2021

    2021

  12. [20]

    Guided-tts: A diffusion model for text-to-speech via classifier guidance,

    H. Kim, S. Kim, and S. Yoon, “Guided-tts: A diffusion model for text-to-speech via classifier guidance,” in Proc. ICML, 2022

    2022

  13. [21]

    Styletts 2: Towards human-level text-to-speech through style dif- fusion and adversarial training with large speech language mod- els,

    Y . A. Li, C. Han, V . Raghavan, G. Mischler, and N. Mesgarani, “Styletts 2: Towards human-level text-to-speech through style dif- fusion and adversarial training with large speech language mod- els,” NeurIPS, 2024

    2024

  14. [22]

    Diff- tts: A denoising diffusion model for text-to-speech,

    M. Jeong, H. Kim, S. J. Cheon, B. J. Choi, and N. S. Kim, “Diff- tts: A denoising diffusion model for text-to-speech,” in Proc. In- terspeech, 2021

    2021

  15. [23]

    Diffvoice: Text-to-speech with latent diffusion,

    Z. Liu, Y . Guo, and K. Yu, “Diffvoice: Text-to-speech with latent diffusion,” in Proc. ICASSP, 2023

    2023

  16. [24]

    Seeing through the conversation: Audio-visual speech separation based on diffu- sion model,

    S. Lee, C. Jung, Y . Jang, J. Kim, and J. S. Chung, “Seeing through the conversation: Audio-visual speech separation based on diffu- sion model,” in Proc. ICASSP, 2024

    2024

  17. [25]

    Flowavse: Efficient audio-visual speech enhancement with conditional flow match- ing,

    C. Jung, S. Lee, J.-H. Kim, and J. S. Chung, “Flowavse: Efficient audio-visual speech enhancement with conditional flow match- ing,” in Proc. Interspeech, 2024

    2024

  18. [26]

    Align your latents: High-resolution video synthesis with latent diffusion models,

    A. Blattmann, R. Rombach, H. Ling, T. Dockhorn, S. W. Kim, S. Fidler, and K. Kreis, “Align your latents: High-resolution video synthesis with latent diffusion models,” in Proc. CVPR, 2023

    2023

  19. [27]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis,

    D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ¨uller, J. Penna, and R. Rombach, “Sdxl: Improving latent diffusion models for high-resolution image synthesis,” in Proc. ICLR, 2025

    2025

  20. [28]

    Mag- icvideo: Efficient video generation with latent diffusion models,

    D. Zhou, W. Wang, H. Yan, W. Lv, Y . Zhu, and J. Feng, “Mag- icvideo: Efficient video generation with latent diffusion models,” arXiv:2211.11018, 2022

    2022 arXiv

  21. [29]

    Audioldm: Text-to-audio generation with latent diffusion models,

    H. Liu, Z. Chen, Y . Yuan, X. Mei, X. Liu, D. Mandic, W. Wang, and M. D. Plumbley, “Audioldm: Text-to-audio generation with latent diffusion models,” in Proc. ICML, 2023

    2023

  22. [30]

    Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models,

    R. Huang, J. Huang, D. Yang, Y . Ren, L. Liu, M. Li, Z. Ye, J. Liu, X. Yin, and Z. Zhao, “Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models,” in Proc. ICML, 2023

    2023

  23. [31]

    V oiceldm: Text-to- speech with environmental context,

    Y . Lee, I. Yeon, J. Nam, and J. S. Chung, “V oiceldm: Text-to- speech with environmental context,” inProc. ICASSP, 2024

    2024

  24. [32]

    Audioldm 2: Learn- ing holistic audio generation with self-supervised pretraining,

    H. Liu, Y . Yuan, X. Liu, X. Mei, Q. Kong, Q. Tian, Y . Wang, W. Wang, Y . Wang, and M. D. Plumbley, “Audioldm 2: Learn- ing holistic audio generation with self-supervised pretraining,” IEEE/ACM Trans. on Audio, Speech, and Language Processing , vol. 32, pp. 2871–2883, 2024

    2024

  25. [33]

    Text-to- audio generation using instruction-tuned llm and latent diffusion model,

    D. Ghosal, N. Majumder, A. Mehrish, and S. Poria, “Text-to- audio generation using instruction-tuned llm and latent diffusion model,” in Proc. ACM MM, 2023

    2023

  26. [34]

    Diffsound: Discrete diffusion model for text-to-sound genera- tion,

    D. Yang, J. Yu, H. Wang, W. Wang, C. Weng, Y . Zou, and D. Yu, “Diffsound: Discrete diffusion model for text-to-sound genera- tion,” IEEE/ACM Trans. on Audio, Speech, and Language Pro- cessing, vol. 31, pp. 1720–1733, 2023

    2023

  27. [35]

    Retrieval-augmented text-to-audio generation,

    Y . Yuan, H. Liu, X. Liu, Q. Huang, M. D. Plumbley, and W. Wang, “Retrieval-augmented text-to-audio generation,” inProc. ICASSP, 2024

    2024

  28. [36]

    Audiogen: Tex- tually guided audio generation,

    F. Kreuk, G. Synnaeve, A. Polyak, U. Singer, A. D ´efossez, J. Copet, D. Parikh, Y . Taigman, and Y . Adi, “Audiogen: Tex- tually guided audio generation,” in Proc. ICLR, 2023

    2023

  29. [37]

    Audiolcm: Text-to-audio generation with latent consistency models,

    H. Liu, R. Huang, Y . Liu, H. Cao, J. Wang, X. Cheng, S. Zheng, and Z. Zhao, “Audiolcm: Text-to-audio generation with latent consistency models,” in Proc. ACM MM, 2024

    2024

  30. [38]

    V oicedit: Dual-condition diffusion transformer for environment- aware speech synthesis,

    J. Jung, J. Ahn, C. Jung, T. D. Nguyen, Y . Jang, and J. S. Chung, “V oicedit: Dual-condition diffusion transformer for environment- aware speech synthesis,” in Proc. ICASSP, 2025

    2025

  31. [39]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in Proc. ICASSP, 2023

    2023

  32. [40]

    Audio generation with multiple conditional diffusion model,

    Z. Guo, J. Mao, R. Tao, L. Yan, K. Ouchi, H. Liu, and X. Wang, “Audio generation with multiple conditional diffusion model,” in Proc. AAAI, 2024

    2024

  33. [41]

    Fifo-diffusion: Generating infinite videos from text without training,

    J. Kim, J. Kang, J. Choi, and B. Han, “Fifo-diffusion: Generating infinite videos from text without training,” in NeurIPS, 2024

    2024

  34. [42]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in Proc. ICLR, 2021

    2021

  35. [43]

    AudioCaps: Generating captions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee, and G. Kim, “AudioCaps: Generating captions for audios in the wild,” in NAACL-HLT, 2019

    2019

  36. [44]

    Audiobox: Unified audio generation with natural language prompts,

    A. Vyas, B. Shi, M. Le, A. Tjandra, Y .-C. Wu, B. Guo, J. Zhang, X. Zhang, R. Adkins, W. Ngan et al., “Audiobox: Unified audio generation with natural language prompts,” arXiv:2312.15821, 2023

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.