{"id":"cfbc58f3-b316-47c4-8569-b3ca8784ca86","arxiv_id":"2505.07615","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Inference energy of seven text-to-audio diffusion models grows linearly with denoising steps, while quality saturates, so the best quality-per-energy settings use 10 to 50 steps.","lead":"This paper measures how much electricity seven text-to-audio AI models use when generating sounds, and tests how changing the number of diffusion steps or the batch size affects energy use. It maps the trade-off between audio quality and energy cost, showing that low step counts deliver most of the quality at a fraction of the energy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pareto step-count conclusion rests on single unreplicated FAD/CLAP measurements; 150/200-step configurations may be dominated only by quality-metric noise.","rationale":"The paper is a useful, reproducible benchmark: energy is averaged over five runs, code is provided, and the linear step-scaling and batching results are plausible. The weakest point is the Pareto conclusion, which is the main actionable message. For that conclusion to hold, the quality scores used to build the frontier must be stable enough that frontier membership is not an artifact of a single draw of 300 generated clips. The paper reports energy standard deviations but no quality-metric uncertainty, so the claim that 150/200 steps never appear on any frontier is more brittle than the paper acknowledges. The reader's weakest assumption focused on CodeCarbon GPU-only measurement; I agree that is also a limitation, but a uniform energy scaling error would preserve the shape of the frontier and the step-count conclusion. Quality-metric noise, in contrast, can directly reshuffle the Pareto frontier and change which step counts are recommended. I also noticed a possible unit inconsistency in Fig. 1 (values appear to be Wh rather than kWh when compared with the Table 1 inference speeds and A40 power), but since a uniform factor of 1000 would not affect linearity, batching trends, or Pareto step membership, I did not make it the primary attack. Overall, I would keep the reader's CONDITIONAL verdict: the empirical claims are likely directionally correct, but the Pareto step-count recommendation needs replication or resampling-based intervals before it can be stated as strongly as it is.","tokens_in":9123,"tokens_out":11938,"duration_ms":130704,"concrete_test":"Using the released code and prompts, regenerate the 50/100/150/200-step configurations for the models near the frontier (e.g., AudioLDM, Tango, Stable Audio Open) on AudioCaps and Clotho. Compute FAD and CLAP on 5 independent 300-sample subsets (or bootstrap resamples of the generated clips) and recompute the Pareto frontier for each replication. If any 150- or 200-step configuration appears on any replication's frontier, the Section 4 claim that these step counts are never Pareto-optimal is not robust to quality-metric variability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing claim is the Pareto analysis in Section 4: configurations with 150 or 200 diffusion steps never appear on the frontier, while 10–50 steps typically do. For each (model, step-count, dataset) cell, CLAP score and FAD are computed once on a single 300-prompt subset, with no reported confidence intervals, bootstrap replicates, or seeds for the quality metrics. Energy is averaged over 5 repeats, but quality is not. FAD on 300 clips and CLAP embedding similarity have nontrivial sampling variance, and the frontier in Fig. 2 is drawn from these point estimates; a configuration can be classified as dominated by a difference of a few hundredths in FAD or CLAP. If the 300-sample subset is resampled or regenerated, frontier membership can change, so the blanket statement that 150/200 steps are never Pareto-optimal may not be supported. The sanity check in Section 4 addresses subset-vs-full-dataset trends using only AudioLDM and Stable Audio Open; it does not quantify FAD/CLAP variance for frontier membership across all models.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a comparative empirical study of the energy consumption of seven text-to-audio (TTA) diffusion models at inference time. Using CodeCarbon on a single NVIDIA A40 GPU, the authors measure GPU energy as a function of the number of denoising steps (10–200) and batch size (1–10), and they combine these energy measurements with CLAP and FAD quality metrics on 300-sample subsets of AudioCaps and Clotho to identify Pareto-optimal configurations. The main claims are: (i) inference energy scales approximately linearly with the number of denoising steps across all models; (ii) batching reduces per-sample energy with diminishing returns beyond batch size 5; and (iii) Pareto-optimal quality/energy trade-offs are achieved with low step counts, typically 10–50, while 150- and 200-step configurations never appear on the frontier. The code is publicly released.","tokens_in":9290,"tokens_out":5776,"duration_ms":50286,"significance":"If the findings hold, this is a useful first step toward energy-aware deployment of TTA models. The study covers seven current models, uses repeated energy measurements, and provides public code, which aids reproducibility. The Pareto analysis is a valuable practical contribution because it offers concrete guidance on step-count choices. However, the central Pareto conclusion is currently supported only by point estimates of CLAP/FAD without uncertainty quantification, and the energy measurements are specific to one GPU and one measurement tool. These limitations materially affect the strength of the claims as stated.","major_comments":[{"comment":"The statement that \"configurations with 150 or 200 diffusion steps never appear on the Pareto frontier\" (last sentence of Section 4) is not supported by the reported data. CLAP and FAD are each computed once per configuration on a 300-sample subset, with no confidence intervals, bootstrap replicates, or seeds reported. The Pareto frontier in Fig. 2 is therefore drawn from point estimates; small FAD/CLAP differences (on the order of 0.01–0.02) between adjacent step counts can change frontier membership under resampling. The sanity check described in Section 4 only compares subset vs full-dataset trends for AudioLDM and Stable Audio Open; it does not quantify the variance of the quality metrics or check frontier membership for all models. Please either provide uncertainty estimates (e.g., bootstrap over the 300 samples) and re-derive the frontier, or soften the claim to \"in our measurements, no 150/200-step configuration appeared on the frontier.\"","section":"4. Pareto Frontier Analysis"},{"comment":"The text states that \"Each experiment was repeated 5 times to compute the average and standard deviation of energy consumption,\" but neither Fig. 1(a), Fig. 1(b), nor Fig. 2 reports any standard deviation, error bars, or numerical values. Without these, the reader cannot assess whether the \"clear linear relationship\" in Section 3.1 and the \"diminishing returns\" in Section 3.2 are statistically distinguishable from noise. Please add error bars or confidence intervals, or provide a table of mean±std for all reported configurations.","section":"3. Experiments and Results"},{"comment":"All conclusions rest on CodeCarbon GPU-only energy readings from a single NVIDIA A40, excluding CPU, RAM, model loading, and storage, and no validation of the tracker against hardware power measurements is reported. Since the Pareto frontier in Fig. 2 is determined by absolute kWh values, the identified \"Pareto-optimal\" step counts may change on other hardware (e.g., consumer GPUs, integrated GPUs, CPU-only inference). The paper should state this as an explicit limitation and, ideally, include a sensitivity analysis or a discussion of how relative model rankings might differ.","section":"2.2 Energy Consumption evaluation"}],"minor_comments":[{"comment":"The y-axis label of Fig. 1 appears garbled (\"/uni00000014/uni00000048/uni00000016\" and \"/uni00000014/uni00000048/uni00000015\"), apparently a PDF rendering issue; please ensure the label reads \"Energy at inference [kWh]\" on both panels.","section":"Figures 1(a) and 1(b)"},{"comment":"The sentence \"we generated a separate 10-second audio sample for each inference step using the same prompt\" is ambiguous: does this mean one sample per step count per repeat, or a single sample per step? Please clarify how the 5 repeats were allocated.","section":"3.1 Does energy consumption scale with inference steps?"},{"comment":"The claim that \"only related work on multimodal image generation has begun to address this issue\" is inconsistent with the earlier citation of Douwes et al. [9] on the energy of neural audio synthesis models; please narrow the claim to text-to-audio diffusion models or explicitly acknowledge [9] as related audio work.","section":"1. Introduction"},{"comment":"Trimming Clotho audio (originally 15–30 s) to 10 s may affect the FAD distribution and the resulting frontier; please discuss or justify this preprocessing choice.","section":"4. Pareto Frontier Analysis"},{"comment":"Table 1 reports inference speed without specifying whether this is wall-clock time from a single run or an average; please clarify the measurement procedure.","section":"2.1 TTA models selected"}],"recommendation":"major_revision","confidential_remarks":"This is a solid empirical benchmark with public code and repeated energy measurements, and the linear step/energy and batching trends are likely reproducible and valuable to the TTA community. The main weakness is that the Pareto-optimality claim is currently stronger than the evidence: quality metrics are unreplicated point estimates, and the energy measurements are hardware- and tool-specific. The authors should either add uncertainty quantification for CLAP/FAD or soften the claim; with that change, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked about Passoni et al. on energy consumption of text-to-audio diffusion models. My short take: this deserves a serious referee, with conditions on the Pareto analysis. The genuinely new artifact is a public, reproducible measurement setup plus the first systematic inference-energy figures for seven current TTA models, including Tango2 and Stable Audio Open. The methodology is inherited from Douwes et al., but applying it to current models is a legitimate contribution, not a flaw. The step-count and batch-size experiments are repeated five times, the code is available, and the core claims there are solid: energy scales roughly linearly with denoising steps, and batching gives diminishing returns after batch size five. Those findings are useful for practitioners and should survive scrutiny.\n\nThe soft spots are real but localized. As the reader notes, all energy numbers are CodeCarbon GPU readings on a single NVIDIA A40, with no calibration against a physical meter and no second GPU. For the relative scaling claims that is minor; for absolute kWh values and for exact Pareto-frontier positions it matters more. The bigger issue, which the stress-test note correctly identifies, is the Pareto analysis in Section 4. FAD and CLAP are computed once per model/step/dataset on a 300-sample subset, with no confidence intervals, bootstrap, or seeds. FAD on 300 clips is noisy, and the frontier in Figure 2 is drawn from point estimates, so the blanket statement that 150- or 200-step configurations are never Pareto-optimal is not fully supported. It may be true in broad strokes, but the current evidence only supports saying those configurations were not on the measured frontier. The sanity check with full datasets uses only two models and does not quantify quality-metric variance. The figures also omit the standard deviations they say they computed, which is an easy fix.\n\nThe citation pattern is honest: the self-citations to Douwes and Serizel are directly relevant, and there is no inflation. The paper overclaims slightly by calling itself a first analysis, but the substance is there. This is a benchmark paper, not a theoretical one, and it is a useful one for anyone choosing a TTA model or thinking about green audio AI. I would send it to peer review and ask for error bars or intervals on quality metrics, the exact prompts/seeds, and a more careful wording of the Pareto conclusions. I would also bring it to a reading group; the measurement discussion is worth having in the open.","headline":"Useful first systematic energy benchmark for seven current text-to-audio diffusion models; the step-scaling and batch findings hold up, but the Pareto claim that 150/200 steps are never optimal rests on unreplicated quality metrics and needs variance bars before it becomes recommendation-grade.","tokens_in":9828,"tokens_out":1556,"would_cite":true,"duration_ms":18163,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Inference energy of text-to-audio diffusion grows linearly with denoising steps, and quality plateaus by 50 steps.","keywords":["text-to-audio generation","diffusion models","inference energy","denoising steps","Pareto frontier","energy efficiency","CLAP score","Fréchet Audio Distance"],"falsifier":"Repeat the step-count and Pareto experiments on a second GPU and with total system power (CPU plus GPU plus RAM) instead of GPU-only readings; if 150- or 200-step configurations ever match the FAD/CLAP-per-kWh of 50-step settings, or if the energy-per-step slope is no longer constant for any model, the central claims fail.","tokens_in":8927,"feed_emoji":"⚡","tokens_out":7400,"duration_ms":65325,"temperature":0.7,"pith_summary":"This paper measures how much electricity seven text-to-audio diffusion models consume when generating sound, and asks whether the energy buys better audio. It reports that inference energy grows almost perfectly linearly with the number of denoising steps for every model tested, that generating several clips in one batch cuts per-clip energy but with sharply diminishing returns beyond a batch of about five, and that the configurations with the best quality-per-watt trade-offs all use between 10 and 50 steps. Settings of 150 or 200 steps never land on that trade-off frontier, so the extra computation fails to translate into measurably better audio. If these measurements hold, users can cut energy use substantially by choosing low-step settings and a modest batch size, and model choice matters: the cheapest model consistently costs a fraction of the most expensive ones.","feed_headline":"Audio diffusion models waste energy past 50 steps","feed_subtitle":"Seven text-to-audio models show energy rising linearly with denoising steps while audio quality plateaus near 50.","key_machinery":"The load-bearing mechanism is the diffusion inference loop itself, whose cost is controlled by the number of denoising steps: each additional step adds roughly constant GPU work, producing the observed linear energy scaling. Energy is counted per generated clip with a GPU-tracking toolkit, while quality is scored by CLAP (semantic alignment between prompt and audio) and Fréchet Audio Distance (distributional similarity to real audio). The Pareto frontier over model, step, and batch configurations is what converts raw energy numbers into an actionable recommendation about which settings dominate others.","core_discovery":"The paper's central claim is that the inference energy of text-to-audio diffusion models is a predictable, tunable quantity: across AudioLDM, AudioLDM2, Make-an-Audio, Make-an-Audio-2, Stable Audio Open, Tango, and Tango2, the energy measured per generated ten-second clip rises linearly with the number of denoising steps, with each model having its own slope. Batching multiple generations lowers energy per sample, with the largest gain from a batch size of 2 and little benefit past 5. Combining these energy readings with two quality metrics, the authors find that Pareto-optimal configurations—those where no further quality can be gained without more energy, or vice versa—are concentrated at 10 to 50 inference steps; 150- and 200-step configurations never appear on the frontier. The paper presents this as the first inference-focused energy analysis of text-to-audio diffusion models and as evidence that step count, batch size, and model architecture jointly determine the environmental cost of generative audio.","pith_inferences":["If GPU power is the dominant term, the linear step-energy law suggests that energy budgets for a generation job could be predicted from step count alone, enabling quota-based scheduling; the paper does not test this extension.","A natural next experiment is dynamic step scheduling: if quality plateaus by 50 steps, adaptive early stopping during denoising could be benchmarked against the fixed-step frontier.","The GPU-only measurement may not transfer to CPU or heterogeneous inference; replicating the Pareto analysis with total system power would show whether the low-step conclusion is hardware-independent.","The same Pareto methodology could be applied to autoregressive text-to-audio models, which the paper explicitly leaves for future work."],"forward_implications":["Users can cut inference energy by roughly the step-count ratio: halving steps from 100 to 50 nearly halves the measured energy for every model.","A batch size of 2 to 5 is the efficiency sweet spot; larger batches add little or no per-sample savings, so generating many clips one by one is the most wasteful strategy.","Defaulting to 150 or 200 denoising steps is hard to justify on quality grounds: those settings never achieved a Pareto-optimal FAD or CLAP score in this study.","Energy-sensitive deployments should prefer the cheapest model architecture, while quality-driven applications can still pick a heavier model as long as steps stay in the 10 to 50 range.","The ranking of models on the frontier depends on the dataset, so application domain should guide model choice as much as raw energy cost."],"supporting_citations":[{"why":"the energy-tracking toolkit that produced every kWh reading in the two experiments and the Pareto analysis","marker":"[30]"},{"why":"the prior study whose Pareto-frontier approach and energy-evaluation setup this paper adapts to text-to-audio models","marker":"[9]"},{"why":"defines CLAP and supplies the embedding space used for the CLAP quality score","marker":"[19]"},{"why":"introduces Fréchet Audio Distance, the second quality metric used in the frontier analysis","marker":"[32]"},{"why":"provides the AudioCaps test subset whose prompts generated audio for the frontier evaluation","marker":"[33]"},{"why":"provides the Clotho evaluation subset used alongside AudioCaps","marker":"[34]"},{"why":"supplies the AudioLDM model architecture and the observation that quality gains from extra steps diminish","marker":"[18]"},{"why":"supplies the Stable Audio Open model and its training-data description, used to explain Clotho performance","marker":"[25]"},{"why":"grounds the decision to report only GPU energy on the assumption that GPU dominates consumption","marker":"[5]"},{"why":"supplies the Tango model, one of the two most energy-intensive systems in the comparison","marker":"[23]"}],"fun_headline_variants":["Audio diffusion energy scales linearly; quality plateaus near 50 steps","Text-to-audio models: Pareto-optimal at 10-50 steps, not 200","Batching cuts energy per clip, but gains fade past batch 5","First inference-energy study of text-to-audio diffusion models","Step count, batch size, and model shape audio energy cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The measurements assume that GPU-only power readings, taken on one NVIDIA A40 while excluding model loading, CPU, RAM, and storage, faithfully represent the inference energy that determines real-world deployment cost.","fun_headline_variants_meta":{"raw":{"variants":["Audio diffusion energy scales linearly; quality plateaus near 50 steps","Text-to-audio models: Pareto-optimal at 10-50 steps, not 200","Batching cuts energy per clip, but gains fade past batch 5","First inference-energy study of text-to-audio diffusion models","Step count, batch size, and model shape audio energy cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000233,"raw_usage":{"total_tokens":1448,"prompt_tokens":852,"completion_tokens":596,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":501}},"tokens_in":468,"tokens_out":596,"duration_ms":5966,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:11:25.857773+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the step-count and Pareto experiments on a second GPU and with total system power (CPU plus GPU plus RAM) instead of GPU-only readings; if 150- or 200-step configurations ever match the FAD/CLAP-per-kWh of 50-step settings, or if the energy-per-step slope is no longer constant for any model, the central claims fail.","supporting_citations":[{"cited_title":"Is quality enoughƒ integrating energy consumption in a large-scale evaluation of neural audio synthesis models,","cited_arxiv_id":null,"evidence_quote":"the prior study whose Pareto-frontier approach and energy-evaluation setup this paper adapts to text-to-audio models"},{"cited_title":"Clap learning audio concepts from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"defines CLAP and supplies the embedding space used for the CLAP quality score"},{"cited_title":"Audiocaps: Generating captions for audios in the wild,","cited_arxiv_id":null,"evidence_quote":"provides the AudioCaps test subset whose prompts generated audio for the frontier evaluation"},{"cited_title":"Clotho: An audio captioning dataset,","cited_arxiv_id":null,"evidence_quote":"provides the Clotho evaluation subset used alongside AudioCaps"},{"cited_title":"AudioLDM: Text-to-audio generation with latent diffusion models,","cited_arxiv_id":null,"evidence_quote":"supplies the AudioLDM model architecture and the observation that quality gains from extra steps diminish"},{"cited_title":"Stable audio open,","cited_arxiv_id":null,"evidence_quote":"supplies the Stable Audio Open model and its training-data description, used to explain Clotho performance"},{"cited_title":"Energy consumption trends in sound event detection systems,","cited_arxiv_id":null,"evidence_quote":"grounds the decision to report only GPU energy on the assumption that GPU dominates consumption"}],"review_version":1}