{"id":"15206304-f9d6-4590-9aa7-57dd8ced54d6","arxiv_id":"2505.13771","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper introduces delta loss, a score-based training objective for EBM TTS that reduces to a special case of flow matching, and reports one-step inference results competitive with sliced score matching.","lead":"This paper proposes a 'delta loss' for training energy-based text-to-speech models, which asks the model's score at a noisy sample to point directly toward the reference sample. The authors report that models trained this way reach good quality after a single inference step, and that the loss is closely related to flow matching.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Delta loss is trained only on force-aligned Tacotron2 hypotheses, so Eq. 8's one-step optimality for 'any noisy speech sample' is unsupported and inference with unconstrained durations starts off-distribution.","rationale":"The reader's weakest assumption is that Tacotron2 hypotheses constitute a representative and sufficiently varied distribution of noise samples, so that the learned score generalizes to arbitrary inference initializations. My reading identifies the same load-bearing point and sharpens it: the training distribution is even narrower than 'Tacotron2 hypotheses' in general, because training uses force-aligned hypotheses with reference durations, while inference uses predicted durations with different lengths. This is stated in the paper itself, so the concern is grounded in the manuscript rather than imported. The one-step optimality of Eq. 8 is only demonstrated for initializations on the training support, and the experiments do not probe off-support starts. Because the paper's headline contribution is that delta loss provides scores suitable for first-order schemes, this unsupported generalization is a genuine correctness risk. The paper's independent support is limited to empirical results without significance testing, and its acknowledged equivalence to flow matching further weakens the novelty claim. The appropriate disposition is CONDITIONAL: the central claim should be accepted only if the one-step behavior is shown to hold for non-training initializations, or if the claim is restricted to the acoustic-model initialization actually used in the system. I therefore agree with the reader's CONDITIONAL verdict.","tokens_in":8347,"tokens_out":3606,"duration_ms":38707,"concrete_test":"Run the trained delta EBM from the released checkpoint and compare one-step inference from four initialization types: (a) Tacotron2 hypotheses with forced reference durations (train-like), (b) Tacotron2 hypotheses with predicted durations (actual inference condition), (c) the same hypotheses plus isotropic Gaussian noise of increasing variance, and (d) random Gaussian mel-spectrograms matched to reference length. Report MCD and UTMOSv2 at 1, 10, and 100 steps. If the one-step improvement over step 0 vanishes or degrades for (b)-(d), Eq. 8 does not generalize beyond the training initialization and the 'any noisy speech sample' claim fails. If metrics remain comparable across all starts, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that delta loss (Eq. 9) learns scores suitable for first-order inference rests on Eq. 8, Y+ = Y- + S_theta(x, Y-), which Section 3.2 states holds 'using any noisy speech sample as initialisation'. But the loss is optimized only for Y- drawn from a single source: Tacotron2 hypotheses that are force-aligned to reference durations (Section 4.1.2). The same section explicitly says 'No such constraint is enforced in inference.' Thus inference initializations have predicted, not reference, durations and therefore lie off the training manifold, with different sequence lengths and alignment. A score trained at the training Y- points need not point toward the reference at these unseen starting points; the one-step optimality in Eq. 8 is exact only for the training pairs, not for arbitrary initializations. The experiments never vary the initialization distribution (no Gaussian noise, no unaligned hypotheses, no random starts), so they cannot distinguish 'delta loss learns globally useful scores' from 'delta loss memorizes a map from this particular acoustic model's hypotheses to references.' This is especially important because Section 3.3 acknowledges delta loss is closely linked to flow matching: it is essentially a t=0 conditional flow-matching loss whose source distribution is the acoustic model. Good one-step results when starting from that same source are then unsurprising and do not validate the stronger 'any initialization' claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 'delta loss' (Eq. 9) for training energy-based text-to-speech (TTS) models. The loss minimizes the squared error between the model score evaluated at a noisy acoustic-model hypothesis Y- and the displacement (Y+ - Y-) to the reference speech. The authors argue that this makes learned scores suitable for one-step first-order inference (Eq. 8), note a close connection to flow matching (Sec. 3.3), and report experiments on LJSpeech comparing delta loss with sliced score matching (SSM), noise contrastive estimation (NCE), and Tacotron2. Objective metrics (MCD, log f0, SpeechBERTScore, UTMOSv2) and a small MOS study are reported; the conclusion claims comparable objective performance and better subjective performance for delta loss.","tokens_in":8630,"tokens_out":5260,"duration_ms":51387,"significance":"If the claims were fully supported, the paper would provide a simple and efficient alternative to NCE and SSM for EBM training and would clarify the relationship between score-based EBM training and flow matching. The derivation is transparent, the SSM baseline for EBM TTS is a legitimate first application, and the paper is honest about the flow-matching connection. However, the empirical evidence is currently insufficient: the headline comparison against NCE is confounded by different evaluation subsets, no uncertainty quantification or significance tests are provided, the subjective evaluation has only three listeners, and the central 'any initialization' claim is only tested on one source distribution. These issues are load-bearing for the paper's main claims.","major_comments":[{"comment":"The claim that Eq. (8) holds 'using any noisy speech sample as initialisation' is not supported by the training objective. Eq. (9) is optimized only over Y- produced by force-aligned Tacotron2 outputs (Sec. 4.1.2), while inference initializations use predicted durations and therefore fall outside this training manifold. One-step optimality is exact only for the training pairs (if the loss reaches zero); for arbitrary Y- it is a generalization assumption. The experiments never vary the initialization distribution (e.g., Gaussian noise, unaligned hypotheses, random starts), so they cannot distinguish a globally useful score from a map memorized for the acoustic-model distribution. Please either add such experiments or restrict the claim to initializations from the same source distribution.","section":"Section 3.2, Eq. (8)"},{"comment":"The comparison of NCE with score-based EBMs is confounded: NCE is evaluated on the full validation set (1,800 clips) while SSM and Delta are evaluated on a 600-clip subset. The text states that score-based training is 'significantly better' than NCE, but no significance test is reported and the evaluation sets differ. All systems must be scored on the same validation set, with paired significance tests or bootstrap confidence intervals, before a superiority claim over NCE can be made.","section":"Table 3, Sec. 4.1.1"},{"comment":"Objective metrics are reported as point estimates without error bars, confidence intervals, or significance tests. Many differences are very small (e.g., MCD 5.292 vs 5.294 for Delta and SSM at one step in Table 2) and could easily be within noise. Moreover, Table 2 shows that SSM also attains its best objective results after one step, so the experiments do not demonstrate that delta loss improves one-step inference relative to SSM. Please provide utterance-level statistics or bootstrap CIs and a paired significance test.","section":"Tables 1-2"},{"comment":"The subjective evaluation rests on only three listeners, and the MOS difference between SSM-1 (3.90 ± 0.06) and Delta-1 (3.95 ± 0.05) is within the reported confidence intervals. The conclusion's claim of 'better subjective evaluations' is therefore not established by the MOS scores alone; it relies on the raw score-count histogram in Figure 3 without any statistical test. A larger listening panel and a significance test (e.g., Wilcoxon signed-rank or paired t-test on utterance-level scores) are needed.","section":"Table 4, Sec. 4.1.3"},{"comment":"The paper correctly acknowledges the close link between delta loss and flow matching, but Eq. (9) is essentially a t=0 conditional flow-matching loss whose source distribution is the acoustic model. Without a comparison to a flow-matching or rectified-flow baseline trained with the same network architecture and data, the experiments cannot attribute the observed behavior to the proposed 'scores suitable for first-order schemes' mechanism rather than to the already-known flow-matching objective. Please add such a baseline or clearly reposition delta loss as an application/reexamination of flow matching for EBM training.","section":"Section 3.3, Eq. (10)"}],"minor_comments":[{"comment":"The description 'Three listeners, who are all native English speakers, were asked to rate ... on the Amazon Mechanical Turk platform' is unclear, since MTurk is typically associated with many anonymous workers. Please clarify the recruitment procedure and listener compensation.","section":"Section 4.1.3"},{"comment":"The asterisk on NCE* is never explained in the text or in the table caption.","section":"Table 3"},{"comment":"The sentence 'Experiments contrasts these approaches for training EBMs' contains a subject-verb agreement error; it should be 'Experiments contrast...'.","section":"Abstract"},{"comment":"The column header 'log fo' should read 'log f0' for consistency with the rest of the paper.","section":"Table 2"},{"comment":"The caption says 'hypotheses list' but the figure illustrates different score/log-likelihood shapes; please rephrase to match the actual content.","section":"Figure 2"},{"comment":"The score notation S_theta(x, Y(N), N) introduces an iteration index that is not used elsewhere in the same way; this may confuse readers comparing it with S_theta(x, Y) in Eq. (8).","section":"Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like a short conference paper. As a journal submission, the experimental rigor needs substantial strengthening: confounded comparisons, missing significance testing, and a very small listening test are the main blockers. The delta loss is a special case of conditional flow matching, and the authors should be careful not to overstate novelty; the contribution is better framed as an application or reexamination for EBM TTS. I do not see integrity concerns; the issue is evidential and presentational."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this before reading: the 'delta loss' in Eq. (9) is exactly the t=0 endpoint of the conditional flow-matching loss in Eq. (10), with Y0 = Y− and no time interpolation. The authors acknowledge the link in Section 3.3, but they still frame delta loss as a new criterion. That framing is the paper's main weakness, not the math.\n\nWhat the paper does well: it is the first to apply sliced score matching (SSM) to EBM TTS, and it gives a clear, readable comparison of NCE, SSM, delta loss, and their combination. The training setup is described in enough detail to reproduce (architecture from Matcha-TTS, LJSpeech split, 50K steps, optimizer settings). The simple one-step result is genuinely interesting: both SSM and delta loss reach competitive objective metrics after a single gradient step, which contrasts with the hundreds of steps often needed for NCE-trained EBMs. The paper is also honest about the connection to flow matching, even if it does not fully embrace the consequence.\n\nThe soft spots are real but not fatal. First, the novelty claim does not survive contact with Eq. (10); delta loss is a special case, not a new objective. Second, Eq. (8) claims one-step optimality 'using any noisy speech sample as initialisation,' but training only sees force-aligned Tacotron2 hypotheses. Inference starts from hypotheses with predicted durations, so the initialization distribution shifts. The authors never test with other initializations, so this central generalization claim is unsupported. Third, Table 3 compares NCE on the full validation set with score-based models on a 600-clip subset, and the objective metrics have no error bars or significance tests. Fourth, the MOS test uses only three listeners, and the 1-step delta vs. SSM difference (3.95 vs. 3.90) is within overlapping error bars; the 'better' claim is weak.\n\nFor whom is this paper useful? People working on EBM or score-based generative models for speech, especially those tired of NCE's negative-sampling pain. It is a good engineering recipe and a decent case study of why evaluation details matter. It is not a foundational contribution.\n\nMy recommendation: send it to peer review, not because the contribution is new, but because the question is relevant and the paper is clearly written. Expect the reviewers to push for a reframed contribution, a clean comparison on the same validation set, more listeners, and at least one experiment that varies the initialization distribution.","headline":"Delta loss is a t=0 flow-matching loss in disguise; the paper is honest about the link but overclaims novelty, and the empirical comparison is confounded, though the study is a useful and readable contribution to EBM TTS training.","tokens_in":9163,"tokens_out":3361,"would_cite":false,"duration_ms":33582,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simple delta loss trains energy-based TTS score functions to point directly from a noisy hypothesis to the reference, yielding one-step-friendly inference and better subjective ratings than sliced score matching.","keywords":["energy-based models","text-to-speech","delta loss","sliced score matching","noise contrastive estimation","flow matching","score function training","first-order inference"],"falsifier":"Use a trained delta-loss EBM and run one-step inference from initializations drawn from a different distribution than the acoustic-model hypotheses used in training, for example references corrupted by Gaussian noise; if the one-step output no longer moves toward the reference (measured by mel-cepstral distortion or a perceptual metric), the claimed one-step suitability does not generalize beyond the training initialization distribution.","tokens_in":8164,"feed_emoji":"🎙️","tokens_out":11131,"duration_ms":92555,"temperature":0.7,"pith_summary":"The paper introduces a new training objective, the delta loss, for energy-based models of text-to-speech, and argues that it learns score functions suited to the first-order inference these models use. It points out that noise contrastive estimation and sliced score matching spend their effort on matching energies or gradients without considering the shape of the log-likelihood, so a perfectly learned score may still fail to move a noisy sample toward the reference in a few steps. The delta loss instead trains the score at a noisy hypothesis to equal the displacement vector from that hypothesis to the reference speech, making the ideal update direction explicit. On a single-speaker benchmark, the resulting model is competitive with sliced-score-matching training on objective measures and receives better subjective naturalness ratings, with fewer low scores and more high scores.","feed_headline":"Delta loss gives TTS energy models one-step-friendly scores","feed_subtitle":"New objective matches sliced score matching objectively and wins listening tests with fewer bad outputs.","key_machinery":"The central object is the delta loss, a regression objective that trains the model score at a noisy sample $Y^-$ to return the displacement vector $Y^+ - Y^-$ that points from that sample to the reference $Y^+$. Its purpose is to make the score realize the one-step identity $Y^+ = Y^- + S_\\theta(x,Y^-)$ for the training pairs, so the first-order update used in inference is aligned with the actual path to the target. The paper shows this target is the same velocity field used by flow matching along the linear path $Y_t = tY^+ + (1-t)Y^-$ evaluated at $t = 0$, which is why the loss is both simple and geometrically meaningful. The paper contrasts this with NCE and SSM, which learn energies or scores without regard for whether the resulting log-likelihood supports few-step gradient ascent.","core_discovery":"The central claim is that an energy-based TTS score function should satisfy the one-step identity $Y^+ = Y^- + S_\\theta(x,Y^-)$ at the noisy starting points $Y^-$ supplied by the acoustic model, and that the delta loss $L^\\Delta_\\theta = \\frac{1}{2}\\lVert S_\\theta(x,Y^-) - (Y^+ - Y^-)\\rVert_2^2$ is the training objective that enforces it. Because a score that satisfies this identity points directly from any training hypothesis toward its reference, the paper argues that the resulting log-likelihood landscape is far friendlier to the first-order gradient updates used in EBM and diffusion inference than landscapes produced by NCE or SSM. The paper also establishes a mathematical link to flow matching, noting that the delta-loss target is the straight-line velocity of the interpolated path between the noisy sample and the reference, evaluated at the starting point. Experimental results show the delta-trained model performs comparably to sliced score matching on objective metrics while receiving better listening-test ratings.","pith_inferences":["Beyond the paper, one could train delta loss at several points along the interpolation between $Y^-$ and $Y^+$, not just at $Y^-$; that would recover full flow matching and might make the score robust to initializations that differ from the acoustic model's training hypotheses.","Beyond the paper, the paper's comparison uses a fixed inference learning rate and a fixed number of inference steps, so sweeping the step size would test whether delta-trained scores really are one-step-optimal or merely improve over SSM at the chosen settings.","Beyond the paper, the one-step identity suggests delta loss could be used to initialize or fine-tune diffusion or flow-based TTS models, since it enforces the straight-line velocity property those models aim to approximate."],"forward_implications":["If the delta-loss claim holds, energy-based TTS training no longer needs carefully engineered negative samples or expensive Hessian-style computations; a simple squared-distance regression on displacement vectors suffices.","The objective results suggest the trained score models reach their best scores after one inference step from the acoustic model's output, so the hundreds of MCMC iterations traditionally used for EBM inference may be unnecessary.","Because delta loss is closely linked to flow matching, advances in flow-matching theory or training practice can be carried over to score-based EBM training, and vice versa.","The subjective results indicate that the delta-trained model reduces the frequency of severely unnatural outputs (scores 1 through 3) and increases the frequency of high ratings (4 and 5), so the practical benefit is concentrated in avoiding bad synthesis rather than in improving average quality."],"supporting_citations":[{"why":"It gives the NCE-trained EBM baseline and the energy-based formulation of speech synthesis that this paper extends.","marker":"[3]"},{"why":"It supplies the sliced score matching loss in Eq. (7) that delta loss is compared against.","marker":"[10]"},{"why":"It presents the flow-based objective in Eq. (10) that the paper shows is closely linked to delta loss.","marker":"[16]"},{"why":"It defines noise contrastive estimation, the training principle behind the NCE baseline in Eq. (4).","marker":"[20]"},{"why":"It provides the score matching unbiased estimator underlying Eq. (6), the starting point for SSM.","marker":"[27]"},{"why":"It describes the pretrained acoustic model used to generate the spectral-feature hypotheses that serve as noisy samples and as inference initializations.","marker":"[21]"},{"why":"It supplies another flow matching formulation supporting the claimed connection between delta loss and flow matching.","marker":"[25]"}],"fun_headline_variants":["Delta objective trains TTS energy models for one-step inference","Score training with delta loss makes TTS EBM one-step ready","Flow-matching-inspired loss improves energy-based TTS scores","One-step score identity lifts listening quality in TTS EBMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the hypothesis-reference pairs seen during training are representative of the starting points used at inference, so a score calibrated to point from the acoustic model's outputs to references also points correctly from other initializations.","fun_headline_variants_meta":{"raw":{"variants":["Delta objective trains TTS energy models for one-step inference","Score training with delta loss makes TTS EBM one-step ready","Flow-matching-inspired loss improves energy-based TTS scores","One-step score identity lifts listening quality in TTS EBMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0005,"raw_usage":{"total_tokens":2428,"prompt_tokens":909,"completion_tokens":1519,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":1449}},"tokens_in":525,"tokens_out":1519,"duration_ms":10639,"temperature":1.0,"reasoning_tokens":1449,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:10:32.429246+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use a trained delta-loss EBM and run one-step inference from initializations drawn from a different distribution than the acoustic-model hypotheses used in training, for example references corrupted by Gaussian noise; if the one-step output no longer moves toward the reference (measured by mel-cepstral distortion or a perceptual metric), the claimed one-step suitability does not generalize beyond the training initialization distribution.","supporting_citations":[{"cited_title":"Section 2 discusses EBMs and related work, including training methods and con- nections between EBMs and diffusion models","cited_arxiv_id":null,"evidence_quote":"It gives the NCE-trained EBM baseline and the energy-based formulation of speech synthesis that this paper extends."},{"cited_title":"Learning deep kernels for exponential family densities,","cited_arxiv_id":null,"evidence_quote":"It supplies the sliced score matching loss in Eq. (7) that delta loss is compared against."},{"cited_title":"Estimation of non-normalized sta- tistical models by score matching","cited_arxiv_id":null,"evidence_quote":"It presents the flow-based objective in Eq. (10) that the paper shows is closely linked to delta loss."},{"cited_title":"Imita- tion with neural density models,","cited_arxiv_id":null,"evidence_quote":"It defines noise contrastive estimation, the training principle behind the NCE baseline in Eq. (4)."}],"review_version":1}