{"id":"90076e17-d5ee-46ee-9561-ae8a89ea749c","arxiv_id":"2505.12800","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"OZSpeech is a one-step zero-shot TTS system that starts from learned content and mean-style codes and uses flow matching to refine them, achieving very low word error rates with a small model.","lead":"This paper presents OZSpeech, a zero-shot text-to-speech model that generates speech in one sampling step by starting from a learned set of speech codes instead of random noise. It reports much lower word error rates than existing models, with smaller size and faster inference, at some cost in naturalness and speaker similarity.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (3)'s reformulation replaces the tied pair (xt, t) with an unobserved (xpr, tau); since no training-time distribution or inference-time value for tau is given, the one-step flow-matching objective is not well-defined as written.","rationale":"The empirical tables, especially the consistent WER across prompt lengths and noise levels, are presented in good faith and the Limitations section honestly reports naturalness degradation and duration-predictor inaccuracies. Those are not the issue. The load-bearing problem is internal to the method: Eq. (3) is presented as a reformulation of the OT-CFM loss, but the substitution from Eq. (2) to Eq. (3) is not a valid change of variables. In Eq. (2), xt and t are linked through xt = t*x1 + (1-t)*x0, which is what makes the target (x1-xt)/(1-t) equal to x1-x0. Replacing xt with the learned xpr and t with an undefined tau breaks that link: there is no stated relation between xpr and tau, no sampling rule for tau during training, and no value for tau during the claimed one-step sampling. The paper's own Appendix B repeats tau in Eq. (13) and in the Anchor Loss but never defines it. This matters because the central novelty is a one-step flow-matching formulation; if the objective is not well-defined, the theoretical basis for the one-step sampler and for the reported 2.7-6.5x speed advantage cannot be assessed. A fixed tau=0 would make the method an empirically plausible residual predictor, but then the 'learned optimal time' claim is unsupported. A sampled tau without a disclosed schedule creates a training-inference mismatch. Thus the paper's current wording is insufficient for reproduction and for validating the central claim. This is the same weak point the reader identified, and the appropriate remedy is a conditional acceptance with a mandatory clarification and/or code release. The missing NaturalSpeech 3 baseline remains a secondary concern about the strength of the empirical comparison, but it does not displace the tau issue as the most load-bearing concern.","tokens_in":20869,"tokens_out":6671,"duration_ms":74735,"concrete_test":"Ask the authors to release the training and inference code, or to state explicitly: (i) the distribution from which tau is drawn in Eq. (3) and Eq. (13), and (ii) the tau value used for one-step sampling at inference. Then run the following ablation: train the vector-field estimator with tau replaced by a constant 0 at both training and inference, keeping everything else fixed. If WER, UTMOS, and SIM-O are unchanged within measurement noise, the paper's learned-time framing is not load-bearing and the current text is a presentation gap. If metrics degrade or training diverges, the paper must disclose the tau schedule and inference value; until then Eq. (3) is not reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Eq. (11) the interpolation parameter t and the state xt are tied: xt = t*x1 + (1-t)*x0, so Eq. (2)'s target (x1 - xt)/(1-t) equals x1 - x0. Section 3.3 then sets xpr = xt and renames t as tau, writing L_CFM = E || v_theta(xpr, tau) - (x1 - xpr)/(1 - tau) ||^2. That substitution is only valid if xpr lies on the linear OT path of some x0 for the same tau used in the denominator, but the paper never says how xpr relates to such a path, how tau is sampled during training, or what tau is used at inference in x1_hat = xpr + (1 - tau) * F^{-1}(v_theta(...)). The sentence about 'implicitly learning the optimal t for each prior' does not supply a mechanism: the network is conditioned on tau, so tau cannot be learned unless it is also an output or estimated by a separate module. If in practice tau is a fixed constant, the 'time variable' is a constant conditioning input and Eq. (3) reduces to a residual prediction objective; if tau is sampled from an undisclosed distribution, the training/inference mismatch is unquantified. Either way the central one-step flow-matching derivation and the reproducibility of the method rest on an unstated premise, and no code is provided to resolve it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OZSpeech, a zero-shot TTS system that combines a learned prior code generator with a one-step optimal-transport conditional flow matching (OT-CFM) module. Speech is decomposed with FACodec into six quantizer sequences (prosody, content, acoustic details) plus a timbre vector; the Prior Codes Generator produces a phoneme-conditioned starting point, and a transformer-based Vector Field Estimator predicts the residual velocity toward the target codes in a single sampling step. The authors report experiments on LibriTTS/LibriSpeech comparing with F5-TTS, VoiceCraft, NaturalSpeech 2, and VALL-E, plus ablations on prompt selection and noise robustness. The main claimed advantages are a multi-fold WER reduction, stable WER across prompt lengths and noise levels, a 2.7-6.5x faster inference speed, and a 29%-71% model-size reduction relative to baselines.","tokens_in":21186,"tokens_out":2227,"duration_ms":24656,"significance":"If the central reformulation is valid, the paper makes a useful practical contribution: it demonstrates that a learned prior derived from a codec can serve as the initialization for a one-step flow-matching TTS, with a favorable efficiency/size trade-off and a noise-robustness analysis that few prior TTS papers provide. The empirical results, especially the consistently low WER and the controlled noise experiment, are potentially impactful for deployment-oriented zero-shot TTS. However, the significance is currently bounded by two gaps: the one-step flow-matching derivation rests on an underspecified quantity tau, and the evaluation omits the strongest published baseline (NaturalSpeech 3), which the paper itself discusses as a canonical factorized-codec system.","major_comments":[{"comment":"The reformulation of the OT-CFM loss is not well-defined as written. The original loss (Eq. (2)) couples the state xt and the interpolation parameter t through the linear path xt = t*x1 + (1-t)*x0; replacing (xt, t) with (xpr, tau) is only valid if xpr lies on the corresponding linear path for the same tau used in the denominator. The paper never states how xpr relates to such a path, how tau is sampled during training, or what value of tau is used at inference in the update x1_hat = xpr + (1-tau) * F^{-1}(v_theta(...)). The statement that the model \"implicitly learns the optimal t for each prior\" is not a mechanism, since tau is a conditioning input to the network and is not produced as a network output or estimated by an auxiliary module. The training objective and the one-step sampling claim therefore rest on an unstated premise; please specify the training distribution of tau and the inference-time choice, or reformulate the loss so that the target does not depend on an unobserved interpolation parameter.","section":"Section 3.3, Eq. (3) and Eq. (13)"},{"comment":"The baseline set omits NaturalSpeech 3 (Ju et al., 2024b), which is cited in the paper as the source of FACodec and which the text identifies as a state-of-the-art zero-shot TTS system built on the same factorized codec representation. Without this comparison, the central claim of \"multi-fold WER improvement\" and the implied SOTA claim are not supported. Please add the published NaturalSpeech 3 numbers (or a reproduction under the same evaluation protocol) to Tables 1 and 2, or explicitly justify its exclusion.","section":"Section 4.1 and Table 1"},{"comment":"No variance or confidence-interval information is reported for any metric, which matters especially for the WER claims. The paper reports identical WER values (0.05) across 1s/3s/5s prompts and across all noise levels (0.05 or 0.06), which is surprising for an ASR-based metric on a test set of 2.2 hours; such stability could arise from a degenerate ASR output (e.g., empty hypotheses or a systematic mis-transcription) rather than robust synthesis. Please report per-utterance score distributions (or at least standard errors and the number of test utterances), and include a worked example or failure analysis that confirms the low WER is not a scoring artifact. The boxplots in Figure 2 are a step in this direction, but they do not report N or the fraction of utterances contributing to the near-zero WER box.","section":"Tables 1, 3, 4 and Section F"},{"comment":"The joint training of the Prior Codes Generator and the Vector Field Estimator is described, but the interaction between the prior and the flow objective is not specified precisely. In particular, Eq. (3) treats xpr as a random variable, yet Appendix C states that the two components are \"jointly trained\" with a total loss Ltotal = Lprior + Ldur + LCFM + Lanchor. If xpr is generated by f_psi during training and gradients flow back through the prior into LCFM, the distribution of xpr is being shaped by the same objective that later depends on tau; if instead xpr is detached, the loss reduces to a regression toward x1 with a bias determined by the prior. The paper does not state which of these regimes is used; please clarify the gradient flow (stop-gradient or not) and explain how the claimed \"learned prior\" behavior is achieved without a separate distillation stage beyond what is already in Lanchor.","section":"Section 3.3, 'One-Step Optimal Transport Flow Matching Formulation' and Appendix C"}],"minor_comments":[{"comment":"The table footnote lists '[star] and [clubs] mean results inferred from official and ufficial checkpoints' — 'ufficial' is a typo and should be 'official' or 'unofficial', depending on intent.","section":"Section 4.1, abbreviations"},{"comment":"The F0 RMSE value is listed as '102,68' (with a comma); this should be '102.68' for consistency with the decimal-point convention used elsewhere in the table.","section":"Table 4, SNR=0dB row for VALL-E"},{"comment":"The paper presents two versions of the conditional flow matching loss (Eq. (12) with the target x1 - x0, and Eq. (2) with the target (x1 - xt)/(1-t)). Please verify the equivalence and consistency of the notation across these equations, since Eq. (2) as written appears to omit the factor that maps the conditional velocity to the unconditional target.","section":"Appendix A.2, Eq. (12) and Eq. (2)"},{"comment":"The sentence describing the retrained F5-TTS reports a WER exceeding 0.95 and excludes the checkpoint from Table 1. This is an important negative result; please provide more detail in an appendix (training steps, data, and the specific failure mode), because as written it is a single unreproduced observation that could be affected by hyperparameter choices.","section":"Section 4.2, retrained F5-TTS discussion"},{"comment":"The claim that the Small model 'outperforms the Base model by 13.6% in F0 RMSE for 1s' needs a unit or direction reference: the table reports lower RMSE as better, so please make explicit that relative improvement means a decrease in RMSE, and similarly for the other directional statements.","section":"Appendix E, Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper's claimed contribution is a one-step flow-matching TTS with a learned prior. The central methodological step (the tau reformulation) is underspecified, and the evaluation omits the most relevant baseline (NaturalSpeech 3). Both issues are fixable with additional exposition and experiments, so I do not see grounds for rejection. However, the WER stability across prompt lengths and noise levels is unusual enough that I would want the authors to report confidence intervals and inspect the ASR outputs for degenerate behavior before the paper is accepted. The paper also cites NaturalSpeech 3 multiple times in the related work, which makes its absence from the comparison tables particularly noticeable to the community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Best to keep this short: OZSpeech is a practical one-step zero-shot TTS that reports strong WER and latency numbers, and the idea of starting flow matching from a learned prior rather than noise is worth taking seriously. But the paper's central derivation has an undefined tau, and the empirical evaluation has gaps. It deserves a serious referee, but it needs revisions.\n\nWhat's actually new: combining a learned prior (from a FastSpeech-like duration predictor plus code generator) with a single flow-matching step over FACodec's factorized tokens. The prior-conditioning is a pragmatic way to shorten the flow path, and the noise-robustness analysis is a nice addition. The authors are honest about limitations—they admit naturalness degradation and duration rounding issues. The efficiency numbers (2.7-6.5x faster, 29-71% model size) are concrete and useful.\n\nSoft spots, in order of concern. First, Eq. (3) is not well-defined as written: tau appears as a prior-dependent time variable, but the paper never specifies how it is sampled during training or what value is used at inference. Since v_theta is conditioned on tau, tau can't simply be 'learned' unless it's also an output or estimated separately. If tau is a constant, the 'one-step flow matching' is really residual prediction with a scaling factor; if it's sampled, the train/inference mismatch is unaddressed. This is the kind of thing that blocks reproducibility, and no code is available to fill the gap. Second, the omission of NaturalSpeech 3—which shares the same FACodec representation and is the strongest diffusion-based zero-shot baseline—is hard to justify. Third, the WER being 0.05 for every prompt length and even at 0dB SNR is surprising; baselines all degrade, and ground truth is 0.02. That needs error bars and a closer check of the ASR's sensitivity. The ablation and noise fine-tuning sections are fine but don't rescue the metric concerns.\n\nBottom line: the idea is plausible and the engineering is real, but the math description is sloppy and the evaluation is incomplete. This is a conditional accept/reject—worth refereeing so the authors can fix tau and add comparisons. I'd read it if you want a fast on-device TTS, but I wouldn't rely on the method details until clarified.","headline":"OZSpeech makes a plausible practical case for one-step zero-shot TTS with a learned prior, but an undefined tau in the flow-matching derivation and missing key baselines keep it from being reproducible as written.","tokens_in":21731,"tokens_out":3419,"would_cite":false,"duration_ms":35118,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A learned prior lets zero-shot TTS finish in one sampling step, with word error rate near 0.05.","keywords":["zero-shot text-to-speech","flow matching","one-step sampling","learned prior","optimal transport","factorized neural codec","FACodec","speech synthesis"],"falsifier":"Retrain the model with $\\tau$ fixed to a few concrete values, such as 0, 0.5, and 0.9, and report whether one-step sampling still gives WER near 0.05 on the LibriSpeech test-clean evaluation set. If no fixed $\\tau$ reproduces the reported results, or if the model only works when $\\tau$ is chosen per sample in a way the paper does not specify, the central reformulation is not reproducible as stated.","tokens_in":20675,"feed_emoji":"🎙️","tokens_out":6836,"duration_ms":63660,"temperature":0.7,"pith_summary":"The paper tries to establish that text-to-speech can start not from Gaussian noise but from a learned prior code that already carries the content, and then reach the target speech in a single flow-matching step. It argues that conditioning the flow on this learned prior removes the need for multi-step diffusion sampling, and the result is a zero-shot voice-cloning system whose word error rate stays near 0.05 even as audio prompts get longer or noisier. A careful reader should care because this is the efficiency claim that matters for real-time TTS: the model is 2.7 to 6.5 times faster at inference and only 29% to 71% the size of competing systems, trained on only 500 hours of speech.","feed_headline":"One-step zero-shot TTS holds WER at 0.05","feed_subtitle":"Learned prior codes skip the noise-to-speech path, making synthesis 2.7–6.5x faster at 29–71% model size.","key_machinery":"The load-bearing object is the learned prior code $x_{\\mathrm{pr}}$: six quantizer sequences from FACodec that decompose speech into prosody, content, and acoustic detail, predicted by a cascaded feed-forward transformer conditioned on phoneme embeddings. Around this the paper builds three mechanisms: the Prior Codes Generator $f_\\psi$ with a duration predictor, the one-step OT-CFM objective $\\mathcal{L}_{\\mathrm{CFM}}(\\theta)=\\mathbb{E}_{x_{\\mathrm{pr}},x_1}\\left\\|v_\\theta(x_{\\mathrm{pr}},\\tau)-\\frac{x_1-x_{\\mathrm{pr}}}{1-\\tau}\\right\\|^2$ in which the interpolation time $t$ is replaced by the unspecified prior-dependent $\\tau$, and the Anchor Loss that prevents embedding collapse and estimates $\\tilde{x}_1=x_{\\mathrm{pr}}+(1-\\tau)v_\\theta$. A folding operation and quantizer embeddings let all six code sequences be modeled in one Transformer pass instead of sequentially.","core_discovery":"The central discovery is a reformulation of optimal transport conditional flow matching in which the source distribution is not the standard Gaussian $x_0$ but a learned prior $x_{\\mathrm{pr}}$ produced by a cascaded code generator aligned to phonemes. The vector field estimator $v_\\theta$ then regresses the velocity between $x_{\\mathrm{pr}}$ and $x_1$, with the usual interpolation time replaced by an unknown prior-dependent time variable $\\tau$. The authors report that one evaluation of this field is enough to synthesize intelligible speech, and they measure a consistent word error rate of about 0.05 across 1s, 3s, and 5s prompts and across prompt signal-to-noise ratios from clean down to 0 dB, with multi-fold WER improvements over F5-TTS, VoiceCraft, NaturalSpeech 2, and VALL-E at the cost of somewhat lower naturalness scores.","pith_inferences":["If the learned prior is doing most of the generation work, the flow-matching component may be closer to a residual correction than a generative model; a clean ablation that replaces $v_\\theta$ with a simple predictor would reveal how much of the one-step success comes from the prior rather than the flow.","The unspecified $\\tau$ suggests a testable extension: infer $\\tau$ from $(x_{\\mathrm{pr}}, x_1)$ with a small regressor or schedule it during training; if any fixed $\\tau$ works as well as the reported results, the prior-dependent-time claim would be strengthened.","The noise-tolerance result points to a broader use: the same conditioning scheme could be applied to speech enhancement or voice conversion, where the prompt is noisy by construction and the content codes are masked.","Because WER stays flat while other metrics degrade with prompt length, the disentangled codec is likely isolating content from prosody; this separation could be verified by probing whether content codes are invariant to prompt noise."],"forward_implications":["At inference, one evaluation of the vector field replaces the 32 to 200 function evaluations used by F5-TTS and NaturalSpeech 2, giving a real-time factor of 0.26 on an A100 GPU.","The model keeps WER near 0.05 for 1s, 3s, and 5s prompts, while baseline WERs rise with prompt length, so content accuracy no longer depends on prompt duration.","With noisy prompts at SNR 0 to 12 dB, OZSpeech's WER stays near 0.05 without fine-tuning; fine-tuning on noisy prompts further lifts naturalness and prosody metrics without hurting WER.","Training on arbitrary prompt segments instead of first segments improves all metrics, indicating that hiding the prompt position is important for attribute transfer.","A 100M-parameter variant performs almost as well as the 145M base model, so the efficiency gain is not tied to the larger configuration."],"supporting_citations":[{"why":"Supplies the original optimal transport conditional flow matching objective and conditional probability path that OZSpeech reformulates.","marker":"(Lipman et al., 2023)"},{"why":"Introduces the rectified flow formulation that also underlies the OT-CFM loss being modified.","marker":"(Liu et al., 2023)"},{"why":"FlowSeq is the flow-matching-for-text adaptation that OZSpeech extends to speech and from which the anchor-loss adaptation is drawn.","marker":"(Hu et al., 2024)"},{"why":"Difformer is the source of the Anchor Loss regularization that stabilizes training on discrete token embeddings.","marker":"(Gao et al., 2024)"},{"why":"FACodec is the factorized neural codec whose six quantizer codes define the data representation and the prior and target spaces.","marker":"(Ju et al., 2024b)"},{"why":"FastSpeech's duration predictor aligns phoneme embeddings to codec code lengths.","marker":"(Ren et al., 2019a)"},{"why":"VALL-E is the autoregressive codec-LM baseline and the evaluation protocol that OZSpeech is compared against.","marker":"(Chen et al., 2025)"},{"why":"F5-TTS is the main flow-matching baseline whose 32-step sampling and 95,000-hour training are contrasted with OZSpeech's one step and 500 hours.","marker":"(Chen et al., 2024c)"},{"why":"NaturalSpeech 2 is the diffusion baseline requiring 200 steps, used in latency and WER comparisons.","marker":"(Shen et al., 2024)"}],"fun_headline_variants":["One-step TTS with learned prior: WER 0.05, 6.5x faster","Skipping Gaussian noise: OZSpeech does TTS in one step","Zero-shot TTS: single flow step, WER 0.05, smaller model","Learned-prior flow matching: one-step synthesis, WER 0.05","OZSpeech: one-step zero-shot TTS, WER 0.05 with prior"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on a number $\\tau$ that the paper never defines concretely: it stands in for how far the learned prior is from real speech, but the paper does not say what value is used when training or when generating in one step.","fun_headline_variants_meta":{"raw":{"variants":["One-step TTS with learned prior: WER 0.05, 6.5x faster","Skipping Gaussian noise: OZSpeech does TTS in one step","Zero-shot TTS: single flow step, WER 0.05, smaller model","Learned-prior flow matching: one-step synthesis, WER 0.05","OZSpeech: one-step zero-shot TTS, WER 0.05 with prior"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000332,"raw_usage":{"total_tokens":1845,"prompt_tokens":943,"completion_tokens":902,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":786}},"tokens_in":559,"tokens_out":902,"duration_ms":8924,"temperature":1.0,"reasoning_tokens":786,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:26:32.006110+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the model with $\\tau$ fixed to a few concrete values, such as 0, 0.5, and 0.9, and report whether one-step sampling still gives WER near 0.05 on the LibriSpeech test-clean evaluation set. If no fixed $\\tau$ reproduces the reported results, or if the model only works when $\\tau$ is chosen per sample in a way the paper does not specify, the central reformulation is not reproducible as stated.","supporting_citations":[],"review_version":1}