Pith. sign in

REVIEW 3 major objections 5 minor 32 references

StreamFlow: Streaming Flow Matching with Block-wise Guided Attention Mask for Speech Token Decoding

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read StreamFlow claims that speech token decoding can run at a constant ~180 ms per chunk, with audio quality on par with non-streaming decoders, by restricting a flow-matching transformer to a fixed block-wise receptive field instead of full…

desk verdict The 180 ms first-packet latency is per-chunk GPU compute, not end-to-end streaming latency, which undercuts the main claim; the block-wise mask design itself is a plausible extension that deserves a referee but needs correction. read the letter →

arxiv 2506.23986 v2 pith:SCWQCGUS submitted 2025-06-30 cs.SD eess.AS

Dake Guo , Jixun Yao , Linhan Ma , He Wang , Lei Xie This is my paper
classification cs.SDeess.AS
keywords streamingspeechsynthesisflowmatchingdiffusiontransformerblock-wiseattentionmasktokendecodinglow-latencygenerationcodeclanguagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

StreamFlow is an architecture for converting discrete speech tokens into audio in real time, filling in the decoder stage that follows a codec language model. The paper argues that turning semantic tokens into a waveform is a local process, so the decoder can attend only to a fixed window of nearby token blocks instead of the whole utterance. It implements this with block-wise attention masks placed at selected transformer layers, giving a receptive field of $(p+q+1)\cdot b$ tokens, and runs inference chunk by chunk with a sliding window. The experiments claim that this streaming scheme matches non-streaming audio quality and beats a causal streaming baseline while holding per-chunk latency roughly constant at 180 ms in long-form generation. If that holds, interactive speech agents could keep generating indefinitely without accumulating latency.

What carries the argument

The central object is the block-wise guided attention mask, a family of three binary masks that constrain transformer attention to block granularity: a Block Mask that isolates each token block, a Backward Mask that lets a block read the preceding block, and a Forward Mask that lets a block read the following block. With $p$ DiT layers using the Backward Mask and $q$ using the Forward Mask, the decoder's overall receptive field is $(p+q+1)\cdot b$ tokens, where $b$ is the block size (24 frames, or 0.24 s, in the experiments). The argument is carried by stacking these mask types hierarchically across the 22 DiT layers (backward masks at layers 7 and 14, forward masks at layers 1 and 22 for the large-receptive-field variant) and pairing the masked flow-matching decoder with sliding-window chunked inference and a chunked BigVGAN vocoder, keeping per-chunk computation constant. The generation side uses optimal-transport conditional flow matching with classifier-free guidance to map Gaussian noise to mel-spectrograms conditioned on semantic tokens and speaker embeddings.

What would settle it

Generate utterances whose acoustically decisive events fall outside the receptive field, for example a contrastive stress, a speaking-rate change, or a speaker switch occurring more than $(p+q+1)\cdot b$ tokens from the block being generated, and compare UTMOS/PESQ and listening scores against a non-streaming decoder; a sharp quality drop or audible block-boundary artifacts would refute the locality assumption. A complementary check is to log per-chunk latency after many minutes of continuous generation to verify it stays near 180 ms rather than growing with history.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that streaming speech token decoding need not trade quality for latency. A conventional streaming decoder attends only to past context and pays growing computational cost as history accumulates; StreamFlow instead treats waveform generation as a local conversion and endows a diffusion-transformer flow-matching decoder with a block-wise guided attention mask. Each token block may read the previous and/or next block depending on mask placement, and choosing how many DiT layers carry backward or forward masks sets the receptive field to exactly $(p+q+1)\cdot b$ tokens. With two past and one future block (StreamFlow-SR) or two past and two future blocks (StreamFlow-LR), subjective naturalness reaches NMOS 4.012 and 4.153, approaching the non-streaming DiT decoder at 4.213 and beating the causal streaming baseline at 3.978, while sliding-window inference keeps each chunk at roughly 180 ms regardless of utterance length. The same chunked treatment extends to the BigVGAN vocoder, whose fixed convolutional receptive field is naturally compatible with the block scheme.

Load-bearing premise

The design assumes that converting speech tokens into a waveform is a local operation, so a fixed window of a few token blocks is enough context for high-quality audio; if long-range context such as speaker prosody or articulation style is actually required, the constant-latency result becomes a quality-versus-latency tradeoff rather than a strict improvement.

Editorial extensions

If this is right

  • Codec-LM speech agents can converse indefinitely with flat per-chunk decoding cost, because the sliding-window mask makes computation independent of how much history has accumulated.
  • Streaming quality no longer depends on replaying history: the large-receptive-field variant scores NMOS 4.153, within 0.06 of the non-streaming DiT-CV and 0.175 above the causal streaming baseline.
  • Receptive-field size is a tunable knob: adding backward or forward masked layers, or enlarging block size from 0.12 s to 0.48 s, lifts PESQ from 1.361 to 1.573, at the cost of more future-token lookahead and higher system-level first-packet latency.
  • Streaming chunking extends to the vocoder stage, not just the flow-matching decoder, so the whole synthesis chain keeps bounded latency.
  • A first-packet latency of about 180 ms meets the real-time budget of interactive spoken-dialogue systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the block-wise locality design should transfer to other streaming generative modalities, such as music, sound effects, or low-bitrate video tokens, wherever outputs depend mostly on local structure; the paper only demonstrates it for speech.
  • Editorial inference: if long-range prosody matters, a hybrid that adds a few sparse global-attention layers or periodically refreshes a global conditioning vector could restore long-range context while keeping per-chunk cost constant; this is not explored in the paper.
  • Editorial inference: because mask placement is a discrete layer choice, a learned placement policy might outperform the hand-set layers (1, 7, 14, 22), which is a testable extension the paper does not run.
  • Editorial inference: the 180 ms figure is measured on an A100 for the decoder alone; end-to-end latency in a deployed agent, including the codec language model and network, would be larger and remains to be measured.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes StreamFlow, a streaming flow-matching decoder for speech token decoding in Codec-LM systems. Instead of full attention over the whole token sequence, it partitions the sequence into blocks and applies three block-wise attention masks (Block, Backward, Forward) at different DiT layers, yielding a bounded receptive field of (p+q+1)*b tokens. At inference, the mel-spectrogram is generated chunk-by-chunk with a sliding window, and BigVGAN is also run in chunks. Experiments compare two StreamFlow variants (small and large receptive field) against non-streaming and streaming baselines using objective metrics and a small listening test, and report per-chunk latency on an A100. The paper claims quality comparable to non-streaming models, better quality than a streaming causal-attention baseline, constant per-chunk inference time, and a first-packet latency of only 180 ms.

Significance. If the claims hold, the block-wise mask design is a simple and useful contribution: it converts a global-receptive-field flow-matching decoder into a streaming module with bounded per-chunk compute, which is relevant for real-time spoken dialogue systems. The paper is honest in describing the architecture, uses a large training corpus, includes a non-streaming DiT baseline and a streaming causal baseline, and reports both objective and subjective metrics. The main significance, however, is conditional: the 180 ms first-packet claim is currently only a per-chunk GPU compute measurement and ignores the lookahead imposed by the forward masks, and the quality advantages over the streaming baseline are small and not supported by significance tests or confidence intervals for the objective metrics. With those issues fixed, the contribution would be credible and useful.

major comments (3)
  1. [Abstract; §5.2] The 'first-packet latency of only 180 ms' is not supported by the measurement described. Section 5.2 states that latency is calculated 'on the NVIDIA A100 GPU device' per chunk, which is decoder compute time, not end-to-end first-packet latency. In the streaming pipeline of Section 3.2 and the mask configurations of Section 4.2, StreamFlow-SR requires one future block and StreamFlow-LR requires two future blocks of semantic tokens before the first chunk can be processed. At a 24-frame block with a 160-sample hop at 16 kHz, that corresponds to at least 0.24 s and 0.48 s of algorithmic lookahead before the 180 ms of compute. The paper itself acknowledges in Section 5.2 that a larger future receptive field 'further extend[s] the system's first-packet latency,' which contradicts the blanket 180 ms claim. Please either report a system-level first-packet latency that includes Codec-LM token generation and the required future context, or clearly relabel Figure 5 as per-chunk decoder compute and state the additional lookahead in the abstract and conclusion.
  2. [Tables 1–3; §5.1] The quality comparisons rely on small metric differences without uncertainty quantification. In Table 1, the gap between DiT-CVS and StreamFlow-SR is 0.013 in STOI, 0.049 in UTMOS, 0.108 in PESQ, and 0.054 in ViSQOL, and no confidence intervals or significance tests are reported for any objective metric. The subjective evaluation in Table 3 uses only 30 sentences and 20 listeners, and the reported intervals overlap between StreamFlow-LR (NMOS 4.153±0.10) and DiT-CV (4.213±0.11), and between StreamFlow-SR (4.012±0.09) and DiT-CVS (3.978±0.10). As published, these results do not demonstrate that StreamFlow is statistically comparable to non-streaming models or statistically better than the streaming baseline. Please add significance tests or confidence intervals for all metrics, and ideally increase the size of the test set and listening panel.
  3. [§3.1; §5.1] The design rests on the assertion in Section 3.1 that semantic-token-to-waveform generation 'can be formatted as a conversion process at localized,' requiring only a fixed receptive field of (p+q+1)*b tokens. This assumption is not validated. The block-size ablation in Table 2 shows that quality increases with block size, and StreamFlow-LR tends to outperform StreamFlow-SR in the subjective results, which suggests that the receptive field is still a limiting factor. Without an ablation that varies the amount of future context or compares against a global-context oracle on long utterances, the paper cannot establish that the fixed local context is sufficient for the 'comparable to non-streaming' claim. Please add an experiment or analysis that directly tests the locality assumption.
minor comments (5)
  1. [Table 3; §5.1.2] The heading 'Subeject evaluation' contains a typo; it should be 'Subjective evaluation.' Similarly, 'Obeject evaluation' in Section 5.1.1 should be 'Objective evaluation.'
  2. [§3.1] The notation is inconsistent: n is used both for the sequence length and for the number of DiT blocks, and the footnote 'q blocks for past and p blocks for future' contradicts the earlier definitions, where Backward Mask accesses preceding blocks and Forward Mask accesses subsequent blocks, as well as the Section 4.2 description in which StreamFlow-SR has two previous blocks and one future block. Please reconcile the p/q notation and the direction labels.
  3. [§4.2; Table 1] The comparison between UNet-CV and the DiT models is confounded by the vocoder: UNet-CV uses HiFiGAN while the DiT models use BigVGAN, which the text itself credits for some of the metric differences. Please state this explicitly in the experimental setup or in the table caption so readers can separate the contributions of the backbone and the vocoder.
  4. [§3.2; §4.2] The chunk-based processing of BigVGAN is described only as 'a similar chunk-based processing strategy,' without giving the chunk size, hop length, or overlap/padding scheme. Please report these details, since boundary artifacts in the vocoder are a common source of popping sounds and are directly relevant to the quality claims.
  5. [§5.2] The word 'paralytically' appears to be a typo for 'particularly' in the sentence about the future receptive field; please fix it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: contributions are an empirical architecture comparison; the reported first-packet latency is a compute-only measurement, which is a validity caveat rather than a circular reduction.

full rationale

The paper does not derive any central result from its own inputs. The block-wise guided attention masks define the receptive field explicitly as (p+q+1)*b tokens (Section 3.1), and quality is assessed with held-out test sentences against external baselines (UNet-CV, DiT-CV, DiT-CVS) using standard objective metrics and listening tests (Tables 1 and 3). Hyperparameters such as block size (0.24 s), chunk size (2 blocks), mask placements, CFG strength, and number of ODE steps are disclosed in Section 4.2 and are not fitted to the evaluation set, so no fitted quantity is renamed as a prediction. The only self-citation ([5]) appears in a generic list of Codec-LM works and is not load-bearing. The weak locality assumption in Section 3.1 is an unvalidated modeling assumption, not a circular step. Section 5.2's 'first-packet latency of approximately 180 ms' is measured per-chunk on an A100 GPU and, because StreamFlow-SR/LR use forward masks requiring future context blocks, the end-to-end first-packet latency would include waiting for those future tokens; this is a measurement-validity caveat, not a circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical entities or latent quantities are introduced; the masks are attention patterns over existing token blocks. The central claim rests on the locality assumption and on hand-picked block, chunk, and mask-configuration hyperparameters.

free parameters (4)
  • block_size = 24 frames (0.24 s)
    Controls the receptive field and latency; Section 4.2 sets b=24, while Table 2 shows larger blocks improve quality, indicating a hand-picked tradeoff rather than a principled choice.
  • chunk_size = 2 blocks
    Sets streaming granularity and first-packet latency; Section 4.2 states chunk size for all streaming models is 2 blocks. No ablations over chunk size are provided.
  • mask_configuration_SR/LR = SR: backward masks at layers 7 and 14, forward at layer 1; LR: backward at 7 and 14, forward at 1 and 22
    The number and placement of forward/backward masks define the receptive field and are chosen ad hoc; no systematic sweep over mask schedules is reported.
  • CFG_strength_and_steps = CFG 0.5, 10 Euler steps
    Inference hyperparameters selected for quality/speed; not part of the novelty but they affect the latency and quality numbers.
assumptions (4)
  • domain assumption Semantic token to mel-spectrogram conversion is a local process, so a fixed block-wise receptive field is sufficient.
    Stated in Section 3.1: 'We consider that the generation from semantic tokens to waveforms can be formatted as a conversion process at localized.' This is load-bearing: if long-range context matters, the mask-based streaming quality claim fails.
  • domain assumption Semantic tokens can be aligned to mel frames by repeated upsampling.
    Footnote 2 states tokens are aligned to mel-spectrogram using repeated upsampling; the flow matching condition c depends on this alignment.
  • domain assumption Chunk-based processing of BigVGAN preserves reconstructing quality.
    Section 3.2 says a similar chunk-based processing strategy is applied to BigVGAN, but no experiment isolates vocoder chunking artifacts.
  • standard math OT-CFM and CFG theory from cited works is correct and applicable.
    Section 2 relies on flow matching and classifier-free guidance from [11,15,16] without reproof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StreamFlow: Streaming Flow Matching with Block-wise Guided Attention Mask for Speech Token Decoding." pith.science (2026). https://pith.science/paper/SCWQCGUS

@misc{pith2026250623986,
  author       = {Pith},
  title        = {Pith review of: StreamFlow: Streaming Flow Matching with Block-wise Guided Attention Mask for Speech Token Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SCWQCGUS}},
  note         = {Machine review of arXiv:2506.23986}
}
read the original abstract

Recent advancements in discrete token-based speech generation have highlighted the importance of token-to-waveform generation for audio quality, particularly in real-time interactions. Traditional frameworks integrating semantic tokens with flow matching (FM) struggle with streaming capabilities due to their reliance on a global receptive field. Additionally, directly implementing token-by-token streaming speech generation often results in degraded audio quality. To address these challenges, we propose StreamFlow, a novel neural architecture that facilitates streaming flow matching with diffusion transformers (DiT). To mitigate the long-sequence extrapolation issues arising from lengthy historical dependencies, we design a local block-wise receptive field strategy. Specifically, the sequence is first segmented into blocks, and we introduce block-wise attention masks that enable the current block to receive information from the previous or subsequent block. These attention masks are combined hierarchically across different DiT-blocks to regulate the receptive field of DiTs. Both subjective and objective experimental results demonstrate that our approach achieves performance comparable to non-streaming methods while surpassing other streaming methods in terms of speech quality, all the while effectively managing inference time during long-sequence generation. Furthermore, our method achieves a notable first-packet latency of only 180 ms.\footnote{Speech samples: https://dukguo.github.io/StreamFlow/}

Figures

Figures reproduced from arXiv: 2506.23986 by the authors.

Figure 1
Figure 1. Overview architecture of our proposed StreamFlow. The overall architecture of StreamFlow is shown in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 4
Figure 4. Streaming Inference process of StreamFlow. 4. Experimental Setup 4.1. Training Setup For training, we use the Chinese and English subsets of Emilia [18], which totals 100,000 hours of speech data. For se￾mantic tokens extraction, we employ 25Hz S3tokenizer4 , while 80-dimensional mel-spectrograms are extracted from 16kHz speech signals, with a frame size of 1024 and a hop size of 160. 4.2. Model Details We use the n… view at source ↗
Figure 2
Figure 2. The details of the fundamental block-wise attention mask. Given a DiT model containing n DiT-blocks, if p blocks use the Backward Mask and q blocks use the Forward Mask while the rest use the Block Mask, the overall receptive field of the model is (p + q + 1) · b tokens 3 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Receptive field of DiT model with 3 DiT-blocks. When all three masks are combined, the final receptive field spans all three blocks. 3.2. Streaming Waveform Generation During inference, we implement a streaming inference pro￾cess by adopting a chunk-by-chunk approach t…
Figure 5
Figure 5. Figure 5: Chunk latency in long-form speech generation. Moreover, both objective and subjective evaluations indi￾cate that expanding the receptive field improves model perfor￾mance. However, in practical applications, when integrating with Codec-LM, a larger receptive field, par…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 20 canonical work pages

  1. [1]

    These systems enable real-time conversations with humans, demon- strating near-human fluency

    Introduction Recent advancements in speech generation and dialogue sys- tems, such as GPT-4o [1] and Moshi [2], have achieved re- markable multimodal speech interaction capabilities. These systems enable real-time conversations with humans, demon- strating near-human fluency. Such human-computer interac- tion requires systems that can perform reasoning di...

    arXiv 2025

  2. [2]

    Flow Matching In this section, we provide a brief description of flow match- ing (FM)

    Preliminaries 2.1. Flow Matching In this section, we provide a brief description of flow match- ing (FM). The goal of FM is to match a probability path that transforms a data distribution pt into a simple distribution p0 (typically p0 ∼ N(0, 1)). It is closely related to Continuous Normalizing Flows(CNFs) [14] but is trained much more effi- ciently in a s...

    work page

  3. [3]

    The overall architecture of StreamFlow is shown in Figure

    StreamFlow Figure 1: Overview architecture of our proposed StreamFlow. The overall architecture of StreamFlow is shown in Figure

    work page

  4. [4]

    which consists of several DiT-blocks

    We use the diffusion transformer (DiT) as the backbone, 2Semantic tokens are aligned to mel-spectrogram using repeated up- sampling. which consists of several DiT-blocks. To enhance stability and controllability during training, we apply zero-initialized adap- tive LayerNorm (adaLN-zero) in the DiT-blocks. StreamFlow takes speech tokens and speaker embedd...

    work page

  5. [5]

    Training Setup For training, we use the Chinese and English subsets of Emilia [18], which totals 100,000 hours of speech data

    Experimental Setup 4.1. Training Setup For training, we use the Chinese and English subsets of Emilia [18], which totals 100,000 hours of speech data. For se- mantic tokens extraction, we employ 25Hz S3tokenizer4, while 80-dimensional mel-spectrograms are extracted from 16kHz speech signals, with a frame size of 1024 and a hop size of 160. 4.2. Model Deta...

    work page

  6. [6]

    Audio Quantity Table 1: Evaluation results for different flow matching models

    Experimental Results 5.1. Audio Quantity Table 1: Evaluation results for different flow matching models. Models Mode STOI↑ UTMOS↑ PESQ↑ ViSQOL↑ SECS↑ UNet-CV Non-Stream0.827 3.671 1.361 4.102 0.743DiT-CV 0.852 3.692 1.581 4.057 0.727 DiT-CVS Stream 0.819 3.618 1.413 4.015 0.717StreamFlow-SR 0.832 3.667 1.521 4.069 0.709StreamFlow-LR 0.829 3.638 1.531 4.05...

    work page

  7. [7]

    We design a local block-wise re- ceptive field method to achieve efficient streaming inference, particularly for long-speech scenarios

    Conclusions In this paper, we propose StreamFlow, a streaming speech to- ken decoding framework based on DiT-driven flow matching, addressing the challenges of real-time speech generation while maintaining high quality. We design a local block-wise re- ceptive field method to achieve efficient streaming inference, particularly for long-speech scenarios. E...

    work page

  8. [8]

    GPT-4o system card,

    OpenAI, “GPT-4o system card,” CoRR, vol. abs/2410.21276, 2024

    arXiv 2024

Show all 32 references
  1. [9]

    Moshi: a speech-text foundation model for real-time dialogue,

    A. D ´efossez, L. Mazar´e, M. Orsini, A. Royer, P. P´erez, H. J´egou, E. Grave, and N. Zeghidour, “Moshi: a speech-text foundation model for real-time dialogue,” CoRR, vol. abs/2410.00037, 2024

    2024 arXiv

  2. [10]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in Proc. ICLR, 2021

    2021

  3. [11]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Proc. ICML, 2021, pp. 5530–5540

    2021

  4. [12]

    The NPU-HWC system for the ISCSLP 2024 inspirational and convincing audio generation challenge,

    D. Guo, J. Yao, X. Zhu, K. Xia, Z. Guo, Z. Zhang, Y . Wang, J. Liu, and L. Xie, “The NPU-HWC system for the ISCSLP 2024 inspirational and convincing audio generation challenge,” 2024

    2024

  5. [13]

    Better speech synthesis through scaling,

    J. Betker, “Better speech synthesis through scaling,” CoRR, vol. abs/2305.07243, 2023

    2023 arXiv

  6. [14]

    BASE TTS: lessons from building a billion-parameter text-to-speech model on 100k hours of data,

    M. Lajszczak, G. C ´ambara, Y . Li, F. Beyhan, A. van Korlaar, F. Yang, A. Joly, ´A. Mart´ın-Cortinas, A. Abbas, A. Michalski, A. Moinet, S. Karlapati, E. Muszynska, H. Guo, B. Putrycz, S. L. Gambino, K. Yoo, E. Sokolova, and T. Drugman, “BASE TTS: lessons from building a bill...

    2024 arXiv

  7. [15]

    Wavchat: A survey of spoken dialogue models,

    S. Ji, Y . Chen, M. Fang, J. Zuo, J. Lu, H. Wang, Z. Jiang, L. Zhou, S. Liu, X. Cheng, X. Yang, Z. Wang, Q. Yang, J. Li, Y . Jiang, J. He, Y . Chu, J. Xu, and Z. Zhao, “Wavchat: A survey of spoken dialogue models,” CoRR, vol. abs/2411.13577, 2024

    2024 arXiv

  8. [16]

    Cosyvoice: A scal- able multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

    Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Ma, Z. Gao, and Z. Yan, “Cosyvoice: A scal- able multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,” CoRR, vol. abs/2407.05407, 2024

    2024 arXiv

  9. [17]

    Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot,

    A. Zeng, Z. Du, M. Liu, K. Wang, S. Jiang, L. Zhao, Y . Dong, and J. Tang, “Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot,” CoRR, vol. abs/2412.02612, 2024

    2024 arXiv

  10. [18]

    Flow matching for generative modeling,

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” inProc. ICLR, 2023

    2023

  11. [19]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models,

    Z. Du, Y . Wang, Q. Chen, X. Shi, X. Lv, T. Zhao, Z. Gao, Y . Yang, C. Gao, H. Wang, F. Yu, H. Liu, Z. Sheng, Y . Gu, C. Deng, W. Wang, S. Zhang, Z. Yan, and J. Zhou, “Cosyvoice 2: Scalable streaming speech synthesis with large language models,” CoRR, vol. abs/2412.10117, 2024

    2024 arXiv

  12. [20]

    Scalable diffusion models with transform- ers,

    W. Peebles and S. Xie, “Scalable diffusion models with transform- ers,” in Proc. ICCV, 2023

    2023

  13. [21]

    Neu- ral ordinary differential equations,

    T. Q. Chen, Y . Rubanova, J. Bettencourt, and D. Duvenaud, “Neu- ral ordinary differential equations,” in Proc. NeurIPS, S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., 2018

    2018

  14. [22]

    Ot-flow: Fast and accurate continuous normalizing flows via optimal transport,

    D. Onken, S. W. Fung, X. Li, and L. Ruthotto, “Ot-flow: Fast and accurate continuous normalizing flows via optimal transport,” in Proc. AAAI, 2021, pp. 9223–9232

    2021

  15. [23]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,”CoRR, vol. abs/2207.12598, 2022

    2022 arXiv

  16. [24]

    Bigv- gan: A universal neural vocoder with large-scale training,

    S. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “Bigv- gan: A universal neural vocoder with large-scale training,” in Proc. ICLR, 2023

    2023

  17. [25]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,

    H. He, Z. Shang, C. Wang, X. Li, Y . Gu, H. Hua, L. Liu, C. Yang, J. Li, P. Shi, Y . Wang, K. Chen, P. Zhang, and Z. Wu, “Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,” CoRR, vol. abs/2407.05361, 2024

    2024 arXiv

  18. [26]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” inProc. NeurIPS, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., 2020

    2020

  19. [27]

    ECAPA-TDNN embeddings for speaker diarization,

    N. Dawalatabad, M. Ravanelli, F. Grondin, J. Thienpondt, B. De- splanques, and H. Na, “ECAPA-TDNN embeddings for speaker diarization,” in Proc. Interspeech, H. Hermansky, H. Cernock ´y, L. Burget, L. Lamel, O. Scharenborg, and P. Motl´ıcek, Eds., 2021

    2021

  20. [28]

    Scaling rectified flow transformers for high-resolution image synthesis,

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boesel, D. Podell, T. Dockhorn, Z. English, and R. Rombach, “Scaling rectified flow transformers for high-resolution image synthesis,” in Proc. ICML, 2024

    2024

  21. [29]

    An al- gorithm for intelligibility prediction of time-frequency weighted noisy speech,

    C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An al- gorithm for intelligibility prediction of time-frequency weighted noisy speech,” IEEE Trans. Speech Audio Process. , pp. 2125– 2136, 2011

    2011

  22. [30]

    Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” in Proc. ICASSP, 2001

    2001

  23. [31]

    Visqol: an objective speech quality model,

    A. Hines, J. Skoglund, A. C. Kokaram, and N. Harte, “Visqol: an objective speech quality model,” EURASIP J. Audio Speech Music. Process., vol. 2015, p. 13, 2015

    2015

  24. [32]

    UTMOS: utokyo-sarulab system for voicemos challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: utokyo-sarulab system for voicemos challenge 2022,” in Proc. Interspeech, 2022

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.