Pith. sign in

REVIEW 4 major objections 4 minor 23 references

Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hierarchy-aware discrete diffusion outperforms autoregressive codec models for text-guided speech inpainting and editing, reporting the best overall results on the RealEdit benchmark.

desk verdict Solid speech-editing diffusion with a real hierarchy gain, but a teacher-forcing gap and missing significance tests keep the causal claim from being proven. read the letter →

arxiv 2608.06424 v1 pith:UGRXJH7G submitted 2026-08-05 cs.SD cs.CLcs.LG

Iftach Shoham , Tali Dror , Oren Gal , Haim Permuter , Gilad Katz , Eliya Nachmani This is my paper · ORCID
classification cs.SDcs.CLcs.LG
keywords speechinpaintingeditingdiscretediffusionresidualvectorquantizationcodectokenstext-guidedclassifier-freeguidanceRealEditbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces SIEDD, a discrete diffusion framework that performs text-guided speech inpainting and editing directly on hierarchical neural codec tokens. The central claim is that explicitly respecting the coarse-to-fine structure of residual vector quantization (RVQ) codebooks—rather than treating all codebook levels symmetrically—lets a diffusion model repair missing or edited spans more faithfully than autoregressive codec language models. On the RealEdit benchmark, the method reports the best overall editing results among the evaluated systems, with the lowest word error rate and mel cepstral distortion and the highest speaker similarity, and it beats the autoregressive baselines across all tested inpainting settings. The authors argue that discrete diffusion's iterative refinement of masked tokens, conditioned on both left and right acoustic context, avoids the irreversible error accumulation of left-to-right generation. If correct, this positions hierarchy-aware discrete diffusion as a strong alternative to autoregressive decoding for context-preserving speech modification.

What carries the argument

The central mechanism is HiCoDD, a hierarchical multi-codebook discrete diffusion transformer. It factorizes the joint distribution over codebooks as $p_\theta(x^{(1)}|P)\prod_{k=2}^K p_\theta(x^{(k)}|x^{(<k)},P)$, where each factor is an absorbing-state reverse-diffusion chain parameterized by a conditional concrete score. A block-triangular attention mask enforces that each diffusion decoder sees only strictly lower committed codebooks as clean context, preventing leakage of the target codebook; within a codebook, all temporal positions are denoised in parallel. Guidance is applied in log-score space with a negative branch that randomizes only the phonemes of the edited span, and a neural duration predictor maps target phonemes to expected codec frame counts so insertion and substitution edits get the right token span length.

What would settle it

Measure SIEDD's editing and inpainting quality when the lower codebooks fed to the clean encoder at inference are corrupted, for example by injecting token substitutions at increasing rates while the diffusion decoder is run unchanged; if WER and speaker similarity degrade sharply as lower-codebook noise grows, the clean-context assumption is doing the work, and a variant that replaces ground-truth lower codebooks with the model's own committed samples during training should be tested against the reported numbers.

Watch

Extended reading notes

Core claim

SIEDD's core discovery is that the RVQ codebook hierarchy should structure the diffusion process itself. Instead of denoising all codebooks jointly or flattening them into one sequence, HiCoDD generates codebooks one level at a time, coarse to fine: at each stage the already-committed lower codebooks are encoded as clean acoustic context, and diffusion is applied only to the current refinement codebook. This separation lets training always condition on exact ground-truth lower codebooks—the clean context the model will have after commitment at inference—while inference commits each codebook before the next is generated. The model further adds phoneme-level conditioning via a pretrained phoneme encoder, a learned duration predictor for variable-length edits, and a span-localized classifier-free guidance that contrasts only the phonemes inside the edited region. On RealEdit, SIEDD reports the best editing performance of the evaluated methods (WER 0.121, speaker similarity 0.98, MCD 270.0) and outperforms the two autoregressive baselines across essentially all single- and multi-gap inpainting settings, with especially large MCD gains for short gaps.

Load-bearing premise

The load-bearing premise is that training on ground-truth lower codebooks transfers to inference, where those codebooks are generated by the model; if the model's own lower-codebook errors are large enough, the reported gains from coarse-to-fine generation could shrink or vanish.

Editorial extensions

If this is right

  • Speech inpainting and editing can be unified under one framework: the same reverse-diffusion update fills fixed-length gaps (inpainting) and variable-length text-driven spans (editing), differing only in how the masked span length is set.
  • Making the codebook hierarchy explicit in the generative model improves both intelligibility and speaker preservation compared with joint or single-stream codebook diffusion, as shown by ablations moving from WER 0.186 to 0.139 to 0.121.
  • Because diffusion refines all masked positions jointly and repeatedly, multi-gap reconstructions stay stable whereas autoregressive baselines show sharp word-error-rate increases as gap count grows.
  • The ordered coarse-to-fine generation scheme means each refinement codebook is generated conditioned on an already-committed acoustic representation, so the approach is compatible with future work on faster sampling and improved duration modeling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same hierarchical conditioning could transfer to other residual tokenizers, such as image or video codecs with coarse-to-fine structure, making hierarchy-aware discrete diffusion a general principle rather than a speech-specific trick.
  • A testable extension is to add self-conditioning at inference: feed the model's own generated lower codebooks back through the clean encoder during refinement, which would directly address the train/inference mismatch without retraining the diffusion backbone.
  • The span-localized classifier-free guidance idea could be applied to other conditional discrete diffusion tasks, such as lyric-corrected music editing, where the negative branch randomizes only the changed words.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces SIEDD, a discrete diffusion framework for text-guided speech inpainting and editing over residual vector-quantized (RVQ) codec tokens. Its core architectural contribution, HiCoDD, generates codebooks in coarse-to-fine order: at each stage only the current codebook is denoised by diffusion, while previously generated codebooks are treated as clean, committed acoustic context. The model combines phoneme-level conditioning with XPhoneBERT, span-localized classifier-free guidance in log-score space, and a learned token-count duration predictor for variable-length edits. Experiments on the RealEdit benchmark compare SIEDD with VoiceCraft, SSR-Speech, and an MMS TTS baseline for speech editing, and with VoiceCraft and SSR-Speech for speech inpainting across several gap durations and numbers of gaps. The paper reports the lowest WER, MCD, and energy distance and the highest speaker similarity for editing, and consistent inpainting gains, with ablations supporting the hierarchical codebook design, duration predictor, and guidance.

Significance. If the central claims hold, SIEDD is a meaningful step toward treating speech editing as a non-causal, jointly refining problem rather than as autoregressive token infilling, and the HiCoDD design offers a reusable template for hierarchical discrete diffusion over RVQ codecs. The paper has several concrete strengths: it evaluates on a public benchmark (RealEdit) with shared EnCodec tokenization for the main editing comparison, releases code and configuration files, and reports detailed hyperparameter tables. The gap-closing train/inference mismatch described below, however, directly bears on the causal claim attached to the hierarchical architecture, and the main editing WER difference is not shown to be statistically reliable. These issues are addressable in a revision, but they currently prevent the paper from fully supporting its headline conclusions.

major comments (4)
  1. [Section 4.1 and Section 4.3] The training objective conditions on ground-truth lower codebooks: Section 4.1 states that 'the model always conditions on the ground-truth lower CBs x_0^{(<k)}' and that this 'gives each refinement level the exact clean context it will have after commitment at inference.' That second statement is not true at inference, because Section 4.3 commits model-generated lower codebooks before denoising higher ones. Generated lower codebooks may contain token errors and are then frozen as context for all higher refinement levels. No scheduled sampling, self-conditioning, or training-time corruption of lower CBs is described, and no analysis is provided of how errors in generated lower CBs propagate. This mismatch is load-bearing for the abstract's causal claim that 'explicitly modeling the codec hierarchy substantially improves' performance: the hierarchical gain in Table 2 (0.152 vs. 0.121) could partly be an artifact of clean oracle context available only in training. Please add an empirical analysis of lower-CB token accuracy at inference, an ablation with scheduled sampling or lower-CB dropout during training, or a direct comparison of inference conditioned on oracle versus generated lower codebooks.
  2. [Table 1 and Section 6] The headline editing WER differences are small relative to the reported variability: SIEDD reports 0.121±0.11 versus VoiceCraft's 0.124±0.11, a difference well within one standard deviation. No paired significance tests, bootstrap confidence intervals, or per-utterance comparisons are reported for WER or for the other metrics that support the 'best overall' claim. Because the editing set is a fixed benchmark with repeated utterances, paired tests are feasible and should be reported. If the WER difference is not significant, the claim of 'lowest WER' should be qualified accordingly, even if other metrics such as MCD show larger absolute differences.
  3. [Section 7, Table 2] The 'Single vs. multi-CB tokenization' ablation confounds the tokenizer with the codebook hierarchy: the single-CB row uses WavTokenizer at 24 kHz, while the multi-CB rows use EnCodec at 16 kHz. The improvement from 0.186 to 0.139 WER can therefore be attributed to the tokenizer change as much as to the presence of multiple RVQ codebooks, and the statement that the result 'indicating that the coarse-to-fine residual structure of RVQ better preserves linguistic content' overstates the evidence. Please either compare against a single-codebook variant of the same EnCodec tokenizer (for example, using only the first RVQ codebook) or remove this row from the hierarchy argument and from the abstract's causal claim.
  4. [Section 5.3 and Table 7] The claim in the abstract and Section 6 that SIEDD 'outperforms the evaluated autoregressive baselines across all speech-inpainting settings' is not fully supported by Table 7 as printed: the 1500 ms block appears to contain only the single-gap condition, while the text claims coverage of multiple gaps across all configurations. If the two- and three-gap 1500 ms results exist, they should be included in the table or supplementary material; otherwise the claim should be restricted to the configurations actually evaluated. Relatedly, it would be helpful to state explicitly whether the baseline numbers were produced with the same utterance-level averaging over five seeds as SIEDD, or whether published baseline outputs were reused.
minor comments (4)
  1. [Eq. (5)] The symbol K is used both for the number of RVQ codebooks and for the function K(a) := a log a − a in the DWDSE loss of Eq. (5); please rename one of them to avoid confusion.
  2. [Figure 2] The inpainting figure plots mean lines across gap configurations but does not show error bars or confidence intervals; since several reported differences are modest relative to the standard deviations in Table 7, adding error visualization or shaded intervals would help the reader assess the stability of the ordering.
  3. [Table 6 and Section 6] Please state whether the sampling hyperparameters (CFG coefficient 1.5, 512 denoising steps, top-k 2) were selected on a validation split of RealEdit or on the reported test set; if they were tuned on the test set, a sensitivity analysis around these values would be valuable.
  4. [Throughout] There are recurring typographical artifacts such as 'V oiceCraft' with a space after the capital V; these should be corrected in the final version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central conclusions rest on external RealEdit comparisons and on architectural ablations, not on self-citations or fitted constants.

full rationale

The paper's core derivation chain is self-contained with respect to external benchmarks. The hierarchical factorization in Eq. 9 is presented as a modeling choice inspired by external work (Arriola et al., 2025), and the conditional concrete score of Eq. 10 generalizes the standard SEDD score rather than being defined in terms of the reported result. The headline editing and inpainting claims are supported by comparisons with VoiceCraft, SSR-Speech, and MMS on the public RealEdit benchmark using external metrics (WER, SIM, MCD, UTMOS), and the ablation that isolates hierarchical versus joint codebook modeling (Table 2: 0.152 to 0.121 WER) is an architectural comparison, not a prediction obtained from a fitted parameter. The only same-group citation, Dror et al. 2026, appears as a related-work mention for discrete-diffusion audio inpainting and is not load-bearing: the method's diffusion machinery is borrowed from SEDD and its factorization from an external Block Diffusion reference. Hyperparameters such as CFG strength, denoising steps, and top-k were fixed for the reported runs, but no reported metric is a function of those hyperparameters by construction, so this is ordinary tuning rather than fitted-input-called-prediction. The train/inference mismatch on lower codebooks (ground-truth conditioning during training versus generated commitment at inference) is a legitimate correctness risk, but it is not circular: the paper does not define the hierarchy gain in terms of that conditioning, and testing robustness to generated lower-codebook errors would be an additional experiment, not a logical reduction of the result to its input. No equation in the paper reduces a claimed prediction to its own input, and no load-bearing inference is carried by a self-citation. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The ledger lists hand-set hyperparameters and learned components the main result depends on. No free physical constants are fitted; in the ML sense the model contains millions of trained weights, which are not enumerated here. The most important ad-hoc assumption is that ground-truth lower codebooks at training are interchangeable with generated lower codebooks at inference, which the paper asserts but does not validate. No new physical or field-level entities are postulated.

free parameters (5)
  • Classifier-free guidance strength alpha = 1.5
    Hand-set coefficient in Eq. (12); the ablation shows removing CFG degrades WER, so the main result depends on this value.
  • Number of denoising steps = 512
    All main results and ablations use 512 reverse steps; no step-count curve is reported, so quality depends on this chosen budget.
  • Sampling top-k and temperature = top-k=2, temperature=1.0
    Inference parameters in Table 6; they shape token sampling and are not derived from theory.
  • Noise schedule bounds sigma_min, sigma_max = 1e-4, 20
    Forward process hyperparameters in Table 4; standard choices, but they control how much masking occurs and were chosen by hand.
  • Duration predictor network weights = Trained on LibriTTS MFA alignments, about 1e5 parameters
    The token-count predictor is learned from data (Eqs. 16-18) and is central to variable-duration edits, so edited span lengths are fitted rather than derived.
assumptions (4)
  • domain assumption RVQ codebook levels are causally ordered so that lower codebooks are valid, sufficient context for higher codebooks.
    EnCodec builds codebooks as sequential residuals; Eq. (9) and the block-triangular attention mask rely on this ordering.
  • ad hoc to paper Ground-truth lower codebooks used in training are a good proxy for model-generated lower codebooks at inference.
    Section 4.1 states the model always sees the exact clean context it will have after commitment, but at inference the committed codebooks are generated and may contain errors; no error analysis is given.
  • standard math Absorbing-state DWDSE training from SEDD transfers to per-codebook conditional scores.
    The paper reuses the score-entropy objective of Eq. (5) from Lou et al. for each factor of Eq. (9); this is prior theory, not re-derived.
  • domain assumption RealEdit benchmark metrics (Whisper WER, WavLM similarity, MCD, UTMOS) capture editing and inpainting quality.
    Used as ground truth for 'best overall'; predicted UTMOS and ASR-based WER may miss perceptually relevant artifacts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing." pith.science (2026). https://pith.science/paper/UGRXJH7G

@misc{pith2026260806424,
  author       = {Pith},
  title        = {Pith review of: Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UGRXJH7G}},
  note         = {Machine review of arXiv:2608.06424}
}
read the original abstract

Speech recordings often contain missing, corrupted, or incorrect regions that must be reconstructed or modified without re-synthesizing the entire utterance. Speech inpainting restores missing segments, whereas speech editing replaces spoken content according to an edited transcript. Both tasks require the generated speech to express the intended words while remaining consistent with the surrounding speaker identity, prosody, timing, and recording conditions. Discrete diffusion is particularly well suited to these tasks because it can iteratively refine masked tokens while jointly conditioning on both left and right acoustic context. We introduce SIEDD, a discrete diffusion framework for text-guided speech inpainting and editing over hierarchical codec tokens. Its core architecture, HiCoDD, follows the RVQ generation order by representing previously generated codebooks as clean, committed acoustic context and applying diffusion only to the current refinement codebook. This separation enables leakage-free joint training while matching sequential coarse-to-fine inference. The model further combines phoneme-level conditioning, span-localized classifier-free guidance, and duration prediction to support both fixed-duration inpainting and variable-duration text edits. On the RealEdit benchmark, SIEDD achieves the best overall speech-editing performance among the evaluated methods. It also outperforms the evaluated autoregressive baselines across all speech-inpainting settings, on both single and multiple gaps. These results demonstrate that explicitly modeling the codec hierarchy substantially improves context-preserving speech reconstruction and editing. See our full code at https://github.com/iftachShoham/SIEDD.

Figures

Figures reproduced from arXiv: 2608.06424 by the authors.

Figure 1
Figure 1. Overview of the proposed architecture and inference procedure. (1) Architecture: for each target CB k, the noised tokens are denoised by the diffusion decoder while the clean lower CBs < k are encoded as committed acoustic context; each block combines temporal self-attention with cross-attention to the lower-CB and phoneme representations.(2) Inference: CBs are generated coarse to fine; at each stage only the edited… view at source ↗
Figure 2
Figure 2. Speech inpainting results across masked-span configurations, ordered by total masked [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 11 canonical work pages

  1. [3]

    Mapache: Masked parallel transformer for advanced speech editing and synthesis

    Guillermo Cámbara, Patrick Lumban Tobing, Mikolaj Babianski, Ravichander Vipperla, Duo Wang, Ron Shmelkin, Giuseppe Coccia, Orazio Angelini, Arnaud Joly, Mateusz Lajszczak, et al. Mapache: Masked parallel transformer for advanced speech editing and synthesis. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICAS...

    work page 2024

  2. [5]

    High fidelity neural audio compression.arXiv preprint arXiv:2210.13438,

    Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression.arXiv preprint arXiv:2210.13438,

    arXiv

  3. [6]

    For substitution and insertion operations, the system must determine the number of codec frames allocated to the edited span before generation

    10 Token-Count Duration Prediction SIEDD performs speech editing directly in the discrete token space of an EnCodec codec. For substitution and insertion operations, the system must determine the number of codec frames allocated to the edited span before generation. A simple approach is to estimate this duration from the number of phonemes in the target t...

    work page 2020

  4. [8]

    Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling.arXiv preprint arXiv:2408.16532,

    Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, et al. Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling.arXiv preprint arXiv:2408.16532,

    arXiv

  5. [11]

    Discrete diffusion modeling by estimating the ratios of the data distribution, 2024.URL https://arxiv

    Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution, 2024.URL https://arxiv. org/abs/2310.16834,

    arXiv 2024

  6. [12]

    Speech inpainting: Context-based speech synthesis guided by video

    Juan F Montesinos, Daniel Michelsanti, Gloria Haro, Zheng-Hua Tan, and Jesper Jensen. Speech inpainting: Context-based speech synthesis guided by video.arXiv preprint arXiv:2306.00489,

    work page Pith review arXiv

  7. [13]

    Transient Noise Removal via Diffusion-based Speech Inpainting

    Mordehay Moradi and Sharon Gannot. Transient noise removal via diffusion-based speech inpainting. arXiv preprint arXiv:2508.08890,

    work page Pith review arXiv

  8. [15]

    Large language diffusion models.arXiv preprint arXiv:2502.09992,

    Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji- Rong Wen, and Chongxuan Li. Large language diffusion models.arXiv preprint arXiv:2502.09992,

    arXiv

Show all 23 references
  1. [16]

    Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu

    URL https://arxiv.org/ abs/2212.04356. Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. Fastspeech 2: Fast and high-quality end-to-end text to speech.arXiv preprint arXiv:2006.04558,

    2006 arXiv

  2. [18]

    Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456,

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456,

    2011 arXiv

  3. [19]

    Score-based continuous-time discrete diffusion models.arXiv preprint arXiv:2211.16750,

    Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, and Hanjun Dai. Score-based continuous-time discrete diffusion models.arXiv preprint arXiv:2211.16750,

    arXiv

  4. [21]

    usee: Unified speech enhancement and editing with conditional diffusion models

    Muqiao Yang, Chunlei Zhang, Yong Xu, Zhongweiyang Xu, Heming Wang, Bhiksha Raj, and Dong Yu. usee: Unified speech enhancement and editing with conditional diffusion models. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pa...

    2024

  5. [22]

    Libritts: A corpus derived from librispeech for text-to-speech.arXiv preprint arXiv:1904.02882,

    Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. Libritts: A corpus derived from librispeech for text-to-speech.arXiv preprint arXiv:1904.02882,

    1904 arXiv

  6. [1981]

    Discrete diffusion for generative modeling of text-aligned speech tokens

    11 Pin-Jui Ku, He Huang, Jean-Marie Lemercier, Subham Sekhar Sahoo, Zhehuai Chen, and Ante Juki´c. Discrete diffusion for generative modeling of text-aligned speech tokens. InICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pa...

    2026

  7. [2011]

    Block diffusion: Interpolating between autoregressive and diffusion language models

    Marianne Arriola, Aaron Gokaslan, Justin Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sahoo, and V olodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. InInternational Conference on Learning Representations, volume 2025, p...

    2025

  8. [2018]

    Fluenteditor: Text-based speech editing by considering acoustic and prosody consistency.arXiv preprint arXiv:2309.11725,

    Rui Liu, Jiatian Xi, Ziyue Jiang, and Haizhou Li. Fluenteditor: Text-based speech editing by considering acoustic and prosody consistency.arXiv preprint arXiv:2309.11725,

    arXiv

  9. [2020]

    Utmos: Utokyo-sarulab system for voicemos challenge 2022.arXiv preprint arXiv:2204.02152,

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hi- roshi Saruwatari. Utmos: Utokyo-sarulab system for voicemos challenge 2022.arXiv preprint arXiv:2204.02152,

    2022 arXiv

  10. [2021]

    Token-based audio inpainting via discrete diffusion

    Tali Dror, Iftach Shoham, Moshe Buchris, Oren Gal, Haim Permuter, Gilad Katz, and Eliya Nachmani. Token-based audio inpainting via discrete diffusion. InInternational Conference on Learning Representations, volume 2026, pages 19136–19151,

    2026

  11. [2022]

    Speechpainter: Text-conditioned speech inpainting.arXiv preprint arXiv:2202.07273,

    Zalán Borsos, Matt Sharifi, and Marco Tagliasacchi. Speechpainter: Text-conditioned speech inpainting.arXiv preprint arXiv:2202.07273,

    arXiv

  12. [2023]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    arXiv

  13. [2024]

    Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio.arXiv preprint arXiv:2106.06909,

    Guoguo Chen, Shuzhou Chai, Guanbo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, et al. Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio.arXiv preprint arXiv:2106.06909,

    arXiv

  14. [2025]

    Xphonebert: A pre-trained multilingual model for phoneme representations for text-to-speech.arXiv preprint arXiv:2305.19709,

    Linh The Nguyen, Thinh Pham, and Dat Quoc Nguyen. Xphonebert: A pre-trained multilingual model for phoneme representations for text-to-speech.arXiv preprint arXiv:2305.19709,

    arXiv

  15. [2026]

    Ssr-speech: Towards stable, safe and robust zero-shot text-based speech editing and synthesis

    Helin Wang, Meng Yu, Jiarui Hai, Chen Chen, Yuchen Hu, Rilin Chen, Najim Dehak, and Dong Yu. Ssr-speech: Towards stable, safe and robust zero-shot text-based speech editing and synthesis. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Process...

    2025

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.