Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

SepALM: Audio Language Models Are Error Correctors for Robust Speech Separation

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SepALM claims that correcting a first separation pass in the text domain, then re-synthesizing the audio with a codec language model, beats audio-domain correction and improves robustness to unseen noise.

desk verdict A clever but under-controlled pipeline: the paper never isolates the text-domain corrector from resynthesis and alignment, so the headline causal claim isn't yet backed by the experiments. read the letter →

arxiv 2505.03273 v2 pith:RIMIJCVG submitted 2025-05-06 cs.SD cs.CLeess.AS

Zhaoxi Mu , Xinyu Yang , Gang Wang This is my paper · ORCID
classification cs.SDcs.CLeess.AS
keywords speechseparationaudiolanguagemodeltext-domainerrorcorrectionneuralcodecre-synthesischain-of-thoughtpromptingknowledgedistillationrobust
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SepALM claims that speech separation improves when a first, imperfect separation is corrected in the text domain rather than re-processed as audio. The pipeline uses a standard separator to produce an initial estimate, an audio language model to transcribe that estimate and fix transcription errors, and a neural-codec language model to re-synthesize cleaner speech from the corrected text plus the initial audio. On Libri2Mix, WHAM!, and WHAMR!, the authors report higher SI-SNRi and SDRi than discriminative and generative baselines, and on MUSAN and DEMAND noise the approach generalizes better than methods that correct directly in the speech domain. If the claim holds, separation systems could become more resilient to unfamiliar real-world acoustics without retraining on every new noise type.

What carries the argument

The load-bearing mechanism is the text-domain correction loop paired with codec-based re-synthesis. The corrector is an audio language model (SpeechGPT2, a LLaMA-based model that accepts both speech and text) prompted with Chain-of-Thought to perform two sub-steps: transcribe the preliminary separated speech, then correct that transcription while still listening to the audio; it is fine-tuned with LoRA using Whisper-Tiny as a knowledge-distillation teacher. The synthesizer treats the corrected transcription and the preliminary separated speech as equal inputs, converting them into phoneme or semantic tokens plus acoustic tokens from DAC (a residual vector-quantization audio codec), and generates new acoustic tokens either autoregressively or with masked non-autoregressive generation before the DAC decoder returns a waveform. The aligner, a small convolutional network, learns time-frequency weights that linearly combine the re-synthesized speech with the preliminary separated speech, restoring the phase information that a codec regeneration lacks. The correction step is what carries the main generalization claim; the synthesis and alignment steps are what turn corrected text back into a metric-friendly waveform.

What would settle it

Score the same separated outputs with an independent ASR model or with human transcripts instead of Whisper-Tiny; if the word-error-rate advantage of SepALM over the separator shrinks or disappears while SI-SNRi gains remain, the specifically textual-correction story is not supported. A stronger version would evaluate the corrected transcription directly against human transcripts in a low-SNR babble condition where the teacher is known to err.

Watch

Extended reading notes

Core claim

The central claim is that a single audio language model can serve as an end-to-end error corrector for speech separation. After a preliminary separator produces a distorted estimate, the ALM first recognizes the speech into a rough transcription and then corrects that transcription using the original audio; the corrected text is fed, together with the preliminary audio tokens, to a neural codec language model that regenerates the signal. The paper argues that text is a lower-resolution representation that is less affected by noise, so correcting there is easier, and that a single ALM avoids the error accumulation and optimization difficulties of cascading a separate ASR model with a large language model. A time-frequency aligner then blends the re-synthesized signal with the preliminary estimate to compensate for missing phase information. The reported results are consistent gains in SI-SNRi and SDRi across Libri2Mix, WHAM!, and WHAMR!, with the largest improvements over baselines in noisy and reverberant conditions, and better out-of-domain performance on MUSAN and DEMAND.

Load-bearing premise

The load-bearing premise is that Whisper-Tiny's transcriptions are a trustworthy stand-in for ground truth when training the corrector and when scoring word error rate, so if the teacher makes the same errors on separated speech and clean speech, the reported intelligibility gains are inflated.

Editorial extensions

If this is right

  • Speech separation systems could be upgraded for new noise environments by improving the correction and synthesis stages, without retraining the separation network itself.
  • The reported experiments indicate that a single ALM corrector with greedy decoding matches or beats a cascaded ASR-plus-LLM corrector at lower inference cost.
  • Because the re-synthesizer learns the distribution of clean speech, the approach may preserve perceived naturalness even when objective sample-level metrics are stressed by phase shifts.
  • The NAR masked-generation variant is reported to give both better quality and faster inference than the AR variant, making it the more practical configuration.
  • The aligner trades a small drop in naturalness for a large gain in SI-SNRi, so the final design is a deliberate balance between subjective quality and objective metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This suggests a general recipe for signal recovery: route an imperfect estimate through a low-resolution semantic channel, such as text or labels, and regenerate the signal with a generative codec model, whenever such a model exists for the modality.
  • If the word-error-rate gain is tied to Whisper-Tiny as both teacher and evaluator, then scoring with an independent ASR system or human transcripts could change the relative ranking of the AR and masked variants, while the SI-SNRi improvements should survive.
  • A natural stress test is to chain SepALM with progressively weaker separators; if the corrector keeps quality high, its contribution is separable from the separator's and the method could be bolted onto any existing separation front end.
  • The same corrector-synthesizer design could be applied to target speaker extraction, where the corrected transcription would also serve as a check that the extracted speaker is the one requested.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SepALM, a speech separation system that first separates a two-speaker mixture with SepFormer, then uses a fine-tuned audio language model (SpeechGPT with LoRA) to transcribe the preliminary separated speech and correct errors in the text domain via chain-of-thought prompting and Whisper-based knowledge distillation, then re-synthesizes the corrected speech with a neural codec language model (AR or NAR masked generation), and finally aligns the re-synthesized signal with the preliminary separated speech in the time-frequency domain. Experiments on Libri2Mix, WHAM!, and WHAMR! report higher SI-SNRi, SDRi, and subjective MOS than several discriminative and generative baselines, and out-of-domain tests on MUSAN and DEMAND noise report improved robustness. The paper also reports ablations of the corrector, synthesizer, and aligner.

Significance. If the central claim is established, the idea of correcting separated speech in the low-resolution text domain with a single end-to-end ALM, rather than in the audio domain or via ASR+LLM cascades, is a potentially useful contribution to robust speech separation, particularly for unknown noise types. The paper also includes a plausibly efficient pipeline and compares AR and NAR codec-based re-synthesis. However, the current evidence does not yet isolate the contribution of the ALM corrector from the contributions of codec re-synthesis and TF-domain alignment, and the WER evaluation is partially circular because the teacher model that generated the training targets is also the evaluation ASR. The main SOTA comparison is also not matched on training data, since SepALM is trained on a combined three-dataset corpus while most baselines are trained on individual datasets. These gaps are load-bearing for the paper's central claims, but they are addressable with additional experiments.

major comments (4)
  1. [§3.3, §3.5, Table 3] No ablation isolates the ALM text-domain corrector from the codec synthesizer and the TF-domain aligner. In Table 3, Exp. (d) ('Separator only') removes the corrector, synthesizer, and aligner together, while Exp. (b)/(c) include all components. Thus the large SI-SNRi/SDRi jump from 13.2/13.8 to 17.4/17.9 cannot be attributed specifically to text error correction; it could be driven by the codec-LM re-synthesis or by the aligner in Eq. (9), which mixes the already-strong preliminary separated speech back into the output with learned weights alpha_1 and alpha_2. The paper reports no learned alpha values and no control condition where the synthesizer receives Whisper-Tiny's uncorrected transcription of the separated speech. I request an ablation that keeps the synthesizer and aligner but disables correction (e.g., feeding the uncorrected preliminary transcription to the synthesizer), and an ablation with alpha_1=1, alpha_2=0 or with alpha_2=0 to show the aligner is not simply recovering the separator output.
  2. [§3.3, §4.3] The WER evaluation is partially circular. The corrector is trained to match Whisper-Tiny's greedy-decoded transcriptions of clean speech via Eqs. (5) and (6), and the WER metric in §4.3 computes WER against Whisper-Tiny's transcription of the clean reference, using the same Whisper-Tiny to transcribe system output. Consequently, the WER numbers in Table 3 measure agreement with the teacher model rather than an independent measure of intelligibility or accuracy. This does not invalidate the SI-SNRi and SDRi results, but it undermines the paper's claim that the method reduces WER of the separated audio. Please re-evaluate with an ASR model not used in training (e.g., Whisper-Large or wav2vec2.0) or with human transcriptions, and report the second WER column for the generated audio separately.
  3. [Table 1, §4.4] The main SOTA comparison in Table 1 is not matched on training data. The table footnote indicates that only SepALM and MossFormer2* are trained on the combined dataset of three sources, while the other baselines (Conv-TasNet, DPRNN, Wavesplit, SepFormer, MossFormer2, DiffSep) are trained on individual datasets. Since SepALM is trained on roughly three times as much data and on a mixture of noisy and reverberant conditions, the reported SI-SNRi/SDRi advantages may partly reflect a training-data effect rather than the proposed method. The single MossFormer2* row is an insufficient control, especially because its numbers are nearly identical to MossFormer2, which raises questions about how the combined-data training was conducted. Please add baselines trained on the same combined corpus, or evaluate SepALM trained per-dataset, so that the comparison isolates the method.
  4. [§3.5, §4.2] The aligner is described in Eq. (9) as a learned linear combination in the STFT domain, but the paper says 'A detailed exposition of the alignment procedure is provided in the technical appendix' and no appendix is present in the manuscript. The architecture (two-layer CNN with residual connections), loss weighting, and any regularization on alpha_1 and alpha_2 are not described, and no learned alpha values are reported. Since the aligner can in principle improve SI-SNRi merely by re-weighting toward the high-quality preliminary separation, it is essential to describe this component and provide an ablation that quantifies its behavior.
minor comments (5)
  1. [§4.2] The sentence 'We set the maximum sequence length to 1024' is ambiguous: it should state whether 1024 refers to ALM text tokens, DAC acoustic tokens, or audio samples, and how length padding is handled across the four components.
  2. [Table 3] The table caption says the first and second WER values correspond to the corrected transcription and the model's output speech, but the columns show values like '3.76/4.79' and '−/5.68'. Please clarify which side of the slash is which and what the '−' entry in Exp. (d) means (presumably no corrected transcription exists), and consider using separate columns for the two WER readings.
  3. [§3.3] The model is introduced as 'SpeechGPT2' but then referred to as 'SpeechGPT' throughout; please use a consistent name and cite the exact model version used.
  4. [§4.4] The claim that rows 5 and 6 of Table 1 demonstrate that a simple increase in training data did not help would be more convincing with a statistical significance test or multiple seeds, since the differences are within 0.1 dB and baseline numbers are single runs.
  5. [§2, Table 1] TokenSplit is described as a closely related approach in Section 2, but it is not included in the experimental comparison. Given the similarity of predicting refined audio tokens conditioned on speech and transcription, a comparison with TokenSplit (or at least an explanation of its absence) would strengthen the evaluation.

Circularity Check

1 steps flagged · score 4.0 of 10

WER evaluation in Table 3 is measured against Whisper-Tiny, the same model that generated the corrector's training targets; the central SI-SNRi/SDRi claims are independent.

  1. self definitional [Sec. 3.3 (Eqs. 5–6) and Sec. 4.3 (Evaluation Metrics); Table 3 caption]
    "Specifically, we engage Whisper [Radford et al., 2023] to execute greedy decoding on both the preliminary separated speech ˆs and the true clean speech s, yielding the target transcriptions ˆt∗ and ˜t∗, respectively. … We utilized Whisper-Tiny to perform ASR on the separated speech to assess the transcription accuracy. To establish a benchmark, we also applied Whisper-Tiny to the original clean speech, treating the resulting transcription as the true reference."

    Equations 5–6 train the ALM corrector to output Whisper-Tiny's transcription of clean speech (˜t∗) and of separated speech (ˆt∗). The reported WER in Table 3 then scores the corrected transcription against the same function: Whisper-Tiny's transcription of the clean reference. The WER column is therefore a measure of how faithfully the model imitates its teacher, not an independent measure of transcription accuracy; improvement on that column is substantially forced by the training loss. The second WER column inherits the loop because the synthesized speech is conditioned on the teacher-matching transcription. The SI-SNRi/SDRi/PESQi/ESTOIi results do not depend on this loop and remain independent evidence for the separation claims.

full rationale

The only circular loop I can exhibit from the paper's own equations is the WER evaluation: the same Whisper-Tiny model provides both the training targets (Eqs. 5–6) and the reference transcription for WER (Sec. 4.3). That makes the WER improvements in Table 3 partially circular as an accuracy claim, though valid as a distillation-fidelity claim. The paper's strongest separation-quality claims rest on SI-SNRi/SDRi and subjective MOS, which are computed from waveforms against true references and are not inside this loop. The skeptic's missing-control objection—that no ablation removes the ALM corrector while keeping the synthesizer and aligner—is a causal-attribution or experimental-design concern, not a circularity: it does not reduce an output to an input by construction, so I do not score it as circular. No load-bearing self-citation or imported uniqueness argument appears; citations are to standard external systems (Whisper, SpeechGPT, SepFormer, DAC). Overall score 4 reflects one partial circularity affecting one metric while the central separation claims retain independent content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The pipeline relies on several pretrained models and hand-chosen hyperparameters rather than a parameter-free derivation. The most consequential assumptions are the fidelity of the Whisper-Tiny teacher and the adequacy of DAC codec resynthesis for SI-SNR evaluation.

free parameters (4)
  • NAR masked inference steps = 25
    Set in Sec. 4.2. Affects synthesis quality and inference speed; chosen by hand.
  • LoRA rank = 8
    Set in Sec. 4.2. Controls corrector adaptation capacity; chosen by hand.
  • ALM decoding hyperparameters = top-k 40, top-p 0.9, temperature 0.1, beam 1
    Set in Sec. 4.2. Greedy-style decoding choices that affect correction quality.
  • Classifier-free guidance drop probability = 0.1
    Set in Sec. 4.2 for synthesizer training; affects condition adherence.
assumptions (4)
  • domain assumption SpeechGPT can perform ASR and grammatical/contextual error correction on 1,000 HuBERT-unit speech inputs after LoRA fine-tuning.
    Sec. 3.3 relies on the ALM's cross-modal ability to map discrete speech units to text and correct that text.
  • domain assumption Whisper-Tiny greedy decoding is a sufficiently accurate teacher for both separated-speech and clean-speech transcriptions.
    Eqs. 5 and 6 use Whisper-Tiny outputs as ground-truth targets; any teacher error propagates into the corrector.
  • domain assumption The DAC neural codec (B=1024, Q=12) preserves enough information for high-quality re-synthesis and SI-SNR evaluation.
    Sec. 3.4 uses DAC tokens as the synthesis representation; if the codec is too lossy, re-synthesis cannot recover original phase and fidelity.
  • domain assumption The aligner's STFT-domain linear combination (Eq. 9) can compensate for phase shifts without distorting magnitude.
    Sec. 3.5 assumes the learned weights alpha1 and alpha2 fix phase mismatch; the detailed mechanism is only in the missing appendix.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SepALM: Audio Language Models Are Error Correctors for Robust Speech Separation." pith.science (2026). https://pith.science/paper/RIMIJCVG

@misc{pith2026250503273,
  author       = {Pith},
  title        = {Pith review of: SepALM: Audio Language Models Are Error Correctors for Robust Speech Separation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RIMIJCVG}},
  note         = {Machine review of arXiv:2505.03273}
}
read the original abstract

While contemporary speech separation technologies adeptly process lengthy mixed audio waveforms, they are frequently challenged by the intricacies of real-world environments, including noisy and reverberant settings, which can result in artifacts or distortions in the separated speech. To overcome these limitations, we introduce SepALM, a pioneering approach that employs audio language models (ALMs) to rectify and re-synthesize speech within the text domain following preliminary separation. SepALM comprises four core components: a separator, a corrector, a synthesizer, and an aligner. By integrating an ALM-based end-to-end error correction mechanism, we mitigate the risk of error accumulation and circumvent the optimization hurdles typically encountered in conventional methods that amalgamate automatic speech recognition (ASR) with large language models (LLMs). Additionally, we have developed Chain-of-Thought (CoT) prompting and knowledge distillation techniques to facilitate the reasoning and training processes of the ALM. Our experiments substantiate that SepALM not only elevates the precision of speech separation but also markedly bolsters adaptability in novel acoustic environments.

Figures

Figures reproduced from arXiv: 2505.03273 by the authors.

Figure 1
Figure 1. The illustration contrasts (a) the conventional method of error correction for preliminary separated speech in the audio domain, and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The structural framework of our proposed SepALM. For [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline

    eess.AS 2025-05 conditional novelty 5.0 of 10

    A cascaded pipeline of audio compression, latent diffusion extraction, and generative correction achieves state-of-the-art target speech extraction quality and intelligibility on Libri2Mix and out-of-domain data.

Reference graph

Works this paper leans on

44 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [2]

    On synthesis for supervised monaural speech sepa- ration in time domain

    [Chenet al., 2020 ] Jingjing Chen, Qirong Mao, and Dong Liu. On synthesis for supervised monaural speech sepa- ration in time domain. InINTERSPEECH, pages 2627–

    work page 2020

  2. [5]

    w2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre- training

    [Chunget al., 2021 ] Yu-An Chung, Yu Zhang, Wei Han, et al. w2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre- training. InASRU, pages 244–250. IEEE,

    work page 2021

  3. [7]

    Tokensplit: Using discrete speech rep- resentations for direct, refined, and transcript-conditioned speech separation and recognition

    [Erdoganet al., 2023 ] Hakan Erdogan, Scott Wisdom, Xu- ankai Chang, et al. Tokensplit: Using discrete speech rep- resentations for direct, refined, and transcript-conditioned speech separation and recognition. InINTERSPEECH, pages 3462–3466. ISCA,

    work page 2023

  4. [8]

    Prompting large language mod- els with speech recognition abilities

    [Fathullahet al., 2024 ] Yassir Fathullah, Chunyang Wu, Egor Lakomkin, et al. Prompting large language mod- els with speech recognition abilities. InICASSP, pages 13351–13355. IEEE,

    work page 2024

  5. [12]

    Classifier-free diffusion guidance.CoRR, abs/2207.12598,

    [Ho and Salimans, 2022] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.CoRR, abs/2207.12598,

    arXiv 2022

  6. [13]

    Hu, Yelong Shen, Phillip Wallis, et al

    [Huet al., 2022 ] Edward J. Hu, Yelong Shen, Phillip Wallis, et al. Lora: Low-rank adaptation of large language models. InICLR. OpenReview.net,

    work page 2022

  7. [14]

    Listen again and choose the right answer: A new paradigm for automatic speech recognition with large lan- guage models

    [Huet al., 2024 ] Yuchen Hu, Chen Chen, Chengwei Qin, et al. Listen again and choose the right answer: A new paradigm for automatic speech recognition with large lan- guage models. InACL (Findings), pages 666–679. Asso- ciation for Computational Linguistics,

    work page 2024

  8. [15]

    Speak, read and prompt: High- fidelity text-to-speech with minimal supervision.Trans

    [Kharitonovet al., 2023 ] Eugene Kharitonov, Damien Vin- cent, Zal´an Borsos, et al. Speak, read and prompt: High- fidelity text-to-speech with minimal supervision.Trans. Assoc. Comput. Linguistics, 11:1703–1718,

    work page 2023

Show all 44 references
  1. [16]

    High-fidelity audio compression with improved RVQGAN

    [Kumaret al., 2023 ] Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, et al. High-fidelity audio compression with improved RVQGAN. InNeurIPS,

    2023

  2. [17]

    End-to-end speech recognition con- textualization with large language models

    [Lakomkinet al., 2024 ] Egor Lakomkin, Chunyang Wu, Yassir Fathullah, et al. End-to-end speech recognition con- textualization with large language models. InICASSP, pages 12406–12410. IEEE,

    2024

  3. [18]

    Schuller

    [Latifet al., 2023 ] Siddique Latif, Moazzam Shoukat, Fahad Shamshad, Muhammad Usama, Heriberto Cuay´ahuitl, and Bj¨orn W. Schuller. Sparks of large audio models: A survey and outlook.CoRR, abs/2308.12792,

    2023 arXiv

  4. [19]

    A simultaneous denoising and dereverberation framework with target decoupling

    [Liet al., 2021 ] Andong Li, Wenzhe Liu, Xiaoxue Luo, Guochen Yu, Chengshi Zheng, and Xiaodong Li. A simultaneous denoising and dereverberation framework with target decoupling. InInterspeech, pages 2801–2805. ISCA,

    2021

  5. [20]

    Conv-tasnet: Surpassing ideal time-frequency magnitude masking for speech separation.IEEE ACM Trans

    [Luo and Mesgarani, 2019] Yi Luo and Nima Mesgarani. Conv-tasnet: Surpassing ideal time-frequency magnitude masking for speech separation.IEEE ACM Trans. Audio Speech Lang. Process., 27(8):1256–1266,

    2019

  6. [22]

    Separate and diffuse: Using a pretrained diffusion model for better source separation

    [Lutatiet al., 2024 ] Shahar Lutati, Eliya Nachmani, and Lior Wolf. Separate and diffuse: Using a pretrained diffusion model for better source separation. InICLR. OpenRe- view.net,

    2024

  7. [23]

    Whamr!: Noisy and reverberant single-channel speech separation

    [Maciejewskiet al., 2020 ] Matthew Maciejewski, Gordon Wichern, Emmett McQuinn, and Jonathan Le Roux. Whamr!: Noisy and reverberant single-channel speech separation. InICASSP, pages 696–700. IEEE,

    2020

  8. [24]

    Separate in the speech chain: Cross-modal conditional audio-visual target speech extraction

    [Mu and Yang, 2024] Zhaoxi Mu and Xinyu Yang. Separate in the speech chain: Cross-modal conditional audio-visual target speech extraction. InIJCAI, pages 6415–6423. ij- cai.org,

    2024

  9. [25]

    Self-supervised disentangled representation learning for robust target speech extraction

    [Muet al., 2024 ] Zhaoxi Mu, Xinyu Yang, Sining Sun, and Qing Yang. Self-supervised disentangled representation learning for robust target speech extraction. InAAAI, pages 18815–18823. AAAI Press,

    2024

  10. [26]

    To- wards real-time single-channel speech separation in noisy and reverberant environments

    [Neri and Braun, 2023] Julian Neri and Sebastian Braun. To- wards real-time single-channel speech separation in noisy and reverberant environments. InICASSP, pages 1–5. IEEE,

    2023

  11. [27]

    GPT-4 technical report.CoRR, abs/2303.08774,

    [OpenAI, 2023] OpenAI. GPT-4 technical report.CoRR, abs/2303.08774,

    2023 arXiv

  12. [28]

    Librispeech: An ASR corpus based on public domain audio books

    [Panayotovet al., 2015 ] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: An ASR corpus based on public domain audio books. In ICASSP, pages 5206–5210. IEEE,

    2015

  13. [30]

    Whispering llama: A cross-modal generative error correction frame- work for speech recognition

    [Radhakrishnanet al., 2023 ] Srijith Radhakrishnan, Chao- Han Huck Yang, Sumeer Ahmad Khan, et al. Whispering llama: A cross-modal generative error correction frame- work for speech recognition. InEMNLP, pages 10007– 10016. Association for Computational Linguistics,

    2023

  14. [31]

    Reading to listen at the cocktail party: Multi-modal speech separation

    [Rahimiet al., 2022 ] Akam Rahimi, Triantafyllos Afouras, and Andrew Zisserman. Reading to listen at the cocktail party: Multi-modal speech separation. InCVPR, pages 10483–10492. IEEE,

    2022

  15. [32]

    [Rouxet al., 2019 ] Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, and John R. Hershey. SDR - half-baked or well done? InICASSP, pages 626–630. IEEE,

    2019

  16. [33]

    Diffusion-based generative speech source separa- tion

    [Scheibleret al., 2023 ] Robin Scheibler, Youna Ji, Soo- Whan Chung, Jaeuk Byun, Soyeon Choe, and Min-Seok Choi. Diffusion-based generative speech source separa- tion. InICASSP, pages 1–5. IEEE,

    2023

  17. [34]

    MUSAN: A music, speech, and noise cor- pus.CoRR, abs/1510.08484,

    [Snyderet al., 2015 ] David Snyder, Guoguo Chen, and Daniel Povey. MUSAN: A music, speech, and noise cor- pus.CoRR, abs/1510.08484,

    2015 arXiv

  18. [35]

    Attention is all you need in speech separation

    [Subakanet al., 2021 ] Cem Subakan, Mirco Ravanelli, Samuele Cornell, Mirko Bronzi, and Jianyuan Zhong. Attention is all you need in speech separation. InICASSP, pages 21–25. IEEE,

    2021

  19. [36]

    Llama: Open and efficient foundation language models.CoRR, abs/2302.13971,

    [Touvronet al., 2023 ] Hugo Touvron, Thibaut Lavril, Gau- tier Izacard, et al. Llama: Open and efficient foundation language models.CoRR, abs/2302.13971,

    2023 arXiv

  20. [37]

    Neural codec language models are zero-shot text to speech synthesizers.CoRR, abs/2301.02111,

    [Wanget al., 2023 ] Chengyi Wang, Sanyuan Chen, Yu Wu, et al. Neural codec language models are zero-shot text to speech synthesizers.CoRR, abs/2301.02111,

    2023 arXiv

  21. [38]

    Noise-robust speech separation with fast genera- tive correction.CoRR, abs/2406.07461,

    [Wanget al., 2024a ] Helin Wang, Jesus Villalba, Laureano Moro-Velazquez, Jiarui Hai, Thomas Thebaud, and Najim Dehak. Noise-robust speech separation with fast genera- tive correction.CoRR, abs/2406.07461,

    arXiv

  22. [39]

    Maskgct: Zero-shot text-to-speech with masked generative codec transformer.CoRR, abs/2409.00750,

    [Wanget al., 2024b ] Yuancheng Wang, Haoyue Zhan, Liwei Liu, et al. Maskgct: Zero-shot text-to-speech with masked generative codec transformer.CoRR, abs/2409.00750,

    arXiv

  23. [40]

    Chain-of-thought prompting elicits reasoning in large language models

    [Weiet al., 2022 ] Jason Wei, Xuezhi Wang, Dale Schuur- mans, et al. Chain-of-thought prompting elicits reasoning in large language models. InNeurIPS,

    2022

  24. [41]

    Wham!: Extending speech sepa- ration to noisy environments

    [Wichernet al., 2019 ] Gordon Wichern, Joe Antognini, Michael Flynn, et al. Wham!: Extending speech sepa- ration to noisy environments. InINTERSPEECH, pages 1368–1372. ISCA,

    2019

  25. [42]

    Wavesplit: End-to-end speech separation by speaker clustering.IEEE ACM Trans

    [Zeghidour and Grangier, 2021] Neil Zeghidour and David Grangier. Wavesplit: End-to-end speech separation by speaker clustering.IEEE ACM Trans. Audio Speech Lang. Process., 29:2840–2849,

    2021

  26. [43]

    Mossformer2: Combining transformer and rnn- free recurrent network for enhanced time-domain monau- ral speech separation

    [Zhaoet al., 2024 ] Shengkui Zhao, Yukun Ma, Chongjia Ni, et al. Mossformer2: Combining transformer and rnn- free recurrent network for enhanced time-domain monau- ral speech separation. InICASSP, pages 10356–10360. IEEE,

    2024

  27. [44]

    Neural target speech extraction: An overview.IEEE Signal Process

    [Zmol´ıkov´aet al., 2023 ] Katerina Zmol ´ıkov´a, Marc Del- croix, Tsubasa Ochiai, Keisuke Kinoshita, Jan Cernock ´y, and Dong Yu. Neural target speech extraction: An overview.IEEE Signal Process. Mag., 40(3):8–29, 2023

    2023

  28. [2014]

    Hershey, Zhuo Chen, Jonathan Le Roux, and Shinji Watanabe

    [Hersheyet al., 2016 ] John R. Hershey, Zhuo Chen, Jonathan Le Roux, and Shinji Watanabe. Deep cluster- ing: Discriminative embeddings for segmentation and separation. InICASSP, pages 31–35. IEEE,

    2016

  29. [2015]

    Robust speech recognition via large-scale weak supervision

    [Radfordet al., 2023 ] Alec Radford, Jong Wook Kim, Tao Xu, et al. Robust speech recognition via large-scale weak supervision. InICML, volume 202 ofProceedings of Machine Learning Research, pages 28492–28518. PMLR,

    2023

  30. [2016]

    Diffusion-based signal refiner for speech separation

    [Hiranoet al., 2023 ] Masato Hirano, Kazuki Shimada, Yuichiro Koyama, Shusuke Takahashi, and Yuki Mitsu- fuji. Diffusion-based signal refiner for speech separation. CoRR, abs/2305.05857,

    2023

  31. [2019]

    Dual-path RNN: efficient long sequence model- ing for time-domain single-channel speech separation

    [Luoet al., 2020 ] Yi Luo, Zhuo Chen, and Takuya Yosh- ioka. Dual-path RNN: efficient long sequence model- ing for time-domain single-channel speech separation. In ICASSP, pages 46–50. IEEE,

    2020

  32. [2020]

    Wavlm: Large-scale self- supervised pre-training for full stack speech processing

    [Chenet al., 2022 ] Sanyuan Chen, Chengyi Wang, Zhengyang Chen, et al. Wavlm: Large-scale self- supervised pre-training for full stack speech processing. IEEE J. Sel. Top. Signal Process., 16(6):1505–1518,

    2022

  33. [2021]

    Librimix: An open-source dataset for generalizable speech separation.CoRR, abs/2005.11262,

    [Cosentinoet al., 2020 ] Joris Cosentino, Manuel Pariente, Samuele Cornell, Antoine Deleforge, and Emmanuel Vin- cent. Librimix: An open-source dataset for generalizable speech separation.CoRR, abs/2005.11262,

    2020 arXiv

  34. [2022]

    Hyporadise: An open baseline for genera- tive speech recognition with large language models

    [Chenet al., 2023 ] Chen Chen, Yuchen Hu, Chao-Han Huck Yang, et al. Hyporadise: An open baseline for genera- tive speech recognition with large language models. In NeurIPS,

    2023

  35. [2023]

    Soundstorm: Efficient parallel audio generation.CoRR, abs/2305.09636,

    [Borsoset al., 2023b ] Zal´an Borsos, Matthew Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. Soundstorm: Efficient parallel audio generation.CoRR, abs/2305.09636,

    arXiv

  36. [2024]

    Multichannel audio database in var- ious acoustic environments

    [Hadadet al., 2014 ] Elior Hadad, Florian Heese, Peter Vary, and Sharon Gannot. Multichannel audio database in var- ious acoustic environments. InIWAENC, pages 313–317. IEEE,

    2014

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.