Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

A single 63M-parameter autoregressive language model can unify speech restoration, target speaker extraction, and speech separation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 08:25 UTC pith:ZIDM5G2S

load-bearing objection Solid multi-task LM speech enhancement with a clean task-token design, but the abstract promises a progressive RL strategy that never appears in the methods. the 3 major comments →

arxiv 2510.20441 v2 pith:ZIDM5G2S submitted 2025-10-23 cs.SD cs.AI

UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement

Haoyin Yan , Chengwei Liu , Shaofei Xue , Xiaotao Liang , Yinghao Liu , Yuxiang Kong , Zheng Xue This is my paper
classification cs.SD cs.AI
keywords speech enhancementautoregressive language modeldecoder-onlyspeech restorationtarget speaker extractionspeech separationneural audio codec
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that a single decoder-only autoregressive language model can perform three related speech-enhancement tasks—restoring clean speech from degraded recordings, extracting a target speaker from a mixture, and separating all speakers—using one set of weights and one training objective. The key move is to treat enhancement as next-token prediction over discrete audio tokens produced by a neural codec, with a small set of task tokens switching the model between behavioral modes. If correct, it means the unification machinery of language models transfers to low-level audio reconstruction, not just content generation. The paper reports that a 63M-parameter model matches or beats larger and task-specific baselines on standard benchmarks, and that multi-task training does not hurt single-task quality.

Core claim

The paper's central claim is that the three sub-tasks can be composed from three operational modes—speech restoration (SR), target speaker extraction (TSE), and reverse TSE (rTSE)—each marked by a learnable task token in the input prefix. The model extracts continuous features from degraded and reference audio with a frozen self-supervised encoder plus a trainable adapter, and autoregressively predicts BiCodec's global and semantic discrete tokens of the desired output speech; a codec decoder reconstructs the waveform. For two-speaker separation, the paper chains the modes: SR isolates the louder speaker, TSE re-extracts that speaker consistently across segments, and rTSE yields the remainin

What carries the argument

The load-bearing mechanism is conditional autoregressive next-token prediction on discrete audio tokens. The input to the decoder-only transformer is a prefix built from a task token, start markers, and continuous features of degraded and reference speech produced by a frozen self-supervised speech encoder with a trainable linear adapter. The target is the sequence of BiCodec tokens—32 global tokens that carry speaker identity and about 50 per-second semantic tokens that carry content—and optimizing the cross-entropy of these tokens is the entire training signal. At inference, a speech-restoration job uses only the degraded-audio prefix, target-speaker extraction adds a reference-audio prefi

Load-bearing premise

Everything rests on BiCodec's discrete tokens being a faithful enough representation of clean target speech that maximizing token likelihood is the same as maximizing perceived speech quality; if the codec throws away important acoustic detail, the language model can never recover it.

What would settle it

Reconstruct a clean utterance by encoding and decoding it with BiCodec alone and score it with the same metrics used in the paper (DNSMOS, NISQA, UTMOS). If that codec-only reconstruction already outscores UniSE's outputs on the same material, then the LM adds little and the reported gains come from the codec; if the codec-only bound is clearly below UniSE's outputs, the model is genuinely learning enhancement beyond the codec.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A single 63M-parameter autoregressive language model can match or exceed both discriminative and generative baselines on speech restoration, so model scale is not the only path to competitive quality.
  • Multi-task unification via task tokens and mode-switching carries no observable performance penalty: the task-specific variants score nearly identically to the unified model.
  • Speech separation can be achieved without a dedicated separation architecture by composing SR, TSE, and inverse-TSE inferences on segmented audio.
  • The codec is a ceiling on attainable quality: swapping in a lower-fidelity codec degrades results, while swapping the LM backbone changes little, so future gains will come from better discrete speech representations.
  • Because the framework generalizes to unseen distortions (codec artifacts, wind noise), the conditional autoregressive formulation learns robust mappings rather than memorizing training distortions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the task-token interface is truly compositional, other audio-to-audio tasks—dereverberation-only, bandwidth extension, denoising with explicit noise-type prompts—could be added by new tokens without retraining the base from scratch; the paper only demonstrates three modes, but the design invites extension.
  • A direct test of the codec-bottleneck hypothesis would be to measure BiCodec's own reconstruction ceiling on the clean test signals; if UniSE's outputs approach that ceiling, then the LM is doing near-maximal work given its interface, and better codecs become a clear, linearly additive improvement.
  • The sequential SR→TSE→rTSE inference for separation may scale to more than two speakers by iterating, but the paper restricts itself to the two-speaker case, so the iterative behavior (error accumulation, speaker identity drift) is an open empirical question.
  • Since reference speech is only used in TSE/rTSE modes, the same backbone could be trained with imperfect or noisy reference audio to probe how well the model trades speaker-identity fidelity against content quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes UniSE, a decoder-only autoregressive language model framework for speech enhancement that unifies speech restoration (SR), target speaker extraction (TSE), and speech separation (SS). A frozen WavLM with a trainable adapter produces conditioning features from degraded/reference speech; a BiCodec supplies discrete global and semantic target tokens; and a 63M-parameter LLaMA-style backbone predicts these tokens autoregressively. Task-specific tokens define SR, TSE, and reverse-TSE modes, and SS is handled by sequential multi-mode inference. The paper reports DNSMOS, NISQA, and UTMOS results on DNS 2020, URGENT 2025, Libri2Mix, and WSJ0-2mix, claiming competitive or superior performance relative to discriminative and generative baselines, and includes ablations on LM backbone and codec choice.

Significance. If the reported results are reproducible, the paper makes a useful empirical contribution: it demonstrates that a relatively small decoder-only AR LM (63M parameters) can jointly handle several speech enhancement sub-tasks with competitive non-intrusive metrics, and it introduces a practical multi-mode inference strategy for separation. The paper is strengthened by the breadth of training distortions, the comparison against a much larger generative baseline (LLaSE-G1, ~1B parameters), and the explicit ablations on LM backbone and codec. The authors are also candid in Table 6 about the codec bottleneck. However, the manuscript's central empirical claim is currently not fully verifiable because the abstract promises a progressive reinforcement learning strategy that is absent from the methodology, and the point estimates in the tables are not accompanied by any variance or significance information.

major comments (3)
  1. [Abstract and §2.3/§3.1] The abstract states: "To further optimize speech quality, we introduce a progressive reinforcement learning strategy with multiple assessment criteria." Yet §2.3 defines only the negative log-likelihood losses in Eqs. (1) and (2), and §3.1 describes only AdamW, 30 epochs, and a learning-rate schedule. There is no reward function, policy update, progressive training schedule, or any other RL component anywhere in the method or implementation. If RL was used to obtain Tables 2–6, the training objective is incompletely specified; if it was not used, the abstract is factually incorrect. Either way, the current manuscript does not specify the exact model that produced the reported results, which is a load-bearing reproducibility gap.
  2. [Tables 2 and 5] The SOTA claims rest on very small DNSMOS differences. For example, Table 2 shows UniSE OVRL 3.40 versus LLaSE-G1 3.33 on the With Reverb subset, and Table 5 shows improvements of about 0.1–0.2 OVRL over MossFormer2. No confidence intervals, standard deviations, multiple-seed results, or significance tests are provided, and DNSMOS is known to be noisy at the 0.01–0.05 scale. The absence of any variance information makes it difficult to assess whether the observed differences are meaningful. Please add error bars or significance tests, or at minimum report scores across multiple checkpoints/seeds.
  3. [§2.2 and Table 6] The framework's ceiling is set by the BiCodec discrete token representation, since the LM can only rearrange or predict tokens that the codec can reconstruct. The paper itself provides evidence of this bottleneck: in Table 6, replacing BiCodec with X-codec2 lowers OVRL by roughly 0.1 on both DNS subsets, and in Table 4 the BiCodec upper bound on clean target speech already yields SIG 3.59 / OVRL 3.30, close to UniSE's 3.62 / 3.33. The paper acknowledges this in a sentence around Table 6, but it is a central limitation that should be analyzed more explicitly: how much of UniSE's output quality comes from the codec's reconstruction ability, and what headroom remains if the codec improves? Please add a discussion of this codec-bound effect, ideally with a direct comparison to coding the degraded input or an oracle-token experiment.
minor comments (5)
  1. [Abstract/footnote and §1] Two inconsistent GitHub URLs are given: the abstract refers to github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE, while the footnote in the abstract/first page and the demo statement refer to github.com/hyyan2k/UniSE. No commit hash is provided. Please unify the URLs and pin a commit for reproducibility.
  2. [§3.1] The description of UniSE-SR and UniSE-TSE variants is terse. Please state explicitly how these variants differ from the full model during training (e.g., only the SR task token is used, or only TSE/rTSE modes are sampled) and report their training setup.
  3. [§3.2 / Table 3] The claim of generalization to "unseen distortions (codec artifacts and wind noise)" is not directly supported in the table. Please specify which URGENT 2025 test-set conditions are unseen relative to the training simulation in Table 1, and how the corresponding degradation types are identified.
  4. [§2.4] The SS inference procedure uses three sequential passes (SR, TSE, rTSE). The computational cost and potential error propagation of this multi-stage inference are not discussed; a sentence on runtime and failure modes would help readers assess practical deployment.
  5. [General] Some table formatting issues: Table 1 omits probability for interference speaker in TSE/rTSE rows in a way that could confuse readers (the value 1.0 is placed across two columns). Consider making the table layout clearer.

Circularity Check

0 steps flagged

No significant circularity: the framework is an empirical system trained on simulated data and tested on held-out benchmarks; the only flagged issue is an abstract claim of progressive RL that never appears in the methods, which is a reproducibility gap rather than a circular derivation.

full rationale

UniSE is an empirical systems paper rather than a derivation from first principles. The training objectives are explicit cross-entropy losses (Eq. 1 and Eq. 2) over BiCodec discrete tokens, with no fitted constants masquerading as predictions and no result that reduces to its inputs by construction. Evaluation is against external benchmarks (DNSMOS, NISQA, UTMOS, SIM) and baselines, and ablations (Table 6) vary LM backbone and codec, while Table 4 includes a BiCodec-only baseline that honestly exposes the codec bottleneck. The only self-citation is reference [3], co-authored by Haoyin Yan, but it is used solely as background for a broadened definition of speech enhancement and is not load-bearing. Per the reviewing rule, I flag the following missing-support inconsistency: the abstract states 'we introduce a progressive reinforcement learning strategy with multiple assessment criteria,' yet Section 2 defines only LSR and LTSE, and Section 3.1 describes only 'AdamW optimizer with 30 epochs' — no reward, policy, or RL procedure is specified. This undermines reproducibility but is not circularity: no claimed prediction is equivalent to an input by construction. Hence the low circularity score.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 1 invented entities

The central claim relies on a set of domain assumptions about codec sufficiency, WavLM feature informativeness, metric validity, and simulation-to-real transfer. The hand-tuned distortion simulation parameters in Table 1 are free parameters that shape the training distribution. The only invented entities are the task tokens, which are standard prompt-style conditioning and have no independent evidence outside this paper.

free parameters (3)
  • Distortion sampling probabilities (noise 0.8, reverb/clipping/bandwidth/packet-loss 0.3, interference 0.2 for SR, 1.0 fo = Table 1: probabilities as listed
    These hand-chosen probabilities define the simulated training distribution and directly affect all reported downstream metrics. No sensitivity analysis is provided.
  • Distortion severity ranges (SNR, SIR, bandwidth, packet-loss rate, clipping quantiles) = SNR [-5,20], SIR [2,20] for SR and [-5,5] for TSE/rTSE, bandwidth {2,4} kHz, packet loss [0.05,0.25], clipping quantiles
    Chosen by hand in Table 1; these ranges set task difficulty and generalization behavior, but the paper does not explore alternatives.
  • Model hyperparameters (12 layers, 8 heads, hidden 512, 5-second segments, 30 epochs, lr 0.001) = 63M parameters
    Architecture and training hyperparameters are stated without a search or ablation. They are design choices that the central multi-task claim depends on.
axioms (5)
  • domain assumption BiCodec's discrete tokens (global and semantic) are a sufficient representation of target speech for high-quality waveform reconstruction.
    Invoked in Section 2.2; the LM predicts these tokens and the BiCodec decoder reconstructs speech, so any information lost by the codec is unrecoverable.
  • domain assumption Frozen WavLM layer-averaged features, after a linear adapter, contain enough acoustic and semantic information to condition speech enhancement for degraded and reference inputs.
    Section 2.1 assumes WavLM features are sufficient for SR, TSE, and SS; no analysis of feature sufficiency is provided.
  • domain assumption Non-intrusive metrics (DNSMOS, NISQA, UTMOS, WavLM-based SIM) adequately measure perceptual quality and speaker similarity for comparing enhancement systems.
    Section 3.1 uses these metrics as the only evaluation instruments, and the paper's 'competitive' claims are based entirely on them.
  • domain assumption The synthetic training degradation pipeline (Table 1) transfers to the real test sets, including the unseen distortions (codec artifacts, wind noise) in the URGENT Challenge.
    Section 3.2 claims generalization to unseen distortions based on this assumption, but no analysis of domain gap is included.
  • domain assumption Autoregressive cross-entropy training on discrete token sequences is an appropriate surrogate objective for speech enhancement quality.
    Section 2.3 reduces enhancement to next-token prediction; this assumes token-level likelihood correlates with perceptual quality.
invented entities (1)
  • Task-specific learnable tokens T_SR, T_TSE, T_rTSE no independent evidence
    purpose: Switch the decoder-only LM between speech restoration, target speaker extraction, and reverse target speaker extraction modes, enabling multi-task unification.
    These tokens are internal learnable embeddings introduced by this paper. Their necessity is not separately validated: the paper shows UniSE-SR and UniSE-TSE, but does not ablate task tokens away entirely or show they beat a non-token conditioning baseline.

pith-pipeline@v1.3.0-alltime-deepseek · 7974 in / 9938 out tokens · 101465 ms · 2026-08-04T08:25:32.777554+00:00 · methodology

0 comments
read the original abstract

Neural audio codecs have largely promoted the application of language models (LMs) for speech applications. However, the effectiveness of autoregressive LM-based models in unifying speech enhancement (SE) tasks remains underexplored. In this work, we propose UniSE, a unified decoder-only LM-based framework to handle different SE tasks including speech restoration, target speaker extraction, and speech separation. Conditioned on input speech features, it autoregressively generates target discrete tokens, facilitating compatibility between distinct learning patterns of multiple tasks. To further optimize speech quality, we introduce a progressive reinforcement learning strategy with multiple assessment criteria. Experiments on several benchmarks show that UniSE achieves competitive performance compared to discriminative and generative baselines, demonstrating the capacity of LMs in unifying SE tasks. The code and demo are available at: https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Post-Training Speech Enhancement Language Models with Perceptual Rewards

    cs.LG 2026-06 unverdicted novelty 6.0

    Post-training autoregressive speech enhancement LMs via GSPO with composite perceptual rewards from DNSMOS, WER, and UTMOS reaches SOTA on DNS2020 and outperforms single-metric variants in human evaluation.

  2. Reducing Linguistic Hallucination in LM-Based Speech Enhancement via Noise-Invariant Acoustic-Semantic Distillation

    eess.AS 2026-05 unverdicted novelty 6.0

    L3-SE reduces linguistic hallucination in LM-based speech enhancement by distilling noise-invariant acoustic-semantic representations from noisy inputs to condition an autoregressive decoder-only language model.

  3. Absorbing Discrete Diffusion for Speech Enhancement

    cs.SD 2026-02 conditional novelty 5.0

    ADDSE performs speech enhancement by absorbing discrete diffusion over neural audio codec tokens, reaching competitive non-intrusive quality and low-SNR robustness in few sampling steps.

  4. GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model

    eess.AS 2025-12 conditional novelty 5.0

    A two-stage decoder-only language model with continuous embeddings and UTMOS-based preference fine-tuning reports improved target-speaker-extraction scores on Libri2Mix.

Reference graph

Works this paper leans on

49 extracted references · 14 linked inside Pith · cited by 4 Pith papers

  1. [1]

    INTRODUCTION In recent years, the concept of speech enhancement (SE) has become broader, ranging from conventional denoising into re- constructing clean target speech from degraded recordings [1, 2, 3]. In this context, SE can include many sub-tasks: speech restoration (SR) that aims to restore speech from the degraded recording with various distortions; ...

    Pith/arXiv arXiv 2025

  2. [2]

    With Reverb

    METHODOLOGY An overview of the proposed UniSE is illustrated in Fig. 1, comprising WavLM with adapter to extract continuous speech feature, a discrete speech codec to produce discrete tokens and reconstruct waveforms and a decoder-only LM backbone to model conditional probability. 2.1. Conditional Feature Extractor To extract features from reference and d...

    2020

  3. [3]

    EXPERIMENTS 3.1. Experimental Setup Datasets:The clean speech data for training is sourced from the V oxBox dataset [10], which integrates multiple publicly available speech datasets after rigorous data cleaning. Our training set contains 760 hours of LibriSpeech [22] data, 1200 hours from the MLS English [23] subset, and 1800 hours of the Emilia ZH [24] ...

    2020

  4. [4]

    UniSE adopts contin- uous features of the degraded speech and inference speech as conditions to generate discrete tokens of target speech via AR modeling

    CONCLUSION In this work, we proposed an SE framework called UniSE, which unifies SR, TSE and SS tasks. UniSE adopts contin- uous features of the degraded speech and inference speech as conditions to generate discrete tokens of target speech via AR modeling. Multiple operational modes are defined by the task token, enabling task unification by the switchin...

  5. [5]

    V oicefixer: Toward general speech restoration with neural vocoder,

    H. Liu, Q. Kong, Q. Tian, et al., “V oicefixer: Toward general speech restoration with neural vocoder,”arXiv preprint arXiv:2109.13731, 2021

    Pith/arXiv arXiv 2021

  6. [6]

    Universal speech enhancement with score-based diffusion,

    J. Serr `a, S. Pascual, J. Pons, et al., “Universal speech enhancement with score-based diffusion,”arXiv preprint arXiv:2206.03065, 2022

    Pith/arXiv arXiv 2022

  7. [7]

    A composite predictive-generative ap- proach to monaural universal speech enhancement,

    J. Zhang, H. Yan, and X. Li, “A composite predictive-generative ap- proach to monaural universal speech enhancement,”IEEE Trans. Au- dio, Speech, Lang. Process., vol. 33, pp. 2312–2325, 2025

    2025

  8. [8]

    Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,

    Y . Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 27, no. 8, pp. 1256–1266, 2019

    2019

  9. [9]

    CMGAN: Conformer-based metric-gan for monaural speech enhancement,

    S. Abdulatif, R.-Z. Cao, and B. Yang, “CMGAN: Conformer-based metric-gan for monaural speech enhancement,”IEEE/ACM Trans. Au- dio, Speech, Lang. Process., vol. 32, pp. 2477–2493, 2024

    2024

  10. [10]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction,

    K. Tian, Y . Jiang, Z. Yuan, et al., “Visual autoregressive modeling: Scalable image generation via next-scale prediction,” inProc. NeurIPS, 2024, vol. 37, pp. 84839–84865

    2024

  12. [12]

    AudioGen: Textually guided audio generation,

    F. Kreuk, G. Synnaeve, A. Polyak, et al., “AudioGen: Textually guided audio generation,” inProc. ICLR, 2023

    2023

  13. [13]

    Audiobox: Unified audio genera- tion with natural language prompts,

    A. Vyas, B. Shi, M. Le, et al., “Audiobox: Unified audio genera- tion with natural language prompts,”arXiv preprint arXiv:2312.15821, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Spark-TTS: An efficient llm- based text-to-speech model with single-stream decoupled speech to- kens,

    X. Wang, M. Jiang, Z. Ma, et al., “Spark-TTS: An efficient llm- based text-to-speech model with single-stream decoupled speech to- kens,”arXiv preprint arXiv:2503.01710, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    SELM: Speech enhancement using discrete tokens and language models,

    Z. Wang, X. Zhu, Z. Zhang, et al., “SELM: Speech enhancement using discrete tokens and language models,” inProc. ICASSP, 2024, pp. 11561–11565

    2024

  16. [16]

    WavLM: Large-scale self- supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, et al., “WavLM: Large-scale self- supervised pre-training for full stack speech processing,”IEEE J. Sel. Top. Signal Process., vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  17. [17]

    GenSE: Generative speech enhance- ment via language models using hierarchical modeling,

    J. Yao, H. Liu, C. CHEN, et al., “GenSE: Generative speech enhance- ment via language models using hierarchical modeling,” inProc. ICLR, 2025

    2025

  18. [18]

    LauraTSE: Target speaker extraction using auto-regressive decoder-only language models,

    B. Tang, B. Zeng, and M. Li, “LauraTSE: Target speaker extraction using auto-regressive decoder-only language models,”arXiv preprint arXiv:2504.07402, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Conformer: Local features coupling global representations for recognition and detection,

    Z. Peng, Z. Guo, W. Huang, et al., “Conformer: Local features coupling global representations for recognition and detection,”IEEE Trans. Pat- tern Anal. Mach. Intell., vol. 45, no. 8, pp. 9454–9468, 2023

    2023

  20. [20]

    MaskSR: Masked language model for full-band speech restoration,

    X. Li, Q. Wang, and X. Liu, “MaskSR: Masked language model for full-band speech restoration,” inProc. Interspeech, 2024, pp. 2275– 2279

    2024

  21. [21]

    MaskGIT: Masked generative image transformer,

    H. Chang, H. Zhang, L. Jiang, et al., “MaskGIT: Masked generative image transformer,” inProc. CVPR, 2022, pp. 11305–11315

    2022

  22. [22]

    LLaSE-G1: Incentivizing gen- eralization capability for LLaMA-based speech enhancement,

    B. Kang, X. Zhu, Z. Zhang, et al., “LLaSE-G1: Incentivizing gen- eralization capability for LLaMA-based speech enhancement,”arXiv preprint arXiv:2503.00493, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    LLaMA: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, et al., “LLaMA: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    FRCRN: Boosting fea- ture representation using frequency recurrence for monaural speech en- hancement.,

    S. Zhao, B. Ma, K. N. Watcharasupat, et al., “FRCRN: Boosting fea- ture representation using frequency recurrence for monaural speech en- hancement.,” inProc. ICASSP, 2022, pp. 9281–9285

    2022

  25. [25]

    AnyEnhance: A unified generative model with prompt-guidance and self-critic for voice enhancement,

    J. Zhang, J. Yang, Z. Fang, et al., “AnyEnhance: A unified generative model with prompt-guidance and self-critic for voice enhancement,” arXiv preprint arXiv:2501.15417, 2025

    arXiv 2025

  26. [26]

    Librispeech: An asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, et al., “Librispeech: An asr corpus based on public domain audio books,” inProc. ICASSP, 2015, pp. 5206–5210

    2015

  27. [27]

    MLS: A large-scale multilingual dataset for speech research.,

    V . Pratap, Q. Xu, A. Sriram, et al., “MLS: A large-scale multilingual dataset for speech research.,” inProc. Interspeech, 2020, pp. 2757– 2761

    2020

  28. [28]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation.,

    H. He, Z. Shang, C. Wang, et al., “Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation.,” inProc. SLT, 2024, pp. 885–890

    2024

  29. [29]

    The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,

    C. K. A. Reddy, V . Gopal, R. Cutler, et al., “The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,” inProc. Interspeech, 2020, pp. 2492–2496

    2020

  30. [30]

    FSD50K: An open dataset of human-labeled sound events.,

    E. Fonseca, X. Favory, J. Pons, et al., “FSD50K: An open dataset of human-labeled sound events.,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 30, pp. 829–852, 2022

    2022

  31. [31]

    WHAM!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, et al., “WHAM!: Extending speech separation to noisy environments,” inProc. Interspeech, 2019, pp. 1368–1372

    2019

  32. [32]

    Sound event detection in domestic environments with weakly labeled data and soundscape syn- thesis,

    N. Turpault, R. Serizel, J. Salamon, et al., “Sound event detection in domestic environments with weakly labeled data and soundscape syn- thesis,” inProc. DCASE, M. I. Mandel, J. Salamon, and D. P. W. Ellis, Eds., 2019, pp. 253–257

    2019

  33. [33]

    The diverse environments multi- channel acoustic noise database: A database of multichannel environ- mental noise recordings,

    J. Thiemann, N. Ito, and E. Vincent, “The diverse environments multi- channel acoustic noise database: A database of multichannel environ- mental noise recordings,”J. Acoust. Soc. Am., vol. 133, pp. 3591–3591, 2013

    2013

  34. [34]

    MUSAN: A music, speech, and noise corpus,

    D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,”arXiv preprint arXiv:1510.08484, 2015

    Pith/arXiv arXiv 2015

  35. [35]

    DNN-based mask estimation for distributed speech enhancement in spatially unconstrained microphone arrays,

    N. Furnon, R. Serizel, S. Essid, et al., “DNN-based mask estimation for distributed speech enhancement in spatially unconstrained microphone arrays,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 2310–2323, 2021

    2021

  36. [36]

    MUSDB18-HQ - an uncom- pressed version of MUSDB18,

    Z. Rafii, A. Liutkus, F.-R. St ¨oter, et al., “MUSDB18-HQ - an uncom- pressed version of MUSDB18,” [Online], Available: https://doi.org/10. 5281/zenodo.3338373

  37. [37]

    A multi-device dataset for urban acoustic scene classification.,

    A. Mesaros, T. Heittola, and T. Virtanen, “A multi-device dataset for urban acoustic scene classification.,” inProc. DCASE, 2018, pp. 9–13

    2018

  38. [38]

    Interspeech 2025 URGENT speech enhancement challenge,

    K. Saijo, W. Zhang, S. Cornell, et al., “Interspeech 2025 URGENT speech enhancement challenge,”arXiv preprint arXiv:2505.23212, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    DNSMOS P.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,

    C. K. A. Reddy, V . Gopal, and R. Cutler, “DNSMOS P.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,” inProc. ICASSP, 2022, pp. 886–890

    2022

  40. [40]

    NISQA: A deep CNN-self- attention model for multidimensional speech quality prediction with crowdsourced datasets,

    G. Mittag, B. Naderi, A. Chehadi, et al., “NISQA: A deep CNN-self- attention model for multidimensional speech quality prediction with crowdsourced datasets,” inProc. Interspeech, 2021, pp. 2127–2131

    2021

  41. [41]

    UTMOS: Utokyo-sarulab system for V oiceMOS challenge 2022,

    T. Saeki, D. Xin, W. Nakata, et al., “UTMOS: Utokyo-sarulab system for V oiceMOS challenge 2022,” inProc. Interspeech, 2022, pp. 4521– 4525

    2022

  42. [42]

    SpEx+: A complete time domain speaker extraction network,

    M. Ge, C. Xu, L. Wang, et al., “SpEx+: A complete time domain speaker extraction network,” inProc. Interspeech, 2020, pp. 1406– 1410

    2020

  43. [43]

    WeSep: A scalable and flexible toolkit towards generalizable target speaker extraction,

    S. Wang, K. Zhang, S. Lin, et al., “WeSep: A scalable and flexible toolkit towards generalizable target speaker extraction,” inProc. Inter- speech, 2024, pp. 4273–4277

    2024

  44. [44]

    TSELM: Target speaker extrac- tion using discrete tokens and language models,

    B. Tang, B. Zeng, and M. Li, “TSELM: Target speaker extrac- tion using discrete tokens and language models,”arXiv preprint arXiv:2409.07841, 2024

    Pith/arXiv arXiv 2024

  45. [45]

    Attention is all you need in speech separation.,

    C. Subakan, M. Ravanelli, S. Cornell, et al., “Attention is all you need in speech separation.,” inProc. ICASSP, 2021, pp. 21–25

    2021

  46. [46]

    MossFormer2: Combining Transformer and RNN-free recurrent network for enhanced time-domain monaural speech separation.,

    S. Zhao, Y . Ma, C. Ni, et al., “MossFormer2: Combining Transformer and RNN-free recurrent network for enhanced time-domain monaural speech separation.,” inProc. ICASSP, 2024, pp. 10356–10360

    2024

  47. [47]

    Qwen2 technical report,

    A. Yang, B. Yang, B. Hui, et al., “Qwen2 technical report,”arXiv preprint arXiv:2407.10671, 2024

    Pith/arXiv arXiv 2024

  48. [48]

    GLM: General language model pre- training with autoregressive blank infilling,

    Z. Du, Y . Qian, X. Liu, et al., “GLM: General language model pre- training with autoregressive blank infilling,” inProc. ACL, 2022, pp. 320–335

    2022

  49. [49]

    Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis,

    Z. Ye, X. Zhu, C.-M. Chan, et al., “Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis,”arXiv preprint arXiv:2502.04128, 2025

    Pith/arXiv arXiv 2025