REVIEW 3 major objections 6 minor 42 references
This paper argues that using a one-second wake-word as the enrollment audio degrades target speech extraction, and that no current model yet beats the raw noisy mixture for speech recognition.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 22:46 UTC pith:ZWCQRSFX
load-bearing objection A useful first benchmark for wake-word enrollment in TSE, but the central negative result is confounded by an unexamined English-to-Chinese domain shift. the 3 major comments →
Enroll-on-Wakeup: A First Comparative Study of Target Speech Extraction for Seamless Interaction in Real Noisy Human-Machine Dialogue Scenarios
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper reports that with EoW-TSE — where the only enrollment is a roughly one-second, noise-and-interference-contaminated wake-up segment — four state-of-the-art extraction systems (three discriminative, one generative) all underperform compared with their usual high-quality enrollment setting. In the hardest far-field, reverberant, low-SNR conditions, the extracted speech receives better perceptual scores, but automatic speech recognition on it yields a word error rate consistently worse than the raw noisy mixture's direct WER. Enrollment augmentation using three LLM-based zero-shot TTS engines — either replacing the wake-up with a clean re-synthesis of its transcript, or concatenating a
What carries the argument
The central mechanism is EoW enrollment: a keyword-spotting segment x_wake replaces the conventional pre-recorded clean enrollment e_pre as the target-speaker clue, making extraction a function F(x_query | x_wake). The comparison is carried by two TTS-based enrollment-augmentation schemes — Clean Re-synthesis, which regenerates the wake-up transcript in clean synthetic speech, and Extended Concatenation, which appends an additional clean synthetic sentence — that isolate how enrollment contamination and duration affect extraction quality and downstream recognition.
Load-bearing premise
The load-bearing premise is that the wake-word's short, noisy identity clue is the sole cause of the observed degradation; the fact that the models were trained on English but tested on Chinese speech could confound that attribution.
What would settle it
Run the same four TSE models on an English EoW test set (English wake-up phrase plus English query) with matched noise, distance, and duration, and compare WER against the raw mixture. If the raw mixture's advantage disappears or shrinks, the paper's verdict is partly a language-transfer artifact; if the raw mixture wins again, the brevity-and-noise explanation is supported. As a second arm, feed the Chinese test a clean 3-second enrollment from the same speaker; if WER still cannot beat the raw mixture, then enrollment quality is not the binding constraint.
If this is right
- Current TSE models, especially generative ones, are not yet suited for EoW deployment: their output can be perceptually cleaner but more error-prone for ASR than the raw noisy input.
- LLM-based TTS can mitigate enrollment contamination perceptually; at least one TTS engine reduces WER relative to the noisy enrollment, but neither Clean Re-synthesis nor Extended Concatenation improves WER relative to the raw mixture baseline.
- Extending enrollment with an extra synthetic sentence (EC) does not compensate for intelligibility loss, so the bottleneck is not simply enrollment duration.
- Future EoW-TSE optimization should include phonetic fidelity and ASR-awareness in the training objective, since signal-level and perceptual metrics do not track recognition accuracy.
- The study offers a baseline protocol: reports on EoW-TSE should state both perceptual scores and WER against the raw-mixture reference, not only extraction metrics.
Where Pith is reading between the lines
- The reported failure may in part reflect cross-lingual transfer: the models were trained on English two-speaker data but tested on Chinese wake-up and query speech, so an English-language control is needed before blaming enrollment brevity alone.
- A practical fallback design suggested (but not tested) by the results: run ASR on both the raw mixture and the extracted output, and keep whichever yields higher confidence — the paper's data imply this hybrid would frequently choose the raw mixture, so improvements must pass that bar.
- The TTS re-synthesis step could double as a speaker-identity check: if the synthesized enrollment fails a speaker-embedding similarity threshold against the noisy wake-up, the system could flag the enrollment as unreliable rather than proceed with extraction.
- The one-second enrollment itself could be studied parametrically: truncating clean enrollments to 1.0s, 0.5s, and 0.3s would reveal the minimum duration at which current models still beat the raw mixture, a quantity the paper does not report.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces Enroll-on-Wakeup (EoW), a TSE setting in which the wake-up word captured at the start of a human-machine dialogue is used as the enrollment reference instead of a pre-recorded clean utterance. The authors evaluate four recent TSE systems (SEF-PNet, LExt, CIE-mDPTNet, and SoloSpeech) on a five-condition internal Chinese corpus with real TV noise, distance/reverberation differences, and SNR 10/5 dB. They also test enrollment augmentation with three zero-shot TTS models, either by re-synthesizing the wake-up transcript (CR) or by concatenating an additional synthesized sentence (EC). The central empirical findings are: (i) TSE models improve perceptual metrics such as DNSMOS/OVRL relative to the noisy mixture, but none improves WER relative to direct recognition of the noisy mixture in the tested conditions; (ii) TTS-augmented enrollment improves perceived quality but does not improve ASR accuracy; and (iii) there is a perceptual-recognition gap, especially for generative TSE.
Significance. If confirmed, the paper would provide a useful negative-result benchmark for zero-effort TSE enrollment and a first systematic comparison of discriminative and generative extractors under EoW. Strengths include use of real recorded adverse conditions, multiple modern TSE models, and a concrete TTS augmentation comparison. However, the central claim is currently supported only by a single test corpus and is subject to a language/domain confound and the absence of a clean-enrollment control; the empirical comparisons also lack uncertainty quantification. With those controls added, the paper would be a solid contribution to the TSE community.
major comments (3)
- [§3.1, §4, §5.2] Language/domain mismatch confound. The test wake-up commands are Chinese ('Hi, Pandora', 'Hello, Cube', Section 3.1), while all TSE models are trained on English Libri2Mix (Section 4). The paper never states the language of the query audio, and no English or language-matched control is presented. Therefore, the headline observation that 'none of these TSE models outperforms the raw noisy mixture's direct WER' could reflect cross-lingual extraction failure rather than the shortness/noise of the wake-word enrollment. Please add an English (or otherwise language-matched) test set/control condition, or provide evidence that the enrollment embeddings are language-independent.
- [§2.2 and §5.2] No clean pre-recorded enrollment control. Eq. (2) defines conventional TSE with clean e_pre, but the experiments never include a clean enrollment utterance from the same speaker on the same test mixtures. Without this baseline, the conclusion that performance degradation is caused by 'the brevity of x_wake' and 'clue contamination' is not supported; the degradation could be due to noise alone or to the model's domain mismatch. Please report an e_pre control for at least one model/scenario.
- [Tables 3/4 and Figures 2/3] No statistical significance or uncertainty. DNSMOS and WER differences are presented as if deterministic, but many differences are small (e.g., Table 4 FarNoiseReverb-10 DNSMOS 1.200 vs 1.202). Please provide per-utterance confidence intervals, bootstrap, or paired significance tests for the main comparisons, especially for the claim that TTS augmentation 'consistently improves DNSMOS' and for the claim that no TSE model outperforms raw-mixture WER.
minor comments (6)
- [Abstract / Section 1] EoW is called a 'novel framework,' but the paper is primarily an evaluation study. Consider calling it a 'paradigm' or 'evaluation protocol' to avoid overclaiming novelty.
- [Section 4] 'mix both(2-speaker + noise)' likely refers to Libri2Mix's 'mix_both' configuration; please use the dataset's standard terminology and explain the 'min' duration mode.
- [Table 2] The STOI value for SoloSpeech is missing; if unavailable, state why.
- [Tables 3/4] Define CR and EC in the captions or table notes; currently they are defined only in Section 3.3.
- [Section 5.3] The sentence 'neither method succeeds in reducing the WER' would benefit from stating the reference (raw noisy mixture) explicitly.
- [References] Some references are preprints with arXiv IDs; ensure version numbers and access links are stable and that the final manuscript lists the versions actually used.
Circularity Check
No significant circularity: the paper is an empirical comparative study whose conclusions rest on direct measurements, not on a derivation-from-claims or fitted parameters.
full rationale
The paper does not derive EoW-TSE behavior from a theory that already assumes its conclusions. Its central claims are empirical: four pretrained TSE models are evaluated on a five-scenario internal dataset, and WER/DNSMOS/SI-SDR/PESQ/STOI are reported as measurements. There is no equation in which the output is reconstructed from the input by definition, no fitted parameter is relabeled as a prediction, and no uniqueness theorem is imported from self-citations. The only same-group tested model is SEF-PNet [9], whose authors include two co-authors (Guan and Long), but it is used as one of several test subjects, not as evidence supporting the paper's conclusions; its prior benchmark results are not load-bearing. The TTS augmentation experiments compare synthetic enrollments against raw noisy enrollments, and the paper explicitly reports that augmentation does not reduce WER, so the conclusion is not forced by the augmentation method. The possible English-training/Chinese-evaluation mismatch is a validity concern, not circularity: it concerns whether the empirical result generalizes or is confounded, but the result itself is measured, not assumed. Therefore the circularity score is 0.
Axiom & Free-Parameter Ledger
axioms (3)
- domain assumption The mixture model x(t) = s(t) + n(t) (Eq. 1) assumes additive noise and a single clean target.
- domain assumption The KWS module segments the wake-word and query reliably.
- domain assumption TSE models trained on English Libri2Mix transfer to Chinese EoW test data.
read the original abstract
Target speech extraction (TSE) typically relies on pre-recorded high-quality enrollment speech, which disrupts user experience and limits feasibility in spontaneous interaction. In this paper, we propose Enroll-on-Wakeup (EoW), a novel framework where the wake-word segment, captured naturally during human-machine interaction, is automatically utilized as the enrollment reference. This eliminates the need for pre-collected speech to enable a seamless experience. We perform the first systematic study of EoW-TSE, evaluating advanced discriminative and generative models under real diverse acoustic conditions. Given the short and noisy nature of wake-word segments, we investigate enrollment augmentation using LLM-based TTS. Results show that while current TSE models face performance degradation in EoW-TSE, TTS-based assistance significantly enhances the listening experience, though gaps remain in speech recognition accuracy.
Figures
Reference graph
Works this paper leans on
-
[1]
Enroll-on-Wakeup
Introduction Target speech extraction (TSE) aims to isolate a specific speaker’s voice from a multi-talker or noisy acoustic environ- ment by leveraging a set of auxiliary clues, such as a ref- erence enrollment utterance from the target speaker. Tradi- tional TSE frameworks typically operate under the assump- tion that a high-quality, pre-recorded enroll...
-
[2]
Problem Definition This section defines theEnroll-on-Wakeup(EoW) TSE frame- work, illustrated in Figure 1. The goal of EoW-TSE is to extract target speech from a continuous noisy mixture by using only the transient wake-word segment as the enrollment reference. 2.1. Conventional TSE Formulation In a traditional TSE task, the observed noisy mixturex(t)in t...
Pith/arXiv arXiv 2026
-
[3]
Hi, Pandora
Enroll-on-Wakeup TSE Systems This section details our experimental framework for evaluating EoW-TSE. We describe five real-world recording scenarios cap- turing diverse acoustic environments, followed by the four ad- vanced TSE models used in our comparative study. Finally, we introduce the integration of LLM-based TTS engines for enroll- ment augmentatio...
-
[4]
For the dis- criminative systems: SEF-PNet, CIE-mDPTNet, and LExt, they all are trained on the Libri2Mixtrain-100subset under the mix both(2-speaker + noise) andminduration modes
Experimental Setup Datasets: We evaluate all models on the five-scenario test set detailed in Table 1 at a 16 kHz sampling rate. For the dis- criminative systems: SEF-PNet, CIE-mDPTNet, and LExt, they all are trained on the Libri2Mixtrain-100subset under the mix both(2-speaker + noise) andminduration modes. For the generative SoloSpeech, we used the origi...
-
[5]
Results and Discussion 5.1. Results on Libri2mix 2spk+noise Condition Table 2 shows the comparative performance of the selected models on the challenging Libri2Mix 2-speaker+noise condi- tion. We evaluate a diverse set of architectures varying in pa- rameter size and computational complexity (MACs). Among discriminative models, LExt achieves the best trad...
arXiv 1972
-
[6]
Conclusion This paper presents the first systematic study of TSE under the Enroll-on-Wakeup paradigm. Our evaluation across five real-world scenarios reveals a clear perceptual-recognition gap: while advanced TSE models, especially generative ones, can notably improve audio quality, they often fail in ASR perfor- mance. We show that current TSE systems fa...
-
[7]
Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,
K. ˇZmol´ıkov´a, M. Delcroix, K. Kinoshita, T. Ochiai, T. Nakatani, L. Burget, and J. ˇCernock`y, “Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,”IEEE Journal of Selected Topics in Signal Processing, vol. 13, no. 4, pp. 800–814, 2019
2019
-
[8]
V oiceFilter: Targeted V oice Separation by Speaker-Conditioned Spectrogram Masking,
Q. Wang, H. Muckenhirn, K. W. Wilsonet al., “V oiceFilter: Targeted V oice Separation by Speaker-Conditioned Spectrogram Masking,” inProc. Interspeech, 2018
2018
-
[9]
Wespeaker: A Research and Production Oriented Speaker Embedding Learning Toolkit,
H. Wang, C.-Y . Liang, S. Wanget al., “Wespeaker: A Research and Production Oriented Speaker Embedding Learning Toolkit,” inProc. ICASSP, 2022, pp. 1–5
2022
-
[10]
ECAPA- TDNN: Emphasized Channel Attention, Propagation and Ag- gregation in TDNN Based Speaker Verification,
B. Desplanques, J. Thienpondt, and K. Demuynck, “ECAPA- TDNN: Emphasized Channel Attention, Propagation and Ag- gregation in TDNN Based Speaker Verification,” inProc. Inter- speech, 2020, p. 3830–3834
2020
-
[11]
SpEx: Multi-Scale Time Do- main Speaker Extraction Network,
C. Xu, W. Rao, E. S. Chnget al., “SpEx: Multi-Scale Time Do- main Speaker Extraction Network,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, p. 1370–1384, 2020
2020
-
[12]
SpEx+: A Complete Time Domain Speaker Extraction Network,
M. Ge, C. Xu, L. Wanget al., “SpEx+: A Complete Time Domain Speaker Extraction Network,” inProc. Interspeech, 2020
2020
-
[13]
MC-SpEx: Towards Effective Speaker Extraction with Multi-Scale Interfusion and Conditional Speaker Modulation,
J. Chen, W. Rao, Z. Wanget al., “MC-SpEx: Towards Effective Speaker Extraction with Multi-Scale Interfusion and Conditional Speaker Modulation,” inProc. Interspeech, 2023
2023
-
[14]
USEF-TSE: Universal Speaker Embedding Free Target Speaker Extraction,
B. Zeng and M. Li, “USEF-TSE: Universal Speaker Embedding Free Target Speaker Extraction,”IEEE Transactions on Audio, Speech and Language Processing, 2025
2025
-
[15]
SEF-PNet: Speaker Encoder-Free Personalized Speech Enhancement with Local and Global Contexts Aggregation,
Z. Huang, H. Guan, H. Wei, and Y . Long, “SEF-PNet: Speaker Encoder-Free Personalized Speech Enhancement with Local and Global Contexts Aggregation,” inProc. ICASSP, 2025, pp. 1–5
2025
-
[16]
Target speaker extrac- tion by directly exploiting contextual information in the time- frequency domain,
X. Yang, C. Bao, J. Zhou, and X. Chen, “Target speaker extrac- tion by directly exploiting contextual information in the time- frequency domain,” inProc. ICASSP, 2024, pp. 10 476–10 480
2024
-
[17]
Listen to Extract: Onset-Prompted Target Speaker Extraction,
P. Shen, K. Chen, S. Heet al., “Listen to Extract: Onset-Prompted Target Speaker Extraction,”IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 4832–4843, 2025
2025
-
[18]
Av-Sepformer: Cross-Attention Sepformer for Audio-Visual Target Speaker Extraction,
J. Lin, X. Cai, H. Dinkelet al., “Av-Sepformer: Cross-Attention Sepformer for Audio-Visual Target Speaker Extraction,” inProc. ICASSP, 2023, pp. 1–5
2023
-
[19]
MoMuSE: Momentum Multi- modal Target Speaker Extraction for Real-time Scenarios with Im- paired Visual Cues,
J. Li, K. Zhang, S. Wanget al., “MoMuSE: Momentum Multi- modal Target Speaker Extraction for Real-time Scenarios with Im- paired Visual Cues,” inProc. IEEE International Conference on Multimedia and Expo (ICME), 2025, pp. 1–6
2025
-
[20]
Scenario-aware audio- visual TF-Gridnet for target speech extraction,
Z. Pan, G. Wichern, Y . Masuyamaet al., “Scenario-aware audio- visual TF-Gridnet for target speech extraction,” inProc. Work- shop on IEEE Automatic Speech Recognition and Understanding (ASRU), 2023, pp. 1–8
2023
-
[21]
ELEGANCE: Effi- cient LLM Guidance for Audio-Visual Target Speech Extraction,
W. Wu, S. Wang, X. Wu, H. Meng, and H. Li, “ELEGANCE: Effi- cient LLM Guidance for Audio-Visual Target Speech Extraction,” arXiv preprint arXiv:2511.06288, 2025
arXiv 2025
-
[22]
L-SpEx: Localized Target Speaker Extraction,
M. Ge, C. Xu, L. Wanget al., “L-SpEx: Localized Target Speaker Extraction,” inProc. ICASSP, 2022, pp. 7287–7291
2022
-
[23]
Location-aware target speaker extraction for hear- ing aids,
D.-J. Alcala Padilla, N. L. Westhausen, S. Vivekananthan, and B. T. Meyer, “Location-aware target speaker extraction for hear- ing aids,” inProc. Interspeech, 2025, pp. 2975–2979
2025
-
[24]
H. Wang, J. Hai, D. Yanget al., “SoloSpeech: Enhancing Intelligi- bility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline,”arXiv preprint arXiv:2505.19314, 2025
Pith/arXiv arXiv 2025
-
[25]
Librimix: An open-source dataset for generalizable speech separation,
J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vin- cent, “Librimix: An open-source dataset for generalizable speech separation,”arXiv preprint arXiv:2005.11262, 2020
Pith/arXiv arXiv 2005
-
[26]
TF-GridNet: In- tegrating full-and sub-band modeling for speech separation,
Z.-Q. Wang, S. Cornell, S. Choiet al., “TF-GridNet: In- tegrating full-and sub-band modeling for speech separation,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 31, pp. 3221–3236, 2023
2023
-
[27]
TF-Locoformer: Transformer with local modeling by convolution for speech sep- aration and enhancement,
K. Saijo, G. Wichern, F. G. Germainet al., “TF-Locoformer: Transformer with local modeling by convolution for speech sep- aration and enhancement,” inProc. Workshop on International Acoustic Signal Enhancement (IWAENC), 2024, pp. 205–209
2024
-
[28]
Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,
J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,” inProc. ICASSP, 2016, pp. 31–35
2016
-
[29]
WHAM!: Extend- ing speech separation to noisy environments,
G. Wichern, J. Antognini, M. Flynnet al., “WHAM!: Extend- ing speech separation to noisy environments,”arXiv preprint arXiv:1907.01160, 2019
Pith/arXiv arXiv 1907
-
[30]
WHAMR!: Noisy and reverberant single-channel speech sepa- ration,
M. Maciejewski, G. Wichern, E. McQuinn, and J. Le Roux, “WHAMR!: Noisy and reverberant single-channel speech sepa- ration,” inProc. ICASSP, 2020, pp. 696–700
2020
-
[31]
Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,
J. Chen, Q. Mao, and D. Liu, “Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,” inProc. Interspeech, 2020, pp. 2642–2646
2020
-
[32]
S. Zhou, Y . Zhou, Y . Heet al., “IndexTTS2: A Break- through in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech,”arXiv preprint arXiv:2506.21619, 2025
Pith/arXiv arXiv 2025
-
[33]
XTTS: a Massively Mul- tilingual Zero-Shot Text-to-Speech Model,
E. Casanova, K. Davis, E. G ¨olgeet al., “XTTS: a Massively Mul- tilingual Zero-Shot Text-to-Speech Model,” inProc. Interspeech, 2024
2024
-
[34]
Cosyvoice 3: Towards in-the- wild speech generation via scaling-up and post-training,
Z. Du, C. Gao, Y . Wanget al., “Cosyvoice 3: Towards in-the- wild speech generation via scaling-up and post-training,”arXiv preprint arXiv:2505.17589, 2025
Pith/arXiv arXiv 2025
-
[35]
A method for stochastic optimiza- tion,
D. Kinga, J. B. Adamet al., “A method for stochastic optimiza- tion,” inProc. International conference on learning representa- tions (ICLR), vol. 5, no. 6, 2015
2015
-
[36]
Why gradient clip- ping accelerates training: A theoretical justification for adaptiv- ity,
J. Zhang, T. He, S. Sra, and A. Jadbabaie, “Why gradient clip- ping accelerates training: A theoretical justification for adaptiv- ity,”arXiv preprint arXiv:1905.11881, 2019
Pith/arXiv arXiv 1905
-
[37]
SDR– half-baked or well done?
J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR– half-baked or well done?” inProc. ICASSP, 2019, pp. 626–630
2019
-
[38]
Per- ceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,
A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Per- ceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,” in Proc. ICASSP, vol. 2, 2001, pp. 749–752
2001
-
[39]
A short- time objective intelligibility measure for time-frequency weighted noisy speech,
C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “A short- time objective intelligibility measure for time-frequency weighted noisy speech,” inProc. ICASSP, 2010, pp. 4214–4217
2010
-
[40]
DNSMOS P. 835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,
C. K. Reddy, V . Gopal, and R. Cutler, “DNSMOS P. 835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,” inProc. ICASSP, 2022, pp. 886–890
2022
-
[41]
K. An, Y . Chen, Z. Chen, C. Denget al., “Fun-asr technical re- port,”arXiv preprint arXiv:2509.12508, 2025
arXiv 2025
-
[42]
MeetEval: A Toolkit for Computation of Word Error Rates for Meeting Transcription Systems,
T. von Neumann, C. Boeddeker, M. Delcroix, and R. Haeb- Umbach, “MeetEval: A Toolkit for Computation of Word Error Rates for Meeting Transcription Systems,” inProc. 7th Interna- tional Workshop on Speech Processing in Everyday Environments (CHiME 2023), 2023, pp. 27–32
2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.