Pith. sign in

REVIEW 2 major objections 5 minor 2 cited by

M3ANet: Multi-scale and Multi-Modal Alignment Network for Brain-Assisted Target Speaker Extraction

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read M3ANet claims that temporally aligning EEG and speech, plus a multi-scale multi-directional speech encoder, produces the best brain-assisted target speaker extraction on three public datasets.

desk verdict A capable engineering paper whose empirical gains are plausible, but the InfoNCE module is described as doing frame-level temporal alignment when it actually performs segment-level contrastive matching; the mechanism story needs revision, not the experiments. read the letter →

arxiv 2506.00466 v1 pith:CDY25KRU submitted 2025-05-31 eess.AS cs.SD

Cunhang Fan , Ying Chen , Jian Zhou , Zexu Pan , Jingjing Zhang , Youdian Gao , Xiaoke Yang , Zhengqi Wen
show 1 more author
Zhao Lv
This is my paper · ORCID
classification eess.AScs.SD
keywords brain-assistedtargetspeakerextractionEEGtemporalalignmentcontrastivelearningInfoNCEmulti-scaleencoderGroupMambastatespacemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that brain-assisted target speaker extraction—pulling one attended voice out of a mixed recording using EEG—is held back by two fixable problems: speech and EEG are out of temporal sync, and typical speech encoders extract shallow features. M3ANet addresses both: a contrastive alignment module pulls EEG and speech frames at the same time step together before fusion, and a speech encoder stacks four convolution scales with GroupMamba blocks that scan features from four directions. On the Cocktail Party, AVED, and MM-AAD datasets, the model reports the highest published SDR, SI-SDR, STOI, ESTOI, and PESQ scores, with relative SI-SDR gains of 8.2%, 12.8%, and 10.4% over the strongest prior baselines. If correct, this shows that cross-modal temporal alignment and deeper speech encoding are concrete levers for the task rather than incidental training choices.

What carries the argument

The load-bearing pieces are the modal alignment module and the multi-scale GroupMamba speech encoder. The alignment module applies InfoNCE contrastive learning at the frame level: for each EEG segment at time t, the mixture speech frame at the same time serves as the positive sample and all other frames in the batch serve as negatives, which is intended to synchronize the two modalities before they are fused. The speech encoder runs four parallel 1D convolutions with different filter lengths (2.5 to 20 ms) to capture short- and long-term features, then passes the concatenated multi-scale features through GroupMamba blocks: the features are split into four channel groups, each processed by a selective state-space block scanning in one of four directions, with a channel affinity modulation layer reweighting information across scales. The speaker extractor combines a convolutional multi-layer cross-attention fusion module with four dual-path RNN layers that estimate a time-domain mask, which is multiplied by the encoded speech and decoded with a transposed convolution.

What would settle it

Retrain M3ANet on the three datasets with the InfoNCE loss modified so each positive is a randomly chosen mixture frame from the same batch, destroying temporal correspondence; if SI-SDR shows no meaningful drop relative to the reported alignment-module ablation, the claimed temporal alignment is not what drives the gain.

Watch

Extended reading notes

Core claim

The central claim is that a brain-assisted target speaker extractor improves when the two modalities are aligned at the frame level before fusion and when the speech encoder is deepened with directional state-space scanning. The alignment module uses an InfoNCE contrastive loss in which each EEG segment is a query, the mixture speech frame at the same time index is the positive sample, and other frames in the batch are negatives; this loss is added to the SI-SDR objective with weight 3. The speech encoder uses four parallel 1D convolutions with filter lengths 2.5, 5, 10, and 20 ms, then GroupMamba blocks that scan the feature map left-to-right, right-to-left, top-to-bottom, and bottom-to-top before a channel affinity modulation layer reweights cross-scale information. The report states that this architecture achieves the best results on all three datasets and that removing either the alignment module or the GroupMamba blocks degrades performance everywhere, with the alignment removal costing up to 0.89 dB in SDR and the GroupMamba removal costing up to 0.97 dB.

Load-bearing premise

The InfoNCE alignment assumes that the mixture speech frame at time t is the correct target for the EEG segment at time t, so the loss teaches true temporal correspondence; if the contrastive learning instead exploits batch statistics or settles on a fixed shift, the alignment story would lose its footing even if the performance numbers survive.

Editorial extensions

If this is right

  • On all three datasets, M3ANet improves signal-level, intelligibility, and perceptual metrics simultaneously, so the alignment and encoder changes add benefits across every measured dimension of extraction quality.
  • Removing the alignment module hurts performance on every dataset, meaning temporal alignment contributes independently of the encoder modification.
  • Removing the GroupMamba blocks hurts performance most on AVED and MM-AAD, suggesting the deeper encoder matters most when the EEG setup has fewer channels or noisier signals.
  • Two GroupMamba layers outperform one, three, four, and five layers, indicating that the encoder benefit comes from a specific depth and complexity balance rather than raw capacity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because EEG is downsampled to 128 Hz while audio is at 14.7 kHz, the 'same time step' used in the alignment is a coarse 7.8 ms bin; the learned alignment probably reflects a general cross-modal correspondence rather than the true auditory latency of tens of milliseconds.
  • Using the mixture frame as the positive sample also aligns interferer energy to the EEG; using the attended clean speech or its envelope as the positive would test whether alignment to the target alone yields even larger gains.
  • A shuffled-positive control, where the positive is drawn from a random time in the batch, would show whether the InfoNCE term is genuinely time-sensitive or is acting as a soft regularizer; the paper does not include this control.
  • The same recipe of contrastive alignment plus a directional state-space encoder could transfer to audio-visual or text-guided target speaker extraction, where modality timing offsets are also known to exist.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes M3ANet, a brain-assisted target speaker extraction (TSE) model that combines a multi-scale convolutional speech encoder with GroupMamba (GM) modules, a graph-convolutional EEG encoder, and a modal alignment module based on an InfoNCE contrastive loss. The stated goal of the alignment module is to temporally align EEG and speech features before fusion, addressing latency between the modalities. The speaker extractor uses a convolutional multi-layer cross-attention (CMCA) fusion module and four DPRNN layers. Experiments on three public datasets (Cocktail Party, AVED, MM-AAD) report consistent improvements over several baselines, with claimed SI-SDR gains of 8.2%, 12.8%, and 10.4% relative to the best baseline on each dataset. The source code is publicly released.

Significance. If the reported results are reliable, M3ANet would constitute a practical advance in EEG-guided speech extraction, with the multi-scale encoder and GM modules providing a parameter-efficient alternative to transformer encoders. The release of source code and the use of three public datasets are strengths, as is the inclusion of some reproduced baselines. However, the paper's central conceptual contribution—the modal alignment module—does not implement the temporal frame-level alignment it claims, and the baseline comparisons are not made under a fully controlled protocol. The conceptual mechanism therefore needs to be substantially re-evaluated even if the numerical results are reproducible.

major comments (2)
  1. [Section 3.3, Eq. (6)] The modal alignment module does not perform the claimed frame-level temporal alignment. After the reshaping described in Section 3.3, the EEG embedding and speech embedding are both flattened to shape (B, Ne×Ts), so each query e_i is an entire 2-second EEG segment and each positive is the entire corresponding speech segment, not a single time frame. The text states that the module 'achiev[es] alignment between the two on a temporal frame level' and 'minimizes the distance between EEG and the corresponding speech... at the same time steps,' but the InfoNCE loss in Eq. (6) compares whole segments. Furthermore, the positive sample is the mixture speech, which contains both the attended and the interfering speaker; the loss can therefore be minimized by exploiting global acoustic statistics rather than by learning a neural-latency-corrected correspondence. The ablation in Table 2 shows only a 0.32–0.85 dB SI-SDR drop when the module is removed, which is consistent with an auxiliary regularizer on the EEG encoder rather than with a temporal alignment mechanism. This is load-bearing because the paper's abstract and introduction motivate the entire approach by the temporal misalignment problem.
  2. [Section 5.1, Table 1] The state-of-the-art claim is not supported by a controlled and statistically grounded comparison. The Cocktail Party baseline results are taken from their original papers, the AVED baseline results are taken from a single previous paper, and only the MM-AAD baselines are reproduced by the authors. Different papers may use different preprocessing, data splits, and evaluation protocols, and the paper provides no error bars or significance tests. The reported improvements (e.g., 1.06 dB SI-SDR on Cocktail Party over MSFNet) could be within subject-level variability. The paper should re-evaluate all baselines under the same protocol used for M3ANet, or at minimum provide per-subject results with paired statistical tests (e.g., Wilcoxon signed-rank test) to show that the improvements are significant.
minor comments (5)
  1. [Section 3.1] There is a notation conflict: the graph is defined as G=(V, E) with E denoting the set of edges, while elsewhere in the paper E denotes the EEG data. Using different symbols (e.g., ε for edges) would avoid ambiguity.
  2. [Section 3.2, Eq. (3)] The argument order in Conv1D(X, 1, Ns, Li) is unclear. Specify whether the arguments are (input, in_channels, out_channels, kernel_size) or some other convention, and ensure it is consistent with the rest of the paper.
  3. [Section 5.2, Table 2] The ablation 'w/o Alignment' removes the modal alignment module, but the paper does not state whether the InfoNCE loss term λL_InfoNCE was also removed from the total loss in Eq. (9). This is important for reproducibility and for interpreting whether the gain comes from the alignment mechanism or from the auxiliary loss itself.
  4. [Section 5.2] There is a typo in the text: 'acilitates' should be 'facilitates.' Additionally, the claim that 'the CAM structure within the GM module acilitates information interaction across temporal scales' would benefit from a more explicit explanation of how the CAM block operates across scales, since Eq. (5) is not fully described.
  5. [Section 5.3, Table 3] The sensitivity analysis for the number of GM layers is performed only on the Cocktail Party dataset. Reporting the corresponding results on at least one other dataset (e.g., AVED) would strengthen the claim of generalizability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical SOTA claims rest on held-out evaluations, not on re-stating fitted losses or self-citations.

full rationale

The paper makes no theoretical derivation; all conclusions are empirical evaluations on public datasets. The main loss (Eq. 9) combines SI-SDR and InfoNCE with a fitted lambda of 3, but the reported SI-SDR gains are measured on held-out test segments and are not a restatement of the loss or of any fitted parameter. The modal alignment module uses InfoNCE (Eq. 6) to maximize cosine similarity between each EEG segment and its corresponding speech segment; this does make the 'alignment' objective definitional to the loss, but the paper's causal claim (that alignment helps extraction) is tested by the ablation in Table 2 and is therefore empirical, not circular. Self-citations to MSFNet [Fan et al., 2024] and the MM-AAD dataset [Fan et al., 2025b] are used as baselines or benchmarks; these are public, code-reproduced (asterisked rows) or externally falsifiable, so they do not constitute load-bearing self-citation. A caveat is that the InfoNCE module operates on whole 2-second segments flattened to (B, Ne x Ts), so the paper's 'temporal frame level' alignment claim is not strictly established, but this is a correctness or interpretability concern, not a circularity. No step in the derivation reduces to its own inputs by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new particles, forces, or entities are introduced; the model components are existing building blocks. The free parameters are standard hyperparameters, with λ being the most notable fitted choice.

free parameters (5)
  • lambda (InfoNCE loss weight) = 3
    Set as λ=3 in Eq. (9), described as 'determined by experiment' in Section 3.5.
  • tau (InfoNCE temperature) = 0.1
    Temperature in the InfoNCE loss (Eq. 6), Section 3.3.
  • Number of GroupMamba layers N = 2
    Chosen from experiments in Table 3; 2 layers gave best SDR/SI-SDR/STOI on Cocktail Party.
  • Speech encoder filter lengths L1-L4 = 2.5, 5, 10, 20 ms
    Multi-scale kernel sizes defined in Section 4.3; not swept.
  • DPRNN chunk size L = 250
    Chunk length for dual-path RNN in Section 4.3.
assumptions (4)
  • domain assumption EEG signals contain decodable information about the attended speaker
    Assumed throughout; based on prior AAD literature cited in Section 2.
  • domain assumption The temporal offset between EEG and speech is consistent enough that frame-level contrastive alignment can correct it
    Central to the modal alignment module (Section 3.3); no explicit delay model is estimated.
  • domain assumption Mixture speech frames are valid positive samples for EEG queries in InfoNCE
    Used in constructing sample pairs (Section 3.3, Figure 1); not validated by a diagnostic experiment.
  • ad hoc to paper GroupMamba (VSSSBlock) features from vision transfer effectively to 1D speech signals
    The GM module is imported from image modeling (Shaker et al. 2024) and applied to time-domain speech without adaptation analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of M3ANet: Multi-scale and Multi-Modal Alignment Network for Brain-Assisted Target Speaker Extraction." pith.science (2026). https://pith.science/paper/CDY25KRU

@misc{pith2026250600466,
  author       = {Pith},
  title        = {Pith review of: M3ANet: Multi-scale and Multi-Modal Alignment Network for Brain-Assisted Target Speaker Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CDY25KRU}},
  note         = {Machine review of arXiv:2506.00466}
}
read the original abstract

The brain-assisted target speaker extraction (TSE) aims to extract the attended speech from mixed speech by utilizing the brain neural activities, for example Electroencephalography (EEG). However, existing models overlook the issue of temporal misalignment between speech and EEG modalities, which hampers TSE performance. In addition, the speech encoder in current models typically uses basic temporal operations (e.g., one-dimensional convolution), which are unable to effectively extract target speaker information. To address these issues, this paper proposes a multi-scale and multi-modal alignment network (M3ANet) for brain-assisted TSE. Specifically, to eliminate the temporal inconsistency between EEG and speech modalities, the modal alignment module that uses a contrastive learning strategy is applied to align the temporal features of both modalities. Additionally, to fully extract speech information, multi-scale convolutions with GroupMamba modules are used as the speech encoder, which scans speech features at each scale from different directions, enabling the model to capture deep sequence information. Experimental results on three publicly available datasets show that the proposed model outperforms current state-of-the-art methods across various evaluation metrics, highlighting the effectiveness of our proposed method. The source code is available at: https://github.com/fchest/M3ANet.

Figures

Figures reproduced from arXiv: 2506.00466 by the authors.

Figure 1
Figure 1. The speech and EEG encoders extract feature repre [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 1
Figure 1. The overall architecture of the proposed method. The colored squares in the bottom right corner explain how the contrast learning [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) VSSSBlock. The 1D-SS refers to the use of a 1D se [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Combined boxplot and half-violin plot comparison of SI [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DMF2Mel: A Dynamic Multiscale Fusion Network for EEG-Driven Mel Spectrogram Reconstruction

    cs.SD 2025-07 reject novelty 5.0 of 10

    DMF2Mel, a dynamic multiscale fusion network, reports the best mel spectrogram reconstruction scores on SparrKULee, though test-set hyperparameter tuning makes the comparison unreliable.

  2. Decoding Speech Envelopes from Electroencephalogram with a Contrastive Pearson Correlation Coefficient Loss

    eess.AS 2026-01 conditional novelty 4.0 of 10

    A contrastive Pearson-correlation loss—maximizing attended minus average unattended envelope correlation—improves EEG auditory attention decoding accuracy in most, but not all, tested model/dataset settings.

Reference graph

Works this paper leans on

38 extracted references · 34 canonical work pages · cited by 2 Pith papers

  1. [1]

    Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narra- tive speech.Current Biology, 28(5):803–809,

    [Brodericket al., 2018 ] Michael P Broderick, Andrew J An- derson, Giovanni M Di Liberto, Michael J Crosse, and Ed- mund C Lalor. Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narra- tive speech.Current Biology, 28(5):803–809,

    work page 2018

  2. [5]

    Improved Feature Extraction Network for Neuro-Oriented Target Speaker Extraction

    [Fanet al., 2025a ] Cunhang Fan, Youdian Gao, Zexu Pan, Jingjing Zhang, Hongyu Zhang, Jie Zhang, and Zhao Lv. Improved feature extraction network for neuro- oriented target speaker extraction.arXiv preprint arXiv:2501.01673,

    work page Pith review arXiv

  3. [7]

    L-spex: Localized target speaker extraction

    [Geet al., 2022 ] Meng Ge, Chenglin Xu, Longbiao Wang, Eng Siong Chng, Jianwu Dang, and Haizhou Li. L-spex: Localized target speaker extraction. InICASSP 2022- 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7287–7291. IEEE,

    work page 2022

  4. [11]

    The cocktail party problem.Neural computation, 17(9):1875– 1902,

    [Haykin and Chen, 2005] Simon Haykin and Zhe Chen. The cocktail party problem.Neural computation, 17(9):1875– 1902,

    work page 2005

  5. [13]

    Speaker-independent brain enhanced speech denoising

    [Hosseiniet al., 2021 ] Maryam Hosseini, Luca Celotti, and ´Eric Plourde. Speaker-independent brain enhanced speech denoising. InICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1310–1314. IEEE,

    work page 2021

  6. [15]

    Cross-modal global interaction and local alignment for audio-visual speech recognition

    [Huet al., 2023 ] Yuchen Hu, Ruizhe Li, Chen Chen, Heqing Zou, Qiushi Zhu, and Eng Siong Chng. Cross-modal global interaction and local alignment for audio-visual speech recognition. In Edith Elkind, editor,Proceedings of the Thirty-Second International Joint Conference on Ar- tificial Intelligence, IJCAI-23, pages 5076–5084. Interna- tional Joint Confere...

    work page 2023

  7. [16]

    [Le Rouxet al., 2019 ] Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, and John R Hershey

    Main Track. [Le Rouxet al., 2019 ] Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, and John R Hershey. Sdr–half-baked or well done? InICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 626–630. IEEE,

    work page 2019

  8. [17]

    Align before fuse: Vision and language representation learning with momentum distilla- tion.Advances in neural information processing systems, 34:9694–9705,

    [Liet al., 2021 ] Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distilla- tion.Advances in neural information processing systems, 34:9694–9705,

    2021

Show all 38 references
  1. [18]

    Audio-visual active speaker extraction for sparsely overlapped multi-talker speech

    [Liet al., 2024 ] Junjie Li, Ruijie Tao, Zexu Pan, Meng Ge, Shuai Wang, and Haizhou Li. Audio-visual active speaker extraction for sparsely overlapped multi-talker speech. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pa...

    2024

  2. [19]

    Av- sepformer: Cross-attention sepformer for audio-visual tar- get speaker extraction

    [Linet al., 2023 ] Jiuxin Lin, Xinyu Cai, Heinrich Dinkel, Jun Chen, Zhiyong Yan, Yongqing Wang, Junbo Zhang, Zhiyong Wu, Yujun Wang, and Helen Meng. Av- sepformer: Cross-attention sepformer for audio-visual tar- get speaker extraction. InICASSP 2023-2023 IEEE In- ternational ...

    2023

  3. [20]

    Development of the audi- tory system.Handbook of clinical neurology, 129:55–72,

    [Litovsky, 2015] Ruth Litovsky. Development of the audi- tory system.Handbook of clinical neurology, 129:55–72,

    2015

  4. [22]

    Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech separation

    [Luoet al., 2020 ] Yi Luo, Zhuo Chen, and Takuya Yosh- ioka. Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech separation. InICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 46–50. IEEE,

    2020

  5. [23]

    Dbpnet: Dual- branch parallel network with temporal-frequency fusion for auditory attention detection

    [Niet al., 2024 ] Qinke Ni, Hongyu Zhang, Cunhang Fan, Shengbing Pei, Chang Zhou, and Zhao Lv. Dbpnet: Dual- branch parallel network with temporal-frequency fusion for auditory attention detection. InProceedings of the In- ternational Joint Conference on Artificial Intelligenc...

    2024

  6. [24]

    Attentional selection in a cocktail party environment can be decoded from single- trial eeg.Cerebral cortex, 25(7):1697–1706,

    [O’sullivanet al., 2015 ] James A O’sullivan, Alan J Power, Nima Mesgarani, Siddharth Rajaram, John J Foxe, Bar- bara G Shinn-Cunningham, Malcolm Slaney, Shihab A Shamma, and Edmund C Lalor. Attentional selection in a cocktail party environment can be decoded from single- tria...

    2015

  7. [25]

    Neural decoding of at- tentional selection in multi-speaker environments without access to clean sources.Journal of neural engineering, 14(5):056001,

    [O’Sullivanet al., 2017 ] James O’Sullivan, Zhuo Chen, Jose Herrero, Guy M McKhann, Sameer A Sheth, Ashesh D Mehta, and Nima Mesgarani. Neural decoding of at- tentional selection in multi-speaker environments without access to clean sources.Journal of neural engineering, 14(5):056001,

    2017

  8. [26]

    Neu- roheed+: Improving neuro-steered speaker extraction with joint auditory attention detection

    [Panet al., 2024b ] Zexu Pan, Gordon Wichern, Franc ¸ois G Germain, Sameer Khurana, and Jonathan Le Roux. Neu- roheed+: Improving neuro-steered speaker extraction with joint auditory attention detection. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and...

    2024

  9. [27]

    Tf-nsse: A time–frequency domain neuro-steered speaker extractor.Applied Acous- tics, 211:109519,

    [Qiuet al., 2023 ] Zelin Qiu, Jianjun Gu, Dingding Yao, Jun- feng Li, and Yonghong Yan. Tf-nsse: A time–frequency domain neuro-steered speaker extractor.Applied Acous- tics, 211:109519,

    2023

  10. [28]

    Phase space graph convolutional network for chaotic time series learning.IEEE Transactions on Industrial Informat- ics,

    [Renet al., 2024 ] Weikai Ren, Ningde Jin, and Lei OuYang. Phase space graph convolutional network for chaotic time series learning.IEEE Transactions on Industrial Informat- ics,

    2024

  11. [29]

    Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs

    [Rixet al., 2001 ] Antony W Rix, John G Beerends, Michael P Hollier, and Andries P Hekstra. Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs. In2001 IEEE international conference on acous- tics, speech, ...

    2001

  12. [31]

    An algorithm for intelligibil- ity prediction of time–frequency weighted noisy speech

    [Taalet al., 2011 ] Cees H Taal, Richard C Hendriks, Richard Heusdens, and Jesper Jensen. An algorithm for intelligibil- ity prediction of time–frequency weighted noisy speech. IEEE Transactions on audio, speech, and language pro- cessing, 19(7):2125–2136,

    2011

  13. [33]

    A study of multichannel spatiotemporal features and knowledge distillation on robust target speaker extraction

    [Wanget al., 2024 ] Yichi Wang, Jie Zhang, Shihao Chen, Weitai Zhang, Zhongyi Ye, Xinyuan Zhou, and Lirong Dai. A study of multichannel spatiotemporal features and knowledge distillation on robust target speaker extraction. InICASSP 2024-2024 IEEE International Conference on A...

    2024

  14. [34]

    Spex: Multi-scale time domain speaker extraction network.IEEE/ACM transactions on audio, speech, and language processing, 28:1370–1384,

    [Xuet al., 2020 ] Chenglin Xu, Wei Rao, Eng Siong Chng, and Haizhou Li. Spex: Multi-scale time domain speaker extraction network.IEEE/ACM transactions on audio, speech, and language processing, 28:1370–1384,

    2020

  15. [35]

    Darnet: Dual attention refine- ment network with spatiotemporal construction for audi- tory attention detection.arXiv preprint arXiv:2410.11181,

    [Yanet al., 2024 ] Sheng Yan, Hongyu Zhang, Xiaoke Yang, Jianhua Tao, Zhao Lv, et al. Darnet: Dual attention refine- ment network with spatiotemporal construction for audi- tory attention detection.arXiv preprint arXiv:2410.11181,

    2024 arXiv

  16. [36]

    Basen: Time-domain brain-assisted speech enhancement network with convolutional cross at- tention in multi-talker conditions.Interspeech 2023,

    [Zhanget al., 2023 ] Jie Zhang, Qing-Tian Xu, Qiu-Shi Zhu, and Zhen-Hua Ling. Basen: Time-domain brain-assisted speech enhancement network with convolutional cross at- tention in multi-talker conditions.Interspeech 2023,

    2023

  17. [37]

    Based on audio-video evoked auditory attention detection electroencephalogram dataset.Journal of Tsinghua University (Science and Tech- nology), 64(11):1919–1926,

    [ZHANGet al., 2024 ] Hongyu ZHANG, Jingjing ZHANG, Xingguang DONG, Zhao L ¨U, Jianhua TAO, Jian ZHOU, Xiaopei WU, and FAN Cunhang. Based on audio-video evoked auditory attention detection electroencephalogram dataset.Journal of Tsinghua University (Science and Tech- nology), 6...

    2024

  18. [38]

    Neural target speech extraction: An overview.IEEE Signal Processing Magazine, 40(3):8–29, 2023

    [Zmolikovaet al., 2023 ] Katerina Zmolikova, Marc Del- croix, Tsubasa Ochiai, Keisuke Kinoshita, Jan ˇCernock`y, and Dong Yu. Neural target speech extraction: An overview.IEEE Signal Processing Magazine, 40(3):8–29, 2023

    2023

  19. [2001]

    Groupmamba: Parameter-efficient and accu- rate group visual state space model.arXiv preprint arXiv:2407.13772,

    [Shakeret al., 2024 ] Abdelrahman Shaker, Syed Talal Wasim, Salman Khan, Juergen Gall, and Fahad Shahbaz Khan. Groupmamba: Parameter-efficient and accu- rate group visual state space model.arXiv preprint arXiv:2407.13772,

    2024 arXiv

  20. [2005]

    Centroid estimation with transformer-based speaker embedder for robust target speaker extraction

    [Heoet al., 2024 ] Woon-Haeng Heo, Joongyu Maeng, Yoseb Kang, and Namhyun Cho. Centroid estimation with transformer-based speaker embedder for robust target speaker extraction. InProc. Interspeech 2024, pages 4333–4337,

    2024

  21. [2011]

    Speech intelligibility predicted from neural entrainment of the speech envelope.Journal of the Association for Re- search in Otolaryngology, 19:181–191,

    [Vanthornhoutet al., 2018 ] Jonas Vanthornhout, Lien De- cruy, Jan Wouters, Jonathan Z Simon, and Tom Francart. Speech intelligibility predicted from neural entrainment of the speech envelope.Journal of the Association for Re- search in Otolaryngology, 19:181–191,

    2018

  22. [2015]

    Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation.IEEE/ACM transactions on audio, speech, and language processing, 27(8):1256– 1266,

    [Luo and Mesgarani, 2019] Yi Luo and Nima Mesgarani. Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation.IEEE/ACM transactions on audio, speech, and language processing, 27(8):1256– 1266,

    2019

  23. [2018]

    Brain-informed speech separation (biss) for enhancement of target speaker in multitalker speech perception.NeuroImage, 223:117282,

    [Ceoliniet al., 2020 ] Enea Ceolini, Jens Hjortkjær, Daniel DE Wong, James O’Sullivan, Vinay S Raghavan, Jose Herrero, Ashesh D Mehta, Shih-Chii Liu, and Nima Mesgarani. Brain-informed speech separation (biss) for enhancement of target speaker in multitalker speech perception....

    2020

  24. [2019]

    Typing to listen at the cocktail party: Text-guided target speaker extraction

    [Haoet al., 2023 ] Xiang Hao, Jibin Wu, Jianwei Yu, Chenglin Xu, and Kay Chen Tan. Typing to listen at the cocktail party: Text-guided target speaker extraction. arXiv preprint arXiv:2310.07284,

    2023 arXiv

  25. [2020]

    Neurospex: Neuro-guided speaker extraction with cross-modal atten- tion.arXiv preprint arXiv:2409.02489,

    [De Silvaet al., 2024 ] Dashanka De Silva, Siqi Cai, Saurav Pahuja, Tanja Schultz, and Haizhou Li. Neurospex: Neuro-guided speaker extraction with cross-modal atten- tion.arXiv preprint arXiv:2409.02489,

    2024 arXiv

  26. [2021]

    End-to-end brain-driven speech enhance- ment in multi-talker conditions.IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:1718– 1733,

    [Hosseiniet al., 2022 ] Maryam Hosseini, Luca Celotti, and Eric Plourde. End-to-end brain-driven speech enhance- ment in multi-talker conditions.IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:1718– 1733,

    2022

  27. [2022]

    Speaker-independent auditory attention decoding with- out access to clean speech sources.Science advances, 5(5):eaav6134,

    [Hanet al., 2019 ] Cong Han, James O’Sullivan, Yi Luo, Jose Herrero, Ashesh D Mehta, and Nima Mesgarani. Speaker-independent auditory attention decoding with- out access to clean speech sources.Science advances, 5(5):eaav6134,

    2019

  28. [2023]

    X-tf-gridnet: A time–frequency domain target speaker extraction network with adaptive speaker embed- ding fusion.Information Fusion, 112:102550,

    [Haoet al., 2024 ] Fengyuan Hao, Xiaodong Li, and Cheng- shi Zheng. X-tf-gridnet: A time–frequency domain target speaker extraction network with adaptive speaker embed- ding fusion.Information Fusion, 112:102550,

    2024

  29. [2024]

    Msfnet: Multi-scale fusion net- work for brain-controlled speaker extraction

    [Fanet al., 2024 ] Cunhang Fan, Jingjing Zhang, Hongyu Zhang, Wang Xiang, Jianhua Tao, Xinhui Li, Jiangyan Yi, Dianbo Sui, and Zhao Lv. Msfnet: Multi-scale fusion net- work for brain-controlled speaker extraction. InProceed- ings of the 32nd ACM International Conference on Mul...

    2024

  30. [2025]

    Spex+: A complete time domain speaker extraction network.arXiv preprint arXiv:2005.04686,

    [Geet al., 2020 ] Meng Ge, Chenglin Xu, Longbiao Wang, Eng Siong Chng, Jianwu Dang, and Haizhou Li. Spex+: A complete time domain speaker extraction network.arXiv preprint arXiv:2005.04686,

    2020 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.