Pith. sign in

REVIEW 3 major objections 6 minor 53 references

FasTUSS: Faster Task-Aware Unified Source Separation

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Tuning TF-Locoformer blocks yields FasTUSS with 81% fewer operations and 1.2 dB average SNR loss.

desk verdict A useful efficiency study with a solid causal variant, but the headline MAC reductions only hold for 1s chunks and the paper needs latency numbers to back 'faster'. read the letter →

arxiv 2507.11435 v1 pith:OI6B2HCX submitted 2025-07-15 cs.SD eess.ASeess.SP

Francesco Paissan , Gordon Wichern , Yoshiki Masuyama , Ryo Aihara , François G. Germain , Kohei Saijo , Jonathan Le Roux This is my paper · ORCID
classification cs.SDeess.ASeess.SP
keywords audiosourceseparationtask-awareunifiedmodelefficiencyMACreductioncausalattentionmaskpromptconditioningTF-Locoformercontinuous
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the task-aware unified source separation model TUSS can be made dramatically cheaper without sacrificing much quality. It profiles where TUSS spends computation, finds that convolutions dominate for short audio chunks, and uses that insight to alter or remove convolutional components. The result is two configurations, FasTUSS-8.3G and FasTUSS-11.7G, that cut multiply-accumulate operations by 81% and 73% while losing only 1.2 dB and 0.4 dB in average SNR across five benchmarks. The paper also studies how prompts and mixture influence each other through attention masks and derives a causal version of TUSS that can use KVCache-style inference. If the MAC savings translate into real speedups, this matters because TF dual-path models are otherwise compute-heavy despite having few parameters.

What carries the argument

The central object is the TF-Locoformer block, a dual-path transformer module that processes the frequency axis and the temporal axis of a time-frequency representation, combining multi-head self-attention with local convolution-based FFNs (Conv-SwiGLU). The paper's optimization targets these Conv-SwiGLU blocks by changing stride, groups, channel shuffle, and the presence of FFN1, and it also introduces a prompt-aware Conv-SwiGLU that processes prompts with linear layers and the mixture with the original convolution. The causal design hinges on the attention mask decomposed into four blocks (prompt-to-prompt, prompt-to-mixture, mixture-to-prompt, mixture-to-mixture); setting the prompt-to-mixture block to zero while keeping the mixture-to-prompt block active yields a mask that is equivalent to caching prompt keys and values and attending one mixture frame at a time.

What would settle it

Run FasTUSS-8.3G and the original TUSS model on identical hardware with representative audio chunks (1 s and 6 s) and measure end-to-end wall-clock inference time with the same batching; if the speedup is much smaller than the 81% MAC reduction, the primary efficiency claim fails. Separately, examine per-benchmark SNR results: if a task such as WHAM! speech separation degrades by far more than the reported 1.2 dB average, then the paper's headline quality-loss claim is misleading even though the average is correct.

Watch

Extended reading notes

Core claim

The paper claims that the TUSS architecture, built on TF-Locoformer blocks, has most of its compute concentrated in the convolutional Conv-SwiGLU parts when processing audio chunks shorter than about 30 seconds, with convolutions taking about 90% of operations for 1-second chunks. By increasing the convolution stride, removing one of the two FFN blocks, using grouped convolutions with channel shuffle, and optionally adding a prompt-aware FFN, the authors derive FasTUSS-11.7G (ID7) and FasTUSS-8.3G (ID8). FasTUSS-11.7G cuts MACs from 43.1 G to 11.7 G (a 73% reduction) with a 0.4 dB average SNR drop, while FasTUSS-8.3G cuts to 8.3 G (an 81% reduction) with a 1.2 dB average drop. The paper further claims that in the cross-prompt module, contrary to the original design, the prompts do not need to see the mixture to maintain most performance, while the mixture must see the prompts; this asymmetry is exploited to build a causal attention mask where prompts are processed once and the mixture is processed frame-by-frame with KVCache.

Load-bearing premise

The central efficiency claim rests on treating multiply-accumulate count as a valid proxy for inference time, but the paper never measures actual wall-clock latency, and MAC reductions do not automatically translate to real speedups on hardware where memory access and parallelism dominate.

Editorial extensions

If this is right

  • FasTUSS-11.7G retains most of TUSS's quality on speech enhancement, music separation, and cinematic separation while using about a quarter of the operations, making it a practical candidate for deployment where the original was too slow.
  • FasTUSS-8.3G pushes the trade-off further with an 81% operation cut at an average 1.2 dB loss, but per-benchmark results can vary considerably, so task-specific evaluation is needed.
  • The causal attention mask enables streaming-style inference with KVCache, where prompts are processed once and the mixture is processed one frame at a time, opening the door to low-latency, real-time unified source separation.
  • The ablation results show that mixture-to-prompt conditioning is nearly irrelevant (0.3 dB drop when removed) while prompt-to-mixture conditioning is critical (3.2 dB drop when removed), which justifies the causal design and also points to where future simplification can focus.
  • Continuous source separation tests show that overlapping chunks improve quality and that performance plateaus at about 6-second chunks, supporting the decision to optimize for short chunks where convolutions dominate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The headline average SNR drop masks large per-benchmark variations; for example, the WHAM! speech-separation score for FasTUSS-8.3G drops by about 1.7 dB relative to TUSS, so a deployment decision should look at task-level numbers rather than the average.
  • Editorial: MAC counts do not directly measure wall-clock latency because memory access patterns, parallelism, and hardware-specific implementations matter; measuring actual inference time on the target device would likely show a speedup, but its exact size is not established by this paper.
  • Editorial: The finding that prompts can be processed independently of the mixture suggests a broader design pattern for task-aware models: encode task context once, then stream the input through a causal decoder, which could apply to other conditional separation or extraction systems beyond TUSS.
  • Editorial: The combination of stride increase, FFN removal, and grouped convolutions is a recipe that could transfer to other TF dual-path architectures, since they share the same bottleneck structure where convolutions dominate short-chunk compute.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies the TUSS task-aware unified source separation architecture and proposes two reduced-complexity configurations, FasTUSS-11.7G and FasTUSS-8.3G, obtained by increasing convolution stride, removing an FFN block, and using grouped or depthwise-separable convolutions. The authors report MAC reductions of 73% and 81% relative to the original model, with average SNR drops of 0.4 dB and 1.2 dB across five benchmarks. They also analyze prompt conditioning through attention-mask ablations and derive a causal variant intended to enable KVCache-style inference. The experimental work is presented as a design analysis over a common training recipe, with per-benchmark results in an appendix.

Significance. If the complexity reductions are robust at the chunk lengths used in practice, the paper offers a useful efficiency analysis of a recent state-of-the-art unified separation model, and the causal prompt-conditioning study is a valuable contribution in its own right. The manuscript has concrete strengths: all configurations are trained with the same hyperparameters, results are reported on five benchmarks with per-benchmark detail, and the attention-mask ablations directly probe a design assumption of the TUSS architecture. The central efficiency claim, however, is currently tied to MAC measured on 1 s chunks, whereas the evaluations and the continuous source separation experiments use 6--12 s chunks; the paper does not report FasTUSS MAC at those lengths. The title's 'Faster' is also supported only by a MAC proxy, with no wall-clock or throughput measurements. These gaps are addressable but affect the headline message.

major comments (3)
  1. [Abstract; Section 6.1; Table 1 caption; Table 2; Fig. 2] The abstract states unqualified reductions of 81% and 73% in operations, but Table 1 reports MAC only for 1.0 s of audio (caption) and Section 6.1 explicitly describes the analysis 'on 1 s of audio'. The models are evaluated at 6 s chunks in Table 3, and the continuous source separation table (Table 2) reports MAC for 60 s sequences only for ID1. Since ID7 and ID8 leave the MHSA blocks unchanged, and since Section 3 states that convolution cost scales linearly while MHSA cost scales quadratically with sequence length, the percentage reduction necessarily decreases with chunk length. Using the paper's own Fig. 2 split (90% convolution / 10% MHSA at 1 s) and the stated scaling, the reduction for FasTUSS-8.3G at 6 s is roughly 54%, not 81%, and it shrinks further at 12 s and 30 s. The manuscript should report MAC for the FasTUSS configurations at the 6--12 s chunk sizes used in Tables 2--3, or explicitly qualify the abstract and Section 6.1 claim as holding only for 1 s chunks.
  2. [Section 3, footnote 1; Section 6.1; title] The title promises a 'Faster' model, but the only efficiency evidence is multiply-accumulate count, which footnote 1 explicitly labels as an indirect measure of inference time. No wall-clock time, real-time factor, memory bandwidth, or hardware-specific measurements are reported. The proposed changes -- stride 4, removal of FFN1, grouped and depthwise-separable convolutions -- can have large effects on memory access patterns and parallel utilization that are not captured by MAC. The authors should either add at least one latency/throughput measurement on a representative device or temper the title and the 'faster' framing to 'fewer operations'.
  3. [Section 5; Section 6.1; Table 1] The configurations ID7 and ID8 are selected after inspecting results on the same five validation benchmarks used for the reported comparisons, and Section 5 states that validation metrics guided the analysis while Section 6.1 refers to 'test set SNR'. If there is no held-out test split, the selection is susceptible to overfitting to the validation set, and the small differences among ID5, ID6, and ID7 (0.3--0.6 dB) may be within run-to-run training variance, as no multiple seeds or error bars are reported. The authors should clarify whether results are on validation or held-out data and, if possible, report variance across training runs for at least the configurations whose performance differences drive the recommendations.
minor comments (6)
  1. [Abstract] The phrases 'reduce the original model's operations by 81% and 73%' should include the condition 'on 1 s audio chunks', consistent with Table 1.
  2. [Fig. 2] The figure lacks axis labels and units, and the displayed percentages are difficult to reconcile with the stated quadratic MHSA scaling: a 90/10 convolution/attention split at 1 s would imply a convolution share near 23% at 30 s, not the shown 54%. Please clarify what compute is included and how the percentages are computed.
  3. [Section 5, Datasets] There is a typo in 'LibriV ox', which should read 'LibriVox'.
  4. [Section 4] In the BLINDPROMPT mask description, the symbol δij is used without definition; it should be defined as the Kronecker delta.
  5. [Table 2; Appendix Table 3] The units for MAC are inconsistent across rows: Table 2 lists MAC with values such as 2.8 and 5.4, while Table 3 lists 2800.0 and 5400.0 for the same settings. Please state whether the unit is G or T and keep it uniform.
  6. [Table 3, ID8 row] The average 1.2 dB drop for FasTUSS-8.3G hides larger per-benchmark degradations, notably 1.7 dB on WHAM! speech separation and 1.5 dB on FUSS. The averaging procedure over benchmarks should be specified, and the abstract's 'minor performance drops' should be qualified with the per-benchmark range.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the FasTUSS configurations are empirically benchmarked, and the quoted MAC reductions are arithmetic from Table 1, not fitted inputs renamed as predictions.

full rationale

The paper's derivation chain is an empirical architecture search over an externally defined baseline (TUSS, ref. [17]) and standard benchmarks (VCTK-DEMAND, WHAM!, FUSS, MUSDB-HQ, DnR), so its central claims are validated against data outside the fitted configurations. The abstract's 81% and 73% MAC reductions are direct arithmetic consequences of Table 1's reported MAC values at 1.0 s audio (8.3/43.1 = 19.3%, i.e., an 80.7% reduction for ID8; 11.7/43.1 = 27.1%, i.e., a 72.9% reduction for ID7), not quantities fitted from the SNR results. Performance drops are measured for the chosen configurations rather than predicted from the same data that selects them; the paper is explicit that FasTUSS-11.7G and FasTUSS-8.3G are defined as configurations ID7 and ID8 'from Table 1' after presenting the benchmark results, which is post-hoc selection rather than circular derivation. The causal attention mask is designed from qualitative inspection of attention maps and then evaluated in ablations; this is a hypothesis-then-test procedure, not a self-justifying reduction. The paper does cite prior work by overlapping authors ([17], [30], [52]) for the baseline architecture, training setup, and positional encoding comparison, but these citations are not load-bearing in a circular way: they supply the baseline and experimental recipe, while the speedup analysis, MAC profiling, and ablations are generated in this paper on external benchmarks. The main caveat, that MAC is reported for 1.0 s chunks while the abstract states an unqualified percentage reduction, is a reporting limitation about whether the reduction persists at longer chunk sizes; it does not make the derivation equivalent to its inputs by construction. No circular step can be exhibited from the paper's own equations or citations.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The central claim rests on a few assumptions: MAC as a proxy for latency, the inherited TUSS benchmark setup, and the validity of the attention mask design. The architectural choices are selected based on the results they produce, which introduces a model selection bias.

free parameters (2)
  • Model configuration choices (stride, groups, FFN removal) = S=4, G=8, no FFN1 for FasTUSS-8.3G; S=4, no FFN1 for FasTUSS-11.7G
    These architectural choices are selected after observing their impact on SNR in Table 1, making them fitted to the validation benchmarks. This is a form of model selection on the test set, which can bias the reported performance.
  • Training hyperparameters (learning rate, batch size, steps) = lr 0.001, batch 4, 375k steps
    These are standard choices, but they are tuned to produce consistent results among model scales, as stated in Section 5.
assumptions (3)
  • domain assumption MAC count is a valid proxy for inference time.
    The paper explicitly states this in Section 3: 'We use MAC as an indirect measure of inference time.' This is a load-bearing assumption because the headline efficiency gains are expressed in MAC reductions, not measured latency.
  • domain assumption The benchmark setup from TUSS is a fair and sufficient evaluation.
    The paper inherits the training and evaluation protocol from TUSS [17], including dataset mixtures and prompt categories. Any biases in that protocol are inherited.
  • domain assumption The qualitative structure of attention maps in TUSS generalizes to the optimized variants.
    The causal mask design is motivated by a qualitative analysis of attention maps (Fig. 3) in the original TUSS model, and then applied to the optimized models. If the attention patterns change after the architectural modifications, the mask design might be suboptimal.
invented entities (1)
  • Prompt-aware Conv-SwiGLU block
    purpose: A modified FFN block that processes prompts with linear layers and the mixture with the original Conv-SwiGLU structure.
    This is a new architectural component proposed in the paper. It has empirical evidence in the paper showing a small improvement, but no external validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FasTUSS: Faster Task-Aware Unified Source Separation." pith.science (2026). https://pith.science/paper/OI6B2HCX

@misc{pith2026250711435,
  author       = {Pith},
  title        = {Pith review of: FasTUSS: Faster Task-Aware Unified Source Separation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OI6B2HCX}},
  note         = {Machine review of arXiv:2507.11435}
}
read the original abstract

Time-Frequency (TF) dual-path models are currently among the best performing audio source separation network architectures, achieving state-of-the-art performance in speech enhancement, music source separation, and cinematic audio source separation. While they are characterized by a relatively low parameter count, they still require a considerable number of operations, implying a higher execution time. This problem is exacerbated by the trend towards bigger models trained on large amounts of data to solve more general tasks, such as the recently introduced task-aware unified source separation (TUSS) model. TUSS, which aims to solve audio source separation tasks using a single, conditional model, is built upon TF-Locoformer, a TF dual-path model combining convolution and attention layers. The task definition comes in the form of a sequence of prompts that specify the number and type of sources to be extracted. In this paper, we analyze the design choices of TUSS with the goal of optimizing its performance-complexity trade-off. We derive two more efficient models, FasTUSS-8.3G and FasTUSS-11.7G that reduce the original model's operations by 81\% and 73\% with minor performance drops of 1.2~dB and 0.4~dB averaged over all benchmarks, respectively. Additionally, we investigate the impact of prompt conditioning to derive a causal TUSS model.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 46 canonical work pages

  1. [1]

    Supervised speech separation based on deep learning: An overview,

    D. Wang and J. Chen, “Supervised speech separation based on deep learning: An overview,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 26, no. 10, pp. 1702–1726, 2018

    work page 2018

  2. [2]

    The Interspeech 2020 Deep Noise Suppression challenge: Datasets, subjective testing framework, and challenge results,

    C. K. Reddy, V . Gopal, R. Cutler, E. Beyrami et al. , “The Interspeech 2020 Deep Noise Suppression challenge: Datasets, subjective testing framework, and challenge results,” in Proc. Interspeech, Oct. 2020

    work page 2020

  3. [3]

    Toward universal speech enhancement for diverse input conditions,

    W. Zhang, K. Saijo, Z.-Q. Wang, S. Watanabe, and Y . Qian, “Toward universal speech enhancement for diverse input conditions,” in Proc. ASRU, 2023

    work page 2023

  4. [4]

    Deep clustering: Discriminative embeddings for segmentation and separation,

    J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clustering: Discriminative embeddings for segmentation and separation,” in Proc. ICASSP, 2016

    work page 2016

  5. [5]

    Permutation invariant training of deep models for speaker-independent multi-talker speech separation,

    D. Yu, M. Kolbæk, Z. H. Tan, and J. Jensen, “Permutation invariant training of deep models for speaker-independent multi-talker speech separation,” in Proc. ICASSP, 2017

    work page 2017

  6. [6]

    Dual-Path RNN: Efficient long sequence modeling for time-domain single-channel speech separation,

    Y . Luo, Z. Chen, and T. Yoshioka, “Dual-Path RNN: Efficient long sequence modeling for time-domain single-channel speech separation,” in Proc. ICASSP, 2020

    work page 2020

  7. [7]

    Attention is all you need in speech separation,

    C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, and J. Zhong, “Attention is all you need in speech separation,” in Proc. ICASSP, 2021

    work page 2021

  8. [8]

    TF-GridNet: Integrating full-and sub-band modeling for speech separation,

    Z.-Q. Wang, S. Cornell, S. Choi, Y . Lee et al. , “TF-GridNet: Integrating full-and sub-band modeling for speech separation,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 31, pp. 3221–3236, 2023

    work page 2023

Show all 53 references
  1. [9]

    Music source separation with band-split RNN,

    Y . Luo and J. Yu, “Music source separation with band-split RNN,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 31, pp. 1893– 1901, 2023

    1901

  2. [10]

    All for one and one for all: Improving music separation by bridging networks,

    R. Sawata, S. Uhlich, S. Takahashi, and Y . Mitsufuji, “All for one and one for all: Improving music separation by bridging networks,” in Proc. ICASSP, Jun. 2021

    2021

  3. [11]

    Universal sound separation,

    I. Kavalerov, S. Wisdom, H. Erdogan, B. Patton et al. , “Universal sound separation,” in Proc. WASPAA, 2019

    2019

  4. [12]

    Improving universal sound separation using sound classification,

    E. Tzinis, S. Wisdom, J. R. Hershey, A. Jansen, and D. P. Ellis, “Improving universal sound separation using sound classification,” in Proc. ICASSP, 2020

    2020

  5. [13]

    Multi-task audio source separation,

    L. Zhang, C. Li, F. Deng, and X. Wang, “Multi-task audio source separation,” in Proc. ASRU, 2021

    2021

  6. [14]

    The cocktail fork problem: Three-stem audio separation for real-world soundtracks,

    D. Petermann, G. Wichern, Z.-Q. Wang, and J. Le Roux, “The cocktail fork problem: Three-stem audio separation for real-world soundtracks,” in Proc. ICASSP, 2022

    2022

  7. [15]

    The Sound Demixing Challenge 2023 – Cinematic Demixing Track,

    S. Uhlich, G. Fabbro, M. Hirano, S. Takahashi et al. , “The Sound Demixing Challenge 2023 – Cinematic Demixing Track,” Transactions of the International Society for Music Information Retrieval , Apr. 2024

    2023

  8. [16]

    GASS: Generalizing audio source separation with large-scale data,

    J. Pons, X. Liu, S. Pascual, and J. Serr `a, “GASS: Generalizing audio source separation with large-scale data,” in Proc. ICASSP, 2024

    2024

  9. [17]

    Task- aware unified source separation,

    K. Saijo, J. Ebbers, F. G. Germain, G. Wichern, and J. Le Roux, “Task- aware unified source separation,” in Proc. ICASSP, 2025

    2025

  10. [18]

    Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,

    K. ˇZmol´ıkov´a, M. Delcroix, K. Kinoshita, T. Ochiai et al., “Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,” IEEE J. Sel. Top. Signal Process. , vol. 13, no. 4, pp. 800–814, 2019

    2019

  11. [19]

    Few-shot musical source separation,

    Y . Wang, D. Stoller, R. M. Bittner, and J. P. Bello, “Few-shot musical source separation,” in Proc. ICASSP, 2022

    2022

  12. [20]

    Zero-shot audio source separation through query-based learning from weakly-labeled data,

    K. Chen, X. Du, B. Zhu, Z. Ma et al., “Zero-shot audio source separation through query-based learning from weakly-labeled data,” in Proc. AAAI, 2022

    2022

  13. [21]

    V oicefilter: Targeted voice separation by speaker-conditioned spectrogram masking,

    Q. Wang, H. Muckenhirn, K. Wilson, P. Sridhar et al. , “V oicefilter: Targeted voice separation by speaker-conditioned spectrogram masking,” in Proc. Interspeech, 2019

    2019

  14. [22]

    Class- conditional embeddings for music source separation,

    P. Seetharaman, G. Wichern, S. Venkataramani, and J. Le Roux, “Class- conditional embeddings for music source separation,” in Proc. ICASSP, 2019

    2019

  15. [23]

    Listen to what you want: Neural network-based universal sound selector,

    T. Ochiai, M. Delcroix, Y . Koizumi, H. Itoet al., “Listen to what you want: Neural network-based universal sound selector,” in Proc. Interspeech, 2020

    2020

  16. [24]

    Heterogeneous target speech separation,

    E. Tzinis, G. Wichern, A. Subramanian, P. Smaragdis, and J. Le Roux, “Heterogeneous target speech separation,” in Proc. Interspeech, 2022

    2022

  17. [25]

    SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learning,

    M. Delcroix, J. B. V ´azquez, T. Ochiai, K. Kinoshita et al. , “SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learning,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 31, pp. 121–136, 2022

    2022

  18. [26]

    Audio query-based music source separation,

    J. H. Lee, H.-S. Choi, and K. Lee, “Audio query-based music source separation,” in Proc. ISMIR, 2019

    2019

  19. [27]

    Text-driven separation of arbitrary sounds,

    K. Kilgour, B. Gfeller, Q. Huang, A. Jansen et al., “Text-driven separation of arbitrary sounds,” in Proc. Interspeech, 2022

    2022

  20. [28]

    Separate what you describe: Language-queried audio source separation,

    X. Liu, H. Liu, Q. Kong, X. Mei et al. , “Separate what you describe: Language-queried audio source separation,” in Proc. Interspeech, 2022

    2022

  21. [29]

    Clapsep: Leveraging contrastive pre-trained model for multi-modal query-conditioned target sound extraction,

    H. Ma, Z. Peng, X. Li, M. Shao et al. , “Clapsep: Leveraging contrastive pre-trained model for multi-modal query-conditioned target sound extraction,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 32, pp. 4945–4960, 2024

    2024

  22. [30]

    TF- Locoformer: Transformer with local modeling by convolution for speech separation and enhancement,

    K. Saijo, G. Wichern, F. G. Germain, Z. Pan, and J. Le Roux, “TF- Locoformer: Transformer with local modeling by convolution for speech separation and enhancement,” arXiv preprint arXiv:2408.03440 , 2024

    2024 arXiv

  23. [31]

    Music source separation with band-split rope transformer,

    W.-T. Lu, J.-C. Wang, Q. Kong, and Y .-N. Hung, “Music source separation with band-split rope transformer,” in Proc. ICASSP, 2024

    2024

  24. [32]

    Efficient memory management for large language model serving with PagedAttention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng et al. , “Efficient memory management for large language model serving with PagedAttention,” Proc. SOSP, 2023

    2023

  25. [33]

    ShuffleNet: An extremely efficient convolutional neural network for mobile devices,

    X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An extremely efficient convolutional neural network for mobile devices,” Proc. CVPR, 2017

    2017

  26. [34]

    Shufflenet v2: Practi- cal guidelines for efficient CNN architecture design,

    N. Ma, X. Zhang, H. Zheng, and J. Sun, “Shufflenet v2: Practi- cal guidelines for efficient CNN architecture design,” arXiv preprint arXiv:1807.11164, 2018

    2018 arXiv

  27. [35]

    MicroNet: Improving image recognition with extremely low FLOPs,

    Y . Li, Y . Chen, X. Dai, D. Chen et al. , “MicroNet: Improving image recognition with extremely low FLOPs,” in Proc. ICCV, 2021

    2021

  28. [36]

    Xception: Deep learning with depthwise separable convolu- tions,

    F. Chollet, “Xception: Deep learning with depthwise separable convolu- tions,” in Proc. CVPR, 2016

    2016

  29. [37]

    MCUNet: Tiny deep learning on IoT devices,

    J. Lin, W.-M. Chen, Y . Lin, J. Cohn et al., “MCUNet: Tiny deep learning on IoT devices,” arXiv preprint arXiv:2007.10319 , 2020

    2007 arXiv

  30. [38]

    PhiNets: A scalable backbone for low-power AI at the edge,

    F. Paissan, A. Ancilotto, and E. Farella, “PhiNets: A scalable backbone for low-power AI at the edge,” ACM Trans. Embed. Comput. Syst. , vol. 21, pp. 1 – 18, 2021

    2021

  31. [39]

    Sudo RM -RF: Efficient networks for universal audio source separation,

    E. Tzinis, Z. Wang, and P. Smaragdis, “Sudo RM -RF: Efficient networks for universal audio source separation,” in Proc. MLSP, 2020

    2020

  32. [40]

    Exploring self-attention mechanisms for speech separation,

    C. Subakan, M. Ravanelli, S. Cornell, F. Grondin, and M. Bronzi, “Exploring self-attention mechanisms for speech separation,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 31, pp. 2169–2180, 2023

    2023

  33. [41]

    LUNA: Linear unified nested attention,

    X. Ma, X. Kong, S. Wang, C. Zhou et al., “LUNA: Linear unified nested attention,” arXiv preprint arXiv:2106.01540 , 2021

    2021 arXiv

  34. [42]

    EfficientViT: Memory efficient vision transformer with cascaded group attention,

    X. Liu, H. Peng, N. Zheng, Y . Yanget al., “EfficientViT: Memory efficient vision transformer with cascaded group attention,” in Proc. CVPR, 2023

    2023

  35. [43]

    The Voice Bank corpus: Design, collection and data analysis of a large regional accent speech database,

    C. Veaux, J. Yamagishi, and S. King, “The Voice Bank corpus: Design, collection and data analysis of a large regional accent speech database,” in Proc. O-COCOSDA/CASLRE, 2013

    2013

  36. [44]

    J. S. Garofolo et al. , CSR-I (WSJ0) Complete LDC93S6A , Linguistic Data Consortium, Philadelphia, 1993, web Download

    1993

  37. [45]

    Urgent challenge: Universality, robustness, and generalizability for speech enhancement,

    W. Zhang, R. Scheibler, K. Saijo, S. Cornell et al. , “Urgent challenge: Universality, robustness, and generalizability for speech enhancement,” in Proc. Interspeech, 2024

    2024

  38. [46]

    Fsd50k: an open dataset of human-labeled sound events,

    E. Fonseca, X. Favory, J. Pons, F. Font, and X. Serra, “Fsd50k: an open dataset of human-labeled sound events,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 30, pp. 829–852, 2021

    2021

  39. [47]

    WHAM!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu et al. , “WHAM!: Extending speech separation to noisy environments,” in Proc. Interspeech, 2019

    2019

  40. [48]

    The diverse environments multi- channel acoustic noise database (DEMAND): A database of multichannel environmental noise recordings,

    J. Thiemann, N. Ito, and E. Vincent, “The diverse environments multi- channel acoustic noise database (DEMAND): A database of multichannel environmental noise recordings,” in Proc. Mtgs. Acoust. , 2013

    2013

  41. [49]

    MUSDB18-HQ - an uncompressed version of MUSDB18,

    Z. Rafii, A. Liutkus, F.-R. St ¨oter, S. I. Mimilakis, and R. Bittner, “MUSDB18-HQ - an uncompressed version of MUSDB18,” Dec. 2019. [Online]. Available: https://doi.org/10.5281/zenodo.3338373

    2019 doi

  42. [50]

    MoisesDB: A dataset for source separation beyond 4-stems,

    I. Pereira, F. Ara´ujo, F. Korzeniowski, and R. V ogl, “MoisesDB: A dataset for source separation beyond 4-stems,” arXiv preprint arXiv:2307.15913 , 2023

    2023 arXiv

  43. [51]

    FMA: A dataset for music analysis,

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “FMA: A dataset for music analysis,” arXiv preprint arXiv:1612.01840 , 2016

    2016 arXiv

  44. [52]

    A comparative study on positional encoding for time-frequency domain dual-path transformer-based source separation models,

    K. Saijo and T. Ogawa, “A comparative study on positional encoding for time-frequency domain dual-path transformer-based source separation models,” arXiv preprint arXiv:2504.19605 , 2025

    2025 arXiv

  45. [53]

    RoFormer: Enhanced trans- former with rotary position embedding,

    J. Su, Y . Lu, S. Pan, B. Wen, and Y . Liu, “RoFormer: Enhanced trans- former with rotary position embedding,” arXiv preprint arXiv:2104.09864, 2021. Appendix A PERFORMANCE BREAKDOWN FOR ALL BENCHMARKS Table 3: Evaluation results of the model configurations in Table 1. SNR [dB...

    2021 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.