Pith. sign in

REVIEW 4 major objections 7 minor 26 references

SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pre-training self-supervised audio models on partially mixed audio—with a teacher that encodes the unmixed sources separately and a source retention loss—improves polyphonic tagging by up to 9.1% mAP while keeping monophonic benchmarks at…

desk verdict Solid new SSL pre-training recipe for polyphonic audio, but the SOTA claim is overblown: no external baselines on polyphonic datasets and a misreported AS-2M gain. read the letter →

arxiv 2506.12222 v1 pith:K3ZWHBQK submitted 2025-06-13 cs.SD cs.AIcs.LGeess.AS

Tony Alex , Sara Ahmed , Armin Mustafa , Muhammad Awais , Philip JB Jackson This is my paper
classification cs.SDcs.AIcs.LGeess.AS
keywords self-supervisedlearningaudiorepresentationpolyphonicsoundeventtaggingmixturesmaskedlatentbootstrappingsourceretentionlossSet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Self-supervised audio models are usually pre-trained and benchmarked on clips with a single dominant sound, but real audio is polyphonic—full of overlapping sources—and the paper argues this mismatch is why frozen SSL encoders underperform in realistic soundscapes. To fix it, SSLAM mixes audio during pre-training: the student encoder receives partially mixed spectrograms while a teacher encoder sees the two unmixed sources separately, and a new source retention loss pulls the student's representation of the mixture toward the average of the teacher's per-source representations. The paper reports that this pre-training recipe improves polyphonic sound-event tagging by up to 9.1% mean average precision (mAP), a standard multilabel tagging score, over prior state of the art in both linear evaluation and fine-tuning, while matching or exceeding monophonic benchmarks and reaching 50.2 mAP on AudioSet-2M. The larger claim is that polyphonic capability can be built into SSL pre-training itself, without labels and without giving up the frozen-encoder performance that makes these models practical.

What carries the argument

The load-bearing mechanism is the pairing of a spectrogram-domain mixing operator with the source retention loss. The mixing operator is an element-wise maximum in the log-mel domain, $S_{\mathrm{mixed}}(f,\tau)=\max(S_1(f,\tau),S_2(f,\tau))$, applied partially over three regions totaling half the clip, so the original audio keeps its identity while still contributing an overlapping event. The source retention loss, Equation (4), takes the student's decoded predictions for masked patches of the mixture and compares them, by mean squared error, with the average of the teacher's representations of the two unmixed sources: $(Z^{S_2}_{(i,k)}+Z^{S_1}_{(i,k)})/2$. This target is what forces the mixed representation to remain resolvable into its two sources; ablations show that mixing alone gives smaller gains and that adding this loss is what closes the gap on polyphonic benchmarks.

What would settle it

A decisive test would be to linear-evaluate SSLAM and its unmixed baseline on naturally recorded audio with verified overlapping events: if the 9.1% polyphonic margin seen on synthetic SPASS, URBAN-SED, and IDMT-DESED-FL does not transfer to real multi-source recordings, the teacher-average target has failed in the exact regime the paper claims to improve.

Watch

Extended reading notes

Core claim

SSLAM's central claim is that learning from audio mixtures, not just from unmixed clips, is what makes a self-supervised audio encoder ready for polyphonic soundscapes, and its ablations identify the source retention loss as the component that delivers that readiness. In the method, each audio clip is converted to a log-mel spectrogram, a time-frequency image of the sound; two clips are partly mixed by taking the element-wise maximum of their spectrograms over three regions covering half the clip's duration. The student encodes the mixed spectrogram, while the teacher separately encodes the two unmixed sources with the unmixed regions dropped, and the student's decoded patch predictions are trained against the average of the teacher's two per-source representations. The paper claims this target keeps the mixture representation decomposable into its constituent events, yielding new state-of-the-art results on polyphonic datasets (up to 9.1% mAP) and a 3.9% improvement on AudioSet-2M to 50.2 mAP, with monophonic benchmark performance maintained.

Load-bearing premise

The load-bearing premise is that averaging the teacher's separate internal representations of two unmixed audio clips is a valid target for the student's representation of their mixture, even though the element-wise max mixing can suppress the quieter source.

Editorial extensions

If this is right

  • On AudioSet-2M, SSLAM reaches 50.2 mAP with a 3.9% improvement over the prior best self-supervised result, so the mixture-based objective does not sacrifice large-scale general-audio tagging.
  • The largest polyphonic gains appear under linear evaluation, meaning the improvement lives in the frozen representations themselves, which is exactly the form in which pre-trained audio encoders are plugged into downstream multimodal systems.
  • The two-stage curriculum—foundational pre-training on unmixed audio followed by mixed-audio pre-training with the source retention loss—can be layered onto existing masked latent bootstrapping models without architectural changes.
  • On polyphonic evaluation sets the method sets new state-of-the-art results in both linear and fine-tuning regimes, with the margin growing as the number of distinct sound events rises, up to 9.7% mAP on clips with 8-9 events.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because the source retention target is an average of teacher features from two sources, the natural extrapolation is to mixtures of more than two sources by averaging $K$ teacher outputs; a testable consequence is that the per-polyphony-level gains should continue to grow with the number of events rather than plateau.
  • Beyond the paper: the element-wise max operator is justified through the ideal binary mask, so swapping it for a learned soft mask would separate the contribution of the loss from the contribution of the mixing operator; if the loss drives the effect, the soft mask should match the 40.9 mAP result on AS-20K without changing the targets.
  • Beyond the paper: the paper motivates frozen encoders for multimodal LLMs and audio-visual tasks but evaluates only tagging benchmarks; a direct transfer test would be to freeze SSLAM and measure captioning or audio-visual segmentation quality, which would show whether polyphonic capability survives outside the tagging setup.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes SSLAM, a self-supervised audio pre-training method that augments masked latent bootstrapping with audio mixtures and a source retention loss. The student is trained on partially mixed log-mel spectrograms (element-wise max, partial mixing), while an EMA teacher processes the unmixed sources and provides averaged feature targets. Training proceeds in two stages: Stage 1 pre-trains on unmixed audio with the standard global/local losses, and Stage 2 adds mixed-audio objectives and the source retention loss. The authors evaluate on standard audio SSL benchmarks (AS-2M, AS-20K, ESC-50, KS1, KS2) and on synthetic polyphonic soundscape datasets (SPASS, IDMT-DESED-FL, URBAN-SED, and a new Degrees-of-Polyphony split), reporting improvements in both linear and fine-tuning settings. The abstract claims new SOTA on AS-2M (50.2 mAP) and on polyphonic datasets (up to 9.1% mAP improvement).

Significance. If substantiated, SSLAM offers a practical and simple pre-training recipe for improving frozen audio encoders on polyphonic content, which is directly relevant to current uses of audio SSL backbones in multimodal systems. The paper's strengths are its systematic internal ablations (Tables 2, 3, 4, 5, 6), the three-seed robustness analysis in Appendix F, the honest reporting of a failed MixIT-based alternative in Appendix C, and the release of code and pre-trained models. The central weakness is that the polyphonic 'new SOTA' claim is supported only by comparisons against the authors' own baselines, not against external SSL models; additionally, all polyphonic evaluation datasets are synthetic, so the real-world generalization claim is not directly evidenced. The method itself is internally consistent and plausible, and the paper is a useful contribution if the comparative claims are properly restated or externally validated.

major comments (4)
  1. [Section 5, Tables 2 and 3] The central comparative claim that SSLAM sets new SOTA on polyphonic datasets is not supported by the experiments as reported. Tables 2 and 3 compare only SSLAM's internal variants (MB-UA, MB-PMA, MB-UA-PMA, SSLAM); no external SSL baseline (e.g., BEATs, Audio-MAE, EAT, ASiT, A-JEPA) is evaluated on SPASS, IDMT-DESED-FL, URBAN-SED, or the Degrees-of-Polyphony split under the same protocol. The 9.1% improvement quoted in the abstract is relative to the MB-UA baseline (Table 2, SPASS Market linear: 68.5 vs 62.8), not to any prior SOTA. Please add external baselines under identical protocols, or revise the abstract, Section 5, and contribution 4 to claim improvement over the authors' baseline rather than new SOTA.
  2. [Abstract and Table 1] The abstract states that SSLAM 'achieves up to a 3.9% improvement on AudioSet-2M,' but Table 1 shows the prior best AS-2M mAP is 48.6 (BEATsiter3+, A-JEPA, EAT), so 50.2 is a 3.3% relative gain, not 3.9%. Similarly, the 'up to 9.1%' polyphonic improvement is relative to the internal MB-UA baseline, not to an external SOTA. Please state the comparison base explicitly and correct the percentages in the abstract and Section 5.
  3. [Section 3.2.2, Eq. (4); Appendix B.0.2] The paper claims improved handling of 'real-world polyphonic audio,' yet every polyphonic evaluation dataset used is synthetic: SPASS, IDMT-DESED-FL, URBAN-SED, and the Degrees-of-Polyphony dataset are all generated by mixing isolated source recordings (Appendix B.0.2). Because both pre-training and evaluation rely on synthetic mixtures, the observed gains may reflect alignment with the synthetic mixing distribution rather than generalizable robustness to naturally polyphonic audio. The authors should either evaluate on a naturally polyphonic benchmark (e.g., AudioSet eval filtered for true polyphony, or FSD50K) or explicitly restrict the real-world claims to synthetic soundscapes.
  4. [Section 3.2.2, Eq. (4)] The SRL regresses the student's patch-level predictions on the mixed audio to the average of the teacher's per-source features (Eq. 4). If the element-wise max mix suppresses the quieter source in some time-frequency bins, the target for those bins is a mean of two source representations, one of which is poorly represented in the student's input. This potential target ambiguity is not analyzed, and the only feature-aggregation ablation (Appendix E.0.2, Table 11) is on AS-20K fine-tuning, where average and max differ by only 0.2 mAP. Please provide an analysis or ablation on polyphonic datasets with controlled source-level ratios to demonstrate that the averaged teacher target remains informative.
minor comments (7)
  1. [Abstract] The abstract contains a duplicated phrase: 'designed to improve, designed to improve the model's ability'; please remove the redundancy.
  2. [Section 3.2.1] The partial-mixing description says mixing is applied to '3 distinct regions, covering a total duration of t/2,' while the remaining audio is '2 × t/4 duration.' This implies five segments; please clarify the exact number and durations of mixed and unmixed regions.
  3. [Section 4.3 and Table 7] The text says Stage 2 variants were pre-trained with batch size 48, but Table 7 lists a pre-training batch size of 12. Please reconcile these numbers.
  4. [Algorithm 1 and Section 3.2.2] The algorithm says 'mask and drop unmixed regions in B post-positional embedding,' while the text says tokens from unmixed regions are discarded before passing through the teacher. Please clarify the order of operations.
  5. [Appendix D] The heading 'Comaprision' should be 'Comparison.'
  6. [Table 2] The SPASS subcolumns (Square, Park, Waterfront, Street, Market) are clear only if the reader infers the grouping from the header; consider adding a spanning header 'SPASS' over those five columns.
  7. [Eq. (4)] The notation 'nM C' appears with inconsistent spacing and is not defined in the equation; define it or use consistent notation from Eqs. (1)-(2).

Circularity Check

1 steps flagged · score 4.0 of 10

AS-20K results are partly selection-tuned; central polyphonic and AS-2M claims retain independent support.

  1. fitted input called prediction [Section 5 ('Additional ablations'), Tables 4-6, and Table 1 (AS-20K column)]
    "All the experiments discussed in this section are evaluated on downstream task AS-20K in the fine-tuning regime. ... We observed that top k for teacher layer averaging is 1 for global loss and 12 for local loss (refer to Table 6); in regard to the extent of spectrogram mixing, partial mixing was found to be better than full mixing (refer to Table 4)."

    The SSLAM configuration is selected by maximizing AS-20K fine-tuning mAP: the top-k layer choice, partial versus full mixing, global-loss inclusion, and feature aggregation strategy are all chosen using AS-20K mAP numbers reported in Tables 4-6, 10, and 11. The same AS-20K mAP (40.9) is then reported in Table 1 as an evaluation result and used as part of the claim that SSLAM 'maintains or exceeds performance on standard audio SSL benchmarks.' The AS-20K figure is therefore not an independent test of the design; it is the metric on which the design was tuned, so this particular benchmark result is partly a fitted input presented as a predictive outcome.

full rationale

The main derivation chain of SSLAM is not circular: the pre-training objectives (Lglobal, Llocal, LSRL) are defined independently of the downstream benchmarks, and the headline AS-2M mAP of 50.2 and the polyphonic-dataset improvements are not used for hyperparameter selection. The most concrete circularity is localized to AS-20K: the paper explicitly states that its additional ablations are evaluated on AS-20K fine-tuning mAP, and those ablations determine key design choices (top-k layers, partial vs. full mixing, feature averaging, global-loss inclusion) of the final SSLAM model. Reporting the resulting AS-20K number in Table 1 as a benchmark outcome is thus selection-tuned rather than an independent evaluation. This does not invalidate the central polyphonic claim, which is supported by internal ablations on SPASS, IDMT-DESED-FL, URBAN-SED, and the Degrees-of-Polyphony dataset, but it does mean the AS-20K-based evidence for 'maintains or exceeds performance on standard benchmarks' is weakened. The self-citations to ASiT (Ahmed et al., 2024) are not load-bearing: ASiT appears as a prior method in Table 1 and as a general reference, not as the justification for SSLAM's design. The lack of external SOTA baselines in the polyphonic tables is a correctness/experimental-design concern about the 'new SOTA' claim, not a circularity, so it is not scored as such here.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No physical or metaphysical entities are introduced. The free parameters are hyperparameters and design choices tuned on AS-20K; the axioms are the methodological assumptions about mixing, teacher targets, dataset representativeness, and the AudioSet polyphony estimate. The new loss function (SRL) is a training objective, not an entity with independent empirical evidence in the sense of this ledger.

free parameters (5)
  • Teacher layer choice (top-k) for global and local losses = global k=1 (final layer), local k=12 (all layers)
    Selected by fine-tuning mAP on AS-20K in Table 6; alternative settings give 38.8-40.5 mAP, so the choice affects the reported result.
  • Partial mixing configuration = 3 mixed regions totaling t/2 duration
    Selected over full mixing in Table 4 based on AS-20K fine-tuning mAP (40.6 vs 40.4 in Stage 2).
  • Mixing operation = element-wise max in log-mel spectrogram domain
    Selected over spectrogram average and waveform average in Table 10 based on AS-20K fine-tuning mAP (40.9 vs 40.8 vs 40.4).
  • SRL feature aggregation = average of the two teacher representations
    Selected over element-wise max of features in Table 11 based on AS-20K mAP (40.9 vs 40.7).
  • Stage 1 / Stage 2 curriculum = 10 epochs unmixed-only, then 5 epochs with half mixed and half unmixed batch
    Chosen by the authors; Appendix D argues the half/half batch split trades polyphonic performance for monophonic robustness, but no sweep over curriculum lengths is reported.
assumptions (5)
  • ad hoc to paper Element-wise max of log-mel spectrograms is a useful proxy for a polyphonic audio mixture.
    Introduced in Eq. 3 and motivated by ideal binary masks; the paper does not compare against additive waveform mixtures in the main pre-training, only in Table 10, and does not show that max mixing preserves both sources.
  • ad hoc to paper Averaging the teacher features of the two unmixed sources gives a valid regression target for the student's representation of the mixed input.
    Used in Eq. 4 for the source retention loss and in Figure 1; if feature averaging loses source identity, the SRL objective would not teach what the paper claims.
  • domain assumption AudioSet's true polyphony is approximately 35.6-42.5% of clips, as estimated by the hierarchy-level analysis.
    Appendix B.1 and Figure 3; this motivates the need for mixture-based pre-training and is an estimate based on the authors' label-removal criteria, not ground truth.
  • domain assumption Synthetic polyphonic datasets (SPASS, URBAN-SED, IDMT-DESED-FL, Degrees of Polyphony) are representative of real-world polyphonic audio.
    Used for all polyphonic evaluations in Tables 2 and 3; the datasets are generated from monophonic source sets, so transfer to real mixtures is assumed rather than measured.
  • standard math Masked latent bootstrapping with inverse block multi-masking is a valid SSL baseline.
    Adopted from Baevski et al. (2023), Chen et al. (2024), and ASiT as the foundation in Section 3.1; this is prior methodology, not novel.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes." pith.science (2026). https://pith.science/paper/K3ZWHBQK

@misc{pith2026250612222,
  author       = {Pith},
  title        = {Pith review of: SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3ZWHBQK}},
  note         = {Machine review of arXiv:2506.12222}
}
read the original abstract

Self-supervised pre-trained audio networks have seen widespread adoption in real-world systems, particularly in multi-modal large language models. These networks are often employed in a frozen state, under the assumption that the SSL pre-training has sufficiently equipped them to handle real-world audio. However, a critical question remains: how well do these models actually perform in real-world conditions, where audio is typically polyphonic and complex, involving multiple overlapping sound sources? Current audio SSL methods are often benchmarked on datasets predominantly featuring monophonic audio, such as environmental sounds, and speech. As a result, the ability of SSL models to generalize to polyphonic audio, a common characteristic in natural scenarios, remains underexplored. This limitation raises concerns about the practical robustness of SSL models in more realistic audio settings. To address this gap, we introduce Self-Supervised Learning from Audio Mixtures (SSLAM), a novel direction in audio SSL research, designed to improve, designed to improve the model's ability to learn from polyphonic data while maintaining strong performance on monophonic data. We thoroughly evaluate SSLAM on standard audio SSL benchmark datasets which are predominantly monophonic and conduct a comprehensive comparative analysis against SOTA methods using a range of high-quality, publicly available polyphonic datasets. SSLAM not only improves model performance on polyphonic audio, but also maintains or exceeds performance on standard audio SSL benchmarks. Notably, it achieves up to a 3.9\% improvement on the AudioSet-2M (AS-2M), reaching a mean average precision (mAP) of 50.2. For polyphonic datasets, SSLAM sets new SOTA in both linear evaluation and fine-tuning regimes with performance improvements of up to 9.1\% (mAP).

Figures

Figures reproduced from arXiv: 2506.12222 by the authors.

Figure 1
Figure 1. Overview of the components in our proposed audio SSL pre-training on unlabeled data [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of novel contributions in SSLAM. The left side demonstrates the partial mix [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Percentage of audio files in AudioSet (Gemmeke et al., 2017) with at least 2 distinct sound [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Overview of concept separation using mixture invariant training loss [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Visualisation of mixing using element-wise maximum of log-mel spectrograms. Each [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 8 canonical work pages

  1. [1]

    In our initial experiments, we observed that this approach yielded worse performance compared to SSLAM on the AS-20K benchmark (39.9 mAP vs

    This assigns each of the K representation vectors to either Z S1 or Z S2. In our initial experiments, we observed that this approach yielded worse performance compared to SSLAM on the AS-20K benchmark (39.9 mAP vs. 40.9 mAP). One key drawback of this approach is that mixture invariant training assumes the independence of individual sources, which is a rea...

    work page 2025

  2. [4]

    This substantiates our argument that relying solely on AudioSet is insufficient for developing models capable of handling polyphonic audio effectively. 16 Published as a conference paper at ICLR 2025 C R EPRESENTATION LEARNING VIA CONCEPT SEPARATION WITH MIXTURE INVARIANT LOSS In the development of SSLAM, we investigated whether separating multiple concep...

    work page 2025

  3. [8]

    Efficient training of audio Transformers with patchout

    Khaled Koutini, Jan Schl ¨uter, Hamid Eghbal-Zadeh, and Gerhard Widmer. Efficient training of audio Transformers with patchout. arXiv preprint arXiv:2110.05069,

    arXiv

  4. [9]

    SGDR: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983,

    arXiv

  5. [10]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101,

    arXiv

  6. [12]

    X-instructblip: A framework for aligning x-modal instruction-aware representations to llms and emergent cross-modal reasoning

    Artemis Panagopoulou, Le Xue, Ning Yu, Junnan Li, Dongxu Li, Shafiq Joty, Ran Xu, Silvio Savarese, Caiming Xiong, and Juan Carlos Niebles. X-instructblip: A framework for aligning x-modal instruction-aware representations to llms and emergent cross-modal reasoning. arXiv preprint arXiv:2311.18799,

    arXiv

  7. [13]

    Specaugment: A simple data augmentation method for automatic speech recognition

    12 Published as a conference paper at ICLR 2025 Daniel S Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D Cubuk, and Quoc V Le. Specaugment: A simple data augmentation method for automatic speech recognition. arXiv preprint arXiv:1904.08779,

    arXiv 2025

  8. [18]

    Video-llama: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858,

    arXiv

Show all 26 references
  1. [19]

    mixup: Beyond empirical risk minimization

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412,

    arXiv

  2. [20]

    Chatbridge: Bridging modalities with large language model as a language catalyst

    Zijia Zhao, Longteng Guo, Tongtian Yue, Sihan Chen, Shuai Shao, Xinxin Zhu, Zehuan Yuan, and Jing Liu. Chatbridge: Bridging modalities with large language model as a language catalyst. arXiv preprint arXiv:2305.16103,

    arXiv

  3. [21]

    13 Published as a conference paper at ICLR 2025 A T RAINING HYPER -PARAMETERS Additional hyper-parameters used in pre-training using AS-2M and fine-tuning of standard audio SSL benchmark datasets are listed in Table

    2025

  4. [22]

    0.8 0.8 0.8 0.8 0.8 Multilabel True True True True True Loss Function BCE BCE BCE BCE BCE Dataset Mean for Normalization -5.275 -5.464 -5.561 -5.216 |-5.659 -4.268 Dataset Std for Normalization 3.268 3.380 2.699 3.376 |2.620 4.569 14 Published as a conference paper at ICLR 202...

    2025

  5. [23]

    Each recording is annotated with a single class

    is a collection of 2000, 5-second environmental sound recordings across 50 classes. Each recording is annotated with a single class. Following previous works Chen et al. (2024; 2022); He et al. (2022), we employ a 5-fold cross- validation setting and report the classification ...

    2022

  6. [24]

    multi-label

    is one of the largest collections of multi-labeled audio files. While the term “multi-label” may suggest the presence of multiple distinct sound events within a single audio file, implying polyphony, this is not always the case. As discussed in the introduction, labels such as...

    2025

  7. [2005]

    Speech commands: A dataset for limited-vocabulary speech recognition

    Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209,

    arXiv

  8. [2011]

    Dropout: a simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958,

    1929

  9. [2014]

    Salmonn: Towards generic hearing abilities for large language models

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. Salmonn: Towards generic hearing abilities for large language models. arXiv preprint arXiv:2310.13289,

    arXiv

  10. [2015]

    Scaper: A library for soundscape synthesis and augmentation

    Justin Salamon, Duncan MacConnell, Mark Cartwright, Peter Li, and Juan Pablo Bello. Scaper: A library for soundscape synthesis and augmentation. In 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 344–348. IEEE,

    2017

  11. [2016]

    Masked autoencoders that listen

    Po-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, et al. Masked autoencoders that listen. In Proc. NeurIPS, 2022a. Po-Yao Huang, Hu Xu, Juncheng Li, et al. Masked autoencoders that listen. arXiv preprint arXiv:2207.06405, 2022b. David S Johnson, Wolfgang Lorenz, Michael Taenze...

    2021 arXiv

  12. [2017]

    AST: Audio spectrogram Transformer.arXiv preprint arXiv:2104.01778,

    Yuan Gong, Yu-An Chung, and James Glass. AST: Audio spectrogram Transformer.arXiv preprint arXiv:2104.01778,

    arXiv

  13. [2020]

    MAE-AST: Masked autoencoding audio spectro- gram Transformer

    Alan Baade, Puyuan Peng, and David Harwath. MAE-AST: Masked autoencoding audio spectro- gram Transformer. arXiv preprint arXiv:2203.16691,

    arXiv

  14. [2021]

    Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, and Kunio Kashino

    URL https: //arxiv.org/abs/2103.06695. Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, and Kunio Kashino. Masked spectrogram modeling using masked autoencoders for learning general-purpose audio represen- tation. In HEAR: Holistic Evaluation of Audio Represent...

    arXiv

  15. [2022]

    Eat: Self-supervised pre- training with efficient audio transformer

    Wenxi Chen, Yuzhe Liang, Ziyang Ma, Zhisheng Zheng, and Xie Chen. Eat: Self-supervised pre- training with efficient audio transformer. arXiv preprint arXiv:2401.03497,

    arXiv

  16. [2023]

    An im- age is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, et al. An im- age is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

    2010 arXiv

  17. [2024]

    11 Published as a conference paper at ICLR 2025 Jort F

    URL https://arxiv.org/abs/2311.15830. 11 Published as a conference paper at ICLR 2025 Jort F. Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter. Audio Set: An ontology and human-labeled dataset for audio ev...

    2025 arXiv

  18. [9304]

    URL http://dx.doi.org/10.1109/TASLP

    doi: 10.1109/taslp.2024.3428908. URL http://dx.doi.org/10.1109/TASLP. 2024.3428908. Augustin Arnault, Baptiste Hanssens, and Nicolas Riche. Urban sound classification: striving to- wards a fair comparison. arXiv preprint arXiv:2010.11805,

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.