Pith. sign in

REVIEW 3 major objections 5 minor 49 references

Self-Supervised Convolutional Audio Models are Flexible Acoustic Feature Learners: A Domain Specificity and Transfer-Learning Study

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The domain of pre-training data has only a small effect on how well a self-supervised audio model transfers to downstream speech and non-speech tasks.

desk verdict Useful and reproducible study of BYOL-A pre-training diets, but the 'non-speech' diet is contaminated with speech, so the domain-specificity claim needs a cleaner test. read the letter →

arxiv 2502.02366 v1 pith:VIEVKEA6 submitted 2025-02-04 eess.AS

Mattson Ogg This is my paper · ORCID
classification eess.AS
keywords self-supervisedlearningaudiorepresentationtransferdomainspecificityBYOL-Arepresentationalsimilarityanalysissoundeventrecognitionspeech
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the domain of a self-supervised audio model's pre-training data limits what tasks it can later be transferred to. Using the BYOL-A convolutional architecture, the author pre-trains three models on matched volumes of speech, non-speech, and combined audio, then evaluates each on held-out speech, non-speech, and voice-activity-detection tasks. The central finding is that pre-training domain has only a small effect: all three models perform well on nearly all tasks, often matching or beating domain-specific baselines such as Yamnet and WavLM in their own or opposite domains. The paper interprets this as evidence that SSL convolutional models learn flexible acoustic features whose transfer value does not strongly depend on matching the pre-training diet to the target domain.

What carries the argument

The load-bearing mechanism is BYOL-A ('bootstrap your own latent for audio'), a self-supervised method that applies random audio augmentations—noise, cropping, resizing, time-frequency stretching, amplitude modulation—to spectrograms and trains a two-block convolutional network to output representations invariant to those manipulations. The transfer protocol then freezes the pre-trained embeddings and learns only a linear layer on top for each downstream task, so downstream performance reflects the quality of the pre-trained features themselves. Representational similarity analysis (RSA) is the interpretative tool: it correlates pairwise distances among a model's embeddings with pairwise differences along acoustic descriptor dimensions, revealing which features the models encode.

What would settle it

Train a BYOL-A model on a rigorously speech-free subset of the non-speech corpora (e.g., AudioSet clips with no speech-related labels, MUSAN noise and music only) and a matched speech-only model, then compare their transfer accuracy on a speech task like Speech Commands. If the clean non-speech model loses most of its speech-task performance, the paper's flexibility conclusion would be substantially weakened.

Watch

Extended reading notes

Core claim

The paper claims that BYOL-A models pre-trained on speech data, non-speech data, or both converge on similar, broadly useful acoustic representations. Across held-out transfer tasks—keyword recognition, speaker recognition, sound event recognition, instrument recognition, and voice activity detection—the three models' accuracies cluster together, with only small advantages for diet-matched tasks. The domain-specific baselines behave differently: Yamnet excels on non-speech tasks but fades on speech, and WavLM excels on speech but struggles on non-speech. Representational similarity analysis shows the BYOL-A models' embedding spaces are highly correlated with one another (Spearman $\rho = 0.88$–$0.95$) and share sensitivity to spectral energy in frequency bands above 600 Hz, spectral variability, overall amplitude, and pitch. The author concludes that this SSL approach yields flexible feature learners for which domain mismatch between pre-training and downstream data is not a major penalty, at least for classification-style transfer.

Load-bearing premise

The clean separation between 'speech' and 'non-speech' pre-training diets assumes the non-speech corpus contains no speech; in fact the MUSAN and AudioSet portions include speech material, so the apparent domain flexibility could partly come from speech leaking into the non-speech model.

Editorial extensions

If this is right

  • Practitioners can pre-train one SSL audio model on whatever unlabeled audio is abundant and expect it to transfer across speech and non-speech classification tasks, reducing the need for domain-matched pre-training data.
  • Domain-specific supervised baselines remain valuable in their home domain, but their out-of-domain brittleness makes SSL models a safer default when the downstream domain is uncertain or mixed.
  • The high inter-model representational similarity ($\rho = 0.88$–$0.95$) suggests that pre-training diet is not the main determinant of what a BYOL-A model learns; augmentation strategy and architecture may matter more.
  • For voice activity detection, all models, including those with little or no speech pre-training, performed well even in the data-limited regime, indicating VAD features are learnable from generic audio.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The convergence of representational spaces across diets hints that BYOL-A's augmentation set imposes a stronger inductive bias than the data distribution; one testable extension would be to ablate individual augmentations and measure how much cross-domain transfer survives.
  • Scale may interact with this result: the paper itself notes WavLM used far more data; at larger scales or with transformer architectures, domain specificity could re-emerge, especially on sequence-level or regression tasks, which were not tested here.
  • The results suggest a practical recipe for data-exploration tools: a single BYOL-A embedding space can serve outlier detection, sound synthesis steering, or bioacoustic monitoring without retraining, provided the target features align with the spectral/source character the model encodes.
  • A stronger test of flexibility would be few-shot or zero-shot probing—evaluating the frozen embeddings with no fine-tuning—where differences between diets might be more visible than after a learned linear layer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper investigates whether the domain of self-supervised pre-training data affects the downstream transfer performance of a convolutional BYOL-A audio model. The author trains three BYOL-A models from scratch on roughly 5,000-hour corpora of speech-only, non-speech-only, and combined audio, then evaluates frozen embeddings via linear probing on seven held-out tasks spanning speech, non-speech, and voice-activity detection, comparing against YAMNet and WavLM baselines. Representational similarity analysis is used to compare the learned embedding spaces across diets. The central claim is that all three BYOL-A models perform well across nearly all tasks with only small domain-specificity advantages, suggesting that SSL convolutional audio models learn flexible representations despite domain mismatch.

Significance. If the conclusion were fully supported, the paper would provide a practical and useful message for transfer learning in audio: that SSL pre-training on either speech or non-speech data may yield representations flexible enough for downstream tasks in both domains. The study uses a clean linear-probing protocol, standard public benchmark datasets, and publicly releases the three pre-trained checkpoints, which are concrete contributions that support reproducibility. The RSA comparison to acoustic descriptors is also a thoughtful addition. However, the central domain-specificity comparison is weakened by a testable confound in the construction of the non-speech pre-training corpus, and the 'small differences' claim lacks uncertainty quantification.

major comments (3)
  1. [II.A, Table I] The non-speech pre-training corpus is contaminated with speech, which confounds the central comparison. MUSAN (ref. 26) is by its own name a Music, Speech, and Noise corpus and contains a dedicated speech subset, while AudioSet (ref. 25) includes speech-related labels such as 'Speech' and 'Human voice'; the paper does not state that these clips were removed. Since AudioSet contributes 4,817 of the roughly 5,200 non-speech hours, even a small speech fraction corresponds to hundreds of hours. Consequently, the BYOL-A model labeled 'Non-Speech' likely saw substantial speech during pre-training, and its strong performance on Speech Commands, VCTK, and AVA-Speech is not clean evidence that pre-training domain has little effect. The claim in Section III that 'the pre-training domain only had a small influence on downstream task performance' is therefore not cleanly tested; the author should either re-train the non-speech model on a verified speech-free corpus or clearly qualify the result.
  2. [III, Fig. 1] The paper reports no uncertainty or significance testing for the differences among the three BYOL-A diets. The conclusion that only 'small' domain-specificity advantages exist appears to be based on visual inspection of single runs. Because pre-training involves stochasticity, and because the claim is specifically about the size of a difference, the paper should provide multiple seeds with confidence intervals or a statistical comparison of the accuracy distributions, at least for the key tasks where the differences are visually small.
  3. [IV] The limitations section does not acknowledge the speech contamination in the non-speech pre-training corpus. Given that this is a direct threat to the main claim about domain specificity, it should be stated explicitly and discussed, along with the potential impact on the interpretation of the RSA results.
minor comments (5)
  1. [Global] The paper uses 'Yamnet' and 'YAMNet' inconsistently; standardize the model name. Also, in Figure 1 the label 'Y amnet' contains an erroneous space.
  2. [References to supplements] The paper references 'Supplemental Table 1' and 'Supplemental Figures 1 and 2' but these are not included in the manuscript; ensure they are available with the submission.
  3. [II.D] The decision to retain only correlations with rs > 0 before FDR correction is unusual and should be justified; a positive threshold can bias the average correlations upward.
  4. [III] Minor grammatical issues: e.g., 'a sounds' overall amplitude' should be 'a sound's overall amplitude'.
  5. [II.B] The acronym 'BYOL-A1' in the text should be 'BYOL-A'; the footnote marker is not needed in the running text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study is an empirical evaluation on held-out benchmarks, and no fitted parameter is renamed as a prediction.

full rationale

The paper reports an empirical transfer-learning study rather than a derivation. Three BYOL-A models are pre-trained from scratch on speech, non-speech, and combined corpora; downstream models are linear probes trained on frozen embeddings and evaluated on held-out test partitions (Section II.C). No parameter is fitted to the downstream test results, and no target metric is used to define the pre-training objective. The only self-citations are [43] and [44], which supply a set of acoustic descriptors for RSA; these are fixed measurement dimensions from prior published work, not quantities fitted to the present results, so they do not make the representational-similarity findings equivalent to their inputs. BYOL-A itself comes from external references [11,12]. The skeptical concern that MUSAN and AudioSet contain speech is a dataset-contamination/validity issue that could weaken the domain-specificity comparison, but it is not circularity: the observed transfer results are not true by construction, and the paper's main flexibility claim has independent content (e.g., the speech-only model performs well on non-speech tasks and vice versa). Therefore no circular steps are present; score 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new theoretical entities. It relies on standard SSL methodology, hand-chosen training hyperparameters, and a set of acoustic descriptors from prior work. The main empirical caveat is the misclassification of MUSAN and AudioSet as 'non-speech' pre-training data, which is a data-selection issue rather than a new free parameter.

free parameters (4)
  • Pre-training epochs = 100
    Chosen by hand; training duration affects representation quality but is not fitted to downstream performance.
  • Fine-tuning learning rate = 0.0001
    Adam optimizer learning rate used for all linear probes; standard value, not tuned per task.
  • Fine-tuning epochs = 50 (25 for NSynth, 10 for AVA-Speech)
    Task-specific training durations; selected by hand.
  • Model width (conv channels/MLP units) = 64 channels, 2048 units
    Architecture inherited from prior BYOL-A work; model capacity could influence domain flexibility.
assumptions (5)
  • domain assumption BYOL-A self-supervised learning produces useful audio representations when trained from scratch on these corpora.
    The work relies on the BYOL-A method [11,12] working as intended; if the pre-training failed to learn, the downstream results would be uninformative.
  • domain assumption Pre-training data volumes are approximately matched across speech and non-speech diets.
    Section II.A states volumes are 'closely matched', but non-speech has slightly longer total duration and speech has more examples; this imbalance could affect the comparison.
  • domain assumption The held-out transfer learning datasets do not overlap with the pre-training corpora.
    The paper asserts held-out status but does not empirically verify zero overlap, especially with AudioSet or FSD Superset, which might contain related YouTube or Freesound content.
  • domain assumption A linear probe on frozen embeddings is a valid measure of representation transferability.
    This is a standard protocol, but it assumes linear separability captures useful feature quality; non-linear fine-tuning might produce different conclusions.
  • domain assumption Acoustic features used in RSA are meaningful perceptual descriptors for speech and non-speech sounds.
    Features are taken from Ogg and Slevc (2019); if these descriptors do not relate to the dimensions humans attend to, the RSA interpretative claims lose force.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Supervised Convolutional Audio Models are Flexible Acoustic Feature Learners: A Domain Specificity and Transfer-Learning Study." pith.science (2026). https://pith.science/paper/VIEVKEA6

@misc{pith2026250202366,
  author       = {Pith},
  title        = {Pith review of: Self-Supervised Convolutional Audio Models are Flexible Acoustic Feature Learners: A Domain Specificity and Transfer-Learning Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VIEVKEA6}},
  note         = {Machine review of arXiv:2502.02366}
}
read the original abstract

Self-supervised learning (SSL) algorithms have emerged as powerful tools that can leverage large quantities of unlabeled audio data to pre-train robust representations that support strong performance on diverse downstream tasks. Up to now these have mostly been developed separately for speech and non-speech applications. Here, we explored the domain specificity of a convolutional model's pre-training data relative to different downstream speech and non-speech tasks using a self-supervised pre-training approach (BYOL-A). We found that these pre-trained models (regardless of whether they were pre-trained on speech data, non-speech data or both) enabled good performance on nearly all downstream tasks, beating or nearly matching the performance of popular domain-specific models. Only small domain-specificity advantages were observed between the different pre-training datasets. The popular domain-specific models used as baselines performed very well in their target domains, but generally faltered outside of them. Together, these results demonstrate that SSL methods can be a powerful way to learn flexible representations for domain specific data without labels. These models can be a powerful resource for later transfer learning, fine-tuning or data exploration applications when the downstream data are similar, but also perhaps when there may be a domain mismatch.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 19 canonical work pages

  1. [1]

    PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition,

    Q. Kong, Y. Cao, T. Iqbal, Y. Wang, W. Wang, and M. D. Plumbley, “PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition,” IEEEACM Trans Audio Speech Lang Proc, vol. 28, pp. 2880–2894, Nov. 2020, doi: 10.1109/TASLP.2020.3030497

    arXiv 2020

  2. [2]

    Embeddings for up to 2000 random samples from the validation partition of select datasets representing speech, non-speech and VAD audio domains

    T-SNE visualization of the BYOL-A (All) embeddings. Embeddings for up to 2000 random samples from the validation partition of select datasets representing speech, non-speech and VAD audio domains. Axes comprise arbitrary units. Examples are colored by the target classes used in the transfer learning experiments in A, B C and E. The separation of classes i...

    work page 2000

  3. [3]

    Using State of the Art Speaker Recognition and Natural Language Processing Technologies to Detect Alzheimer’s Disease and Assess its Severity,

    R. Pappagari, J. Cho, L. Moro-Velázquez, and N. Dehak, “Using State of the Art Speaker Recognition and Natural Language Processing Technologies to Detect Alzheimer’s Disease and Assess its Severity,” in Interspeech 2020, ISCA, Oct. 2020, pp. 2177–2181. doi: 10.21437/Interspeech.2020-2587

    doi:10.21437/interspeech.2020-2587 2020

  4. [4]

    wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations,

    A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2020, pp. 12449–12460. Accessed: Nov. 26,

    work page 2020

  5. [5]

    Characterizing soundscapes across diverse ecosystems using a universal acoustic feature set,

    S. S. Sethi et al., “Characterizing soundscapes across diverse ecosystems using a universal acoustic feature set,” Proc. Natl. Acad. Sci., vol. 117, no. 29, pp. 17049–17055, Jul. 2020, doi: 10.1073/pnas.2004702117

    work page doi:10.1073/pnas.2004702117 2020

  6. [6]

    Soundscapes and deep learning enable tracking biodiversity recovery in tropical forests,

    J. Müller et al., “Soundscapes and deep learning enable tracking biodiversity recovery in tropical forests,” Nat. Commun., vol. 14, no. 1, p. 6191, Oct. 2023, doi: 10.1038/s41467-023-41693-w

    doi:10.1038/s41467-023-41693-w 2023

  7. [7]

    Using X-Vectors to Automatically Detect Parkinson’s Disease from Speech,

    L. Moro-Velazquez, J. Villalba, and N. Dehak, “Using X-Vectors to Automatically Detect Parkinson’s Disease from Speech,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2020, pp. 1155–1159. doi: 10.1109/ICASSP40776.2020.9053770

    arXiv 2020

  8. [8]

    Representation Learning with Contrastive Predictive Coding,

    A. van den Oord, Y. Li, and O. Vinyals, “Representation Learning with Contrastive Predictive Coding,” Jan. 22, 2019, arXiv: arXiv:1807.03748. doi: 10.48550/arXiv.1807.03748

    arXiv doi:10.48550/arxiv.1807.03748 2019

Show all 49 references
  1. [9]

    Unsupervised Cross-lingual Representation Learning for Speech Recognition,

    A. Conneau, A. Baevski, R. Collobert, A. Mohamed, and M. Auli, “Unsupervised Cross-lingual Representation Learning for Speech Recognition,” Dec. 15, 2020, arXiv: arXiv:2006.13979. doi: 10.48550/arXiv.2006.13979

    2020 doi arXiv

  2. [10]

    Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training,

    W.-N. Hsu et al., “Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training,” Sep. 08, 2021, arXiv: arXiv:2104.01027. doi: 10.48550/arXiv.2104.01027

    2021 doi arXiv

  3. [11]

    BYOL for Audio: Exploring Pre-Trained General-Purpose Audio Representations,

    D. Niizumi, D. Takeuchi, Y. Ohishi, N. Harada, and K. Kashino, “BYOL for Audio: Exploring Pre-Trained General-Purpose Audio Representations,” IEEEACM Trans Audio Speech Lang Proc, vol. 31, pp. 137–151, Nov. 2022, doi: 10.1109/TASLP.2022.3221007

    2022

  4. [12]

    BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation,

    D. Niizumi, D. Takeuchi, Y. Ohishi, N. Harada, and K. Kashino, “BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation,” Apr. 21, 2021, arXiv: arXiv:2103.06695. doi: 10.48550/arXiv.2103.06695

    2021 doi arXiv

  5. [13]

    WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing,

    S. Chen et al., “WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing,” Jun. 17, 2022, arXiv: arXiv:2110.13900. doi: 10.48550/arXiv.2110.13900

    2022 doi

  6. [14]

    The fifth ‘CHiME’ Speech Separation and Recognition Challenge: Dataset, task and baselines,

    J. Barker, S. Watanabe, E. Vincent, and J. Trmal, “The fifth ‘CHiME’ Speech Separation and Recognition Challenge: Dataset, task and baselines,” Mar. 28, 2018, arXiv: arXiv:1803.10609. doi: 10.48550/arXiv.1803.10609

    2018 doi arXiv

  7. [15]

    Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus,

    J. Carletta, “Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus,” Lang. Resour. Eval., vol. 41, no. 2, pp. 181–190, 2007, doi: 10.1007/s10579-007-9040-x

    2007 doi

  8. [16]

    Common Voice: A Massively-Multilingual Speech Corpus,

    R. Ardila et al., “Common Voice: A Massively-Multilingual Speech Corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Conference, N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, ...

    2020

  9. [17]

    Available: https://aclanthology.org/2020.lrec-1.520

    [Online]. Available: https://aclanthology.org/2020.lrec-1.520

    2020

  10. [18]

    Recognition and understanding of meetings the AMI and AMIDA projects,

    S. Renals, T. Hain, and H. Bourlard, “Recognition and understanding of meetings the AMI and AMIDA projects,” in 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU), The Westin Miyako Kyoto: IEEE, 2007, pp. 238–247. doi: 10.1109/ASRU.2007.4430116

    2007

  11. [19]

    Enhancing the TED-LIUM Corpus with Selected Data for Language Modeling and More TED Talks,

    A. Rousseau, P. Deléglise, and Y. Estève, “Enhancing the TED-LIUM Corpus with Selected Data for Language Modeling and More TED Talks,” in Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), N. Calzolari, K. Choukri, T. Declerck, H....

    2014

  12. [20]

    VoxCeleb: A Large-Scale Speaker Identification Dataset,

    A. Nagrani, J. S. Chung, and A. Zisserman, “VoxCeleb: A Large-Scale Speaker Identification Dataset,” in Interspeech 2017, ISCA, Aug. 2017, pp. 2616–2620. doi: 10.21437/Interspeech.2017-950

    2017 doi

  13. [21]

    Librispeech: An ASR corpus based on public domain audio books,

    V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr. 2015, pp. 5206–5210. doi: 10.1109/ICASSP.2015.7178964

    2015

  14. [22]

    General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline,

    E. Fonseca et al., “General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline,” Oct. 07, 2018, arXiv: arXiv:1807.09902. doi: 10.48550/arXiv.1807.09902

    2018 doi arXiv

  15. [23]

    Audio tagging with noisy labels and minimal supervision,

    E. Fonseca, M. Plakal, F. Font, D. P. W. Ellis, and X. Serra, “Audio tagging with noisy labels and minimal supervision,” Jan. 20, 2020, arXiv: arXiv:1906.02975. doi: 10.48550/arXiv.1906.02975

    2020 doi arXiv

  16. [24]

    The Multilingual TEDx Corpus for Speech Recognition and Translation,

    E. Salesky et al., “The Multilingual TEDx Corpus for Speech Recognition and Translation,” Jun. 15, 2021, arXiv: arXiv:2102.01757. doi: 10.48550/arXiv.2102.01757

    2021 doi arXiv

  17. [25]

    SONYC-UST-V2: An Urban Sound Tagging Dataset with Spatiotemporal Context,

    M. Cartwright et al., “SONYC-UST-V2: An Urban Sound Tagging Dataset with Spatiotemporal Context,” Sep. 11, 2020, arXiv: arXiv:2009.05188. doi: 10.48550/arXiv.2009.05188

    2020 doi arXiv

  18. [26]

    MUSAN: A Music, Speech, and Noise Corpus,

    D. Snyder, G. Chen, and D. Povey, “MUSAN: A Music, Speech, and Noise Corpus,” Oct. 28, 2015, arXiv: arXiv:1510.08484. doi: 10.48550/arXiv.1510.08484

    2015 doi arXiv

  19. [27]

    An open dataset for research on audio field recording archives: freefield1010,

    D. Stowell and M. D. Plumbley, “An open dataset for research on audio field recording archives: freefield1010,” Oct. 01, 2013, arXiv: arXiv:1309.5275. doi: 10.48550/arXiv.1309.5275

    2013 doi arXiv

  20. [28]

    FSD50K: An Open Dataset of Human-Labeled Sound Events,

    E. Fonseca, X. Favory, J. Pons, F. Font, and X. Serra, “FSD50K: An Open Dataset of Human-Labeled Sound Events,” IEEEACM Trans Audio Speech Lang Proc, vol. 30, pp. 829–852, Dec. 2021, doi: 10.1109/TASLP.2021.3133208

    2021

  21. [29]

    to operate over audio by converting the acoustic time series to a spectrogram. This method applies different transformations to each input example and trains a 2-dimensional convolutional neural network (CNN) to produce a representation of the input audio that is invariant to ...

  22. [30]

    Audio Set: An ontology and human-labeled dataset for audio events,

    J. F. Gemmeke et al., “Audio Set: An ontology and human-labeled dataset for audio events,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar. 2017, pp. 776–780. doi: 10.1109/ICASSP.2017.7952261

    2017

  23. [31]

    CNN architectures for large-scale audio classification,

    S. Hershey et al., “CNN architectures for large-scale audio classification,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar. 2017, pp. 131–135. doi: 10.1109/ICASSP.2017.7952132

    2017

  24. [32]

    WHAM!: Extending Speech Separation to Noisy Environments,

    G. Wichern et al., “WHAM!: Extending Speech Separation to Noisy Environments,” Jul. 02, 2019, arXiv: arXiv:1907.01160. doi: 10.48550/arXiv.1907.01160

    2019 doi arXiv

  25. [33]

    Bootstrap your own latent a new approach to self-supervised learning,

    J.-B. Grill et al., “Bootstrap your own latent a new approach to self-supervised learning,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, in NIPS ’20. Red Hook, NY, USA: Curran Associates Inc., Dec. 2020, pp. 21271–21284

    2020

  26. [34]

    MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition,

    S. Majumdar and B. Ginsburg, “MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition,” in Interspeech 2020, ISCA, Oct. 2020, pp. 3356–3360. doi: 10.21437/Interspeech.2020-1058

    2020 doi

  27. [35]

    Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition,

    P. Warden, “Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition,” Apr. 09, 2018, arXiv: arXiv:1804.03209. doi: 10.48550/arXiv.1804.03209

    2018 doi arXiv

  28. [36]

    Environmental sound classification with convolutional neural networks,

    K. J. Piczak, “Environmental sound classification with convolutional neural networks,” in 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), Sep. 2015, pp. 1–6. doi: 10.1109/MLSP.2015.7324337

    2015

  29. [37]

    A Dataset and Taxonomy for Urban Sound Research,

    J. Salamon, C. Jacoby, and J. P. Bello, “A Dataset and Taxonomy for Urban Sound Research,” in Proceedings of the 22nd ACM international conference on Multimedia, in MM ’14. New York, NY, USA: Association for Computing Machinery, Nov. 2014, pp. 1041–1044. doi: 10.1145/2647868.2655045

    2014

  30. [38]

    Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders,

    J. Engel et al., “Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders,” in Proceedings of the 34th International Conference on Machine Learning, PMLR, Jul. 2017, pp. 1068–1077. Accessed: Nov. 26,

    2017

  31. [40]

    CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit (version 0.92),

    J. Yamagishi, C. Veaux, and K. MacDonald, “CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit (version 0.92),” Rainbow Passage Which Speak. Read Can Be Found Int. Dialects Engl. Arch. Httpwebkueduideareadingsrainbowhtm, Nov. 2019, doi: 10.7488/ds/2645

    2019 doi

  32. [41]

    AVA-Speech: A Densely Labeled Dataset of Speech Activity in Movies,

    S. Chaudhuri et al., “AVA-Speech: A Densely Labeled Dataset of Speech Activity in Movies,” Aug. 23, 2018, arXiv: arXiv:1808.00606. doi: 10.48550/arXiv.1808.00606

    2018 doi arXiv

  33. [42]

    Representational geometry: integrating cognition, computation, and the brain,

    N. Kriegeskorte and R. A. Kievit, “Representational geometry: integrating cognition, computation, and the brain,” Trends Cogn. Sci., vol. 17, no. 8, pp. 401–412, Aug. 2013, doi: 10.1016/j.tics.2013.06.007

    2013 doi

  34. [43]

    The Timbre Toolbox: Extracting audio descriptors from musical signals,

    G. Peeters, B. L. Giordano, P. Susini, N. Misdariis, and S. McAdams, “The Timbre Toolbox: Extracting audio descriptors from musical signals,” J. Acoust. Soc. Am., vol. 130, no. 5, pp. 2902–2916, Nov. 2011, doi: 10.1121/1.3642604

    2011 doi

  35. [44]

    The Modulation Transfer Function for Speech Intelligibility,

    T. M. Elliott and F. E. Theunissen, “The Modulation Transfer Function for Speech Intelligibility,” PLOS Comput. Biol., vol. 5, no. 3, p. e1000302, Mar. 2009, doi: 10.1371/journal.pcbi.1000302

    2009 doi

  36. [45]

    YIN, a fundamental frequency estimator for speech and music,

    A. de Cheveigné and H. Kawahara, “YIN, a fundamental frequency estimator for speech and music,” J. Acoust. Soc. Am., vol. 111, no. 4, pp. 1917–1930, Apr. 2002, doi: 10.1121/1.1458024

    1917 doi

  37. [46]

    Acoustic Event Detection Using Speaker Recognition Techniques: Model Optimization and Explainable Features,

    M. Ogg and B. Skerritt-Davis, “Acoustic Event Detection Using Speaker Recognition Techniques: Model Optimization and Explainable Features,” in Proceedings of the 6th Detection and Classification of Acoustic Scenes and Events 2021 Workshop (DCASE2021), Barcelona, Spain, 2021, p...

    2021 doi

  38. [47]

    Acoustic Correlates of Auditory Object and Event Perception: Speakers, Musical Timbres, and Environmental Sounds,

    M. Ogg and L. R. Slevc, “Acoustic Correlates of Auditory Object and Event Perception: Speakers, Musical Timbres, and Environmental Sounds,” Front. Psychol., vol. 10, Jul. 2019, doi: 10.3389/fpsyg.2019.01594

    2019

  39. [48]

    AVES: Animal Vocalization Encoder based on Self-Supervision,

    M. Hagiwara, “AVES: Animal Vocalization Encoder based on Self-Supervision,” Oct. 26, 2022, arXiv: arXiv:2210.14493. doi: 10.48550/arXiv.2210.14493

    2022 doi arXiv

  40. [49]

    SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network,

    W. Chan, D. Park, C. Lee, Y. Zhang, Q. Le, and M. Norouzi, “SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network,” Apr. 27, 2021, arXiv: arXiv:2104.02133. doi: 10.48550/arXiv.2104.02133

    2021 doi arXiv

  41. [2024]

    Available: https://proceedings.neurips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html

    [Online]. Available: https://proceedings.neurips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html

    2020

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.