Pith. sign in

REVIEW 4 major objections 4 minor 40 references

Evaluation of Deep Audio Representations for Hearables

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BEATs, an AudioSet-trained audio model, outperforms speech-focused rivals on every task in the new DEAR benchmark.

desk verdict A genuinely useful hearable-focused audio benchmark with a consistent BEATs advantage, but the 'significantly' claim lacks statistics and the ACE contradiction needs a fix. read the letter →

arxiv 2502.06664 v2 pith:BDHTKK6C submitted 2025-02-10 cs.SD cs.AIcs.LG

Fabian Gröger , Pascal Baumann , Ludovic Amruthalingam , Laurent Simon , Ruksana Giurda , Simone Lionetti This is my paper · ORCID
classification cs.SDcs.AIcs.LG
keywords audiorepresentationlearninghearabledevicesself-supervisedbenchmarkdatasetBEATsacousticpropertyestimationreverberationtimedirect-to-reverberantratio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces DEAR, a benchmark dataset of 1,158 synthetic 30-second audio tracks built by mixing anechoic speech into real ambisonic sound scenes, with labels for scene context, speech presence and speaker count, and technical acoustic properties (DRR, RT60, SNR). It then evaluates four public self-supervised audio representation models—Wav2Vec2.0, HuBERT, WavLM, and BEATs—using frozen features with linear and k-nearest-neighbor probes. The central claim is that BEATs outperforms all other models on every task, often by a large margin, and for reverberation-related properties the gap can reach 30 percentage points. If this holds, models trained on diverse audio collections like AudioSet encode the environment properties hearable devices need better than speech-specialized models, and the benchmark provides a controlled way to measure progress.

What carries the argument

The load-bearing mechanism is the DEAR generation pipeline: proprietary anechoic monologues are convolved with impulse responses and mixed into 4th-order ambisonics recordings from the HOA-SSR library, producing 1,158 mono tracks whose acoustic labels (DRR, RT60, SNR, environment, speaker counts) are known exactly by construction. The evaluation protocol then freezes each pre-trained transformer (12 layers, 768 hidden units, 90 million parameters) and trains only a linear or k-nearest-neighbor probe on the representations, so any measured performance difference reflects what the representation itself captured during pre-training. BEATs uses acoustic tokenizers to turn audio into discrete tokens and predicts masked tokens, which the paper identifies as the key contrast with the contrastive or masked-cluster objectives of the other three models.

What would settle it

Record the same speech-in-noise scenes with in-ear microphones in real rooms, measure the actual DRR, RT60, and SNR, and rerun the eight DEAR tasks on those recordings; if BEATs' advantage shrinks or reverses relative to the synthetic-label ranking, the synthetic-label assumption fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that BEATs, a masked-token prediction model pre-trained on AudioSet, encodes the information needed to steer hearable devices substantially better than three speech-focused self-supervised models (Wav2Vec2.0, HuBERT, and WavLM) across all eight prospective DEAR tasks and the two retrospective tasks. The advantage is largest for technical acoustic properties: reverberation-property differences reach 30 percentage points, and BEATs with kNN or linear regression achieves a qualitatively higher level of performance. The authors attribute the result to BEATs' label-prediction objective from acoustic tokenizers combined with the large and varied AudioSet corpus, while noting that DRR estimation remains difficult and deserves separate investigation.

Load-bearing premise

The synthetic mixing pipeline produces ground-truth acoustic labels (DRR, RT60, SNR) that faithfully represent real-world hearable conditions, so a model ranking on DEAR transfers to actual devices.

Editorial extensions

If this is right

  • If BEATs' margin is real, hearable developers can prefer AudioSet-trained masked-token models over speech-only self-supervised models when choosing a frozen representation for environment steering.
  • BEATs' encoding of SNR and RT60 without explicit supervision suggests general-purpose audio foundation models can supply acoustic-property estimates useful for source localization, distance estimation, and speech intelligibility prediction.
  • The mono downmix and controlled labels make DEAR a leakage-resistant benchmark, because the speech, backgrounds, and impulse responses are proprietary or licensed, so public models cannot have trained on the exact test tracks.
  • The finding that speaker counting is much harder on DEAR than on LibriCount indicates that realistic background noise, rather than training overlap, is the bottleneck for speech-source tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper treats synthetic labels as ground truth; a direct extension would verify BEATs' advantage with in-situ microphone measurements in real rooms, since rankings could in principle shift if the simulation misses device-relevant effects.
  • The result suggests a testable prediction beyond the paper: other AudioSet-trained masked-audio-token models, or larger BEATs variants, should also outperform the speech-focused trio on DEAR.
  • Because DEAR currently downmixes to mono, extending the benchmark to binaural or multi-channel inputs could probe spatial cues like interaural differences, which the paper notes are affected by the very DRR and RT60 parameters it labels.
  • The paper evaluates only frozen representations; fine-tuning BEATs on DEAR's prospective tasks would likely raise acoustic-property accuracy further, but that remains an untested consequence.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces DEAR, a new benchmark dataset of 1,158 synthetically mixed 30-second audio tracks for evaluating self-supervised audio representations in hearable contexts. Eight prospective tasks cover scene context, speech presence and speaker count, and acoustic properties (indoor/outdoor, stationary/transient, DRR, RT60, SNR), with TUT2017 and LibriCount added as retrospective comparison tasks. The authors evaluate Wav2Vec2.0, HuBERT, WavLM, and BEATs using frozen representations with linear and kNN probes, and report that BEATs outperforms the other models across all tasks, with the largest margins on acoustic-property regression. The dataset and associated code are released.

Significance. If the findings are robust, DEAR would be a useful community resource: it is the first benchmark targeted at hearable-relevant acoustic properties, it uses controlled label generation with disjoint monologue, room, and background splits to avoid leakage, and it selects kNN hyperparameters on a validation split, strengthening internal validity. The release of dataset and code is a concrete contribution. However, the headline claim of significant superiority is not supported by statistical evidence, and the transfer from mono synthetic tracks to binaural hearable use is asserted rather than tested. The benchmark contribution is sound in principle, but the paper's conclusions about hearable applicability are stronger than the current evidence.

major comments (4)
  1. [Section III / Fig. 1] The abstract and Section III state that BEATs 'significantly surpasses' its counterparts, and Section IV cites differences of up to 30 percentage points, but no confidence intervals, error bars, or significance tests are reported anywhere. Because the probe evaluation appears to be a single run on one split, the word 'significantly' is unsupported as a statistical claim. Please add bootstrap or permutation tests over tracks and 1-second segments, and report the resulting uncertainty for each metric, or reword the claim.
  2. [Sections II-A, II-B, and IV] The benchmark is constructed entirely from mono down-mixed synthetic mixtures: Section II-A down-mixes to 'a single reference track at 44.1 kHz,' and Section II-B defines every task on these tracks or 1-second segments. Yet the introduction motivates DRR, RT60, and SNR through their effects on binaural cues and hearable steering. The conclusion that BEATs 'confirms applicability to hearable steering' therefore assumes that a ranking obtained on mono synthetic audio transfers to binaural in-ear signals with head-related filtering, device processing, and realistic background dynamics. This transfer is untested. I recommend adding a binaural validation condition, using measured or simulated binaural impulse responses, or clearly scoping the conclusion to mono reference signals.
  3. [Section II-A] The paper states that tracks are stored at 44.1 kHz, but the evaluated models (Wav2Vec2, HuBERT, WavLM, BEATs) are typically pre-trained at 16 kHz. The manuscript does not specify whether representations were extracted directly at 44.1 kHz or after resampling to each model's native rate. This detail is essential for reproducibility and could affect the relative ranking if the models differ in tolerance to sample-rate mismatch. Please document the exact preprocessing, including resampling and any normalization, for each model.
  4. [Section II-A] The ground-truth acoustic labels (DRR, RT60, SNR) are generated by construction, from convolution with impulse responses and controlled mixing, rather than verified by acoustic measurement or independent estimation. This is acceptable for an internal model comparison, but the benchmark's claim to encode 'essential acoustic properties' for hearables would be materially strengthened by validating a subset of labels against in-situ microphone measurements or established blind estimators. Without such validation, the reported performance may partly reflect recovery of the synthetic mixing pipeline rather than of real-room acoustic properties.
minor comments (4)
  1. [Section II-B] The sentence describing SNR regression contains a duplicated clause ('on segments with only one speech source active'), which should be corrected to a single condition statement.
  2. [Table I] The table rows are visually collapsed in the manuscript text, making it difficult to see which task names correspond to which columns; please ensure each row and column is clearly separated.
  3. [Fig. 1 caption] The caption says 'markers missing from the plot have worse-than-random performance,' but it does not explain how worse-than-random is represented visually; please clarify the marker convention or add a legend entry for missing values.
  4. [Section II-D] The evaluation section does not specify how segment-level predictions are aggregated to track-level predictions for tasks with whole-track labels, nor how many segments are used per task; this information should be stated for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DEAR benchmark evaluation is an external comparison, and the BEATs ranking is not forced by construction, fitting, or self-citation.

full rationale

The paper's central claim is an external benchmark comparison, not a derivation from fitted inputs. DEAR labels (DRR, RT60, SNR, environment, speech presence, speaker count) are generated by construction from known mixing parameters and impulse responses, but no model parameter is fitted to those labels. The evaluated representations are frozen public checkpoints, and the downstream linear/kNN probes are trained on disjoint development splits, with the number of neighbors selected on a held-out validation split; test tracks are held out and used only for final scoring. The ranking of BEATs against HuBERT, Wav2Vec2.0, and WavLM therefore does not reduce to any equation or fitted quantity in the paper. The prospective tasks are complemented by retrospective tasks on TUT2017 and LibriCount, which provide independent comparison points even though those datasets may overlap with pre-training data. The only self-citation is reference [21] on reverberation effects on binaural cues, and it is contextual motivation rather than a load-bearing argument. The transfer-to-binaural real-world limitation is an external-validity concern, not a circularity: all models see identical mono, synthetically mixed audio, so the internal ranking is unaffected. No step in the paper defines a predicted quantity in terms of itself, and no load-bearing assumption is imported from the authors' own prior work. The benchmark and evaluation are self-contained, and the score is accordingly 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The benchmark itself is a dataset, not an entity postulated to explain observations. The only free parameter affecting reported results is the kNN neighborhood size, selected by validation.

free parameters (1)
  • kNN neighborhood size k = selected per model and task from {1, 2, 5, 10, 20, 50} on validation
    The reported kNN results depend on this hyperparameter choice, though it is selected on a held-out validation split rather than the test labels.
assumptions (4)
  • domain assumption Frozen-feature linear and kNN evaluation is a valid proxy for downstream task performance.
    Section II-D cites Caron et al. [39]; this is standard practice in representation evaluation, but a model that scores well under linear probing may not fine-tune best in real hearable systems.
  • domain assumption Labels computed from the mixing process are accurate ground truth for DRR, RT60, SNR, and speech activity.
    Section II-A constructs labels from known convolution and mixing parameters; no in-situ microphone verification is reported.
  • domain assumption The pretrained models have no prior exposure to DEAR data.
    Section II-B says the commercial HOA-SSR library and proprietary monologues make prior training or evaluation unlikely, but this cannot be externally verified.
  • domain assumption Disjoint splits of monologues, background recordings, and impulse responses prevent train/test leakage.
    Section II-A and II-D describe the split design, but the implementation details are not checkable from the manuscript text alone.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluation of Deep Audio Representations for Hearables." pith.science (2026). https://pith.science/paper/BDHTKK6C

@misc{pith2026250206664,
  author       = {Pith},
  title        = {Pith review of: Evaluation of Deep Audio Representations for Hearables},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BDHTKK6C}},
  note         = {Machine review of arXiv:2502.06664}
}
read the original abstract

Effectively steering hearable devices requires understanding the acoustic environment around the user. In the computational analysis of sound scenes, foundation models have emerged as the state of the art to produce high-performance, robust, multi-purpose audio representations. We introduce and release Deep Evaluation of Audio Representations (DEAR), the first dataset and benchmark to evaluate the efficacy of foundation models in capturing essential acoustic properties for hearables. The dataset includes 1,158 audio tracks, each 30 seconds long, created by spatially mixing proprietary monologues with commercial, high-quality recordings of everyday acoustic scenes. Our benchmark encompasses eight tasks that assess the general context, speech sources, and technical acoustic properties of the audio scenes. Through our evaluation of four general-purpose audio representation models, we demonstrate that the BEATs model significantly surpasses its counterparts. This superiority underscores the advantage of models trained on diverse audio collections, confirming their applicability to a wide array of auditory tasks, including encoding the environment properties necessary for hearable steering. The DEAR dataset and associated code are available at https://dear-dataset.github.io.

Figures

Figures reproduced from arXiv: 2502.06664 by the authors.

Figure 1
Figure 1. Comparison of the general-purpose audio representation models on a suite of downstream tasks. Evaluation is performed [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages

  1. [1]

    World report on hearing,

    WHO, “World report on hearing,” World Health Organization, 2021

    work page 2021

  2. [2]

    Hearing loss in adults,

    L. L. Cunningham and D. L. Tucci, “Hearing loss in adults,” New England Journal of Medicine , vol. 377, no. 25, pp. 2465–2473, 2017

    2017

  3. [3]

    Hearing loss is associated with cortical thinning in cognitively normal older adults,

    J. Ha, Y .-S. Cho, S. Kim, S. Cho, J. Kim, Y . Jung, H. Jang, H.-Y . Shin, F. R. Lin, D. L. Na et al. , “Hearing loss is associated with cortical thinning in cognitively normal older adults,” European Journal of Neurology, vol. 27, no. 6, pp. 1003–1009, 2020

    work page 2020

  4. [4]

    Impairments in hearing and vision impact on mortality in older people: the ages-reykjavik study,

    D. Fisher, C.-M. Li, M. S. Chiu, C. L. Themann, H. Petersen, F. J ´onasson, P. V . J´onsson, J. E. Sverrisdottir, M. Garcia, T. B. Harris et al., “Impairments in hearing and vision impact on mortality in older people: the ages-reykjavik study,” Age and ageing , vol. 43, no. 1, pp. 69–76, 2014

    work page 2014

  5. [5]

    Us population data on hearing loss, trouble hearing, and hearing-device use in adults: National health and nutrition examination survey, 2011–12, 2015–16, and 2017–20,

    L. E. Humes, “Us population data on hearing loss, trouble hearing, and hearing-device use in adults: National health and nutrition examination survey, 2011–12, 2015–16, and 2017–20,” Trends in hearing, vol. 27, p. 23312165231160978, 2023

    work page 2011

  6. [6]

    Emerging technologies, market segments, and marketrak 10 insights in hearing health technology,

    B. Edwards, “Emerging technologies, market segments, and marketrak 10 insights in hearing health technology,” in Seminars in hearing , vol. 41, no. 01. Thieme Medical Publishers, 2020, pp. 037–054

    work page 2020

  7. [7]

    Speech en- hancement for hearing-impaired listeners using deep neural networks with auditory-model based features,

    T. Goehring, X. Yang, J. J. Monaghan, and S. Bleeck, “Speech en- hancement for hearing-impaired listeners using deep neural networks with auditory-model based features,” in 2016 24th European signal processing conference (EUSIPCO). IEEE, 2016, pp. 2300–2304

    work page 2016

  8. [8]

    A deep learning based segregation algorithm to increase speech intelligibility for hearing- impaired listeners in reverberant-noisy conditions,

    Y . Zhao, D. Wang, E. M. Johnson, and E. W. Healy, “A deep learning based segregation algorithm to increase speech intelligibility for hearing- impaired listeners in reverberant-noisy conditions,” The Journal of the Acoustical Society of America , vol. 144, no. 3, pp. 1627–1637, 2018

    work page 2018

Show all 40 references
  1. [9]

    An effectively causal deep learning algorithm to increase intelligibility in untrained noises for hearing-impaired listeners,

    E. W. Healy, K. Tan, E. M. Johnson, and D. Wang, “An effectively causal deep learning algorithm to increase intelligibility in untrained noises for hearing-impaired listeners,” The Journal of the Acoustical Society of America, vol. 149, no. 6, pp. 3943–3953, 2021

    2021

  2. [10]

    Creating clarity in noisy environments by using deep learning in hearing aids,

    A. H. Andersen, S. Santurette, M. S. Pedersen, E. Alickovic, L. Fiedler, J. Jensen, and T. Behrens, “Creating clarity in noisy environments by using deep learning in hearing aids,” in Seminars in Hearing , vol. 42, no. 03. Thieme Medical Publishers, Inc., 2021, pp. 260–281

    2021

  3. [11]

    Audio self-supervised learning: A survey,

    S. Liu, A. Mallol-Ragolta, E. Parada-Cabaleiro, K. Qian, X. Jing, A. Kathan, B. Hu, and B. W. Schuller, “Audio self-supervised learning: A survey,” Patterns, vol. 3, no. 12, 2022

    2022

  4. [12]

    Restoring speech intelligibility for hearing aid users with deep learn- ing,

    P. U. Diehl, Y . Singer, H. Zilly, U. Sch ¨onfeld, P. Meyer-Rachner, M. Berry, H. Sprekeler, E. Sprengel, A. Pudszuhn, and V . M. Hofmann, “Restoring speech intelligibility for hearing aid users with deep learn- ing,” Scientific Reports, vol. 13, no. 1, p. 2719, 2023

    2023

  5. [13]

    Towards learning a universal non- semantic representation of speech,

    J. Shor, A. Jansen, O. Lang, O. Tuval, F. de Chaumont Quitry, M. Tagliasacchi, and D. Emanuel, “Towards learning a universal non- semantic representation of speech,” in Proc. Interspeech, no. 2020, 2020

    2020

  6. [14]

    Lebench- mark: A reproducible framework for assessing self-supervised represen- tation learning from speech,

    S. Evain, H. Nguyen, H. Le, M. Z. Boito, S. Mdhaffar, S. Alisamir, Z. Tong, N. Tomashenko, M. Dinarelli, T. Parcollet et al. , “Lebench- mark: A reproducible framework for assessing self-supervised represen- tation learning from speech,” in INTERSPEECH 2021: Conference of the ...

    2021

  7. [15]

    Superb: Speech processing universal performance benchmark,

    S. W. Yang, P. H. Chi, Y . S. Chuang, C. I. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G. T. Lin et al., “Superb: Speech processing universal performance benchmark,” in 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH

  8. [16]

    Towards learning universal audio representations,

    L. Wang, P. Luc, Y . Wu, A. Recasens, L. Smaira, A. Brock, A. Jaegle, J.- B. Alayrac, S. Dieleman, J. Carreira et al., “Towards learning universal audio representations,” in ICASSP 2022-2022 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP) . ...

    2022

  9. [17]

    Hear: Holistic evaluation of audio representations,

    J. Turian, J. Shier, H. R. Khan, B. Raj, B. W. Schuller, C. J. Steinmetz, C. Malloy, G. Tzanetakis, G. Velarde, K. McNally et al., “Hear: Holistic evaluation of audio representations,” in NeurIPS 2021 Competitions and Demonstrations Track. PMLR, 2022, pp. 125–145

    2021

  10. [18]

    Decorrelating feature spaces for learning general-purpose audio representations,

    S. Ghosh, A. Seth, and S. Umesh, “Decorrelating feature spaces for learning general-purpose audio representations,” IEEE Journal of Se- lected Topics in Signal Processing, vol. 16, no. 6, pp. 1402–1414, 2022

    2022

  11. [19]

    Benchmarking representations for speech, music, and acoustic events,

    M. La Quatra, A. Koudounas, L. Vaiani, E. Baralis, L. Cagliero, P. Garza, and S. M. Siniscalchi, “Benchmarking representations for speech, music, and acoustic events,” arXiv preprint arXiv:2405.00934 , 2024

    2024 arXiv

  12. [20]

    Estimation of Room Acoustic Parameters: The ACE Challenge,

    J. Eaton, N. D. Gaubitch, A. H. Moore, and P. A. Naylor, “Estimation of Room Acoustic Parameters: The ACE Challenge,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 24, no. 10, pp. 1681–1693, Oct. 2016, conference Name: IEEE/ACM Transactions on Audio, S...

    2016

  13. [21]

    Reverberation effects on source localization and beamforming for hearing aid applications,

    R. Giurda, E. Georganti, and N. Dillier, “Reverberation effects on source localization and beamforming for hearing aid applications,” 2018

    2018

  14. [22]

    Effects of room reflectance and background noise on perceived auditory distance,

    D. H. Mershon, W. L. Ballenger, A. D. Little, P. L. McMurtry, and J. L. Buchanan, “Effects of room reflectance and background noise on perceived auditory distance,” Perception, vol. 18, no. 3, pp. 403–416, 1989

    1989

  15. [23]

    Sound localization: Effects of reverberation time, speaker array, stimulus frequency, and stimulus rise/decay,

    C. Gigu `ere and S. M. Abel, “Sound localization: Effects of reverberation time, speaker array, stimulus frequency, and stimulus rise/decay,” The Journal of the Acoustical Society of America , vol. 94, no. 2, pp. 769– 776, 1993

    1993

  16. [24]

    Effects of reverberation and noise on speech intelligibility in normal-hearing and aided hearing-impaired listeners,

    J. Xia, B. Xu, S. Pentony, J. Xu, and J. Swaminathan, “Effects of reverberation and noise on speech intelligibility in normal-hearing and aided hearing-impaired listeners,” The Journal of the Acoustical Society of America, vol. 143, no. 3, pp. 1523–1533, 2018

    2018

  17. [25]

    Don’t make your llm an evaluation benchmark cheater,

    K. Zhou, Y . Zhu, Z. Chen, W. Chen, W. X. Zhao, X. Chen, Y . Lin, J.-R. Wen, and J. Han, “Don’t make your llm an evaluation benchmark cheater,” arXiv preprint arXiv:2311.01964 , 2023

    2023 arXiv

  18. [26]

    Tut database for acoustic scene classification and sound event detection,

    A. Mesaros, T. Heittola, and T. Virtanen, “Tut database for acoustic scene classification and sound event detection,” in 2016 24th European Signal Processing Conference (EUSIPCO) . IEEE, 2016, pp. 1128– 1132

    2016

  19. [27]

    Libricount, a dataset for speaker count estimation,

    F.-R. St ¨oter, S. Chakrabarty, E. Habets, and B. Edler, “Libricount, a dataset for speaker count estimation,” 2018

    2018

  20. [28]

    Librispeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210

    2015

  21. [29]

    Audio set: An ontology and human- labeled dataset for audio events,

    J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human- labeled dataset for audio events,” in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2017,...

    2017

  22. [30]

    ESC: Dataset for environmental sound classification,

    K. J. Piczak, “ESC: Dataset for environmental sound classification,” in Proceedings of the 23rd ACM international conference on Multimedia , 2015, pp. 1015–1018

    2015

  23. [31]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449– 12 460, 2020

    2020

  24. [32]

    HuBERT: Self-Supervised Speech Representation Learn- ing by Masked Prediction of Hidden Units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learn- ing by Masked Prediction of Hidden Units,” IEEE/ACM transactions on audio, speech, and language processing , vol. 29, pp. 3451–3460, 2021

    2021

  25. [33]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al. , “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  26. [34]

    Beats: Audio pre-training with acoustic tokenizers,

    S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, and F. Wei, “Beats: Audio pre-training with acoustic tokenizers,” arXiv preprint arXiv:2212.09058, 2022

    2022 arXiv

  27. [35]

    wav2vec: Unsupervised pre-training for speech recognition,

    S. Schneider, A. Baevski, R. Collobert, and M. Auli, “wav2vec: Unsupervised pre-training for speech recognition,” arXiv preprint arXiv:1904.05862, 2019

    1904 arXiv

  28. [36]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

    2018 arXiv

  29. [37]

    Gigaspeech: An evolving, multi- domain asr corpus with 10,000 hours of transcribed audio,

    G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhang et al. , “Gigaspeech: An evolving, multi- domain asr corpus with 10,000 hours of transcribed audio,” arXiv preprint arXiv:2106.06909, 2021

    2021 arXiv

  30. [38]

    V oxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,

    C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, and E. Dupoux, “V oxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” arXiv preprint arXiv:2101.00390 , 2021

    2021 arXiv

  31. [39]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9650–9660

    2021

  32. [2021]

    3161–3165

    International Speech Communication Association, 2021, pp. 3161–3165

    2021

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.