Pith. sign in

REVIEW 2 major objections 4 minor 36 references

Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Heart rate can be estimated from heart-sound recordings using representations from pre-trained audio foundation models, and one contrastively trained encoder beats the acoustic-feature baseline at 1.88 vs 1.91 bpm mean absolute error.

desk verdict A useful layer-wise benchmark of audio FMs for PCG heart rate estimation, but the headline CLAP advantage is a post-hoc 0.03 bpm gap with no significance testing; the 'comparable performance' finding is solid, the 'outperforms' claim is not. read the letter →

arxiv 2505.20745 v2 pith:QVA4C26U submitted 2025-05-27 cs.SD cs.LGeess.AS

Jingping Nie , Dung T. Tran , Karan Thakkar , Vasudha Kowtha , Jon Huang , Carlos Avendano , Erdrin Azemi , Vikramjit Mitra This is my paper · ORCID
classification cs.SDcs.LGeess.AS
keywords heartrateestimationphonocardiogramauscultationacousticfoundationmodelsself-supervisedlearninglayer-wiseanalysisCLAPtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether hidden representations of self-supervised audio foundation models, originally trained on speech and general audio events, carry enough information about heart sounds to estimate heart rate from phonocardiogram recordings. Using a public PCG dataset split six ways, the authors compare representations from every encoder layer of six foundation models against a handcrafted acoustic-feature baseline. They find the foundation-model representations are broadly comparable to the baseline, and the audio encoder of an in-house CLAP model trained on general audio events achieves a lower mean absolute error (1.88 bpm) than the baseline (1.91 bpm) across all splits, despite the domain mismatch. A layer-wise analysis shows that mid-level layers of this CLAP encoder carry the most useful cardiac information, and that deeper layers of speech-oriented models lose cardiorespiratory content. The result matters because it suggests general audio foundation models can be reused for vital-sign monitoring without retraining the encoder.

What carries the argument

The machinery is the layer-wise extraction of hidden representations: for each foundation model, the output of the $n$-th embedding layer of its audio encoder is taken from 5-second PCG snippets and fed into a downstream 2D CNN classifier that predicts heart rate as one of 141 classes ($HR \in [40,180]$ bpm) with cross-entropy loss. The in-house CLAP model, which carries the main positive result, is a 12-layer ViT audio encoder with 768 hidden dimensions, pre-trained in two stages: first by masked spectrogram reconstruction in the style of AudioMAE on roughly three million audio samples, then by contrastive alignment with a text encoder on about five million captions. This two-stage, general-audio-event pretraining is what the paper credits for the encoder's ability to preserve cardiac cues in mid-level layers.

What would settle it

Rerun the same layer-wise comparison on the same snippets but with heart-rate labels obtained from an independent reference, such as a synchronized ECG recording or expert re-annotation of S1/S2 boundaries; the central claim fails if the in-house CLAP's 1.88 bpm result no longer beats the baseline's 1.91 bpm under independently verified labels.

Watch

Extended reading notes

Core claim

The central claim is that pre-trained acoustic foundation-model representations are a viable feature source for heart-rate estimation from auscultation, and that a contrastively trained audio encoder can beat a purpose-built acoustic-feature method. In the paper's layer-wise evaluation, each FM's embedding-layer output feeds a small 2D CNN that classifies heart rate into 141 bins from 40 to 180 bpm; the best single result is the sixth embedding layer of the in-house CLAP model, which reaches an average MAE of 1.88 bpm across six train/validation/test splits, versus 1.91 bpm for the baseline. The paper further reports that the shallow layers of ASR-oriented models (HuBERT, wav2vec2, WavLM, Whisper) generally perform better than deep layers, that larger model size does not guarantee better heart-rate estimates, and that averaging the baseline's predictions with FM-based predictions improves accuracy on every split. The authors attribute the in-house CLAP advantage to its training on a wide range of audio events rather than speech-dominated corpora.

Load-bearing premise

The ground-truth heart rate of each snippet is derived from the S1 onset annotations in the public PCG dataset using the labeling method of the baseline paper, so if those onset annotations carry systematic bias or noise, every reported MAE is measured against an unreliable target.

Editorial extensions

If this is right

  • General audio foundation models can be used as frozen feature extractors for heart-rate estimation, avoiding task-specific pretraining and achieving accuracy comparable to handcrafted acoustic features.
  • The in-house CLAP's advantage over the baseline (1.88 vs 1.91 bpm MAE) identifies contrastive audio-event pretraining, rather than speech-oriented pretraining, as the more transferable recipe for cardiac acoustics.
  • The layer-wise curves provide a direct fine-tuning target: for ASR-based models, shallow and middle layers retain cardiorespiratory information while deep linguistic layers lose it, so adaptation should focus on the earlier layers.
  • Averaging FM-based and acoustic-feature predictions improves accuracy on every split, suggesting ensemble schemes that combine both feature families are a practical route to more robust HR estimation.
  • Evaluation across multiple data splits is necessary; the paper finds that data-split variation can affect model comparisons more than model size, and larger FMs do not systematically help.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: probe the same six encoders on other cardiorespiratory sounds (breathing, cough, murmurs); the layer-wise pattern predicts that mid-level layers of contrastive audio encoders will again carry the most signal.
  • A concrete next experiment: concatenate the baseline acoustic features with the in-house CLAP layer-6 vector before the downstream CNN; the paper's averaging result suggests fusion would lower the MAE below 1.88 bpm.
  • Because the reported difference is only 0.03 bpm, an independent-label rerun (ECG-synchronized) is needed to know whether the true effect is bigger, smaller, or absent; the paper does not provide that check.
  • The paper's attribution of the advantage to audio-event pretraining breadth could be isolated by ablating the two pretraining stages of the in-house CLAP encoder, but that ablation is not reported here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper benchmarks hidden representations from six pre-trained acoustic foundation models (HuBERT, wav2vec2, wavLM, Whisper, CLAP, and an in-house CLAP) for heart rate (HR) estimation from phonocardiogram snippets. Using the CirCor dataset, heart sound snippets are generated with 5-second windows, HR targets are derived from S1-onset annotations via the method of Nie et al. [1], and a downstream 2D CNN is trained on embeddings from each layer of each model. The reported result is that FM representations generally achieve MAE comparable to the acoustic-feature baseline (average MAE 1.91 bpm), with the 6th layer of in-house CLAP achieving an average MAE of 1.88 bpm, which the abstract and conclusions describe as outperforming the baseline.

Significance. If the findings are properly supported, the paper would provide a useful benchmark showing that frozen self-supervised audio representations are competitive with handcrafted acoustic features for auscultation-based HR estimation. The layer-wise analysis across six models and six held-out splits is a useful contribution, and the reporting of model sizes and per-split variance is commendable. The strongest parts are the breadth of the layer sweep and the use of a public dataset with multiple splits. However, the headline superiority claim for in-house CLAP rests on a 0.03 bpm difference that is not statistically tested, and the label-generation procedure is not independently validated; these issues are load-bearing for the paper's central claim.

major comments (2)
  1. [Section 5, Table 1, Abstract] The claim that in-house CLAP 'outperforms' the baseline is not statistically supported. The evidence is min(MAE)=1.88 for the 6th layer versus baseline MAE=1.91, a difference of 0.03 bpm, while the reported split-level spread is σ=0.37 for in-house CLAP at that layer and STD=0.32 for the baseline. The 6th layer was selected as the best among 12 embedding layers after examining test performance on the same six splits, so the minimum-of-12 comparison is subject to selection bias. No paired test, confidence interval, or multiple-comparison correction is reported for this key comparison. The authors should report per-split paired differences and a paired significance test, or use a held-out layer-selection procedure; otherwise the conclusion should be limited to 'comparable performance.'
  2. [Section 2 and Section 5] The ground-truth HR for each snippet is generated from S1-onset annotations in the CirCor dataset using the method of the authors' prior paper [1], and the manuscript itself acknowledges 'biases and errors in segmentation annotations' (Section 2) and 'annotation bias' (Section 5). Because both the baseline method and the FM-based models are evaluated against these derived labels, the reliability of the label-generation procedure is load-bearing for every reported MAE. No independent validation of the derived HR targets is provided (for example, against ECG-derived HR or manual reference measurements). The authors should either provide such validation or explicitly and consistently frame the results as relative performance under this particular label-generation protocol.
minor comments (4)
  1. [Section 1] The phrase 'train/validation/data splits' should read 'train/validation/test splits.'
  2. [Section 4, Eq. (1)] The displayed cross-entropy formula has mismatched subscripts (x_{a,c} versus x_{n,i}) and is not written in standard form; please correct the notation so the sum over classes and the softmax argument are unambiguous.
  3. [Table 1] Please clarify that min(MAE) is the minimum over embedding layers of the mean MAE across splits, and that σ_{MAE,j*} is the standard deviation across splits at the layer achieving that minimum; the current header is dense and could be misinterpreted as a minimum over individual split results.
  4. [Section 5, Figure 5] The shaded standard-deviation bands in Figure 5 are useful, but the figure legends do not state the number of splits underlying each mean; adding that information would improve interpretability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the FM representations are evaluated independently against an implemented acoustic baseline on labels derived from external S1-onset annotations.

full rationale

The paper's central comparison is not circular. The heart-rate labels are computed from adjacent S1-onset intervals in the CirCor dataset using the method of [1], but those onsets are external annotations, not outputs of the baseline or of the foundation models being evaluated. The baseline itself is implemented from [1] as an acoustic-feature 2D CNN, and the FM embeddings are passed through a separately trained downstream classifier on the same six splits. No parameter is fitted on the test set in a way that defines the reported outcome; the 'outperforms' claim is based on selecting the minimum mean MAE across embedding layers, which is a model-selection and statistical-inference concern rather than a circularity. The paper's own caveat about annotation bias in the CirCor dataset (Section 5) is a data-quality limitation, not a step that reduces the derivation to its inputs. The self-citations to [1] are used as an external baseline and label-generation recipe, but the baseline is implemented and the labels are objectively derived, so the cited prior work does not carry the argument by assumption. Overall, the derivation chain is self-contained for the purposes of circularity analysis.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper's main results depend on the validity of the annotation-derived heart rate labels and on the choice of downstream architecture, but the study is an empirical benchmark with no fitted parameters in a derivation sense.

assumptions (2)
  • domain assumption The S1 onset annotations in the CirCor dataset, combined with the averaging method from [1], produce a valid ground-truth heart rate for each 5-second snippet.
    Invoked in Section 2, data preparation, where average HR is computed from adjacent S1 onsets. If the annotations are biased, all reported MAE values are comparisons against a biased target.
  • domain assumption Representations from pre-trained foundation models, despite domain mismatch with cardiac acoustics, can be fed into a 2D CNN for heart rate classification.
    Invoked in Section 4, downstream model, where the n-th embedding layer output is passed into the 2D CNN. The paper does not justify why this input representation is sufficient beyond empirical results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation." pith.science (2026). https://pith.science/paper/QVA4C26U

@misc{pith2026250520745,
  author       = {Pith},
  title        = {Pith review of: Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QVA4C26U}},
  note         = {Machine review of arXiv:2505.20745}
}
read the original abstract

Auscultation, particularly heart sound, is a non-invasive technique that provides essential vital sign information. Recently, self-supervised acoustic representation foundation models (FMs) have been proposed to offer insights into acoustics-based vital signs. However, there has been little exploration of the extent to which auscultation is encoded in these pre-trained FM representations. In this work, using a publicly available phonocardiogram (PCG) dataset and a heart rate (HR) estimation model, we conduct a layer-wise investigation of six acoustic representation FMs: HuBERT, wav2vec2, wavLM, Whisper, Contrastive Language-Audio Pretraining (CLAP), and an in-house CLAP model. Additionally, we implement the baseline method from Nie et al., 2024 (which relies on acoustic features) and show that overall, representation vectors from pre-trained foundation models (FMs) offer comparable performance to the baseline. Notably, HR estimation using the representations from the audio encoder of the in-house CLAP model outperforms the results obtained from the baseline, achieving a lower mean absolute error (MAE) across various train/validation/test splits despite the domain mismatch.

Figures

Figures reproduced from arXiv: 2505.20745 by the authors.

Figure 1
Figure 1. The overall goal of this project. pre-trained acoustic FMs can improve emotion recogni￾tion performance compared to the performance when using acoustic features [15–18]. It has been shown that the layer￾wise progression of representations follows an acoustic￾linguistic hierarchy, with shallow layers encoding acoustic features, followed by phonetic, word identity, and meaning information in pre-trained wav2vec2 [19].… view at source ↗
Figure 2
Figure 2. The data preparation process. across various train/validation/test splits despite the domain mismatch. This work establishes a foundational bench￾mark for future research and potential adaptations of FMs in heart sound analysis, broader cardiorespiratory sound as￾sessment, and clinical applications. Given the widespread adoption of these FMs across diverse domains, our find￾ings can be seamlessly integrated into exi… view at source ↗
Figure 4
Figure 4. The representation vectors of n th embedding layer in the audio encoder of the foundation model are passed into a downstream 2D convolutional neural network (2dCNN) for HR estimation [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (2 more)
Figure 6
Figure 6. Figure 6: , the deeper layers of the Librispeech fine-tuned wav2vec2 Base model encode less cardiorespiratory infor￾mation, suggesting that the shift toward linguistic represen￾tations may reduce the model’s sensitivity to physiological signals. We also noted that larger FMs do …
Figure 5
Figure 5. Figure 5: The Mean Absolute Error (MAEi,j ) across six data splits for different feature models (FMs) at various em￾bedding layers. We found that shallower layers of ASR-based FMs gen￾erally perform better in HR estimation. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p0…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 34 canonical work pages

  1. [1]

    Introduction Health-related acoustic sounds have significant potential for health, fitness, and wellbeing. Heart sound auscul- tation (from phonocardiograms, a.k.a PCGs) is a non- invasive method that offers crucial vital sign data (such as heart rate (HR) information), and machine learning can be used to explore features and enable high-value use cases. ...

    work page Pith review arXiv 2025

  2. [2]

    Each PCG recording spans from 5.1 to 64.5 seconds, totaling about 20 hours

    Data The PCG dataset used in our study is theCirCor DigiScope Phonocardiogram dataset, containing 3,163 heart sound recordings from 942 subjects collected across four main auscultation sites in hospitals. Each PCG recording spans from 5.1 to 64.5 seconds, totaling about 20 hours. All recordings in this dataset are low-pass filtered with a cut- off frequen...

    work page

  3. [3]

    16 kHz raw audio using a 25 ms Hann window and 10 ms hops

    In-House CLAP Model Audio Encoder Architecture:The audio encoder takes a 128 LogMel spectrogram as input, computed from mono Figure 3:The heart rate distribution for 6 training, valida- tion, and test splits and the number of unique subjects. 16 kHz raw audio using a 25 ms Hann window and 10 ms hops. The model is a 12-layer ViT-B with around 86 mil- lion ...

    work page

  4. [4]

    Downstream Model and Training Since HR is typically represented as an integer in beats per minute, we followed a similar approach to [1], treating HR estimation as a 141-class classification problem, where HR∈[40,180](bpm). The cross-entropy (CE) loss func- tion was used for HR estimation: CE= PA a=1(− PB b=1 log exp (xa,c)PB i=1 exp (xn,i) ya,b),(1) wher...

    work page

  5. [5]

    The thick purple line represents the average MAE ( M AEj) across the 6 splits, with the light purple shaded area indi- cating the standard deviation (σ M AEj )

    Results Figure 5 shows theM AEi,j for splitiusing the represen- tations from embedding layerjof the different FMs. The thick purple line represents the average MAE ( M AEj) across the 6 splits, with the light purple shaded area indi- cating the standard deviation (σ M AEj ). Table 1 further il- lustrates the performance of using the representations from d...

    work page

  6. [6]

    Conclusion and Future Work In this study, we evaluated and benchmarked the ability of pre-trained acoustic foundation models (FMs) to en- code heart sound auscultation for heart rate (HR) estima- tion. Through a layer-wise investigation of six FMs, such as HuBERT, wav2vec2, wavLM, Whisper, CLAP, and an in-house CLAP model, we demonstrated that these FMs o...

    work page

  7. [7]

    Model-driven heart rate estimation and heart murmur detection based on phonocardiogram,

    J. Nie, R. Liu, B. Mahasseni, and V . Mitra, “Model-driven heart rate estimation and heart murmur detection based on phonocardiogram,” in2024 IEEE 34th International Work- shop on Machine Learning for Signal Processing (MLSP), 2024, pp. 1–6

    work page 2024

  8. [8]

    Heart rate recovery as a guide to monitor fa- tigue and predict changes in performance parameters,

    R. Lamberts, J. Swart, B. Capostagno, T. Noakes, and M. Lambert, “Heart rate recovery as a guide to monitor fa- tigue and predict changes in performance parameters,”Scan- dinavian journal of medicine & science in sports, vol. 20, no. 3, pp. 449–457, 2010

    work page 2010

Show all 36 references
  1. [9]

    Heart rate vs stress indicator for short term mental stress,

    P. Fauquet-Alekhine, L. Rouillac, J. Bertoni, and J.-C. Granry, “Heart rate vs stress indicator for short term mental stress,”British Journal of Medicine and Medical Research, vol. 17, no. 7, pp. 1–11, 2016

    2016

  2. [10]

    Resting heart rate in cardiovascular disease,

    K. Fox, J. S. Borer, A. J. Camm, N. Danchin, R. Ferrari, J. L. Lopez Sendon, P. G. Steg, J.-C. Tardif, L. Tavazzi, M. Ten- deraet al., “Resting heart rate in cardiovascular disease,” Journal of the American College of Cardiology, vol. 50, no. 9, pp. 823–830, 2007

    2007

  3. [11]

    Short- segment heart sound classification using an ensemble of deep convolutional neural networks,

    F. Noman, C.-M. Ting, S.-H. Salleh, and H. Ombao, “Short- segment heart sound classification using an ensemble of deep convolutional neural networks,” inICASSP 2019-2019 IEEE international conference on acoustics, speech and signal pro- cessing (ICASSP). IEEE, 2019, pp. 1318–1322

    2019

  4. [12]

    Study selectively: An adaptive knowledge distillation based on a voting network for heart sound classification,

    X. Qiu, L. Zhu, Z. Song, Z. Chen, H. Zhang, K. Qian, Y . Zhang, B. Hu, Y . Yamamoto, and B. W. Schuller, “Study selectively: An adaptive knowledge distillation based on a voting network for heart sound classification,” inProceed- ings of Interspeech 2024, 2024, pp. 137–141

    2024

  5. [13]

    Heart rate estimation from phonocardiogram signals using non-negative matrix factorization,

    N. Dia, J. Fontecave-Jallon, P.-Y . Gum ´ery, and B. Rivet, “Heart rate estimation from phonocardiogram signals using non-negative matrix factorization,” inICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 1293–1297

    2019

  6. [14]

    Heart sound classifi- cation based on improved mel-frequency spectral coefficients and deep residual learning,

    F. Li, Z. Zhang, L. Wang, and W. Liu, “Heart sound classifi- cation based on improved mel-frequency spectral coefficients and deep residual learning,”Frontiers in Physiology, vol. 13, p. 1084420, 2022

    2022

  7. [15]

    Ar- steth: Enabling home self-screening with ar-assisted intelli- gent stethoscopes,

    K. Hou, S. Xia, E. Bejerano, J. Wu, and X. Jiang, “Ar- steth: Enabling home self-screening with ar-assisted intelli- gent stethoscopes,” inProceedings of the 22nd International Conference on Information Processing in Sensor Networks, 2023, pp. 205–218

    2023

  8. [16]

    The interspeech 2018 compu- tational paralinguistics challenge: Atypical & self-assessed affect, crying & heart beats,

    B. Schuller, S. Steidl, A. Batliner, P. B. Marschik, H. Baumeister, F. Dong, S. Hantke, F. B. Pokorny, E.- M. Rathner, K. D. Bartl-Pokorny, C. Einspieler, D. Zhang, A. Baird, S. Amiriparian, K. Qian, Z. Ren, M. Schmitt, P. Tzirakis, and S. Zafeiriou, “The interspeech 2018 comp...

    2018

  9. [17]

    Opensmile: the mu- nich versatile and fast open-source audio feature extractor,

    F. Eyben, M. W ¨ollmer, and B. Schuller, “Opensmile: the mu- nich versatile and fast open-source audio feature extractor,” inProceedings of the 18th ACM international conference on Multimedia, 2010, pp. 1459–1462

    2010

  10. [18]

    An ensemble of transfer, semi-supervised and su- pervised learning methods for pathological heart sound clas- sification,

    A. I. Humayun, M. T. Khan, S. Ghaffarzadegan, Z. Feng, and T. Hasan, “An ensemble of transfer, semi-supervised and su- pervised learning methods for pathological heart sound clas- sification,” inProceedings of Interspeech 2018, 2018, pp. 127–131

    2018

  11. [19]

    Learning image-based representations for heart sound classification,

    Z. Ren, N. Cummins, V . Pandit, J. Han, K. Qian, and B. Schuller, “Learning image-based representations for heart sound classification,” inProceedings of the 2018 interna- tional conference on digital health, 2018, pp. 143–147

    2018

  12. [20]

    Predicting heart activity from speech using data-driven and knowledge- based features,

    G. Elbanna, Z. Mostaani, and M. Magimai-Doss, “Predicting heart activity from speech using data-driven and knowledge- based features,” inProceedings of Interspeech 2024, 2024, pp. 4758–4762

    2024

  13. [21]

    Investigating salient repre- sentations and label variance in dimensional speech emotion analysis,

    V . Mitra, J. Nie, and E. Azemi, “Investigating salient repre- sentations and label variance in dimensional speech emotion analysis,” inICASSP 2024-2024 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 11 111–11 115

    2024

  14. [22]

    Pre-trained model representations and their robust- ness against noise for speech emotion analysis,

    V . Mitra, V . Kowtha, H.-Y . S. Chien, E. Azemi, and C. Aven- dano, “Pre-trained model representations and their robust- ness against noise for speech emotion analysis,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE...

    2023

  15. [23]

    Speech emotion recognition using self- supervised features,

    E. Morais, R. Hoory, W. Zhu, I. Gat, M. Damasceno, and H. Aronowitz, “Speech emotion recognition using self- supervised features,” inICASSP 2022-2022 IEEE Interna- tional Conference on Acoustics, Speech and Signal Process- ing (ICASSP). IEEE, 2022, pp. 6922–6926

    2022

  16. [24]

    Multi-modal dataset across exertion levels: Capturing post-exercise speech, breathing, and phonocardio- gram,

    J. Nie, Y . Fan, M. Zhao, R. Wan, Z. Xuan, M. Preindl, and X. Jiang, “Multi-modal dataset across exertion levels: Capturing post-exercise speech, breathing, and phonocardio- gram,” inProceedings of the 23rd ACM Conference on Em- bedded Networked Sensor Systems, 2025, pp. 297–304

    2025

  17. [25]

    Layer-wise anal- ysis of a self-supervised speech representation model,

    A. Pasad, J.-C. Chou, and K. Livescu, “Layer-wise anal- ysis of a self-supervised speech representation model,” in 2021 IEEE Automatic Speech Recognition and Understand- ing Workshop (ASRU). IEEE, 2021, pp. 914–921

    2021

  18. [26]

    Pre-trained founda- tion model representations to uncover breathing patterns in speech,

    V . Mitra, A. Chatterjee, K. Zhai, H. Weng, A. Hill, N. Hay, C. Webb, J. Cheng, and E. Azemi, “Pre-trained founda- tion model representations to uncover breathing patterns in speech,”arXiv preprint arXiv:2407.13035, 2024

    2024 arXiv

  19. [27]

    Exploring wav2vec 2.0 model for heart murmur detection,

    D. S. Panah, A. Hines, and S. McKeever, “Exploring wav2vec 2.0 model for heart murmur detection,” in2023 31st European Signal Processing Conference (EUSIPCO). IEEE, 2023, pp. 1010–1014

    2023

  20. [28]

    Foundation models for cardiovascular disease detection via biosignals from digital stethoscopes,

    G. Mathew, D. Barbosa, J. Prince, and S. Venkatraman, “Foundation models for cardiovascular disease detection via biosignals from digital stethoscopes,”npj Cardiovascular Health, vol. 1, no. 1, p. 25, 2024

    2024

  21. [29]

    The circor digiscope phonocardiogram dataset,

    J. Oliveira, F. Renna, P. Costa, M. Nogueira, A. C. Oliveira, A. Elola, C. Ferreira, A. Jorge, A. B. Rad, M. Reynaet al., “The circor digiscope phonocardiogram dataset,”version 1.0. 0, 2022

    2022

  22. [30]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhut- dinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” inIEEE Transactions on Audio, Speech, and Lan- guage Processing, vol. 29, 2021, pp. 3451–3460

    2021

  23. [31]

    wav2vec 2.0: A framework for self-supervised learning of speech rep- resentations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech rep- resentations,” inAdvances in Neural Information Processing Systems, vol. 33, 2020, pp. 12 449–12 460

    2020

  24. [32]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, J. Li, M. Zhou, and F. Wei, “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  25. [33]

    Robust speech recognition via large-scale weak supervision,

    A. Radfordet al., “Robust speech recognition via large-scale weak supervision,” https://github.com/openai/whisper, 2022

    2022

  26. [34]

    Clap learning audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang, “Clap learning audio concepts from natural language supervision,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  27. [35]

    Masked autoencoders that listen,

    P.-Y . Huang, H. Xu, J. Li, A. Baevski, M. Auli, W. Galuba, F. Metze, and C. Feichtenhofer, “Masked autoencoders that listen,”Advances in Neural Information Processing Systems, vol. 35, pp. 28 708–28 720, 2022

    2022

  28. [36]

    Hear–health acoustic representations,

    S. Baur, Z. Nabulsi, W.-H. Weng, J. Garrison, L. Blanke- meier, S. Fishman, C. Chen, S. Kakarmath, M. Maimbolwa, N. Sanjaseet al., “Hear–health acoustic representations,” arXiv preprint arXiv:2403.02522, 2024

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.