REVIEW 4 major objections 4 minor 35 references
Explainability of CNN Based Classification Models for Acoustic Signal
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Fusing Grad-CAM and DeepLIFT heatmaps reveals the song regions a CNN uses to separate Bewick's wren variants.
desk verdict A competent applied XAI case study on Bewick's wren dialects whose central biological interpretation is plausible but under-supported because recording-site provenance is not controlled in the train/test split. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the ensemble saliency map built from Grad-CAM and DeepLIFT heatmaps. Grad-CAM weights the last convolutional layer's feature maps by pooled gradients, giving a coarse localization of the class-relevant image region; DeepLIFT back-propagates contributions relative to a reference input, here a plain white image, giving fine-grained relevance at the input pixels. The two normalized maps are fused by Eq. (1) weighted average and Eq. (2) element-wise maximum, with the max strategy doing the main work of preserving the strongest activated regions of either method. The second piece of machinery is t-SNE on the CNN's flattened penultimate-layer features, which reveals the sub-clusters that the subsequent per-cluster heatmaps are aligned with.
What would settle it
Retrain the identical CNN on the same spectrograms with the song pixels masked out; if classification accuracy remains near 94.8% or the ensemble heatmaps still highlight non-song regions, the model is exploiting background cues rather than the vocal features the paper interprets.
Extended reading notes
Core claim
The central claim is that no single XAI technique is sufficient for interpreting a CNN trained on spectrogram images, but a deliberately fused saliency map is. Grad-CAM supplies coarse class-discriminative localization from the last convolutional layer, while DeepLIFT supplies fine-grained, direction-sensitive attribution from a reference input; after normalizing both to $[0,1]$, the paper combines them either as a weighted average $H_c = 0.5H_{\mathrm{CAM}} + 0.5H_{\mathrm{LDF}}$ or as an element-wise maximum $H_{\max}(i,j) = \max(H_{\mathrm{CAM}}(i,j), H_{\mathrm{LDF}}(i,j))$. In the paper's reading, the maximum ensemble captures all key activation regions from either method and, by a threshold-based pixel count, activates more regions than either standalone heatmap. Model-agnostic LIME and SHAP explanations were less consistent, and DeepLIFT was the most interpretable to bird-song experts. A separate t-SNE latent-space analysis then shows multiple distinct clusters within each of the two song variants, with heatmaps consistent within each cluster, which the paper takes as evidence of biologically meaningful sub-populations rather than noisy classifications.
Load-bearing premise
The CNN separates Eastern from Mexican songs because of song content itself, not because recordings from the two groups carry different environmental backgrounds, so the heatmaps reveal biologically meaningful vocal features.
Editorial extensions
If this is right
- A single XAI method can miss regions the model relies on; the element-wise maximum fusion is a cheap way to get a more complete saliency map without retraining.
- DeepLIFT stands out as the most interpretable single method for experts reading spectrograms, so bioacoustic studies may adopt it as a default.
- Cluster-specific explanations imply that a model's global explanations for a whole class can be misleading if the class contains acoustic sub-populations.
- Terminal trills, highlighted across methods, are the acoustic features the CNN appears to use to distinguish Eastern from Mexican songs, matching human listening.
- The same ensemble-XAI pipeline can be applied to other species or acoustic tasks without model changes.
Reading between the lines
- This is an inference: a natural next test would be to compare t-SNE cluster assignments against recording locality, date, or individual bird identity; the paper does not report such matching, so whether the sub-populations are geographic dialects, individual signatures, or recording-site artifacts remains open.
- This is an inference: the threshold-pixel-count metric used to compare ensemble versus standalone heatmaps measures spatial coverage but not whether the added pixels are biologically informative; coverage should be validated against expert-labeled song regions or a removal test.
- This is an inference: if the CNN is chiefly keying on terminal trills, the same fusion method could be used to quantify how much each spectro-temporal element contributes to dialect differences in other song-learning birds, generating comparative hypotheses the paper only gestures at.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper trains a CNN to classify spectrograms of Eastern and Mexican Bewick's wren songs, reaching 94.8% accuracy on a mixed-background dataset, and applies four XAI methods (LIME, SHAP, Grad-CAM, DeepLIFT) to interpret predictions. The authors propose ensemble saliency maps combining Grad-CAM and DeepLIFT via weighted averaging and element-wise maximum, reporting that the max ensemble highlights more regions above importance thresholds. They also apply PCA and t-SNE to the latent space and claim that t-SNE reveals sub-populations within each song variant, supported by qualitative inspection of Grad-CAM/DeepLIFT heatmaps for selected cluster samples.
Significance. If the evidence were sound, the paper would offer a useful comparative evaluation of XAI techniques in bioacoustics and a practical ensemble strategy for generating more complete saliency maps, as well as a hypothesis-generating latent-space analysis. The work addresses an under-explored application of XAI to bird song classification, and the qualitative biological observations (e.g., emphasis on terminal trills) are interesting. However, the current manuscript does not yet substantiate its central claims because key quantitative support is partly definitional and several analyses rely on very small sample sizes without control for recording provenance.
major comments (4)
- [III-B] The manuscript never describes how the 1854 training and 806 test samples were split with respect to recording sessions, individual birds, or geographic locations. Since all recordings were collected in Arizona and New Mexico (Section III-A), a random clip-level split can place the same bird or the same ambient recording conditions in both train and test, which would inflate the reported 94.8% accuracy and make the saliency maps reflect recording-site or background-acoustic artifacts rather than song content. This is load-bearing for the biological interpretation of the explanations and for the t-SNE sub-population claim. The authors should clarify the split procedure and, if possible, perform a group-wise split (e.g., by recording session or location) and report the resulting metrics, or provide control analyses ruling out site and background cues.
- [IV-B, Figure 6] The quantitative support for the max ensemble is definitional. Since Hmax(i,j) = max(H_CAM(i,j), H_LDF(i,j)) by Equation 2, the fraction of pixels above any threshold t is at least as large for the max ensemble as for either Grad-CAM or DeepLIFT individually, simply because the max of two values exceeds t whenever either component exceeds t. Therefore, Figure 6's observation that 'Ensemble Max' consistently activates more regions above thresholds is mathematically guaranteed and does not independently establish that the ensemble captures more meaningful or discriminative features. The authors should either replace this metric with a more substantive evaluation (e.g., agreement with expert-annotated song regions, or a controlled comparison of downstream tasks) or explicitly reframe the claim as a qualitative property rather than empirical evidence of superiority.
- [IV-A, Figures 3-4] The central qualitative claims about the XAI outputs are based on a single randomly selected test sample per class. Statements such as 'both Grad-CAM and DeepLIFT consistently highlight low- to mid-frequency elements' and 'DeepLIFT produced the most interpretable explanations for bird song experts' are made from exactly one Eastern and one Mexican example. This is not sufficient to establish consistency or to justify general conclusions about which XAI method is more interpretable. The authors should report aggregated statistics over a larger set of samples (e.g., multiple examples per class, or quantitative overlap of highlighted regions with the spectrogram signal), or at minimum present several representative samples to show that the patterns are reproducible.
- [IV-C, Figures 5 and 7] The sub-population claim rests on visual t-SNE clusters without any quantitative validation. t-SNE often produces apparent clusters even on random or uniformly distributed data, and the manuscript provides no stability analysis (e.g., repeated runs with different perplexities), no cluster quality metric (e.g., silhouette score), and no control for recording site, individual identity, or ambient noise. The conclusion that the clusters represent 'meaningful subgroups' within Eastern and Mexican song variants is therefore not supported. The authors should add quantitative cluster validation and control analyses, and should temper the claims if such analyses are not feasible with the current dataset.
minor comments (4)
- [III-A] There is a typo: 'The STF transformation' should read 'The STFT transformation'.
- [Abstract] The abstract says 'improve trust and interoperability' where 'interpretability' is presumably intended.
- [References] References [7] and [32] appear to cite the same paper (Das et al., 'Exploring explainable AI methods for bird sound-based species recognition systems'); please consolidate or differentiate them.
- [Table I] The classification metrics in Table I are reported without confidence intervals or multiple-seed standard deviations; adding these would help assess significance of the small differences across background conditions.
Circularity Check
The element-wise-max ensemble's threshold superiority is entailed by Eq. (2), making its 'more comprehensive' quantitative support tautological.
-
self definitional
[Section III-C, Eq. (2); Section IV-B, Fig. 6]
"Hmax(i,j) = max (H CAM(i,j), H LDF(i,j)) (2) ... To quantify the density of strongly activated regions, we calculated the percentage of heatmap cells above varying importance thresholds (from 0.4 to 0.9). As shown in Figure 6, the ensemble-max saliency map consistently activates a higher proportion of regions across thresholds compared to either Grad-CAM or DeepLIFT alone, suggesting its effectiveness in capturing both localized and distributed cues."
Because both heatmaps are normalized to [0,1], Eq. (2) forces Hmax >= H_CAM and H_max >= H_LDF at every pixel. Therefore the set of pixels above any threshold for the max ensemble necessarily contains the corresponding sets for both component heatmaps, so the measured 'higher proportion of regions' is a logical consequence of the definition, not an empirical finding. The paper presents this as evidence that the ensemble 'captures a more comprehensive set of discriminative features,' but the quantitative threshold curve in Fig. 6 is fixed by Eq. (2) once the two input heatmaps are chosen. The qualitative visual claim about biological relevance may be independently judged, but the specific quantitative support is tautological.
full rationale
The central ensemble construction in Eq. (2) makes the max-based ensemble's larger threshold-exceedance areas true by definition, so the paper's Figure 6 evidence for a 'more comprehensive' explanation reduces to the construction itself. This is a genuine partial circularity in the quantitative support for the ensemble claim. However, the rest of the paper is largely self-contained: the DeepLIFT-versus-Grad-CAM qualitative comparisons, the t-SNE sub-population visualization, and the biological interpretation do not depend on Eq. (2), and no load-bearing self-citation chain or fitted-parameter-as-prediction pattern is present. The recording-provenance and generalization concerns are correctness risks rather than circularity. Overall, one central quantitative claim reduces by construction, but the derivation is not entirely equivalent to its inputs, giving a partial-circularity score of 6.
Assumptions & free parameters
free parameters (5)
- CNN architecture and training hyperparameters =
16/32/64/128/256 conv channels, 3x3 kernels, 2x2 pooling, FC 512/1024, lr=0.001, weight decay=1e-5, 20 epochs, batch…
- STFT/spectrogram parameters =
window 512 samples, 95% overlap, dynamic range -40 to 5 dB, resolution 480x960
- LIME superpixel count and perturbation number =
100 superpixels, 1000 perturbations
- SHAP background sample count =
50
- Ensemble weights w1, w2 =
0.5, 0.5
assumptions (5)
- domain assumption Eastern and Mexican song variants are the true class labels and recordings are not confounded with recording site.
- domain assumption Spectrogram images retain the acoustic information needed for classification.
- domain assumption Saliency and attribution maps faithfully indicate which input pixels the CNN uses for its decision.
- domain assumption t-SNE clusters correspond to biological sub-populations rather than embedding artifacts.
- domain assumption DeepLIFT reference baseline (white image) is a reasonable neutral input.
Cite this review
Pith. "Pith review of Explainability of CNN Based Classification Models for Acoustic Signal." pith.science (2026). https://pith.science/paper/SYY4XMFF
@misc{pith2026250908717,
author = {Pith},
title = {Pith review of: Explainability of CNN Based Classification Models for Acoustic Signal},
year = {2026},
howpublished = {\url{https://pith.science/paper/SYY4XMFF}},
note = {Machine review of arXiv:2509.08717}
}
read the original abstract
Explainable Artificial Intelligence (XAI) has emerged as a critical tool for interpreting the predictions of complex deep learning models. While XAI has been increasingly applied in various domains within acoustics, its use in bioacoustics, which involves analyzing audio signals from living organisms, remains relatively underexplored. In this paper, we investigate the vocalizations of a bird species with strong geographic variation throughout its range in North America. Audio recordings were converted into spectrogram images and used to train a deep Convolutional Neural Network (CNN) for classification, achieving an accuracy of 94.8\%. To interpret the model's predictions, we applied both model-agnostic (LIME, SHAP) and model-specific (DeepLIFT, Grad-CAM) XAI techniques. These techniques produced different but complementary explanations, and when their explanations were considered together, they provided more complete and interpretable insights into the model's decision-making. This work highlights the importance of using a combination of XAI techniques to improve trust and interoperability, not only in broader acoustics signal analysis but also argues for broader applicability in different domain specific tasks.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
R. D. Kent and Y . Kim, “Acoustic analysis of speech,”The handbook of clinical linguistics, pp. 360–380, 2008
work page 2008
-
[2]
Biosignal sensors and deep learning-based speech recognition: A review,
W. Lee, J. J. Seong, B. Ozlu, B. S. Shim, A. Marakhimov, and S. Lee, “Biosignal sensors and deep learning-based speech recognition: A review,”Sensors, vol. 21, no. 4, p. 1399, 2021
work page 2021
-
[3]
C. Zhang, A. A. Mousavi, S. F. Masri, G. Gholipour, K. Yan, and X. Li, “Vibration feature extraction using signal processing techniques for structural health monitoring: A review,”Mechanical Systems and Signal Processing, vol. 177, p. 109175, 2022
work page 2022
-
[4]
C. Kroos, O. Bones, Y . Cao, L. Harris, P. J. Jackson, W. J. Davies, W. Wang, T. J. Cox, and M. D. Plumbley, “Generalisation in environ- mental sound classification: the ‘making sense of sounds’ data set and challenge,” inICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 8082–8086
work page 2019
-
[5]
D. Kroodsma and B. Byers, “The function(s) of bird song,”American Zoologist, vol. 31, pp. 318–328, 04 1991
work page 1991
-
[6]
A survey of audio classification using deep learning,
K. Zaman, M. Sah, C. Direkoglu, and M. Unoki, “A survey of audio classification using deep learning,”IEEE Access, vol. 11, pp. 106 620– 106 649, 2023
work page 2023
-
[7]
Exploring ex- plainable ai methods for bird sound-based species recognition systems,
N. Das, N. Padhy, N. Dey, H. Paul, and S. Chowdhury, “Exploring ex- plainable ai methods for bird sound-based species recognition systems,” Multimedia Tools and Applications, vol. 83, no. 24, pp. 64 223–64 253, 2024
work page 2024
-
[8]
Audio explainable artificial intelligence: A review,
A. Akman and B. W. Schuller, “Audio explainable artificial intelligence: A review,”Intelligent Computing, vol. 2, p. 0074, 2024
work page 2024
Show all 35 references
-
[9]
Song learning, dialects, and dispersal in the bewick’s wren,
D. E. Kroodsma, “Song learning, dialects, and dispersal in the bewick’s wren,”Zeitschrift f ¨ur Tierpsychologie, vol. 35, no. 4, pp. 352–380, 1974
1974
-
[10]
Seewave, a free modular tool for sound analysis and synthesis,
J. Sueur, T. Aubin, and C. Simonis, “Seewave, a free modular tool for sound analysis and synthesis,”Bioacoustics, vol. 18, no. 2, pp. 213–226, 2008
2008
-
[11]
Why should i trust you? explaining the predictions of any classifier,
M. T. Ribeiro, S. Singh, and C. Guestrin, “Why should i trust you? explaining the predictions of any classifier,” inProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1135–1144
2016
-
[12]
A unified approach to interpreting model predictions,
S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,”Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017
2017
-
[13]
Grad-cam: Visual explanations from deep networks via gradient-based localization,
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 618–626
2017
-
[14]
Learning important features through propagating activation differences,
A. Shrikumar, P. Greenside, and A. Kundaje, “Learning important features through propagating activation differences,”arXiv preprint arXiv:1704.02685, 2017
2017 arXiv
-
[15]
Visualizing data using t-sne
L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.”Journal of machine learning research, vol. 9, no. 11, 2008
2008
-
[16]
Principal component analysis,
H. Abdi and L. J. Williams, “Principal component analysis,”Wiley interdisciplinary reviews: computational statistics, vol. 2, no. 4, pp. 433– 459, 2010
2010
-
[17]
Environmental sound classification with convolutional neural networks,
K. J. Piczak, “Environmental sound classification with convolutional neural networks,” in2015 IEEE 25th international workshop on machine learning for signal processing (MLSP). IEEE, 2015, pp. 1–6
2015
-
[18]
Automatic acoustic detection of birds through deep learning: the first bird audio detection challenge,
D. Stowell, M. Wood, H. Pamuła, and Y . Stylianou, “Automatic acoustic detection of birds through deep learning: the first bird audio detection challenge,”Methods in Ecology and Evolution, vol. 7, no. 9, pp. 1175– 1187, 2016
2016
-
[19]
Birdnet: A deep learning solution for avian diversity monitoring,
S. Kahl, C. M. Wood, M. Eibl, and H. Klinck, “Birdnet: A deep learning solution for avian diversity monitoring,”Ecological Informatics, vol. 61, p. 101236, 2021
2021
-
[20]
A machine learning approach for classifying and quantifying acoustic diversity,
S. C. Keen, K. J. Odom, M. S. Webster, G. M. Kohn, T. F. Wright, and M. Araya-Salas, “A machine learning approach for classifying and quantifying acoustic diversity,”Methods in Ecology and Evolution, vol. 12, no. 7, pp. 1213–1225, 2021
2021
-
[21]
Deep convolutional neural networks and data augmentation for environmental sound classification,
J. Salamon and J. P. Bello, “Deep convolutional neural networks and data augmentation for environmental sound classification,” in2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, pp. 3581–3585
2017
-
[22]
Birdsong classification based on ensemble multi-scale convolutional neural network,
J. Liu, Y . Zhang, D. Lv, J. Lu, S. Xie, J. Zi, Y . Yin, and H. Xu, “Birdsong classification based on ensemble multi-scale convolutional neural network,”Scientific Reports, vol. 12, no. 1, p. 8636, 2022
2022
-
[23]
Bird species recognition using support vector machines,
S. Fagerlund, “Bird species recognition using support vector machines,” EURASIP Journal on Advances in Signal Processing, vol. 2007, no. 1, p. 038637, 2007. [Online]. Available: https://doi.org/10.1155/2007/38637
2007 doi
-
[24]
Computational bioacoustics with deep learning: A review and roadmap,
D. Stowell, “Computational bioacoustics with deep learning: A review and roadmap,”PeerJ, vol. 10, p. e13152, 2022
2022
-
[25]
Explainable artificial intelligence (xai): Concepts, tax- onomies, opportunities and challenges toward responsible ai,
A. B. Arrieta, N. D ´ıaz-Rodr´ıguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. Garcia, S. Gil-Lopez, D. Molina, R. Benjamins, and F. Herrera, “Explainable artificial intelligence (xai): Concepts, tax- onomies, opportunities and challenges toward responsible ai,”Informa...
2020
-
[26]
Axiomatic attribution for deep networks,
M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,”International conference on machine learning, pp. 3319– 3328, 2017
2017
-
[27]
The mythos of model interpretability,
Z. C. Lipton, “The mythos of model interpretability,”arXiv, 2017. [Online]. Available: http://arxiv.org/abs/1606.03490
2017 arXiv
-
[28]
Jeyasothy, T
A. Jeyasothy, T. Laugel, M.-J. Lesot, C. Marsala, and M. Detyniecki, Integrating Prior Knowledge in Post-hoc Explanations, 01 2022, pp. 707–719
2022
-
[29]
Artificial intelligence in landscape ecology: Recent advances, perspectives, and opportunities,
A. E. Frazier and L. Song, “Artificial intelligence in landscape ecology: Recent advances, perspectives, and opportunities,”Current Landscape Ecology Reports, vol. 10, no. 1, p. 1, 2024
2024
-
[30]
Explainable artificial intelligence enhances the ecological interpretability of black-box species distribution models,
M. Ryo, B. Angelov, S. Mammola, J. M. Kass, B. M. Benito, and F. Hartig, “Explainable artificial intelligence enhances the ecological interpretability of black-box species distribution models,”Ecography, vol. 44, no. 2, pp. 199–205, 2021
2021
-
[31]
Categorizing shallow marine soundscapes using explained clusters,
C. Parcerisas, I. T. Roca, D. Botteldooren, P. Devos, and E. Debusschere, “Categorizing shallow marine soundscapes using explained clusters,” Journal of Marine Science and Engineering, vol. 11, no. 3, p. 550, 2023
2023
-
[32]
Exploring explainable ai methods for bird sound-based species recognition systems,
N. Das, N. Padhy, N. Deyet al., “Exploring explainable ai methods for bird sound-based species recognition systems,”Multimedia Tools and Applications, vol. 83, pp. 64 223–64 253, 2024
2024
-
[34]
Bridging ai and ecology: Cilnn and xai for acoustic based prediction of dangerous wild animals,
G. Govindaprabhu, M. Sumathi, S. Neyvasagam, and N. A. J. Kumar, “Bridging ai and ecology: Cilnn and xai for acoustic based prediction of dangerous wild animals,”International Research Journal of Multidisciplinary Studies, vol. 6, no. 1, pp. 1280–1298, 2025. [Online]. Availabl...
2025
-
[35]
Audioprotopnet: An interpretable deep learning model for bird sound classification,
W. Zhang, J. Smith, and A. Patel, “Audioprotopnet: An interpretable deep learning model for bird sound classification,” inProceedings of the International Conference on Machine Learning for Bioacoustics. ICMLB, 2024, pp. 345–356. [Online]. Available: https://arxiv.org/abs/ 2401.12345
2024 arXiv
-
[2024]
Available: https://arxiv.org/abs/2406.01253
[Online]. Available: https://arxiv.org/abs/2406.01253
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.