{"id":"c72ed0a2-550c-445f-8550-73178a584141","arxiv_id":"2412.02006","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A cross-attention framework aligns Wav2Vec 2.0 speech embeddings with clinical features to yield interpretable, competitive Parkinson's detection across five languages.","lead":"This paper combines self-supervised speech embeddings with 35 clinical speech features using cross-attention, letting the model show which features matter for Parkinson's detection and when they matter in an utterance. It matches the accuracy of black-box SSL baselines on most tasks while offering two views of what the model focuses on.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Interpretability claim rests on attention maps from one seed and correct test samples only; no null-model or statistical baseline shows the task-aligned patterns are actually learned rather than selected.","rationale":"The accuracy results are credible, the code is released, and the paper is candid about the attention-interpretability debate and the absence of clinical validation. Those are real strengths. My concern is limited to the interpretability claim as stated in Section VI-B: 'effectively identifies distinct speech patterns ... align with the expected speech dimensions.' The supporting evidence is the averaged attention maps in Figures 2-5, computed on one corpus, one seed, and correctly predicted test samples only, with no null model or statistical test. The paper's own caveats in Sections III-B, VI-B, VI-C, and VI-D show that the authors are aware of the limitation, but the headline claim is still phrased more strongly than the evidence warrants. The decisive experiment is a control: label-shuffled and/or randomly initialized modules. If the same task-aligned attention patterns persist under those controls, the patterns are not diagnostic evidence. If they disappear, the concern is resolved. This reinforces, rather than changes, the reader's conditional verdict: the framework is promising but the interpretability claim needs a stronger empirical basis before it is positioned as clinically transparent.","tokens_in":24500,"tokens_out":5587,"duration_ms":60372,"concrete_test":"Using the released code, rerun the GITA interpretability pipeline (Sections VI-B and VI-C) under two control conditions: (1) randomly initialized cross-attention modules with the same input features; (2) training with shuffled PD/HC labels. Then recompute Fig. 2 and the prosody-peak analysis. If either control yields task-aligned attention patterns of similar magnitude (e.g., same top-3 features per task, or same prosody peak on stressed words), the reported patterns are not evidence of learned diagnostic relevance. Additionally, recompute Fig. 2 across all 5 seeds and including misclassified test samples; if the task-aligned pattern is not stable across seeds or disappears when misclassified samples are included, the selection criterion is driving the result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the cross-attention framework 'effectively identifies distinct speech patterns ... align[ed] with the expected speech dimensions of each assessment task' (Section VI-B). The load-bearing step is treating learned attention weights as evidence about which informed clinical features drive PD detection. Three conditions would have to hold: (i) the attention matrices S_emb and S_temp reflect causally relevant alignment between SSL embeddings and informed features; (ii) averaging those weights over groups is statistically meaningful; (iii) the reported pattern is not an artifact of evaluation choices. Condition (iii) is where the argument is weakest. All embedding-level and temporal interpretability analyses use only the GITA corpus, only the best-performing random seed, and only test samples that were correctly predicted (Sections VI-B and VI-C). Selecting on correctness makes the attention maps conditional on the model's own decision rule; a model that is right for spurious dataset-specific reasons can still produce clean-looking HC/PD attention differences. The paper itself reports that other corpora do not always show consistent patterns (VI-B), that 'not all samples reflect this expected behavior' (VI-C), and that high attention does not imply presence/severity of impairment (III-B, VI-D). Because attention weights are softmax-normalized over the 35 informed features and optimized jointly with the classifier, there is no null baseline showing that the task-aligned patterns in Fig. 2 would not also appear with label-shuffled or randomly initialized modules. Without such a baseline, the qualitative agreement with clinical expectations is not sufficient to separate learned diagnostic structure from SSL geometry and selection bias.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an interpretable framework for Parkinson's Disease (PD) detection that combines self-supervised speech representations (XLS-R Wav2Vec2.0 embeddings) with 35 clinically informed DisVoice features through two cross-attention modules. The embedding-level module yields an attention matrix over SSL embedding dimensions and informed features, and the temporal-level module yields a time-step-by-feature attention matrix; both feed a linear classifier. The method is evaluated on five PD speech corpora (NeuroVoz, GITA, FraLusoPark, GermanPD, CzechPD) across six assessment tasks using speaker-independent nested cross-validation. Classification results are competitive with an SSL-only baseline, with some noticeable drops on FraLusoPark and CzechPD, and the method shows strong cross-lingual performance on the MONOLOGUE task. Interpretability analyses are performed on the GITA corpus and show attention patterns that the authors argue align with expected speech dimensions per task, plus temporal case studies of individual PD subjects.","tokens_in":24761,"tokens_out":3595,"duration_ms":37450,"significance":"If the interpretability claims were fully supported, this would be a useful contribution toward transparent SSL-based clinical speech analysis: it is one of few works to explicitly inject clinically informed features into an attention mechanism over SSL embeddings, it evaluates across five languages and multiple tasks, and it honestly reports the accuracy/transparency trade-off. The paper also ships a public code repository and uses a task-specific evaluation protocol that facilitates comparison. However, the central claim that the framework 'effectively identifies distinct speech patterns' rests on post-hoc visual inspection of attention maps from a single corpus, a single random seed, and only correctly predicted test samples, with no null model or statistical validation. The authors themselves acknowledge the ongoing debate on attention interpretability and state that the explanations have not undergone medical validation. As a result, the interpretability evidence is currently suggestive rather than demonstrative, and the paper would need additional validation to support its main claim.","major_comments":[{"comment":"The central claim that the framework 'effectively identifies distinct speech patterns ... align[ed] with the expected speech dimensions of each assessment task' is supported only by attention maps computed on the GITA corpus, using the best-performing random seed and only the test samples that were correctly predicted. This selection is acknowledged in the text, but it conditions the explanation on the model's own decision rule and on the most favorable run. As a result, the reported HC/PD differences and task-specific alignments could be artifacts of this selection. Please report the stability of the attention patterns across all seeds and folds, include misclassified samples for comparison, and provide a null baseline (e.g., permuted labels or randomly initialized attention) to demonstrate that the observed alignment is not produced by chance.","section":"Section VI-B"},{"comment":"The paper cites the debate on whether attention weights are faithful explanations [46]-[48] and correctly states that high attention does not imply the presence or severity of a speech impairment, but then interprets high mean attention on logE, F1, and glottal features as evidence that the model identified phonatory and glottal dynamics in the VOWELS task. Because the key projection is the identity matrix in Eq. (2), the attention scores are softmax-normalized alignments between projected SSL embeddings and static informed features; they have no demonstrated causal link to the classification outcome or to the clinical constructs. A concrete fidelity test is needed, such as comparing attention-based feature rankings with gradient-based attributions on the same model, or ablating the highest-attended features and measuring the effect on classification performance, to substantiate that attention is an explanation rather than only an alignment score.","section":"Section III-B and VI-B"},{"comment":"The claimed differentiation between HC and PD groups in embedding-level attention is descriptive only: no error bars, confidence intervals, or significance tests are reported, and the paper concedes that analyses across the other corpora 'did not always show a consistent pattern.' Without quantifying variability across seeds, folds, or corpora, the robustness of the interpretability finding is not established. Please report per-seed and per-fold distributions, add a statistical comparison (e.g., permutation test or effect sizes) for the GITA results, and explicitly discuss how the inconsistent behavior in the other four corpora bears on the claim of robustness.","section":"Section VI-B, Figure 3"}],"minor_comments":[{"comment":"There is a typo in the first sentence of Section IV-A: 'black-blox' should be 'black-box'.","section":"Section IV-A"},{"comment":"In Section IV-B, the word 'intepretability' appears in the sentence beginning 'Our first experiments utilized the full range...'; it should be 'interpretability'.","section":"Section IV-B"},{"comment":"Figures 2 and 3 are extremely dense, with axis labels and tick labels too small to read at normal print size; the figures should be restructured or rendered with larger fonts, and error bars should be added if the displayed values are averages over samples.","section":"Figures 2 and 3"},{"comment":"Figures 4 and 5 present single-subject temporal analyses; the text should more explicitly frame these as illustrative case studies rather than generalizable group findings, given the acknowledged heterogeneity across PD subjects.","section":"Section VI-C"},{"comment":"The hyperparameter search is described only as 'preliminary,' with the final learning rate and number of epochs given; please report the search range or the values that were tried, to support reproducibility.","section":"Section V-D"}],"recommendation":"major_revision","confidential_remarks":"The accuracy results are honestly reported and the multi-corpus evaluation is a strength. My concern is confined to the interpretability evidence, which is the paper's central contribution; the current post-hoc selection and lack of statistical validation do not yet support the strength of the claims in the abstract and Section VI-B. I see no issues with the citation pattern or novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a competent and honest empirical paper. The dual-branch cross-attention design — aligning SSL embeddings with 35 clinically informed DisVoice features at both embedding and temporal levels — is genuinely new relative to prior work that used self-attention, phoneme-level neuron analysis, or gradient methods. The authors report accuracy competitively, acknowledge the transparency/performance trade-off frankly (e.g., FraLusoPark drops, CzechPD READ-TEXT/MONOLOGUE), and they ship code. That alone earns them a serious read.\n\nThe best part is the temporal analysis: forcing alignment to the SSL encoder layer 7, using DTW and forced alignment to show prosody peaks on stressed words is a concrete, falsifiable pattern. The embedding-level heatmaps (Fig. 2) align with clinical expectations for sustained vowels and DDK in the GITA corpus. That is real evidence, not hand-waving.\n\nSoft spots, in proportion:\n\n1. The interpretability analysis rests on one seed, one corpus (GITA), and only correctly predicted test samples. The authors state this openly (Sections VI-B, VI-C), but it means the attention patterns are conditioned on the model's own decisions. No label-shuffled or random-init baseline shows that these task-aligned patterns would not appear with a model that is right for spurious dataset-specific reasons. This is the load-bearing weakness. It does not destroy the paper, but it caps the strength of the central claim to \"suggestive\" rather than \"demonstrated.\"\n\n2. The paper cites the attention-interpretability debate [46]-[48] and explicitly states \"we cannot guarantee that a high attention ... directly corresponds to higher occurrence of that feature\" (Section III-B). That is good scholarly hygiene, but the abstract and title still sell \"interpretability\" as if attention weights are explanations. The gap between the strong framing and the careful caveats is noticeable.\n\n3. The cross-lingual robustness section is thin: leave-one-out on monologue only, no cross-validation, \"though not significant\" stated honestly. Fine, but it should be labeled exploratory rather than robustness evidence.\n\n4. Feature selection (35 features from 655) is justified clinically, but the selection itself was presumably done with hindsight on test performance. The authors do not discuss whether this selection was locked before seeing test data. Minor, but worth flagging.\n\nWho benefits: researchers in pathological speech and interpretable ML for healthcare; clinicians thinking about decision support. The paper deserves a serious referee — the method is reproducible, the reporting is honest, and the question matters. A referee should push for a null-model baseline and multi-seed attention aggregates before the interpretability claim is taken as established.\n\nRecommendation: send it to peer review. The central mechanism is sound and the limitations are stated; what is missing is standard statistical discipline around the explanation analysis, not a fundamental flaw.","headline":"A useful, honest interpretability framework for SSL-based PD detection, but the headline claim about attention-as-explanation is softer than the title suggests.","tokens_in":25325,"tokens_out":713,"would_cite":true,"duration_ms":9535,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cross-attention makes black-box speech embeddings explainable for Parkinson's diagnosis.","keywords":["Parkinson's Disease","Self-Supervised Speech Representations","Cross-Attention Mechanisms","Interpretability","Deep Learning","Speech Biomarkers","Pathological Speech Analysis"],"falsifier":"A clinical validation study comparing the attention-highlighted features (e.g., amplitude perturbation quotient in diadochokinetic tasks, pause-duration variability in read speech) against independently measured acoustic or perceptual ratings of the same patients would settle the matter. If the top-attended features do not correlate with those independent measures on held-out data, or if temporal attention peaks do not align with the moments a speech therapist marks as pathological, the interpretability claim is falsified.","tokens_in":24330,"feed_emoji":"🗣️","tokens_out":9102,"duration_ms":77953,"temperature":0.7,"pith_summary":"The paper tries to establish that the opaque embeddings produced by a self-supervised speech model (one pre-trained on large amounts of unlabeled audio) can be made transparent enough to support Parkinson's disease diagnosis. Its mechanism is a cross-attention training setup that aligns the 1024-dimensional Wav2Vec2.0 embedding sequence with 35 static, clinically informed speech features covering articulation, glottal, phonation, and prosody. The resulting attention weights are read at two levels: which embedding dimensions track which clinical feature, and which moments of the utterance matter most. Across five PD speech corpora and six assessment tasks, the attended representations stay competitive with the SSL-only black-box baseline, and in spontaneous monologue they transfer across languages. A sympathetic reader would care because this is a concrete route from black-box deep speech models to a tool a clinician can question.","feed_headline":"Attention maps reveal which speech traits flag Parkinson's","feed_subtitle":"The model's attention weights point clinicians to the speech dimensions behind each Parkinson's diagnosis.","key_machinery":"The carrying device is a single-head scaled dot-product cross-attention mechanism in which the query and value projections are learned while the key projection is fixed to the identity matrix: $Q = X_{\\mathrm{ssl}} W_Q$, $K = X_{\\mathrm{inf}}$, $V = X_{\\mathrm{ssl}} W_V$. Because the keys are the 35 clinically informed features kept unchanged, the two resulting attention matrices are directly readable: the embedding-level matrix $S_{\\mathrm{emb}}$ says which Wav2Vec2.0 dimensions align with which clinical feature, and the temporal matrix $S_{\\mathrm{temp}}$ says which time step of the utterance aligns with which feature. The informed feature set is repeated to match either the time axis or the embedding axis, which is what allows a static feature vector to serve as an interpretable key in both perspectives. The classification module then consumes averaged, enriched representations from both branches, so the same trained model that produces the explanation also produces the prediction.","core_discovery":"On its own terms, the paper claims that the internal dimensions of a frozen XLS-R Wav2Vec2.0 representation do not encode interpretable speech features as isolated values, but that these features can nevertheless be retrieved by attention when the embeddings are aligned, as queries and values, against a static set of 35 clinically informed features acting as keys with an identity projection. The learned embedding-level attention matrix $S_{\\mathrm{emb}} \\in \\mathbb{R}^{D\\times F}$ quantifies, for each of the 1024 embedding dimensions, the relevance of each informed feature, while the temporal matrix $S_{\\mathrm{temp}} \\in \\mathbb{R}^{T\\times F}$ attributes the same features to each 20 ms time step. The paper's evidence that this works is behavioral: in the GITA corpus, attention weights concentrate on phonation and glottal features for sustained vowels, shift to prosodic features including pause variability in read text and sentences, and, in a contrastive healthy-control analysis, prosody peaks align with the words marked for emphasis in the GITA protocol, all while classification F1 stays broadly on par with the SSL-only baseline.","pith_inferences":["We infer that the attention matrices could be turned into per-patient speech profiles and tested against independent acoustic measurements to determine whether attention tracks the underlying signal rather than a class-discriminative shortcut.","We infer that ablating individual informed features should shift the attention pattern in a predictable direction, given that the identity key projection is what preserves interpretability; the paper does not run this check.","We infer that the same cross-attention design would carry over to other neurodegenerative or cognitive-communication conditions if the informed feature set were expanded with macro-descriptors such as word-finding difficulty.","We suspect the accuracy drops on FraLusoPark and CzechPD mark cases where the static informed features are weak anchors, which would motivate adding dynamic clinical features as keys."],"forward_implications":["A frozen self-supervised encoder can be interpreted without fine-tuning, since only the attention projections and the classifier are trained on PD data.","The explanation is task-sensitive: sustained-vowel tasks pull attention toward phonation and glottal dimensions, while continuous speech tasks pull attention toward prosodic dimensions such as pause duration and speech rate.","Temporal attention, contrasted against healthy controls, localizes relevant moments at word and phoneme level, with prosody peaking at words marked for emphasis.","The interpretable model stays competitive with the SSL-only black-box baseline on most tasks and transfers notably well in cross-lingual spontaneous monologue.","Swapping the static informed feature set re-targets the framework to other assessment tasks or conditions."],"supporting_citations":[{"why":"Supplies the frozen XLS-R Wav2Vec2.0 model from which the 1024-dimensional SSL embeddings are extracted.","marker":"[32]"},{"why":"Provides the DisVoice toolkit from which the 35 clinically informed speech features are selected.","marker":"[30]"},{"why":"Establishes that SSL representations outperform knowledge-based features in cross-lingual PD detection, the baseline the paper builds on.","marker":"[38]"},{"why":"Shows attention-based interpretability in cleft-lip-and-palate speech, the direct precedent for using attention as explanation.","marker":"[50]"},{"why":"Defines the scaled dot-product attention operation that both cross-attention modules implement.","marker":"[53]"},{"why":"Layer-wise analysis of SSL speech models used to justify choosing the 7th encoder layer.","marker":"[43]"},{"why":"Introduces the contrastive healthy-control reference approach used to highlight PD-specific temporal attention.","marker":"[82]"},{"why":"Supplies the phoneme-level alignments used for the fine-grained temporal interpretability analysis.","marker":"[84]"},{"why":"The 'attention is not explanation' argument the paper acknowledges when treating attention scores as clinically meaningful.","marker":"[46]"}],"fun_headline_variants":["Attention weights pinpoint Parkinson's speech features in SSL models","Cross-attention reveals which speech cues signal Parkinson's","SSL speech attention maps expose Parkinson's diagnostic patterns","Attention to prosody and phonation flags Parkinson's in speech"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the attention weights learned for classification genuinely reflect clinically meaningful speech attributes, rather than dataset-specific cues or spurious correlations, a premise the paper itself flags as debated and still awaiting clinical validation.","fun_headline_variants_meta":{"raw":{"variants":["Attention weights pinpoint Parkinson's speech features in SSL models","Cross-attention reveals which speech cues signal Parkinson's","SSL speech attention maps expose Parkinson's diagnostic patterns","Attention to prosody and phonation flags Parkinson's in speech"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1348,"prompt_tokens":964,"completion_tokens":384,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":320}},"tokens_in":580,"tokens_out":384,"duration_ms":4001,"temperature":1.0,"reasoning_tokens":320,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:55:22.414097+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A clinical validation study comparing the attention-highlighted features (e.g., amplitude perturbation quotient in diadochokinetic tasks, pause-duration variability in read speech) against independently measured acoustic or perceptual ratings of the same patients would settle the matter. If the top-attended features do not correlate with those independent measures on held-out data, or if temporal attention peaks do not align with the moments a speech therapist marks as pathological, the interpretability claim is falsified.","supporting_citations":[{"cited_title":"XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale,","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen XLS-R Wav2Vec2.0 model from which the 1024-dimensional SSL embeddings are extracted."},{"cited_title":"DisV oice,","cited_arxiv_id":null,"evidence_quote":"Provides the DisVoice toolkit from which the 35 clinically informed speech features are selected."},{"cited_title":"Interpretable speech features vs. DNN embed- dings: What to use in the automatic assessment of Parkinson’s disease in multi-lingual scenarios,","cited_arxiv_id":null,"evidence_quote":"Establishes that SSL representations outperform knowledge-based features in cross-lingual PD detection, the baseline the paper builds on."},{"cited_title":"Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech,","cited_arxiv_id":null,"evidence_quote":"Shows attention-based interpretability in cleft-lip-and-palate speech, the direct precedent for using attention as explanation."},{"cited_title":"Attention Is All You Need,","cited_arxiv_id":null,"evidence_quote":"Defines the scaled dot-product attention operation that both cross-attention modules implement."},{"cited_title":"Layer-Wise Analysis of a Self- Supervised Speech Representation Model,","cited_arxiv_id":null,"evidence_quote":"Layer-wise analysis of SSL speech models used to justify choosing the 7th encoder layer."},{"cited_title":"Towards Reference Speech Characterization for Health Applications,","cited_arxiv_id":null,"evidence_quote":"Introduces the contrastive healthy-control reference approach used to highlight PD-specific temporal attention."},{"cited_title":"Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi,","cited_arxiv_id":null,"evidence_quote":"Supplies the phoneme-level alignments used for the fine-grained temporal interpretability analysis."},{"cited_title":"Attention Is Not Explanation,","cited_arxiv_id":null,"evidence_quote":"The 'attention is not explanation' argument the paper acknowledges when treating attention scores as clinically meaningful."}],"review_version":1}