{"id":"3d210c2f-8ee6-41e8-bb07-f60fbfbce4e3","arxiv_id":"2507.09009","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Self-supervised embeddings of PSG signals, projected onto disease-specific latent directions, add modest predictive value to the Framingham Risk Score for cardiovascular outcomes.","lead":"This study uses a self-supervised deep learning model to turn overnight sleep recordings (brain, heart, and breathing signals) into cardiovascular risk scores. The scores modestly improve risk prediction when added to the standard Framingham score, with external validation in a second sleep cohort.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Domain shift in Eq. 5 undermines external validation: WSC embeddings may not share SHHS's latent distribution, and WSC-only outcomes leave disease-vector provenance unclear; reported AUC gains need a frozen-vector/permutation check.","rationale":"The strongest claim has two parts: projection scores add to FRS, and external validation confirms it. The second part is the load-bearing one because the model was never retrained for WSC; the transfer of Eq. 5 is the only mechanism by which SHHS-learned knowledge reaches WSC. If that direction is corrupted by cohort/protocol shift, the WSC AUCs in Table 4 are not evidence for the framework. I agree with the reader's weakest_assumption; I sharpen it with the provenance gap for WSC-specific outcomes and the absence of statistical intervals. The paper deserves credit for using a held-out SHHS test and an independent external cohort, and for releasing source code; those are real strengths. But the external validation section does not state whether disease vectors for WSC-only outcomes were derived on WSC, and no domain-shift diagnostic appears anywhere in the manuscript. This is not an internal inconsistency, but it is an unverified transfer assumption. The proposed test—frozen SHHS vectors versus WSC-derived vectors, plus permutation nulls and DeLong CIs—would settle it. Since the reader already returned CONDITIONAL, my read does not change that verdict; it reinforces it.","tokens_in":11511,"tokens_out":7247,"duration_ms":90115,"concrete_test":"Run the released checkpoint (sleep-ssl) on the WSC cohort using disease vectors frozen from SHHS for every shared outcome, and separately compute WSC-derived disease vectors from the same labels; for each outcome compare (i) SHHS-frozen vs WSC-derived projection-score AUCs, (ii) DeLong/bootstrap 95% CIs for the FRS-composite minus FRS-only ΔAUC, and (iii) a permutation null in which disease-vector labels are shuffled on SHHS before computing the frozen vector. If SHHS-frozen transfer is not significantly better than the permutation null, or if WSC-derived vectors give materially different AUCs, the external validation claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central external-validation claim (Abstract; §3.3, Table 4) presupposes that the SHHS-trained disease vector v_disease = μ_positive − μ_negative (Eq. 5) remains a meaningful direction in WSC's embedding space. The paper provides no diagnostic for distribution shift across the two PSG protocols: SHHS is in-home at 125/10 Hz, WSC is in-laboratory, and demographics differ (Table 1 vs Table 2). Because μ_positive and μ_negative are unadjusted means, the vector can encode cohort-specific age/sex/BMI confounds rather than disease physiology; when projected onto WSC embeddings, the resulting scores can remain predictive through demographic transfer while having nothing to do with the disease-specific physiology claimed. The concern is sharpened by an unaddressed provenance gap: Table 4 evaluates WSC outcomes (incident CVD, prevalent CVD, CAD) that are not in the SHHS outcome set of Table 3, so it is not specified whether the disease vectors used for those WSC scores came from SHHS at all. If any WSC outcome used WSC-computed disease vectors, the 'external validation' is no longer independent. No confidence intervals are given for the ΔAUCs in Tables 3–4, so the small observed gains (e.g., +0.007 for WSC hypertension, +0.012 for WSC CVD) cannot be distinguished from sampling noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a self-supervised Residual-Transformer model that encodes 30-second EEG, ECG, and respiratory PSG segments into a shared 256-dimensional latent space, trained with masked-segment reconstruction and a total coding rate penalty. For each cardiovascular outcome, a disease vector is computed as the difference between the mean embeddings of training-set cases and controls (Eq. 5), and each subject receives per-modality projection scores obtained by averaging the top three segment-level dot products onto the disease vector. These scores are then entered into logistic regression models, alone or combined with the Framingham Risk Score, and evaluated on a held-out SHHS test set and an external Wisconsin Sleep Cohort (WSC) sample. The central claims are that the projection scores are interpretable, capture complementary risk information, and that adding them to the FRS consistently improves AUC across prevalent and incident CVD outcomes in both internal and external cohorts.","tokens_in":11845,"tokens_out":5035,"duration_ms":61889,"significance":"If the results hold, the framework would be a useful step toward leveraging routine PSG data for cardiovascular risk stratification without requiring manual sleep staging, and the interpretable projection scores could aid clinical adoption. The study uses two community-based cohorts, reports multimodal comparisons, and makes source code available, which are genuine strengths. However, the current support is weakened by the very small number of events in the internal test set, the absence of confidence intervals for the AUC comparisons, the unexplained provenance of the disease vectors in the external cohort, and an unexamined selection statistic (top-three segments). These issues are load-bearing for the abstract's claim of robust external validation and consistent incremental value.","major_comments":[{"comment":"The manuscript does not state whether the disease vectors used to score WSC subjects were derived from SHHS training labels or recomputed within WSC. This matters because the WSC outcomes in Table 4 (incident CVD, prevalent CVD, CAD) do not appear in the SHHS outcome set in Table 3, so it is unclear what outcome labels would have been available to construct corresponding SHHS-trained vectors. If any WSC disease vector was derived from WSC labels, the external validation is no longer independent and the abstract's claim of robust external replication is unsupported. Please specify the exact provenance of each WSC vector; if SHHS-derived, describe the label mapping; if WSC-derived, re-run the external analysis with frozen SHHS vectors or present it as a second internal validation.","section":"§2.2.2 and Table 4"},{"comment":"No confidence intervals or significance tests are reported for any AUC or for the increments in AUC when projection scores are added to the Framingham Risk Score. This is a serious issue because the SHHS test set contains only 13 AF, 13 CHF, and 11 CVD-mortality cases (Table 1), so the rank-based AUCs are highly unstable; a single reordering of a few subjects can move the AUC by several hundredths. The WSC increments are also small (e.g., hypertension +0.007, CVD +0.012, Table 4) and are within plausible sampling noise. Please provide bootstrap or DeLong confidence intervals for the AUCs and, in particular, for the FRS-composite versus FRS-only differences; otherwise the 'consistently improved predictive performance' claim is not statistically grounded.","section":"§3.3, Tables 3 and 4"},{"comment":"The subject-level projection score is defined as the average of the top three highest segment-level projections, but this choice is presented without any ablation or justification. Because this statistic is the sole input to the downstream logistic regression, the results could be highly sensitive to the arbitrary choice k=3. The manuscript should report results for alternative choices (e.g., top-1, top-5, mean over all segments) to show that the conclusions are not an artifact of this selection. Additionally, Eq. (6) uses an undefined quantity V_p and a nonstandard product notation; the authors should define all terms and write the projection as a dot product of normalized vectors.","section":"§2.2.3"},{"comment":"The external validation assumes that the SHHS-trained disease vector v_disease = μ_positive − μ_negative transfers to the WSC embedding space, but the two cohorts differ in acquisition protocol (in-home versus in-laboratory PSG), age composition, and BMI distribution (Tables 1 and 2), and the disease vector is an unadjusted centroid difference that can encode demographic and protocol confounds rather than disease-specific physiology. No diagnostic is provided for distribution shift between the cohorts, so the WSC projection scores could be predictive through demographic transfer even if the disease-related physiological signal does not transfer. Please include embedding-space distribution comparisons (e.g., MMD or per-modal PCA overlays) and/or a permutation test that evaluates WSC predictions using a randomly reoriented or demographic-only disease vector.","section":"§2.1, §2.2.2 and §3.3"},{"comment":"The paper repeatedly describes the framework as not relying on labels (e.g., 'without relying on labels' in §1), but Eq. (5) constructs disease vectors directly from the training-set outcome labels. While the embedding itself is self-supervised, the projection scores that are actually evaluated are supervised quantities. This overstates the label-free nature of the pipeline and should be corrected to 'without manual sleep-stage annotations' or 'self-supervised embeddings with supervised disease-vector construction.'","section":"Abstract, §1, and §2.2.2"}],"minor_comments":[{"comment":"The notation 'Signal * V_disease / V_p' is ambiguous; V_p is not defined and the product symbol conflates element-wise and vector operations. Please rewrite using an explicit dot product and define the normalization steps.","section":"Eq. (6) and surrounding text"},{"comment":"The footnotes for Table 4 are misnumbered: 'FRS Score1' is followed by footnote 1 that says 'Baseline includes age, sex, and BMI,' and 'Baseline2' is followed by a footnote that says 'FRS Score.' The mapping should be corrected.","section":"Table 4 footnotes"},{"comment":"The sentence 'individuals in the negative outcome groups consistently demonstrated elevated projection scores compared to controls' is confusing because the negative outcome group and the controls appear to be the same group; please clarify the comparison.","section":"§3.2, Figure 5 caption"},{"comment":"The significance threshold is stated as p < 0.05 in Methods but Figure 8 is described with p < 0.005; the two thresholds should be reconciled, and the issue of multiple testing across many modality-outcome pairs should be addressed.","section":"§2.2.4 and §3.4"},{"comment":"The manuscript does not specify whether the centroids μ_positive and μ_negative are computed over all segments pooled from the subjects, or over per-subject averaged embeddings; this affects the interpretation of the disease vector and should be stated.","section":"§2.2.2"}],"recommendation":"major_revision","confidential_remarks":"The WSC disease-vector provenance issue is the most consequential point; if the authors cannot clarify that the vectors were frozen from SHHS, the external-validation claim should be substantially downgraded. The paper is otherwise a reasonable fit for the journal, and the source-code availability should be verified during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is a credible first attempt at using self-supervised PSG embeddings for cardiovascular risk profiling, and it has one genuinely good feature: the evaluation is out-of-sample, including an external cohort (WSC) with a different PSG protocol. That is more than many clinical ML papers do. It also ships code and data access, and the interpretability attempt (projection scores, patient report card) is useful.\n\nWhat is new: combining EEG, ECG, and respiratory signals in one SSL model, deriving disease vectors from centroid differences, and testing projection scores against FRS on multiple CVD outcomes. The finding that ECG embeddings carry most of the signal, with EEG adding value for hypertension and mortality, is consistent with the literature and plausible.\n\nNow the soft spots, in rough order of severity. First, the external validation for WSC outcomes has a provenance gap. Table 4 includes CAD and CVD outcomes that are not in the SHHS outcome set. It is not specified whether the disease vectors for those outcomes were computed from SHHS at all. If any WSC outcome used WSC-derived vectors, the external validation is no longer independent. This needs to be clarified before the results can be trusted.\n\nSecond, no confidence intervals are reported for the AUCs or ΔAUCs. With 13 AF, 13 CHF, and 11 CVD mortality events in the SHHS test set, the rank-based AUCs are unstable. The +0.007 and +0.012 gains on WSC hypertension and CVD could easily be noise. A permutation test or bootstrap would settle this.\n\nThird, the disease vector is a difference of unadjusted centroids. It can encode cohort-specific age, sex, and BMI differences rather than disease physiology. The paper claims the vectors transfer across cohorts, but gives no diagnostic for distribution shift. This is a real concern, though the out-of-sample split does rule out the most circular form of leakage.\n\nFourth, the top-three segment selection is an unablated choice. The authors do not test whether averaging the top one, top five, or all segments changes results. Minor.\n\nFinally, the 'self-supervised' framing oversells: the SSL phase is label-free, but the risk score pipeline uses disease labels to build vectors. That is fine, but the abstract should not say 'without relying on labels' without qualification.\n\nOverall, the central idea is sound and the evaluation is mostly honest. The paper deserves a serious referee, but the version I saw needs revision: clarify the WSC vector provenance, add uncertainty intervals, and ablate the top-3 summary. I would bring it to a reading group if you want to discuss clinical ML evaluation design.","headline":"Reasonable first application of SSL PSG embeddings to CVD risk, with a genuinely external cohort, but the external validation has a provenance gap and the claimed gains lack confidence intervals.","tokens_in":12342,"tokens_out":1984,"would_cite":false,"duration_ms":22077,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Projection scores from self-supervised PSG embeddings add predictive value to the Framingham Risk Score for cardiovascular outcomes.","keywords":["self-supervised learning","cardiovascular disease risk","polysomnography","latent representation","disease vector projection","EEG","ECG","respiratory signals"],"falsifier":"Compute disease vectors from the external cohort's own embeddings and compare the resulting projection-score AUCs with those obtained using the training cohort's vectors; if the transfer assumption holds the two sets of AUCs should be close, whereas a large drop or a reversal in score distributions would show the external results depend on domain-specific artifacts. Alternatively, permute outcome labels when estimating the disease vector: if projection scores derived from permuted centroids achieve similar AUCs to the reported ones, the signal is not disease-specific.","tokens_in":11354,"feed_emoji":"🪀","tokens_out":7310,"duration_ms":80844,"temperature":0.7,"pith_summary":"This paper argues that an overnight polysomnography recording contains cardiovascular risk information that standard summary metrics and the Framingham Risk Score leave untapped. The authors train a self-supervised residual-transformer model on raw EEG, ECG, and respiratory signals, then derive a disease vector for each outcome as the difference between average embeddings of affected and unaffected people. Each person's projection score onto that vector measures how closely their sleep physiology aligns with the disease phenotype. Combining these projection scores with the Framingham Risk Score improved discrimination, with AUCs from 0.607 to 0.965 across outcomes, and the improvement persisted in an independent external cohort. If the finding holds, routine sleep studies could double as individualized cardiovascular risk profiles.","feed_headline":"Sleep signals sharpen heart-risk scores beyond Framingham","feed_subtitle":"Overnight EEG, ECG, and respiratory recordings yield scores that improve cardiovascular risk prediction, confirmed in an external cohort.","key_machinery":"The machinery is the disease-vector projection. After self-supervised pretraining that reconstructs randomly masked 30-second PSG segments under a total-coding-rate penalty, the model maps each modality (EEG, ECG, respiratory) to a 256-dimensional embedding. For each outcome, the disease vector is defined as $\\vec{v}_{\\mathrm{disease}} = \\mu_{\\mathrm{positive}} - \\mu_{\\mathrm{negative}}$, the normalized difference between group centroids; a subject's projection score is the normalized dot product of their segment embeddings with this vector, averaged over their top three segments. This turns an unsupervised representation into one interpretable scalar per modality and outcome, which logistic regression can then combine with age, sex, BMI, and the Framingham Risk Score. The same vectors are applied unchanged to the external cohort, which is why the transferability of $\\vec{v}_{\\mathrm{disease}}$ is the load-bearing step.","core_discovery":"The paper's central discovery is that a single direction in a learned embedding space, namely the vector between average representations of people with and without a given outcome, carries enough disease signal to predict both prevalent and incident cardiovascular conditions from raw sleep recordings. ECG-derived projections discriminate right bundle branch block (AUC 0.997) and atrial fibrillation (AUC 0.961) nearly on their own; EEG-derived projections associate with hypertension and CVD mortality; respiratory projections add incremental value when combined with ECG. Adding all modality projection scores to the Framingham Risk Score consistently improved discrimination in the held-out test set (AUCs 0.607–0.965), and the additive benefit persisted in an independent external cohort, where combined models reached AUCs of 0.753 for incident CVD and 0.807 for coronary artery disease.","pith_inferences":["Beyond the paper, the same disease-vector construction could be applied to other outcomes encoded in sleep physiology, such as diabetes or cognitive decline, since the pretraining phase needs no labels.","Beyond the paper, a testable extension is to recompute disease vectors from the external cohort's embeddings and compare per-outcome AUCs with the transferred vectors; close agreement would support the vectors as biomarkers, while divergence would implicate acquisition-protocol artifacts.","Beyond the paper, if the vectors are stable across cohorts, PSG-derived projection scores could be issued on every routine sleep study as a low-cost, repeatable cardiovascular risk screen without changing the clinical protocol."],"forward_implications":["ECG-derived projection scores can act as near-standalone markers for rhythm and conduction abnormalities, with AUCs above 0.96 for atrial fibrillation and right bundle branch block.","EEG-derived scores carry independent risk information for hypertension and cardiovascular mortality, supporting EEG as a digital biomarker for vascular risk.","Respiratory signals, weak alone, add complementary value when combined with ECG, improving discrimination of congestive heart failure and myocardial infarction.","Adding three-modality projection scores to the Framingham Risk Score, rather than replacing it, gives the best or near-best AUC for most outcomes in both internal and external cohorts.","Because the encoder needs no manual sleep staging, the method can produce risk scores from raw PSG signals at an inference cost of under five seconds per patient."],"supporting_citations":[{"why":"Supplies the masked-reconstruction self-supervised training recipe that the backbone extends to multimodal PSG data.","marker":"[12]"},{"why":"Shows that EEG, ECG, and respiratory signals can be embedded jointly; the paper builds its multimodal latent space on this idea.","marker":"[14]"},{"why":"Establishes the precedent that an unsupervised embedding of physiological signals can be turned into scalar CVD risk scores, which the paper adapts from ECG to PSG.","marker":"[18]"},{"why":"Provides the independent external cohort whose different acquisition protocol and demographics test the transferability of the disease vectors.","marker":"[19]"},{"why":"Contributes the total coding rate penalty used to prevent representational collapse during self-supervised training.","marker":"[20]"},{"why":"Defines the Framingham Risk Score, the clinical baseline that the projection scores are combined with and compared against.","marker":"[21]"},{"why":"Defines the Framingham Risk Score, the clinical baseline that the projection scores are combined with and compared against.","marker":"[22]"}],"fun_headline_variants":["Self-supervised sleep AI sharpens heart-risk beyond Framingham","EEG and ECG sleep features predict CVD better than Framingham","Sleep ECG/EEG signals improve Framingham CVD scores","Sleep-derived AI risk scores add to Framingham","Self-supervised sleep signals boost heart-risk prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the disease vector computed from the training cohort's embeddings, the average difference between people with and without each outcome, points in a direction that generalizes to an independent cohort recorded with different equipment and in a different population, without any adaptation.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised sleep AI sharpens heart-risk beyond Framingham","EEG and ECG sleep features predict CVD better than Framingham","Sleep ECG/EEG signals improve Framingham CVD scores","Sleep-derived AI risk scores add to Framingham","Self-supervised sleep signals boost heart-risk prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002303,"raw_usage":{"total_tokens":8878,"prompt_tokens":927,"completion_tokens":7951,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":7872}},"tokens_in":543,"tokens_out":7951,"duration_ms":58586,"temperature":1.0,"reasoning_tokens":7872,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:06:35.832059+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute disease vectors from the external cohort's own embeddings and compare the resulting projection-score AUCs with those obtained using the training cohort's vectors; if the transfer assumption holds the two sets of AUCs should be close, whereas a large drop or a reversal in score distributions would show the external results depend on domain-specific artifacts. Alternatively, permute outcome labels when estimating the disease vector: if projection scores derived from permuted centroids achieve similar AUCs to the reported ones, the signal is not disease-specific.","supporting_citations":[{"cited_title":"MTS -LOF: medical time -series representation learning via occlusion -invariant features","cited_arxiv_id":null,"evidence_quote":"Supplies the masked-reconstruction self-supervised training recipe that the backbone extends to multimodal PSG data."},{"cited_title":"SleepFM: multi -modal representation learning for sleep across brain activity, ECG and respiratory signals","cited_arxiv_id":null,"evidence_quote":"Shows that EEG, ECG, and respiratory signals can be embedded jointly; the paper builds its multimodal latent space on this idea."},{"cited_title":"Unsupervised deep learning of electrocardiograms enables scalable human disease profiling","cited_arxiv_id":null,"evidence_quote":"Establishes the precedent that an unsupervised embedding of physiological signals can be turned into scalar CVD risk scores, which the paper adapts from ECG to PSG."},{"cited_title":"Burden of sleep apnea: rationale, design, and major findings of the Wisconsin Sleep Cohort study","cited_arxiv_id":null,"evidence_quote":"Provides the independent external cohort whose different acquisition protocol and demographics test the transferability of the disease vectors."},{"cited_title":"Segmentation of multivariate mixed data via lossy data coding and compression","cited_arxiv_id":null,"evidence_quote":"Contributes the total coding rate penalty used to prevent representational collapse during self-supervised training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Framingham Risk Score, the clinical baseline that the projection scores are combined with and compared against."},{"cited_title":"General cardiovascular risk profile for use in primary care: The Framingham heart study","cited_arxiv_id":null,"evidence_quote":"Defines the Framingham Risk Score, the clinical baseline that the projection scores are combined with and compared against."}],"review_version":1}