{"id":"1258fd1f-5fb9-4f9f-adce-1f42d83b8b00","arxiv_id":"2506.00730","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Pretraining a transformer on EEG and fine-tuning on PPG and respiration improved sleep stage classification accuracy from 67.6 to 76.6 percent in a held-out test set.","lead":"This study tested whether a neural network pretrained on brain wave (EEG) sleep data can then learn to stage sleep from wrist-worn sensor signals such as pulse and breathing. The authors report that this transfer learning approach raises sleep stage classification accuracy from 67.6 to 76.6 percent compared with training only on those peripheral signals.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 9-point accuracy gain may be an artifact of subject leakage: MESA is hosted on NSRR, and the paper never states that the EEG pretraining corpus excludes the 1,559 MESA subjects used for fine-tuning and held-out testing.","rationale":"I read the paper as attempting to demonstrate that pretraining a transformer on high-fidelity EEG and fine-tuning on peripheral signals improves sleep-stage decoding accuracy. The central empirical evidence is the reported improvement from 67.6% (peripheral-only baseline) to 76.6% (pretrained then fine-tuned). The most load-bearing condition for this claim is that the EEG pretraining corpus and the MESA fine-tuning/evaluation cohort be disjoint; otherwise, the improvement can be explained by subject overlap rather than by transferable sleep representations. The text is explicit that MESA is hosted on NSRR, while pretraining data are described only as assembled from NSRR datasets, with details deferred to a companion paper. No exclusion of MESA is stated. This is a concrete, testable provenance gap, not merely a disagreement with prevailing practice. Secondary weaknesses include the lack of confidence intervals or repeated-seed results, the unexplained jump in N1 accuracy from 40.9% to 89.1%, and the absence of code or pretrained weights, but the overlap issue is the one that could invalidate the central comparison outright. If the proposed cohort-intersection check shows no overlap, the result would be substantially more credible; if overlap exists, the central claim fails. I therefore agree with the reader's REJECT verdict for the manuscript as written.","tokens_in":6217,"tokens_out":3783,"duration_ms":38549,"concrete_test":"From Ogg and Coon 2024 (or the released code, if available), compile the list of NSRR dataset names and subject/session IDs used for EEG pretraining. Intersect this list with the 1,559 MESA subject IDs used for fine-tuning, and especially with the 161 held-out test IDs. If any intersection exists, retrain and evaluate on a strictly disjoint subset and report the resulting accuracy; if the 76.6%-67.6% gap collapses, the transfer-learning claim is refuted by leakage. If the lists are disjoint, the concern is resolved and the headline result becomes credible.","verdict_should_be":"REJECT","load_bearing_attack":"For the central claim to hold, the pretraining and fine-tuning cohorts must be disjoint. Section II-A says pretraining used 10,897 sessions from 9,013 individuals from NSRR datasets, with dataset details deferred to Ogg and Coon [10]. Section II-B says fine-tuning used 1,559 MESA subjects, and describes MESA as hosted by the NSRR [11]. Because MESA is an NSRR cohort containing PSG with EEG, it could plausibly be part of the pretraining corpus. The paper never states an explicit exclusion, and the companion reference does not appear in this manuscript as a dataset listing. If the same subjects' EEG recordings were seen during pretraining, the held-out 161 subjects would not be truly unseen, and the improvement from 67.6% to 76.6% could reflect memorized subject-specific sleep structure rather than transferable sleep representations. This is not a stylistic quibble: it is a concrete data-provenance gap that directly determines whether the strongest empirical claim is valid. The absence of code, pretrained weights, and error bars makes the gap harder to resolve from the paper alone, but the overlap issue is the load-bearing concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates whether transfer learning from a transformer model pretrained on large-scale EEG sleep recordings can improve sleep-stage classification from peripheral wearable signals (PPG and respiration). The authors pretrain a transformer on 10,897 EEG/EOG sessions from 9,013 individuals (datasets not enumerated, deferred to a preprint), then fine-tune on peripheral signals from 1,559 MESA subjects, using a held-out set of 161 subjects for evaluation. They report an improvement in overall classification accuracy from 67.6% (baseline trained from scratch on peripheral signals) to 76.6% with transfer learning, with large gains in REM and N1. The paper includes a detailed architecture description and a clear baseline comparison.","tokens_in":6420,"tokens_out":4965,"duration_ms":47016,"significance":"If the central claim holds, the result would be practically valuable: it suggests that high-fidelity EEG knowledge can be transferred to noisy peripheral signals, potentially improving consumer wearable sleep staging without hardware changes. The study uses a large pretraining corpus and a reasonably detailed architectural description, which are strengths. However, the current manuscript does not provide enough information to validate the claim: the pretraining dataset composition is undisclosed, there is no uncertainty quantification, and the possibility of subject overlap between pretraining and fine-tuning corpora is not addressed. These gaps currently prevent the paper from supporting its headline accuracy improvement.","major_comments":[{"comment":"The pretraining and fine-tuning cohorts may overlap. The pretraining data are described as sourced from NSRR with details deferred to Ogg and Coon [10], while the fine-tuning data come from MESA, which is also hosted by NSRR. The manuscript never states that the 1,559 MESA subjects were excluded from the pretraining set. If MESA EEG recordings were part of pretraining, the held-out 161 subjects would not be truly unseen, and the 9-point accuracy gain could reflect subject-specific memorization rather than transferable sleep representations. The authors must enumerate the pretraining datasets and explicitly confirm the disjointness of subject cohorts; merely citing an external preprint is not sufficient.","section":"Section II-A and II-B"},{"comment":"The results are reported as single accuracies on a held-out set of 161 subjects, with no confidence intervals, standard errors, or significance tests. The overall difference between 76.6% and 67.6% cannot be assessed without uncertainty measures, especially since per-stage results are mixed (N3 accuracy drops from 39.8% to 36.6% with transfer learning). The authors should provide per-subject accuracy distributions, bootstrap or binomial confidence intervals, or a statistical comparison (e.g., paired test across subjects).","section":"Section III"},{"comment":"The description of the validation scheme is ambiguous. The text states that all performance data were derived from the 'unseen' validation data and that model selection was based on the minimum validation loss, but it is not clear which split is used for model selection (the internal 90/10 validation) and which for the reported results (the external 161-subject held-out set). If model selection used any performance information from the external set, the reported accuracies are optimistic. The authors should clearly distinguish the internal validation set used for early stopping/model selection from the external held-out set used for final evaluation, and confirm that the latter was not used for any training decisions.","section":"Section II-B and III"}],"minor_comments":[{"comment":"The word 'Artherosclerosis' is misspelled; it should be 'Atherosclerosis' in the text and in Reference [14].","section":"Section II-B, Ref. [14]"},{"comment":"'pulse pleysmography' should be 'pulse plethysmography'.","section":"Fig. 1 caption"},{"comment":"The text references 'Fig. III' when describing the baseline result; this should be 'Fig. 3' for consistency with the other figure reference in the same paragraph.","section":"Section III"},{"comment":"The phrase 'transfer learning, a technique commonly used in generative artificial intelligence' is imprecise; transfer learning is a general machine learning technique and is not specific to generative AI. The sentence should be reworded.","section":"Abstract"},{"comment":"The pretraining dataset composition is deferred entirely to a bioRxiv preprint [10]. Since the data provenance is critical to the paper's claim, the manuscript itself should list the specific NSRR datasets used and their subject counts.","section":"Section II-A"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the potential subject leakage between pretraining and fine-tuning datasets. The authors should be asked to provide the full dataset list and an explicit statement of non-overlap. The lack of code and pretrained weights further hampers verification, but this is secondary to the data-provenance issue. If the overlap exists, the paper's central claim would be unsupported; a major revision is appropriate rather than outright rejection because the issue is fixable with a clear disclosure and, if needed, a re-analysis on a suitably disjoint pretraining set."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The cross-modal transfer idea (EEG pretraining to PPG/respiration fine-tuning) is a reasonable extension of existing sleep staging work, and the paper is transparent about architecture and training. But the central claim—76.6% vs 67.6%—rests on an assumption the authors never state: that the NSRR pretraining corpus excludes the 1,559 MESA subjects used for fine-tuning. Since MESA is hosted on NSRR and the pretraining dataset details are deferred to an unpublished companion paper, the reader cannot check this. That is a concrete data-provenance gap, not a stylistic quibble. If those subjects' EEGs were in the pretraining set, the held-out 161 subjects are not truly unseen and the 9-point gain could just be memorized subject-specific structure.\n\nThe other soft spots are real but smaller: no confidence intervals or significance tests, no comparison against existing PPG-only sleep staging (despite citing Korkalainen), and the per-stage results are unexplained—N1 leaps from 40.9% to 89.1%, while N3 drops from 39.8% to 36.6%. That pattern is odd enough that the paper should at least discuss it.\n\nWhat the paper does well: the architecture description is detailed, the baseline and transfer conditions are clearly defined, and the writing is straightforward. If the provenance gap were closed, the result would be a modest but useful data point for wearable sleep staging. As is, I would not cite it yet.\n\nMy recommendation: send it to peer review, because the claim is important and the fix is addressable. A good referee should require (a) explicit confirmation that no fine-tuning subject appeared in the pretraining corpus, (b) error bars or bootstrapped CIs on the accuracy estimates, and (c) a comparison with an existing peripheral-only model on the same held-out set. If the authors can do those, this becomes publishable. If not, the central claim falls.","headline":"Cross-modal transfer for sleep staging is a worthwhile idea, but the paper never proves the pretraining and fine-tuning cohorts are disjoint, so the 9-point accuracy gain is currently unsupported.","tokens_in":6942,"tokens_out":3203,"would_cite":false,"duration_ms":29578,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pretraining a transformer on EEG improves sleep-stage decoding from peripheral wearables from 67.6% to 76.6%.","keywords":["transfer learning","sleep stage classification","wearable devices","electroencephalography","photoplethysmography","transformer","fine-tuning","peripheral signals"],"falsifier":"Compute subject-level overlap between the pretraining corpus and the 1,559 MESA subjects; if any MESA identifiers appear in pretraining, retrain both the baseline and transferred models with strict subject exclusion and check whether the 76.6% versus 67.6% gap persists.","tokens_in":6004,"feed_emoji":"💤","tokens_out":6051,"duration_ms":55984,"temperature":0.7,"pith_summary":"This paper claims that a neural network trained first on clinical EEG can transfer its learned picture of sleep structure to the harder task of scoring sleep stages from the pulse and breathing signals available in consumer wearables. On a held-out set of older adults, the transferred model reaches 76.6% overall accuracy versus 67.6% for an identical model trained only on peripheral signals. The gains are largest for the light sleep stages that peripheral devices struggle with most: N1 accuracy rises from 40.9% to 89.1% and REM from 54.1% to 68.8%. If the result holds, better sleep staging from wearables would not require new sensors, only a better-trained algorithm.","feed_headline":"EEG-pretrained transformer lifts wearable sleep staging by 9 points","feed_subtitle":"A model fine-tuned from EEG beats a from-scratch model on pulse and breathing data, with big gains in light sleep stages.","key_machinery":"The machinery is a two-stage transfer-learning setup: a transformer-based sequence classifier with about 3.9 million parameters, built from seven 1D convolutional layers followed by four transformer encoder layers with positional encoding, is first pretrained on 10,897 EEG/EOG sleep recordings from 9,013 individuals and then fine-tuned, with all weights updated, on abdomen respiration and pulse plethysmography signals. The transferable object is the internal representation of sleep architecture learned from EEG, which the fine-tuning stage adapts to noisy peripheral inputs while retaining the learned transition structure of sleep stages.","core_discovery":"The central claim is that pretrained transformer-based models can transfer high-fidelity EEG-derived knowledge to improve sleep stage decoding from peripheral wearables. The paper demonstrates this by comparing a model fine-tuned from EEG pretraining to an identical architecture trained from scratch on the same PPG and respiration inputs; the transferred model reaches 76.6% overall accuracy versus 67.6% baseline on an unseen held-out set of 161 subjects. The improvements are concentrated in REM and N1, with N1 accuracy rising from 40.9% to 89.1% and REM from 54.1% to 68.8%, while N3 accuracy is essentially unchanged.","pith_inferences":["Editorial extension: The reported N1 jump from 40.9% to 89.1% is far larger than typical transfer-learning gains, so an independent replication on non-NSRR wearable data would clarify whether this reflects genuine knowledge transfer or corpus leakage.","Editorial extension: If the result is real, the same two-stage recipe could be applied to wrist accelerometry, skin temperature, or single-lead ECG, and to targets beyond sleep staging such as apnea severity or circadian phase.","Editorial extension: The paper never reports a subject-exclusion check between the pretraining corpus and the MESA fine-tuning data, so the first experiment a reader should run before building on the claim is a strict subject-level overlap analysis."],"forward_implications":["Consumer wearables could report five-stage hypnograms at clinically more useful accuracy without any hardware changes.","The largest benefit applies to light-stage detection, N1 and REM, which are the categories most often collapsed or misread by commercial devices.","Modality transfer from a high-fidelity signal (EEG) to a lower-fidelity one (PPG and respiration) becomes a general recipe for improving other physiological decoding tasks.","Because both training stages are supervised, the paper's own next step is self-supervised pretraining on unlabeled sleep data, which it argues is likely to raise accuracy further.","The approach could make large longitudinal sleep datasets collected by wearables more scientifically usable for research and personalized health monitoring."],"supporting_citations":[{"why":"Supplies the NSRR pretraining corpus of 10,897 EEG sleep sessions.","marker":"[11]"},{"why":"Supplies the MESA peripheral PPG and respiration records used for fine-tuning and held-out validation.","marker":"[14]"},{"why":"Documents the specific NSRR datasets used for pretraining; relevant because exclusion of MESA subjects is untested.","marker":"[10]"},{"why":"Details the transformer-based EEG sleep-stage classifier whose pretrained weights are transferred.","marker":"[6]"},{"why":"Defines the transformer encoder and positional encoding architecture the model builds on.","marker":"[15]"},{"why":"Supplies the MNE-Python preprocessing pipeline that produced the input epochs.","marker":"[12]"}],"fun_headline_variants":["Transfer learning lifts wearable sleep staging accuracy by 9 points","EEG pretraining gives wearable sleep staging a 9-point boost","From EEG to wearables: transfer learning improves sleep staging","Sleep staging on wearables sharpens with transfer learning","Smartwatch sleep staging improves using EEG-pretrained model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes the pretraining corpus and the fine-tuning/test subjects do not overlap, but since both are drawn from the same NSRR repository, which hosts the MESA dataset used for fine-tuning, and the paper never states that MESA subjects were excluded from pretraining, part of the 9-point gain could be explained by the model having already seen these subjects' sleep data.","fun_headline_variants_meta":{"raw":{"variants":["Transfer learning lifts wearable sleep staging accuracy by 9 points","EEG pretraining gives wearable sleep staging a 9-point boost","From EEG to wearables: transfer learning improves sleep staging","Sleep staging on wearables sharpens with transfer learning","Smartwatch sleep staging improves using EEG-pretrained model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00068,"raw_usage":{"total_tokens":3067,"prompt_tokens":900,"completion_tokens":2167,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":2085}},"tokens_in":516,"tokens_out":2167,"duration_ms":16103,"temperature":1.0,"reasoning_tokens":2085,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:58:54.545860+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute subject-level overlap between the pretraining corpus and the 1,559 MESA subjects; if any MESA identifiers appear in pretraining, retrain both the baseline and transferred models with strict subject exclusion and check whether the 76.6% versus 67.6% gap persists.","supporting_citations":[{"cited_title":"Racial/Ethnic Differences in Sleep Disturbances: The Multi-Ethnic Study of Atherosclerosis (MESA),","cited_arxiv_id":null,"evidence_quote":"Supplies the MESA peripheral PPG and respiration records used for fine-tuning and held-out validation."},{"cited_title":"Laying the foundation: Modern transform- ers for gold-standard sleep analysis,","cited_arxiv_id":null,"evidence_quote":"Documents the specific NSRR datasets used for pretraining; relevant because exclusion of MESA subjects is untested."},{"cited_title":"Laying the Foundation: Modern Transformers for Gold-Standard Sleep Analysis and Beyond,","cited_arxiv_id":null,"evidence_quote":"Details the transformer-based EEG sleep-stage classifier whose pretrained weights are transferred."}],"review_version":1}