{"id":"44315ff1-eb8c-4d35-8e04-c020487e067e","arxiv_id":"2411.17785","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A dual-queue, weighted-sampling test-time adaptation framework improves blood pressure prediction from ECG/PPG when occasional labeled samples are available during deployment.","lead":"Occasional labeled samples, such as a manual blood pressure reading, can be mixed with continuous unlabeled biosignals to keep a deployed model accurate. The paper tests this idea on ECG/PPG data and reports lower blood pressure prediction error as labels become more frequent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing pure fine-tuning baseline conflates label supervision with the proposed dual-queue/weighted-sampling design; the claimed advantage over conventional TTA is not isolated.","rationale":"The reader's conditional verdict is appropriate. My stress-test finds the same soft spot: the comparison against conventional TTA does not control for the presence of supervised labels, and there is no pure fine-tuning baseline or per-subject variance reporting. The paper's mechanism is therefore plausible but unproven. I do not find a stronger internal inconsistency; the main fix is additional controls and variance reporting. The table's sparse-label row showing a regression relative to conventional TTA reinforces the point that the claimed benefit is not uniform. Therefore no adjustment to the verdict is needed beyond keeping it conditional.","tokens_in":5635,"tokens_out":6794,"duration_ms":64086,"concrete_test":"Re-run the PulseDB protocol adding three matched controls: (i) plain supervised fine-tuning on the injected labeled samples with the same SGD and 10 updates per batch, no SSL head and no dual-queue; (ii) SSL+SL with uniform sampling and no weighted batch sampling; (iii) the existing conventional SSL-only TTA as the N/A row with zero initial labels. Report per-subject mean and standard deviation plus paired significance tests for SBP/DBP MAE. Also evaluate on a held-out recording session per subject rather than the same session used for adaptation. If control (i) or (ii) matches the proposed results in the 20/10 injection rows, the dual-queue and weighted-sampling mechanism is not supported as the source of the gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 4.2 is that the proposed OTTA framework 'outperforms the conventional TTA approach by effectively utilizing newly injected labeled data.' Table 1 compares label-injection settings against the no-label 'N/A' row, so the comparison bundles two changes: adding supervised updates on target-subject labels, and the dual-queue buffer with weighted sampling. No baseline runs plain supervised fine-tuning on the same injected labels with the same SGD and 10 updates per batch but without the buffer and sampling strategy, nor a uniform-sampling SSL+SL control. The improvement attributed to the framework could therefore be produced by the labels alone, and the proposed mechanisms are never isolated. The promised ablation on buffer size and sampling ratio is absent from the paper. The within-subject evaluation excludes only the labeled samples used for updates, so the model is updated repeatedly on segments from the same subject and then scored on that subject's other segments; for time-correlated biosignals this can reward distributional memorization rather than a transportable adaptation rule. Table 1 also shows the benefit is not uniform: with injection frequency 100 and no initial labels, the proposed setting gives SBP MAE 14.25, worse than the conventional TTA value of 13.56, so 'occasional labels' only help once labels are dense.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a test-time adaptation scenario for biosignal models in which the model receives a continuous stream of unlabeled samples plus occasional labeled samples during deployment, motivated by periodic blood-pressure calibration. The method extends the Test-Time Training (TTT) framework with a dual-head network (signal reconstruction and blood-pressure regression) and adds a dual-queue buffer for labeled and unlabeled samples, together with weighted batch sampling. Experiments on the PulseDB dataset report SBP/DBP MAE and correlation under different label-injection frequencies and numbers of initial labeled samples. The authors conclude that frequent label injection and more initial labeled samples improve prediction accuracy, and that the proposed framework outperforms conventional test-time adaptation.","tokens_in":5867,"tokens_out":5858,"duration_ms":52124,"significance":"If the comparison were properly controlled, the scenario would be practically relevant to wearable and ICU monitoring, and the dual-queue/weighted-sampling design is a reasonable engineering response to scarce labeled samples. The paper is clearly written, uses a public benchmark (PulseDB), and reports a coherent set of experiments. However, the current evidence does not isolate the proposed mechanisms: there is no pure fine-tuning baseline, no ablation of buffer size or sampling ratio, no comparison with established OTTA methods, and no error bars or statistical tests. The central claim therefore rests on comparisons that conflate label supervision with the proposed design. The absence of code further limits reproducibility. With added controls and ablations the contribution could be a useful empirical study, but as it stands the inference is not yet supported.","major_comments":[{"comment":"The headline claim that the proposed framework 'outperforms the conventional TTA approach' is not substantiated by Table 1 because the N/A row is the same TTT-based model run without label injection, not an independent conventional OTTA baseline. The table therefore demonstrates a sensitivity analysis of label availability within the proposed framework, not an improvement over established test-time adaptation methods such as TENT, CoTTA, or TTT. A comparison with at least one standard OTTA baseline, or an explicit statement that the N/A row is intended to represent the conventional method, is needed before this claim can be evaluated.","section":"§4.2, Table 1"},{"comment":"There is no pure supervised fine-tuning baseline. The improvements in Table 1 could be produced by the injected labels alone, without any contribution from the dual-queue buffer or weighted batch sampling. To attribute the gains to the proposed design, the authors should compare against plain SGD fine-tuning on the same labeled samples with the same SL loss, the same 10 update repetitions, and the same initialization, but without the unlabeled queue, and preferably also against a uniform-sampling SSL+SL control. Without these controls, the central mechanism is not isolated.","section":"§4.1, §4.2"},{"comment":"The within-subject evaluation can reward memorization rather than transportable adaptation. The model is updated 10 times per batch on segments from a given subject and then evaluated on other segments from the same subject, with only the samples used for updates excluded from the metric. For time-correlated biosignals, this setup does not distinguish adaptation to the subject's overall distribution from memorization of that subject's local signal patterns. A pure fine-tuning baseline and/or a held-out-subject generalization check are needed to establish that the method provides a general adaptation benefit.","section":"§4.1"},{"comment":"The Introduction promises 'ablation studies highlighting the buffer size and sampling strategy's trade-offs,' but no ablation table or sensitivity analysis appears anywhere in the paper. The queue size ratio (2:1), sampling ratio (3:1), total batch size (32), and the number of test-time update repetitions (10) are fixed without justification. These are the defining hyperparameters of the proposed method, so their effect must be reported before the method can be considered validated.","section":"§1, §4.1"},{"comment":"The conclusion that 'increasing label injection frequency and initial labeled samples improves blood pressure prediction' is not fully supported by Table 1. At injection frequency 100 with zero initial labels, SBP MAE is 14.25 and DBP MAE is 8.64, both worse than the corresponding N/A row (13.56 and 8.12), so more frequent labels do not always help. The paper's use of 'generally' in Section 4.2 hedges this, but the conclusion should explicitly acknowledge the non-monotonicity. In addition, Table 1 reports only point estimates with no standard deviations, confidence intervals, or numbers of subjects/test segments, making it impossible to judge whether differences of 0.1-0.3 mmHg are meaningful.","section":"§5, Table 1"}],"minor_comments":[{"comment":"There are formatting typos: 'T raining setup' and 'F requency' in Table 1 should be 'Training setup' and 'Frequency'.","section":"§4.1"},{"comment":"The color coding of cells (red/green/blue) is described only in the text and is not visible in a monochrome printout; the table caption should include a legend and the row 'Model performance without any adaptation' should be explicitly labeled as a baseline row.","section":"§4.1, Table 1"},{"comment":"The experimental setup does not state how many subjects were used for testing, how many 10-second segments were processed per subject, or how the source/target split was constructed. Adding these details would improve reproducibility.","section":"§4.1"},{"comment":"The Vollmer et al. reference is formatted oddly ('Physionet, 101(23):215-220, 2022') and appears to have incorrect venue information; please correct it.","section":"References"},{"comment":"The paper does not provide code or a link to an implementation. Since the method is empirical and has several tunable components, releasing code would substantially strengthen the contribution.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"This is a short workshop-style empirical paper. The main barriers are the missing pure fine-tuning baseline and the missing ablation of the proposed buffer and sampling strategy; without these, the central claim is not isolated. I would encourage the authors to add these controls, error bars, and a comparison with at least one standard OTTA method. The novelty of 'occasional labels at test time' should also be positioned against the existing continual test-time adaptation literature, which already uses memory and replay mechanisms; the paper currently does not cite such work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible but under-supported extension of test-time training to a healthcare setting with sparse labels. The new piece is the scenario itself—occasional labeled samples arriving during an unlabeled stream—plus a dual-queue buffer with weighted sampling to mix SSL and SL updates. That is a modest composition of known ideas, but for blood-pressure monitoring from ECG/PPG it is a real practical situation, and the paper's framing is clear. Credit where due: the PulseDB evaluation covers several injection frequencies and initial-label counts, and the trend—more labels, better pressure prediction—is sensible. The exclusion of update samples from evaluation is the right instinct.\n\nThe problems are in the isolation of effects. Table 1 compares label-injection settings against the no-label N/A row, which bundles two changes: adding supervised updates on target labels and the proposed queue/sampling machinery. There is no plain fine-tuning baseline on the same injected labels without the buffer, and no uniform-sampling control. So the claim that the framework 'outperforms the conventional TTA approach' isn't actually tested. The promised ablation on buffer size and sampling ratio is missing. On top of that, there are no error bars or significance tests, and the densest schedule (inject every 10th sample) is hardly 'occasional' calibration. The within-subject evaluation updates the model 10 times per batch on that subject's segments, then scores on other segments from the same subject; for time-correlated biosignals this can reward memorizing the subject's distribution rather than a transferable adaptation rule. The table itself shows the benefit is not uniform: injection frequency 100 with no initial labels is worse than the no-label row for SBP, so the headline monotonic improvement only appears once labels are fairly dense.\n\nOverall, the concept is worth airing, but the evidence as reported doesn't yet support the central claim. A serious referee would need the missing baseline, an ablation, and variance estimates to judge it. My recommendation: send it to peer review, but expect heavy revision and likely a request for the fine-tuning control and reproducibility artifacts. I wouldn't cite it in its current form.","headline":"A plausible but under-evidenced extension of test-time training to sparse-label biosignal calibration; the claimed advantage over conventional TTA is not isolated.","tokens_in":6413,"tokens_out":2173,"would_cite":false,"duration_ms":19749,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a test-time scenario in which a biosignal model adapts to a stream of unlabeled samples plus occasional labeled calibration samples, and reports that this improves blood-pressure prediction on PulseDB.","keywords":["test-time adaptation","online test-time adaptation","biosignals","blood pressure prediction","self-supervised learning","dual-queue buffer","weighted batch sampling","electrocardiogram"],"falsifier":"Run the same model and label-injection schedule in three modes: the proposed SL+SSL dual-queue method, a pure-supervised fine-tuning method with identical label exposure and the same exclusion of updated samples, and a no-label TTA method. If pure fine-tuning matches or beats the proposed method on held-out test samples from the same subjects, then the specific dual-queue, weighted-sampling, reconstruction-loss machinery is not the cause of the improvement.","tokens_in":5438,"feed_emoji":"🩺","tokens_out":7190,"duration_ms":61897,"temperature":0.7,"pith_summary":"The paper introduces a new test-time scenario for biosignal models: a continuous stream of unlabeled data, like an ECG or PPG monitor, is punctuated by occasional labeled samples, like periodic cuff-based blood-pressure measurements. It argues that a test-time adaptation framework with a dual-queue buffer and weighted batch sampling lets a pretrained model use both streams at once, and that this outperforms a conventional adaptation approach that ignores intermittent labels. On the PulseDB cuff-less blood-pressure dataset, more frequent label injection and larger initial labeled sets lower the mean absolute error and raise the correlation of predicted blood pressure, though with diminishing returns. If correct, deployed biosignal monitors could stay accurate under drift using only sparse calibration labels.","feed_headline":"Calibration labels during testing cut blood-pressure error","feed_subtitle":"A dual-queue, weighted-sampling adaptation scheme uses scarce labels to keep biosignal models accurate as data drift.","key_machinery":"The load-bearing mechanism is a dual-head test-time training network: a Vision Transformer encoder with a masked-signal reconstruction decoder for self-supervised learning and a regressor for blood-pressure prediction. During testing a dual-queue buffer keeps labeled samples from being rapidly depleted, and weighted batch sampling mixes unlabeled and labeled samples in a fixed ratio so that scarce labels can influence adaptation without being swamped. This buffer-plus-sampling design is what allows the model to update on both data types during deployment.","core_discovery":"The central claim is that online test-time adaptation for biosignals should exploit both continuous unlabeled samples and rare labeled samples by jointly optimizing a self-supervised reconstruction head and a supervised regression head. The paper's framework stores unlabeled and labeled samples in separate queues sized in a 2:1 ratio, then samples from them in a 3:1 ratio within batches of 32, updating the model ten times per batch during testing. On PulseDB, this protocol lowers systolic and diastolic blood-pressure mean absolute error relative to no adaptation, and it beats the conventional TTA setting that discards injected labels; increasing label-injection frequency and the number of initial calibration labels improves accuracy up to a plateau.","pith_inferences":["A direct ablation against pure supervised fine-tuning on the same injected labels would isolate whether the gain comes from combining SL and SSL in the dual-queue scheme or simply from exposing the model to labeled samples; the paper does not report such a baseline.","Because the evaluation excludes samples used for updates, but updates are repeated ten times per batch per subject, a held-out subject-level test would reveal whether the advantage generalizes across patients or only within each patient's distribution.","If confirmed on held-out subjects, the same protocol could apply to continuous glucose monitoring or ICU vital-sign prediction, where calibration labels are similarly scarce.","The diminishing-returns pattern suggests an adaptive injection policy could stop requesting labels once error stabilizes, cutting annotation cost in real deployments."],"forward_implications":["If the scenario is adopted, biosignal monitors can be calibrated in the field with sparse manual measurements rather than requiring offline retraining on collected data.","Within the reported setup, more frequent label injection and more initial calibration labels give the lowest blood-pressure error, so system designers can trade label cost against accuracy.","The framework is model-agnostic: any network with a supervised and a self-supervised head can be plugged in, so the gains should transfer to other signal-reconstruction tasks.","Because performance plateaus, there is a ceiling to label usefulness; beyond a threshold, collecting more calibration labels buys little accuracy.","The dual-queue design directly addresses the scarcity imbalance in healthcare streams, protecting rare labels from being overshadowed by abundant unlabeled data."],"supporting_citations":[{"why":"Provides the PulseDB benchmark of 5.2 million ECG/PPG/ABP segments used for the blood-pressure experiments.","marker":"Wang et al. (2023)"},{"why":"Supplies the test-time training framework, masked-signal reconstruction head, and per-batch update logic that the proposed method adapts.","marker":"Gandelsman et al. (2022)"},{"why":"Introduces the dual-head supervised/self-supervised approach that the paper extends to this new test-time scenario.","marker":"Liu et al. (2021)"},{"why":"Provides the foundational test-time training with self-supervision that motivates the dual-head adaptation strategy.","marker":"Sun et al. (2020)"},{"why":"MIMIC-III is one of the source databases composing PulseDB.","marker":"Johnson et al. (2016)"},{"why":"VitalDB is the other source database composing PulseDB.","marker":"Lee et al. (2022)"},{"why":"Supplies the shrinkage loss used for the supervised blood-pressure regression head.","marker":"Lu et al. (2018)"},{"why":"LAMB optimizer is used in the pretraining protocol that the paper adopts.","marker":"You et al. (2019)"}],"fun_headline_variants":["Rare labels at test time cut blood-pressure error","Dual-queue adaptation sharpens biosignal models","Test-time labels trim BP error with smart sampling","Weighted sampling learns from scarce labels during test"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that excluding the initial and injected labeled samples from the metrics yields a fair measure of adaptation, even though the model is updated ten times per batch per subject and no pure fine-tuning baseline is included; if those updates merely memorize the target subject's distribution, the reported gains are not a general adaptation advantage.","fun_headline_variants_meta":{"raw":{"variants":["Rare labels at test time cut blood-pressure error","Dual-queue adaptation sharpens biosignal models","Test-time labels trim BP error with smart sampling","Weighted sampling learns from scarce labels during test"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000119,"raw_usage":{"total_tokens":987,"prompt_tokens":751,"completion_tokens":236,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":367,"completion_tokens_details":{"reasoning_tokens":175}},"tokens_in":367,"tokens_out":236,"duration_ms":3352,"temperature":1.0,"reasoning_tokens":175,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:03:38.777148+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same model and label-injection schedule in three modes: the proposed SL+SSL dual-queue method, a pure-supervised fine-tuning method with identical label exposure and the same exclusion of updated samples, and a no-label TTA method. If pure fine-tuning matches or beats the proposed method on held-out test samples from the same subjects, then the specific dual-queue, weighted-sampling, reconstruction-loss machinery is not the cause of the improvement.","supporting_citations":[{"cited_title":"Pulsedb: A large, cleaned dataset based on mimic-iii and vitaldb for benchmarking cuff-less blood pressure estimation methods","cited_arxiv_id":null,"evidence_quote":"Provides the PulseDB benchmark of 5.2 million ECG/PPG/ABP segments used for the blood-pressure experiments."},{"cited_title":"Test-time training with masked autoencoders","cited_arxiv_id":null,"evidence_quote":"Supplies the test-time training framework, masked-signal reconstruction head, and per-batch update logic that the proposed method adapts."},{"cited_title":"Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34: 0 21808--21820, 2021","cited_arxiv_id":null,"evidence_quote":"Introduces the dual-head supervised/self-supervised approach that the paper extends to this new test-time scenario."},{"cited_title":"Test-time training with self-supervision for generalization under distribution shifts","cited_arxiv_id":null,"evidence_quote":"Provides the foundational test-time training with self-supervision that motivates the dual-head adaptation strategy."},{"cited_title":"Vitaldb, a high-fidelity multi-parameter vital signs database in surgical patients","cited_arxiv_id":null,"evidence_quote":"VitalDB is the other source database composing PulseDB."}],"review_version":1}