Pith. sign in

REVIEW 5 major objections 5 minor 21 references

New Test-Time Scenario for Biosignal: Concept and Its Approach

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes a test-time scenario in which a biosignal model adapts to a stream of unlabeled samples plus occasional labeled calibration samples, and reports that this improves blood-pressure prediction on PulseDB.

desk verdict A plausible but under-evidenced extension of test-time training to sparse-label biosignal calibration; the claimed advantage over conventional TTA is not isolated. read the letter →

arxiv 2411.17785 v1 pith:ANXUHUAS submitted 2024-11-26 eess.SP cs.LG

Yong-Yeon Jo , Byeong Tak Lee , Beom Joon Kim , Jeong-Ho Hong , Hak Seung Lee , Joon-myoung Kwon This is my paper · ORCID
classification eess.SPcs.LG
keywords test-timeadaptationonlinebiosignalsbloodpressurepredictionself-supervisedlearningdual-queuebufferweightedbatchsamplingelectrocardiogram
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a new test-time scenario for biosignal models: a continuous stream of unlabeled data, like an ECG or PPG monitor, is punctuated by occasional labeled samples, like periodic cuff-based blood-pressure measurements. It argues that a test-time adaptation framework with a dual-queue buffer and weighted batch sampling lets a pretrained model use both streams at once, and that this outperforms a conventional adaptation approach that ignores intermittent labels. On the PulseDB cuff-less blood-pressure dataset, more frequent label injection and larger initial labeled sets lower the mean absolute error and raise the correlation of predicted blood pressure, though with diminishing returns. If correct, deployed biosignal monitors could stay accurate under drift using only sparse calibration labels.

What carries the argument

The load-bearing mechanism is a dual-head test-time training network: a Vision Transformer encoder with a masked-signal reconstruction decoder for self-supervised learning and a regressor for blood-pressure prediction. During testing a dual-queue buffer keeps labeled samples from being rapidly depleted, and weighted batch sampling mixes unlabeled and labeled samples in a fixed ratio so that scarce labels can influence adaptation without being swamped. This buffer-plus-sampling design is what allows the model to update on both data types during deployment.

What would settle it

Run the same model and label-injection schedule in three modes: the proposed SL+SSL dual-queue method, a pure-supervised fine-tuning method with identical label exposure and the same exclusion of updated samples, and a no-label TTA method. If pure fine-tuning matches or beats the proposed method on held-out test samples from the same subjects, then the specific dual-queue, weighted-sampling, reconstruction-loss machinery is not the cause of the improvement.

Watch

Extended reading notes

Core claim

The central claim is that online test-time adaptation for biosignals should exploit both continuous unlabeled samples and rare labeled samples by jointly optimizing a self-supervised reconstruction head and a supervised regression head. The paper's framework stores unlabeled and labeled samples in separate queues sized in a 2:1 ratio, then samples from them in a 3:1 ratio within batches of 32, updating the model ten times per batch during testing. On PulseDB, this protocol lowers systolic and diastolic blood-pressure mean absolute error relative to no adaptation, and it beats the conventional TTA setting that discards injected labels; increasing label-injection frequency and the number of initial calibration labels improves accuracy up to a plateau.

Load-bearing premise

The evaluation assumes that excluding the initial and injected labeled samples from the metrics yields a fair measure of adaptation, even though the model is updated ten times per batch per subject and no pure fine-tuning baseline is included; if those updates merely memorize the target subject's distribution, the reported gains are not a general adaptation advantage.

Editorial extensions

If this is right

  • If the scenario is adopted, biosignal monitors can be calibrated in the field with sparse manual measurements rather than requiring offline retraining on collected data.
  • Within the reported setup, more frequent label injection and more initial calibration labels give the lowest blood-pressure error, so system designers can trade label cost against accuracy.
  • The framework is model-agnostic: any network with a supervised and a self-supervised head can be plugged in, so the gains should transfer to other signal-reconstruction tasks.
  • Because performance plateaus, there is a ceiling to label usefulness; beyond a threshold, collecting more calibration labels buys little accuracy.
  • The dual-queue design directly addresses the scarcity imbalance in healthcare streams, protecting rare labels from being overshadowed by abundant unlabeled data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct ablation against pure supervised fine-tuning on the same injected labels would isolate whether the gain comes from combining SL and SSL in the dual-queue scheme or simply from exposing the model to labeled samples; the paper does not report such a baseline.
  • Because the evaluation excludes samples used for updates, but updates are repeated ten times per batch per subject, a held-out subject-level test would reveal whether the advantage generalizes across patients or only within each patient's distribution.
  • If confirmed on held-out subjects, the same protocol could apply to continuous glucose monitoring or ICU vital-sign prediction, where calibration labels are similarly scarce.
  • The diminishing-returns pattern suggests an adaptive injection policy could stop requesting labels once error stabilizes, cutting annotation cost in real deployments.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper proposes a test-time adaptation scenario for biosignal models in which the model receives a continuous stream of unlabeled samples plus occasional labeled samples during deployment, motivated by periodic blood-pressure calibration. The method extends the Test-Time Training (TTT) framework with a dual-head network (signal reconstruction and blood-pressure regression) and adds a dual-queue buffer for labeled and unlabeled samples, together with weighted batch sampling. Experiments on the PulseDB dataset report SBP/DBP MAE and correlation under different label-injection frequencies and numbers of initial labeled samples. The authors conclude that frequent label injection and more initial labeled samples improve prediction accuracy, and that the proposed framework outperforms conventional test-time adaptation.

Significance. If the comparison were properly controlled, the scenario would be practically relevant to wearable and ICU monitoring, and the dual-queue/weighted-sampling design is a reasonable engineering response to scarce labeled samples. The paper is clearly written, uses a public benchmark (PulseDB), and reports a coherent set of experiments. However, the current evidence does not isolate the proposed mechanisms: there is no pure fine-tuning baseline, no ablation of buffer size or sampling ratio, no comparison with established OTTA methods, and no error bars or statistical tests. The central claim therefore rests on comparisons that conflate label supervision with the proposed design. The absence of code further limits reproducibility. With added controls and ablations the contribution could be a useful empirical study, but as it stands the inference is not yet supported.

major comments (5)
  1. [§4.2, Table 1] The headline claim that the proposed framework 'outperforms the conventional TTA approach' is not substantiated by Table 1 because the N/A row is the same TTT-based model run without label injection, not an independent conventional OTTA baseline. The table therefore demonstrates a sensitivity analysis of label availability within the proposed framework, not an improvement over established test-time adaptation methods such as TENT, CoTTA, or TTT. A comparison with at least one standard OTTA baseline, or an explicit statement that the N/A row is intended to represent the conventional method, is needed before this claim can be evaluated.
  2. [§4.1, §4.2] There is no pure supervised fine-tuning baseline. The improvements in Table 1 could be produced by the injected labels alone, without any contribution from the dual-queue buffer or weighted batch sampling. To attribute the gains to the proposed design, the authors should compare against plain SGD fine-tuning on the same labeled samples with the same SL loss, the same 10 update repetitions, and the same initialization, but without the unlabeled queue, and preferably also against a uniform-sampling SSL+SL control. Without these controls, the central mechanism is not isolated.
  3. [§4.1] The within-subject evaluation can reward memorization rather than transportable adaptation. The model is updated 10 times per batch on segments from a given subject and then evaluated on other segments from the same subject, with only the samples used for updates excluded from the metric. For time-correlated biosignals, this setup does not distinguish adaptation to the subject's overall distribution from memorization of that subject's local signal patterns. A pure fine-tuning baseline and/or a held-out-subject generalization check are needed to establish that the method provides a general adaptation benefit.
  4. [§1, §4.1] The Introduction promises 'ablation studies highlighting the buffer size and sampling strategy's trade-offs,' but no ablation table or sensitivity analysis appears anywhere in the paper. The queue size ratio (2:1), sampling ratio (3:1), total batch size (32), and the number of test-time update repetitions (10) are fixed without justification. These are the defining hyperparameters of the proposed method, so their effect must be reported before the method can be considered validated.
  5. [§5, Table 1] The conclusion that 'increasing label injection frequency and initial labeled samples improves blood pressure prediction' is not fully supported by Table 1. At injection frequency 100 with zero initial labels, SBP MAE is 14.25 and DBP MAE is 8.64, both worse than the corresponding N/A row (13.56 and 8.12), so more frequent labels do not always help. The paper's use of 'generally' in Section 4.2 hedges this, but the conclusion should explicitly acknowledge the non-monotonicity. In addition, Table 1 reports only point estimates with no standard deviations, confidence intervals, or numbers of subjects/test segments, making it impossible to judge whether differences of 0.1-0.3 mmHg are meaningful.
minor comments (5)
  1. [§4.1] There are formatting typos: 'T raining setup' and 'F requency' in Table 1 should be 'Training setup' and 'Frequency'.
  2. [§4.1, Table 1] The color coding of cells (red/green/blue) is described only in the text and is not visible in a monochrome printout; the table caption should include a legend and the row 'Model performance without any adaptation' should be explicitly labeled as a baseline row.
  3. [§4.1] The experimental setup does not state how many subjects were used for testing, how many 10-second segments were processed per subject, or how the source/target split was constructed. Adding these details would improve reproducibility.
  4. [References] The Vollmer et al. reference is formatted oddly ('Physionet, 101(23):215-220, 2022') and appears to have incorrect venue information; please correct it.
  5. [General] The paper does not provide code or a link to an implementation. Since the method is empirical and has several tunable components, releasing code would substantially strengthen the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports an empirical comparison on a public benchmark and contains no derivation that reduces to its inputs.

full rationale

I walked the claimed derivation chain. The paper makes empirical claims: the proposed OTTA framework improves accuracy on PulseDB; increasing label injection frequency and initial labeled samples improves blood pressure prediction; and buffer size and sampling strategy involve trade-offs. None of these claims is derived by construction. Table 1 is an external benchmark evaluation: MAE and correlation are computed against held-out arterial blood pressure values, and the paper states that samples used for adaptation were excluded ('we excluded the initial labeled samples and those used for label injections from performance evaluations'). The adaptation procedure cites the TTT backbone to Gandelsman et al. (2022), an external prior work, and no load-bearing self-citation appears. The concern that no pure fine-tuning baseline isolates the effect of the dual-queue buffer and weighted sampling is a legitimate experimental-validity criticism, but it is not an exhibit of a claim reducing by definition to its inputs: the reported improvements are not equal to any fitted parameter or to the label-injection schedule by construction. Hence no circular step can be quoted.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim depends on treating PulseDB as representative of deployment shifts, on the TTT reconstruction loss as a useful adaptation signal, and on hand-chosen queue and sampling ratios (2:1 and 3:1) plus 10 update repetitions per batch. No new physical or ontological entities are introduced.

free parameters (6)
  • Unlabeled to labeled queue size ratio = 2:1
    Hand-selected in Section 4.1; no sensitivity analysis is presented.
  • Unlabeled to labeled batch sampling ratio = 3:1
    Hand-selected in Section 4.1; no sensitivity analysis is presented.
  • Total batch size = 32
    Set in Section 4.1; no justification is given.
  • Test-time SGD update repetitions = 10
    Set in Section 4.1; no justification is given.
  • Label injection frequencies = N/A, 100, 50, 20, 10
    Controlled experimental schedule; the reported benefit is a function of this choice, and the densest schedule is not clinically realistic.
  • PBT-tuned learning rate and weight decay = not reported
    Tuned per task with Population Based Training, but exact values are not disclosed.
assumptions (4)
  • domain assumption PulseDB's ECG, PPG, and ABP segments are an adequate proxy for real-world biosignal deployment shifts.
    Section 4.1 selects PulseDB as the only evaluation dataset; the real-world claims rest on this assumption.
  • domain assumption The per-subject test-time update protocol reflects how a deployed model would be calibrated.
    Section 4.1 updates the model individually for each subject, which may not match multi-subject deployment patterns.
  • domain assumption Masked reconstruction loss is a useful self-supervised adaptation signal for blood pressure regression.
    Borrowed from the TTT protocol of Gandelsman et al. (2022) without independent justification for blood pressure prediction.
  • domain assumption Labels used for injection are accurate and immediately available.
    No label noise, delay, or measurement error is modeled; real calibration measurements are imperfect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of New Test-Time Scenario for Biosignal: Concept and Its Approach." pith.science (2026). https://pith.science/paper/ANXUHUAS

@misc{pith2026241117785,
  author       = {Pith},
  title        = {Pith review of: New Test-Time Scenario for Biosignal: Concept and Its Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ANXUHUAS}},
  note         = {Machine review of arXiv:2411.17785}
}
read the original abstract

Online Test-Time Adaptation (OTTA) enhances model robustness by updating pre-trained models with unlabeled data during testing. In healthcare, OTTA is vital for real-time tasks like predicting blood pressure from biosignals, which demand continuous adaptation. We introduce a new test-time scenario with streams of unlabeled samples and occasional labeled samples. Our framework combines supervised and self-supervised learning, employing a dual-queue buffer and weighted batch sampling to balance data types. Experiments show improved accuracy and adaptability under real-world conditions.

Figures

Figures reproduced from arXiv: 2411.17785 by the authors.

Figure 1
Figure 1. Example of data flow and approaches for new test-time paradigm. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Proposed OTTA framework. By mixing these samples, the model continuously learns from recent data inputs while maintaining ac￾curacy through the effective use of labeled data. This ensures that the model adapts to new patterns while retaining the valuable information provided by the labeled samples. 4. Experiments 4.1. Experimental Setup Data: We selected the PulseDB dataset specifically designed for benchmarking cuf… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 16 canonical work pages

  1. [1]

    Contrastive test-time adaptation

    Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 295--305, 2022

    work page 2022

  2. [2]

    Improved test-time adaptation for domain generalization

    Liang Chen, Yong Zhang, Yibing Song, Ying Shan, and Lingqiao Liu. Improved test-time adaptation for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24172--24182, 2023

    work page 2023

  3. [3]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  4. [4]

    Continuous glucose monitoring: the achievement of 100 years of innovation in diabetes technology

    Rodolfo J Galindo and Grazia Aleppo. Continuous glucose monitoring: the achievement of 100 years of innovation in diabetes technology. Diabetes research and clinical practice, 170: 0 108502, 2020

    work page 2020

  5. [5]

    Decorate the newcomers: Visual domain prompt for continual test time adaptation

    Yulu Gan, Yan Bai, Yihang Lou, Xianzheng Ma, Renrui Zhang, Nian Shi, and Lin Luo. Decorate the newcomers: Visual domain prompt for continual test time adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7595--7603, 2023

    work page 2023

  6. [6]

    Test-time training with masked autoencoders

    Yossi Gandelsman, Yu Sun, Xinlei Chen, and Alexei Efros. Test-time training with masked autoencoders. Advances in Neural Information Processing Systems, 35: 0 29374--29385, 2022

    work page 2022

  7. [7]

    Mimic-iv-ecg-diagnostic electrocardiogram matched subset

    Brian Gow, Tom Pollard, Larry A Nathanson, Alistair Johnson, Benjamin Moody, Chrystinne Fernandes, Nathaniel Greenbaum, Seth Berkowitz, Dana Moukheiber, Parastou Eslami, et al. Mimic-iv-ecg-diagnostic electrocardiogram matched subset. Type: dataset, 2023

    work page 2023

  8. [8]

    Population based training of neural networks

    Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, et al. Population based training of neural networks. arXiv preprint arXiv:1711.09846, 2017

    arXiv 2017

Show all 21 references
  1. [9]

    Mimic-iii, a freely accessible critical care database

    Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3 0 (1): 0 1--9, 2016

    2016

  2. [10]

    Vitaldb, a high-fidelity multi-parameter vital signs database in surgical patients

    Hyung-Chul Lee, Yoonsang Park, Soo Bin Yoon, Seong Mi Yang, Dongnyeok Park, and Chul-Woo Jung. Vitaldb, a high-fidelity multi-parameter vital signs database in surgical patients. Scientific Data, 9 0 (1): 0 279, 2022

    2022

  3. [11]

    Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34: 0 21808--21820, 2021

    Yuejiang Liu, Parth Kothari, Bastien Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34: 0 21808--21820, 2021

    2021

  4. [12]

    Deep regression tracking with shrinkage loss

    Xiankai Lu, Chao Ma, Bingbing Ni, Xiaokang Yang, Ian Reid, and Ming-Hsuan Yang. Deep regression tracking with shrinkage loss. In Proceedings of the European conference on computer vision (ECCV), pages 353--369, 2018

    2018

  5. [13]

    Efficient test-time model adaptation without forgetting

    Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International Conference on Machine Learning (ICML), pages 8187--8202. PMLR, 2022

    2022

  6. [14]

    Samsung health monitor, 2024

    Samsung Electronics . Samsung health monitor, 2024. URL https://www.samsung.com/ca/apps/samsung-health-monitor/. Accessed: 2024-08-31

    2024

  7. [15]

    Test-time training with self-supervision for generalization under distribution shifts

    Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learnin...

    2020

  8. [16]

    Critical care nursing, diagnosis and management, 7: critical care nursing

    Linda Diann Urden, Kathleen M Stacy, and Mary E Lough. Critical care nursing, diagnosis and management, 7: critical care nursing. Elsevier Health Sciences, 2013

    2013

  9. [17]

    Simultaneous physiological measurements with five devices at different cognitive and physical loads

    M Vollmer, D Bl \"a sing, J Reiser, M Nisser, and A Buder. Simultaneous physiological measurements with five devices at different cognitive and physical loads. Physionet, 101 0 (23): 0 215--220, 2022

    2022

  10. [18]

    Tent: Fully test-time adaptation by entropy minimization

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726, 2020

    2006 arXiv

  11. [19]

    Continual test-time domain adaptation

    Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7201--7211, 2022

    2022

  12. [20]

    Pulsedb: A large, cleaned dataset based on mimic-iii and vitaldb for benchmarking cuff-less blood pressure estimation methods

    Weinan Wang, Pedram Mohseni, Kevin L Kilgore, and Laleh Najafizadeh. Pulsedb: A large, cleaned dataset based on mimic-iii and vitaldb for benchmarking cuff-less blood pressure estimation methods. Frontiers in Digital Health, 4: 0 1090854, 2023

    2023

  13. [21]

    Large batch optimization for deep learning: Training bert in 76 minutes

    Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. Large batch optimization for deep learning: Training bert in 76 minutes. arXiv preprint arXiv:1904.00962, 2019

    1904 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.