{"id":"f0fe93f6-bbed-4799-8c36-9af16bc402d1","arxiv_id":"2504.21242","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"The Fitbit Body Response Algorithm combines EDA, skin temperature, heart rate, and HRV signals to detect autonomic arousal events, with balanced accuracy around 0.69 in free-living validation.","lead":"This paper describes the Fitbit Body Response Algorithm, a machine-learning system that uses wrist-worn sensor data to detect stress-related autonomic arousal in daily life. It reports validation in a lab stress test and in a week of free-living data, with modest but above-chance performance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Free-living labels are triggered by the same HR/HRV and EDA signals the model uses, so the reported free-living balanced accuracy may reflect label-selection bias rather than valid measurement of perceived stress.","rationale":"The paper's central claim is that the Fitbit Body Response algorithm provides continuous remote measurement of ANS activation, validated by a controlled TSST experiment and a free-living EMA experiment. For that claim to hold, the free-living labels must be a reasonably unbiased sample of moments of perceived stress and non-stress. The design violates this condition: stress-log notifications are triggered by an HR/HRV detector, and retrospective EDA surveys are generated from automatically detected EDA events, both of which are in the signal family used as model inputs. This is not a disagreement with consensus; it is an internal design property stated in the methods. The reported permutation test, while careful, only randomizes the temporal placement of detected stress events; it does not randomize the label-generation process, so it cannot undo the selection bias. The reader's weakest_assumption focused on the TSST manual labels; both concerns are related forms of circularity, and the reader also noted the free-living trigger bias. I rate agreement as 'partial' because I would place the primary weight on the free-living label-selection bias: the TSST arm has an independent conventional labeling scheme (traditional TSST periods) that already produces high AUC, whereas the free-living arm has no independent label source. The decision to use a higher probability threshold of 0.72 and the ±10-minute tolerance further complicate interpretation, but those are secondary. The paper does contain real supporting evidence: the TSST subjective EMAs changed in the expected direction, physiological signal trajectories around manually labeled events show expected HR/EDA/HRV patterns, and the traditional-period model performs well. These prevent me from rejecting outright, but they do not resolve the free-living circularity. The appropriate outcome is the reader's CONDITIONAL verdict: the central claim should not be treated as established until the free-living evaluation is repeated on randomly sampled EMA labels, or another physiology-independent criterion, and until confidence intervals are reported. Since my concern does not change the reader's verdict, I set verdict_should_be to UNCHANGED.","tokens_in":23475,"tokens_out":6083,"duration_ms":68598,"concrete_test":"Restrict the free-living evaluation to stress-log notifications that were delivered by the random 2.75-hour rule (II.B.2), excluding all HR/HRV-triggered notifications and all retrospective EDA-survey labels, and evaluate the three logistic-regression models and the full Body Response algorithm on these randomly-sampled perceived-stress labels only. Recompute balanced accuracy with the production threshold 0.72, first using the same event-overlap rule as Table 6 and then without the ±10-minute adjustment, and report confidence intervals and permutation p-values. If balanced accuracy drops to near chance or the permutation test loses significance, the free-living validation claim is an artifact of label-selection bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is in the free-living validation (Section IV.D, Table 6), because the labels are generated from the same physiological signals that the Body Response algorithm consumes. Per II.B.2, stress-log notifications are sent when a 'previously created HR/HRV-based stress detection algorithm' detects physiological stress; if that detector fires, a notification is sent and the user rates stress. Per II.B.3, retrospective surveys are auto-generated for automatically detected EDA events, and Stress/not-Stress labels are assigned according to the user's attribution of each EDA event. Thus positive stress labels are preferentially located at moments where HR/HRV or EDA are already elevated, exactly the EDA, HR, and HRV feature families used by the three logistic-regression models. A classifier can score well by learning the trigger conditions of the labeling process rather than by measuring ANS activation or perceived stress. The no-stress labels from EDA surveys are themselves EDA-event minutes labeled Heat/Exertion/Humidity/Unknown, so the distinction is between user-attributed causes of EDA events, not between arousal and non-arousal. The permutation test shuffles detected events in time but does not remove this selection bias; it only shows temporal alignment with labels is better than random, which is expected when labels are placed at signal events. The ±10-minute matching window adds further leniency. This makes the headline free-living metrics (accuracy 0.85, balanced accuracy 0.69, sensitivity 0.47) insufficient to support ecological validity as reported. The TSST manual-label circularity (II.B.1) is related but less decisive because the traditional TSST period labels provide a partially independent criterion and yield high ROC AUC (0.87); a reanalysis on period labels can test that arm. The free-living arm has no such independent label source in the current design.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Fitbit Body Response Algorithm, an ensemble of logistic-regression models that classify minutes of autonomic arousal from wrist-worn EDA, skin temperature, HR, and HRV signals. The pipeline includes preprocessing, per-user z-score normalization, tsfresh-derived features with univariate selection, confounder filtering (exercise, water exposure, loose wear), and post-processing that discards short predictions and stitches nearby events. Validation is attempted in two settings: a virtual Trier Social Stress Test (TSST) with n=45, using manually created stress/no-stress labels, and a one-week free-living ecological momentary assessment (EMA) study with n=87, using stress-log notifications and retrospective EDA surveys as labels. The paper reports TSST balanced accuracy of 0.73 for the full logistic-regression model and free-living accuracy of 0.85, balanced accuracy of 0.69, sensitivity of 0.47, and specificity of 0.91 for the combined Body Response Algorithm, with permutation-test p-values below 0.001.","tokens_in":23821,"tokens_out":5203,"duration_ms":57658,"significance":"If the validation were sound, this would be a significant contribution: it describes a deployed consumer-wearable stress-detection algorithm with a transparent signal-processing chain, explicit handling of real-world confounders, a four-classifier comparison, and a relatively large free-living EMA dataset. The authors are candid about label noise, the arbitrary ±10-minute tolerance, and the limitations of virtual TSST delivery. However, both validation arms are compromised by label construction that depends on the same physiological signals the model uses as features. As a result, the reported effect sizes cannot currently be interpreted as evidence for accurate measurement of autonomic arousal or perceived stress; the contribution at this stage is primarily an engineering description rather than a validated measurement method.","major_comments":[{"comment":"The TSST ground-truth labels are circular. Section II.B.1 states that \"minutes which showed changes in the physiological signals commensurate with an autonomic arousal event were labeled stress,\" and these signals are exactly the EDA, HR, HRV, and skin-temperature inputs used by the model (Section III.A). The LOSO metrics in Tables 3–5 therefore measure how well the model reproduces the annotators' visual pattern recognition, not how well it detects an independent criterion of autonomic arousal. The statement in Section IV.C that the manual labels were not cherry-picked does not address this issue: manual annotation still requires an external reference such as traditional TSST period labels, cortisol or subjective ratings, or pre-registered event criteria with inter-rater reliability. Please report the traditional-label model as the primary TSST validation and treat the manually labeled model as descriptive rather than as evidence of criterion validity.","section":"Section II.B.1 and Section IV.C"},{"comment":"The free-living labels are generated from the same physiological signals the model consumes. Stress-log notifications are triggered by a previously created HR/HRV-based stress detection algorithm (Section II.B.2), and retrospective EDA surveys are automatically generated for detected EDA events, with stress/no-stress labels derived from the user's attribution of those events (Section II.B.3). Because EDA, HR, and HRV are model features, positive stress labels are preferentially located at moments when these signals are elevated. A classifier can obtain high performance by learning the trigger conditions of the labeling process rather than by learning to measure autonomic arousal or perceived stress. Moreover, the no-stress labels from the EDA surveys are minutes with EDA events attributed to Heat/Exertion, Humidity, or Unknown, so the contrast is between user-attributed causes of EDA events, not between arousal and non-arousal. Please re-analyze using only the randomly timed notifications that the protocol already includes, or explicitly model the label-generation process.","section":"Section II.B.2, II.B.3, and IV.D"},{"comment":"The event-matching rule and the permutation test do not remove the label-selection bias. Section III.C states that a detected stress event within ±10 minutes of an actual stress event is considered correct, and that if one or more predicted stress events overlap an actual event, the entire actual event is marked correct while the predicted events are discarded. This lenient scoring can inflate accuracy, sensitivity, and specificity. The permutation test shuffles detected stress responses in time within subject-days, but the null distribution inherits the same sampling preference for signal elevations, so a significant p-value only shows that temporal alignment with labels is better than random; it does not demonstrate that predictions capture stress rather than the trigger characteristics of the labeling process. Please report metrics with no tolerance, with strict minute-level matching, and without the event-discard rule, and provide a null model that simulates labels generated by the same triggering process.","section":"Section III.C and IV.D"},{"comment":"Threshold selection and feature selection may leak information into the reported performance. The production probability threshold of 0.72 is described as selected \"through extensive testing of subject preferences,\" but the specificity of 0.90 reported in Table 4 is computed on the TSST training dataset, which suggests the threshold was chosen on the same data used for evaluation. In addition, the univariate feature selection with Benjamini-Hochberg correction (Section III.A) appears to be applied to the entire TSST dataset before the LOSO cross-validation loop; if so, the LOSO performance estimates in Tables 3–5 are optimistic. Please clarify the exact cross-validation nesting and, if necessary, perform feature selection and threshold selection inside each training fold.","section":"Section III.B and IV.C"}],"minor_comments":[{"comment":"The function name \"energey_ratio_by_chunks\" appears to be a typo for \"energy_ratio_by_chunks.\"","section":"Table 2"},{"comment":"The figure shows only mean time series relative to stress onsets; adding confidence intervals or per-subject traces would be needed to support the claim of large inter-individual variability.","section":"Figure 1"},{"comment":"The headline accuracy of 0.85 in the abstract is driven by class imbalance and the lenient matching rule; please report balanced accuracy, sensitivity, and specificity alongside accuracy in the abstract.","section":"Abstract and Table 6"},{"comment":"The number and timing of EMAs and STAI administrations is described inconsistently (\"first and second EMAs\" versus a \"3rd stress EMA\" after recovery); please clarify the exact timeline of survey administration.","section":"Section II.A.2.i"},{"comment":"For reproducibility, please state whether the data and analysis code are available or will be made available, given the study's corporate funding and the description of a proprietary algorithm.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript describes a commercially deployed algorithm, and the validation is substantially weakened by circular label construction. I recommend major revision rather than rejection because the dataset plausibly supports re-analysis with the randomly timed notifications and with the traditional TSST labels, which would provide non-circular evidence. The editor may also wish to consider whether the journal is the right venue for a single-company product evaluation without independent data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper describes a real product-grade stress-sensing pipeline, and the system-level work is worth reading. The free-living validation, however, has a label-selection problem that undercuts the headline numbers, and the TSST arm is partly circular for the same reason.\n\nWhat's genuinely new and good: the Fitbit Body Response algorithm is a complete deployed system, combining EDA, skin temperature, HR, and HRV in an ensemble of three logistic regression models with model selection, post-processing, and filters for exercise, water exposure, and loose wear. That integration is real engineering, and handling those real-world confounders is a contribution. The data collection is also substantial: 45 people in a virtual TSST and 87 people over a week of free living, with detailed model comparisons, LOSO cross-validation, and permutation tests. The limitations section is unusually candid, acknowledging the arbitrary 10-minute tolerance and label quality issues.\n\nThe soft spot is load-bearing. Free-living stress labels come from two sources: stress-log notifications triggered by a previously created HR/HRV-based stress detection algorithm, and retrospective EDA surveys triggered by automatically detected EDA events. So positive stress labels are placed exactly where the model's own input signals are already elevated. A classifier can score well by learning the trigger conditions of the labeling process rather than by measuring perceived stress or autonomic arousal. The no-stress labels from EDA surveys are themselves EDA events labeled Heat/Exertion/Humidity/Unknown, so that contrast is about user attribution of EDA events, not arousal versus non-arousal. The +/-10-minute matching window and the practice of discarding a predicted event when any prediction overlaps a label add further leniency. The permutation test shuffles detected events in time but cannot remove this selection bias; it only shows temporal alignment is better than random, which is expected under this design.\n\nThe TSST arm also has manual stress/no-stress labels created after inspecting the physiological signals, which is circular in the same way. But the authors did run a comparison against traditional TSST period labels and got ROC AUC 0.87, so that arm could be rescued by reanalysis on period labels or cortisol. The free-living arm has no comparable independent label source as currently designed.\n\nI believe the algorithm may work, but this paper's free-living evidence does not demonstrate ecological validity. The fix is not difficult in principle: reanalyze with traditional TSST period labels or cortisol, use randomly sampled EMAs, report confidence intervals, and avoid letting the model's own triggers define the labels.\n\nWho is this for? Practitioners in wearable affective computing will get value from the system details and confounder handling. For evaluators, it is a useful cautionary example of validation leakage. It deserves peer review, not desk rejection, because the engineering is substantial and the central flaw is fixable. I would send it to referees, but I would not let the free-living validity claims stand in their current form.","headline":"A substantial product-grade stress-sensing pipeline, but the free-living validation is undermined by labels generated from the same physiology the model consumes; the headline balanced accuracy of 0.69 should not be taken at face value.","tokens_in":24393,"tokens_out":2727,"would_cite":false,"duration_ms":32471,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Using only wrist-worn sensors, the Fitbit Body Response Algorithm detects acute autonomic arousal and predicts perceived stress in daily life with 85% accuracy, significantly above chance.","keywords":["wearable health monitoring systems","wearable devices","psychological stress","biosensors","autonomic nervous system","electrodermal activity","heart rate variability","ecological momentary assessment"],"falsifier":"A controlled TSST replication in which stress minutes are defined by the protocol's fixed phases or by an independent biomarker such as salivary cortisol, without the annotators seeing the wrist signals, would settle whether the model detects arousal rather than the annotators' visual labels; if balanced accuracy falls to the permutation null, circular labeling is the source of the signal. Likewise, a free-living validation using only randomly timed EMA prompts would test whether the reported 0.69 balanced accuracy survives unbiased sampling.","tokens_in":1718,"feed_emoji":"⌚","tokens_out":4076,"duration_ms":96926,"temperature":0.7,"pith_summary":"The paper seeks to establish that acute autonomic nervous system arousal, the body's stress response, can be measured continuously and passively from wrist-worn consumer sensors in everyday settings, not just in the laboratory. To do this it introduces the Fitbit Body Response Algorithm, which combines heart rate, heart rate variability, electrodermal activity, and skin temperature into ensemble logistic-regression models, and validates the approach in a controlled stress-induction study and a week-long free-living study. In free-living conditions the algorithm predicts perceived stress with accuracy 0.85, balanced accuracy 0.69, sensitivity 0.47, and specificity 0.91, all significantly above chance by permutation test. The authors argue that each sensor is a fallible indicator of the latent stress construct, and that combining them, together with filters for exercise, water exposure, and loose device wear, yields a stable real-world measure that lab-only methods cannot provide.","feed_headline":"Wrist sensors flag stress moments in daily life at 85% accuracy","feed_subtitle":"The algorithm merges heart rate, HRV, skin temperature and sweat signals to flag acute arousal in daily life.","key_machinery":"The central object is the Fitbit Body Response Algorithm, a signal-processing and classification pipeline for wrist-worn devices. It takes 14 minutely input signals, consisting of nine heart-rate-variability metrics, mean heart rate, electrodermal slope and magnitude, and skin-temperature slope and magnitude, z-scores them per user, forms 31-minute sliding windows, and extracts time-series features from each window. Univariate selection keeps the top 20 features per signal group, and three nested logistic-regression models are trained leave-one-subject-out on the Trier Social Stress Test data: electrodermal activity plus temperature, adding heart rate, and adding heart rate variability. At run time the largest model with available sensors is selected, so the ensemble covers 100% of minutes that any model can cover. Confounder filters remove minutes with exercise, water exposure, and loose device wear before classification, and post-processing discards stress predictions shorter than 3 minutes and stitches events closer than 5 minutes. The mechanism's job is to separate true autonomic events from the noise and artifacts that dominate real-world wear.","core_discovery":"The central claim is that autonomic arousal can be indexed in real-world conditions from wrist sensors alone, and that a modular ensemble of physiological signals does this better than any single modality. The paper reports that a logistic-regression model trained on manually labeled acute-arousal minutes from a virtual Trier Social Stress Test learns the dynamics of arousal onset, including rising heart rate and electrodermal activity and falling heart rate variability, rather than mere baseline shifts between stress and recovery periods. In one week of free living, the full Body Response Algorithm detected perceived-stress events with accuracy 0.85, balanced accuracy 0.69, sensitivity 0.47, specificity 0.91, and F1 0.47; every metric exceeded the permutation-test null at p<0.001. Adding cardiovascular signals to electrodermal activity and skin temperature increased sensitivity from 0.25 to 0.47 at the cost of specificity falling from 0.97 to 0.91, which the authors present as consistent with their latent-variable premise that sympathetic and parasympathetic indicators are weak signals that become more stable when combined.","pith_inferences":["Because the TSST labels were created by visually inspecting the same physiological traces used as model inputs, the criterion-validation numbers may partly reflect the authors' labeling conventions; a replication with labels fixed by protocol phase or by an independent marker such as cortisol would test whether the algorithm detects arousal rather than the annotators' judgments.","The free-living labels were not sampled uniformly: notifications were triggered by an HR/HRV-based detector and by nightly EDA-event surveys, so the validation set oversamples times the sensors already thought were stressful; a study with purely random EMA prompts would give a cleaner estimate of real-world sensitivity and specificity.","The same availability-aware nested-model design could generalize to other mobile-health sensing problems in which sensors intermittently fail or are intentionally turned off, such as seizure detection or sleep staging.","Because physiological arousal also accompanies positive excitement, a natural extension is to test whether the algorithm can distinguish negative stress from positive arousal or to add subjective valence labels; the paper's discussion of positive appraisals leaves this as an open problem."],"forward_implications":["A consumer smartwatch can passively flag likely stress episodes throughout the day without requiring the user to answer questions or interact with the device.","Combining electrodermal activity and skin temperature with cardiovascular signals raises sensitivity for detecting arousal, so multi-sensor ensembles should be preferred over single-signal stress detectors.","Algorithmic availability can be kept near 100% by running the largest model whose inputs are currently valid, which matters because noisy or missing heart rate variability is common in real life, occurring in 38.9% of minutes in this dataset.","Real-world confounders such as exercise, water exposure, and loose device wear can be handled with explicit filters before classification, which is required for acceptable specificity in free-living use.","Detected stress events can trigger just-in-time interventions such as mood logging, guided breathing, or mindfulness prompts."],"supporting_citations":[{"why":"Establishes the TSST as the gold-standard stress induction whose physiological markers the model is trained to detect.","marker":"[6]"},{"why":"Documents that perceived and physiological stress responses are only modestly correlated, motivating the separate free-living validation against perceived stress.","marker":"[8]"},{"why":"Supplies the wrist-worn electrodermal sensing approach that provides the EDA signal.","marker":"[11]"},{"why":"Supports the validity of the remote video-conferencing TSST adaptation used here.","marker":"[15]"},{"why":"Validates the heart-rate extraction from photoplethysmography used as a model input.","marker":"[16]"},{"why":"Defines the nine standard heart-rate-variability metrics used as input signals.","marker":"[17]"},{"why":"Provides the median-filter outlier removal used to clean R-R intervals before HRV computation.","marker":"[18]"},{"why":"Describes the EDA smoothing and processing pipeline for the minutely EDA features.","marker":"[19]"},{"why":"Shows that ensembles of physiological signals reflect acute mental stress, the premise for combining modalities.","marker":"[20]"},{"why":"Explains electrodermal lability, accounting for why EDA-only models are less sensitive in some people.","marker":"[21]"}],"fun_headline_variants":["Wrist sensors flag stress in daily life at 85% accuracy","Real-world stress detection: wrist sensors hit 85% accuracy","Wearable algorithm reads arousal outside the lab at 85% accuracy","Combining wrist signals boosts stress detection to 85% accuracy"],"cache_read_input_tokens":26496,"weakest_assumption_plain":"The load-bearing premise is that labels created by visually inspecting the same physiological signals the model sees, and by self-reports triggered by those signals, actually mark true autonomic arousal rather than encoding the algorithm's own assumptions.","fun_headline_variants_meta":{"raw":{"variants":["Wrist sensors flag stress in daily life at 85% accuracy","Real-world stress detection: wrist sensors hit 85% accuracy","Wearable algorithm reads arousal outside the lab at 85% accuracy","Combining wrist signals boosts stress detection to 85% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000269,"raw_usage":{"total_tokens":1614,"prompt_tokens":929,"completion_tokens":685,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":611}},"tokens_in":545,"tokens_out":685,"duration_ms":7180,"temperature":1.0,"reasoning_tokens":611,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:08:55.502680+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled TSST replication in which stress minutes are defined by the protocol's fixed phases or by an independent biomarker such as salivary cortisol, without the annotators seeing the wrist signals, would settle whether the model detects arousal rather than the annotators' visual labels; if balanced accuracy falls to the permutation null, circular labeling is the source of the signal. Likewise, a free-living validation using only randomly timed EMA prompts would test whether the reported 0.69 balanced accuracy survives unbiased sampling.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents that perceived and physiological stress responses are only modestly correlated, motivating the separate free-living validation against perceived stress."}],"review_version":1}