{"id":"d759ec43-803f-48e4-8103-15e98675d579","arxiv_id":"2411.11764","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"FOGSense detects freezing of gait from one vertical accelerometer channel using GAF images and a CNN, reporting F1 0.963 and FPR 3.2% on the tdcsfog dataset.","lead":"A Parkinson's gait monitor called FOGSense uses a Gramian Angular Field transform plus a neural network to detect freezing episodes from a single lower-back accelerometer axis. On a public dataset it reports F1 0.963 and low false alarms, with a federated learning variant that keeps sensor data on the patient's device.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline metrics mix incompatible configurations: AccV F1 is from Table III window-level; the 3.2% FPR is from Table II all-channel episode-level, with the channel selected on the test set.","rationale":"The reader identified DHWT oversampling as the weakest assumption. I agree that the training-only 50% FOG overlap deserves scrutiny, but it does not account for the cross-configuration metric mixing in the paper's own tables. The strongest load-bearing problem is that the central numbers are not produced by a single fixed evaluation pipeline. Algorithm 1 explicitly sorts models and channels by test-set F1, so the AccV result is a post hoc best-of-N choice on the test split. More importantly, the 96.27% F1 and the 3.2% FPR are taken from different runs: Table III's AccV row has no FPR, while Table II's 3.2% FPR is for the all-channel episode-level model, whose F1 is 90.86. Table IV then presents one FOGSense row combining the AccV window-level F1 with the all-channel episode-level FPR. This makes the claimed 22.2% F1 improvement and 74.53% FPR reduction over state-of-the-art an apples-to-oranges comparison. A single-configuration evaluation, with channel selection done on validation and all metrics from one test confusion matrix, would settle whether the AccV-only model actually achieves both high F1 and low FPR. The paper's code is available, so this check is feasible and should be required before the headline is accepted. If the numbers hold under that protocol, the central claim remains credible; if not, the abstract and SOTA comparison overstate the result.","tokens_in":10874,"tokens_out":6008,"duration_ms":56736,"concrete_test":"Using the released GitHub code, rerun a fixed protocol: (1) choose the channel and detection type using only the 10% validation split; (2) with AccV and window-level detection fixed, evaluate the same trained model on the 20% test split and report accuracy, F1, precision, sensitivity, and FPR from a single confusion matrix; (3) repeat for the all-channel episode-level configuration so Table II and Table IV use the same setup. If the AccV model's test FPR is not 3.2%, or if its F1 drops by more than 1 to 2 points when the channel is chosen on validation rather than on the test set, the composite headline is not reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV.A and Algorithm 1 select the input channel by test-set F1: the models are sorted by their test F1 scores, and the best channel (AccV) is then reported as the system. That is test-set model selection, which inflates the reported 96.27% F1 by an unquantified winner's-curse amount. More concretely, the numeric headline does not come from a single configuration. Table III's AccV row (window-level, single-channel) reports ACC 96.3, F1 96.3, PRE 95.8, SEN 96.3 but lists no FPR. The 3.2% FPR quoted in the abstract and Table IV is taken from Table II's episode-level, all-three-channel configuration, which has F1 90.86. Table IV's FOGSense row mixes these two runs: SEN/PRE/F1 from the AccV window-level model and FPR from the all-channel episode-level model. Thus the claimed advantage over state-of-the-art is a composite of the best numbers from different experimental setups, and the claim that using only AccV achieves both 96.27% F1 and 3.2% FPR is unsupported by any single run. The DHWT training-window overlap is a secondary concern; even if it is valid, the metric-mixing issue in the reported headline remains.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"FOGSense is a FOG detection pipeline that converts lower-back accelerometer windows from the tdcsfog dataset into Gramian Angular Field images, classifies them with a multi-branch 2D CNN, and optionally trains under a federated averaging scheme over five simulated clients. The paper reports that the vertical-axis (AccV) single-channel model achieves 96.25% accuracy and 96.27% F1, that a federated variant reaches 90.47% F1, and that FOGSense reduces false-positive rate by 74.53% compared with state-of-the-art methods. A dynamic weight transfer mechanism is proposed to switch to a different channel when one fails, and code is made publicly available.","tokens_in":11213,"tokens_out":6267,"duration_ms":55544,"significance":"The intended contribution addresses a real clinical problem and has a plausible practical impact: single-sensor, low-false-positive FOG detection with a privacy-preserving learning option would be useful for home monitoring. The use of a public dataset, free-living conditions, and the release of code are clear strengths. However, the current evaluation does not support the headline quantitative claims: channel selection is performed on the test set, the abstract and Table IV combine metrics from different detection levels and channel configurations, and the comparison with prior work is confounded by an augmentation applied to FOGSense but not to the baselines. Because these problems are methodological and fixable, the result cannot be accepted as reported.","major_comments":[{"comment":"The best channel (AccV) is selected by evaluating models on the test set: Algorithm 1 trains one model per channel, sorts by test F1, and the paper then reports the top-ranked channel as the system. This is test-set model selection, so the reported 96.27% F1 is an optimistic estimate whose bias is unquantified. The authors should use a held-out validation split (or nested cross-validation) for channel selection and report the performance of the pre-selected channel on the test set.","section":"Section IV.A, Algorithm 1, Table III"},{"comment":"The headline figure mixes incompatible runs: Table III's AccV row is a window-level single-channel model with no FPR reported, while the 3.2% FPR quoted in the abstract and Table IV comes from the episode-level all-channel model in Table II (F1 90.86). Consequently, no single experiment supports the claim that using only AccV attains both 96.27% F1 and 3.2% FPR. The authors should present a single consistent configuration (e.g., the AccV model with its own window- and episode-level FPR) and revise the comparative claims accordingly.","section":"Abstract, Tables II-IV"},{"comment":"The proposed method trains on an augmented distribution in which FOG windows are oversampled with 50% overlap (DHWT), raising the FOG window count by 82%. Table IV compares FOGSense with methods that (with the possible exception of LIFT-PD [29]) are not described as using this augmentation. The reported gains may then be due in part to the augmented training data rather than to GAF or CNN choices. The authors should retrain the comparators under the same DHWT-augmented distribution, or report FOGSense without DHWT, and discuss the augmentation's contribution to the F1 improvement.","section":"Section III.D and Table IV"},{"comment":"Results are averaged over three subject-based splits, but no variance (standard deviation, range, or confidence interval) is reported for any metric in Tables II-IV. Given the small number of patients (62) and the large-scale subject splits, the sensitivity of the headline F1 to the split should be quantified; the authors should report per-split metrics and a measure of dispersion for at least the main configurations.","section":"Section III.C"}],"minor_comments":[{"comment":"The description of the federated clients is too vague: saying clients are assigned 'different combinations of gait data' does not specify whether the partition is by subject, by recording session, or by window, and without this detail the simulated non-IID setting cannot be assessed.","section":"Section III.A"},{"comment":"The normalization description is inconsistent: Eq. (1) maps to [0,1] and Eq. (2) to [-1,1], but θ_i is defined from x'_i, yielding angles in [0,π/2] rather than the usual GAF angle range; the intended normalization should be clarified.","section":"Section III.E, Equations (1)-(2)"},{"comment":"The dual-channel AccML-AccV result is described as 95.30% accuracy and 90.75% F1-score, while Table III reports 95.3 and 90.8; these numbers should be reconciled.","section":"Section IV.A text after Table III"},{"comment":"The FOGSense row reports '–' for DFE but the FPR 3.2% is taken from the episode-level configuration; use consistent column definitions and indicate the detection level for each metric.","section":"Table IV"},{"comment":"The '22.2% improvement in F1-score' is not precisely consistent with the 0.963 versus 0.79 comparison in Table IV (a relative gain of about 21.9%); specify whether the gain is absolute or relative.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the asymmetry between the evaluation of the proposed method and that of the baselines, especially the DHWT augmentation and test-set channel selection. I encourage the editor to ask the authors to provide the exact protocol used for each row of Table IV and the per-split results before a revised version is sent for review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear X,\n\nYou should know two things about this paper before spending time on it. First, the headline number — 96.27% F1, 3.2% FPR, using only the vertical accelerometer axis — does not come from a single run. The F1 is from Table III's window-level single-channel AccV model; the FPR is from Table II's episode-level all-three-channel model. Table IV concatenates the two. There is no run in the paper that achieves both numbers on the same configuration. Second, the channel selection is done on the test set: Algorithm 1 sorts channels by test F1 and picks the best. So the reported AccV result is a post-selected maximum, and the winner's-curse inflation is unquantified.\n\nWhat is genuinely worth credit: the paper attacks a real deployment problem — single low-back sensor, low false positive rate, privacy-preserving updates — and the GAF + CNN pipeline is a sensible combination for time-series imaging. The observation that the vertical axis alone might suffice is interesting if it holds up. The code is public. The DHWT oversampling of FOG windows on the training set is disclosed, and it is a legitimate technique for imbalance, though it deserves scrutiny.\n\nThe soft spots, in order of seriousness. The metric mixing is load-bearing: the central claim in the abstract and conclusion is unsupported by any single experimental configuration. No variance is reported despite three stratified splits; you cannot judge stability. The federated learning part is a simulation with five synthetic clients, not a real deployment. The comparison to prior work is uneven — LIFT-PD's F1 of 0.79 is cited, but the comparison runs may not have used the same preprocessing or DHWT augmentation, so the 22.2% improvement claim is not a controlled comparison.\n\nFor a reader: this is a useful cautionary example of how easy it is to produce a composite headline. I would not cite the numbers as reported, but I would read it to understand the GAF + FL pipeline and the channel-reduction question. The paper deserves peer review, but it needs a major revision before the numbers can be trusted: rerun the evaluation with held-out channel selection, report per-split variance, and report all metrics from the same configuration.\n\nMy recommendation: engage with it as a reviewer, but treat every headline number as unverified until the evaluation protocol is fixed.","headline":"The headline F1 and FPR come from different experimental runs, and the channel is selected on the test set, so the paper's central claim is not supported by any single configuration.","tokens_in":11731,"tokens_out":2392,"would_cite":false,"duration_ms":22711,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A system called FOGSense claims to detect freezing of gait in Parkinson's patients from the vertical axis of a single lower-back accelerometer, reaching a 96.27% F1-score.","keywords":["freezing of gait","Parkinson's disease","Gramian angular field","federated learning","wearable accelerometer","convolutional neural network","false positive rate","single-sensor monitoring"],"falsifier":"Re-run the evaluation on held-out subjects with no overlapping FOG-window augmentation in training, and also compute F1 per FOG episode rather than per window; if the single-axis model's F1 falls far below 0.96 or the per-episode metric is materially worse, the headline claim is inflated.","tokens_in":1598,"feed_emoji":"📱","tokens_out":1764,"duration_ms":59512,"temperature":0.7,"pith_summary":"FOGSense claims that freezing of gait in Parkinson's disease can be detected accurately in free-living conditions from just one axis of a single lower-back accelerometer, using image-style encoding of the signal rather than hand-built features. The paper shows that the vertical acceleration axis alone reaches 96.25% accuracy and a 96.27% F1-score, with a false positive rate of 3.2%, and that a federated-learning version still reaches 90.47% F1 while keeping patient data on-device. If true, this would let FOG monitoring run on a single inexpensive wearable, reduce sensor-failure points, and avoid unnecessary intervention alerts, which matter because too many false alarms can desensitize patients.","feed_headline":"One accelerometer axis detects freezing of gait at 96% F1","feed_subtitle":"A single lower-back sensor plus image-style encoding cuts false alarms by ~74% versus earlier methods.","key_machinery":"Gramian Angular Field (GAF) transformation: a 1D time series is normalized to $[-1,1]$, mapped to polar coordinates (angle $\\theta_i = \\cos^{-1}(x'_i)$, radius as normalized time index), and then the pairwise cosines of summed angles form a 2D matrix $\\text{GASF}(i,j) = \\cos(\\theta_i + \\theta_j)$ that preserves temporal order and correlations. This matrix is fed to a CNN as an image. The paper also uses an oversampling step that creates overlapping FOG windows in training, a weighted-averaging aggregation strategy for federated learning, and dynamic weight transfer, where per-channel models are ranked by test F1 and a failed channel is replaced by the best functional one.","core_discovery":"On the paper's own terms, the central discovery is that a single channel—the vertical (AccV) axis of a waist-worn triaxial accelerometer—carries enough information to detect FOG episodes, and that converting 4-second windows of that signal into Gramian Angular Field images lets a multi-branch CNN outperform all three-channel and two-channel configurations. The reported numbers are 96.25% accuracy, 96.27% F1, 95.8% sensitivity, and a 3.2% false positive rate on a public dataset with 62 Parkinson's patients in uncontrolled settings. A federated variant using weighted averaging of client model updates reaches 86.98% accuracy and 90.47% F1 while preserving data privacy, and a dynamic weight-transfer scheme lets the system fall back to any functional channel if one sensor axis fails.","pith_inferences":["The claim that one axis is enough suggests a testable extension: other wearable sites, such as the wrist or chest, may also have a single dominant axis, and the GAF-plus-CNN recipe could transfer with minimal changes.","The reported numbers are window-level; a per-episode or per-patient aggregation could show different performance, so that is where an independent reader should look first.","Because the oversampling of FOG windows is applied only to training, a benchmark with no overlapping augmentation and strictly held-out subjects would clarify how much of the F1 gain comes from the augmentation rather than from the GAF representation.","The federated result is simulated across synthetic clients; deployment on real phones with real battery and connectivity constraints is an open extension."],"forward_implications":["A waist-worn single-axis accelerometer is sufficient for FOG detection, so multi-sensor anklet or insole systems may be unnecessary for most monitoring.","Low false-positive rates make automated cueing interventions less likely to lose effectiveness from habituation.","Federated training lets the model improve across patients without moving raw sensor data off the phone, supporting long-term monitoring as symptoms evolve.","If any sensor axis fails, the system can switch to another single-channel model, so reliability does not depend on all channels working.","The GAF-to-CNN pipeline needs only about 11 seconds per training epoch for the single-channel model, making it feasible for resource-limited deployment."],"supporting_citations":[{"why":"Supplies the free-living Parkinson's dataset with 62 patients and a lower-back accelerometer on which all results are reported.","marker":"[34]"},{"why":"Provides the windowing and oversampling preprocessing and the prior baseline that FOGSense claims to beat.","marker":"[29]"},{"why":"Establishes the GAF imaging approach for turning time series into images for classification and imputation.","marker":"[15]"},{"why":"Provides the weighted-averaging aggregation strategy used for the federated learning component.","marker":"[33]"},{"why":"Is a single-inertial-sensor multi-head CNN baseline that FOGSense compares against on F1 and false positive rate.","marker":"[40]"},{"why":"Is a one-class classifier baseline used in the state-of-the-art comparison.","marker":"[27]"},{"why":"Motivates the dynamic weight transfer design by showing that multi-sensor failure degrades accuracy by up to 65%.","marker":"[10]"},{"why":"Justifies the 4-second window choice for FOG detection.","marker":"[36]"}],"fun_headline_variants":["Single-axis sensor detects gait freezing at 96% F1","Federated learning cuts FOG false alarms by 74%","GAF images from one accelerometer spot freezing episodes","Privacy-preserving AI spots FOG with 74% fewer false alarms","One waist sensor + GAF catches Parkinson's freezing episodes"],"cache_read_input_tokens":13824,"weakest_assumption_plain":"It assumes that the 50%-overlap oversampling of FOG training windows mimics real test conditions and that three random splits of the 62 patients are representative, so the reported F1 is not inflated by memorized overlapping segments or an unrepresentative split.","fun_headline_variants_meta":{"raw":{"variants":["Single-axis sensor detects gait freezing at 96% F1","Federated learning cuts FOG false alarms by 74%","GAF images from one accelerometer spot freezing episodes","Privacy-preserving AI spots FOG with 74% fewer false alarms","One waist sensor + GAF catches Parkinson's freezing episodes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001146,"raw_usage":{"total_tokens":4801,"prompt_tokens":1043,"completion_tokens":3758,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":3672}},"tokens_in":659,"tokens_out":3758,"duration_ms":27461,"temperature":1.0,"reasoning_tokens":3672,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:09:39.232298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the evaluation on held-out subjects with no overlapping FOG-window augmentation in training, and also compute F1 per FOG episode rather than per window; if the single-axis model's F1 falls far below 0.96 or the per-episode metric is materially worse, the headline claim is inflated.","supporting_citations":[{"cited_title":"Parkinson’s freezing of gait prediction","cited_arxiv_id":null,"evidence_quote":"Supplies the free-living Parkinson's dataset with 62 patients and a lower-back accelerometer on which all results are reported."},{"cited_title":"Real-time detection of freezing of gait in parkinson’s disease using multi-head convolutional neural networks and a single inertial sensor,","cited_arxiv_id":null,"evidence_quote":"Is a single-inertial-sensor multi-head CNN baseline that FOGSense compares against on F1 and false positive rate."},{"cited_title":"Towards real-time prediction of freezing of gait in patients with parkinson’s disease: a novel deep one- class classifier,","cited_arxiv_id":null,"evidence_quote":"Is a one-class classifier baseline used in the state-of-the-art comparison."},{"cited_title":"Designing deep neural networks robust to sensor failure in mobile health environ- ments,","cited_arxiv_id":null,"evidence_quote":"Motivates the dynamic weight transfer design by showing that multi-sensor failure degrades accuracy by up to 65%."},{"cited_title":"Prediction of freezing of gait in parkinson’s disease using statistical inference and lower–limb acceleration data,","cited_arxiv_id":null,"evidence_quote":"Justifies the 4-second window choice for FOG detection."}],"review_version":1}