REVIEW 4 major objections 6 minor 29 references
FedFitTech: A Baseline in Federated Learning for Fitness Tracking
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A client-side early-stopping rule that halts federated training when a wearable's validation F1-score plateaus cuts communication by 13% in a fitness-tracking baseline, with mean F1 dropping only one point.
desk verdict A genuinely useful open-source baseline for federated fitness tracking, but the headline 13%/1% trade-off rests on a validation split that is never defined. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the client-side early stopping rule, applied to the validation F1-score over a sliding window (patience 5, threshold 0.01): after each communication round a client evaluates its local model and stops participating once F1 has plateaued. This is the Prechelt early-stopping heuristic, here used not to halt a single model's training but to let individual clients exit the federated training loop, so the server aggregates only the remaining clients' updates. The surrounding baseline stack — Flower for orchestration, FedAvg for aggregation, TinyHAR as the on-device model, and the WEAR dataset with 24 clients — is the setting in which the rule produces the reported savings.
What would settle it
Repeat the FedFitTech experiment with an explicitly separate validation split (e.g., a held-out session per client that is not part of the first-20% test data) and check whether clients that stop early would have improved their test F1 had they kept training; if they would have improved, the stopping rule is stopping too soon, and if randomly dropped clients match the reported F1, the F1-stability criterion is not responsible for the savings.
Extended reading notes
Core claim
The paper's central claim is that a client-side early stopping rule based on the stability of the local validation F1-score can cut redundant communication in federated fitness activity recognition without materially degrading the global model. In the FedFitTech case study, each client computes its F1 on its own data after each round and exits the federated loop when the F1 has not changed by more than 0.01 over the last 5 rounds; this is the Prechelt-style stability criterion adapted from validation error to F1. Using the TinyHAR model on the WEAR dataset with 24 clients, FedAvg aggregation, and 100 rounds, 9 of 24 clients stopped early, the earliest at round 40, for an overall 13% reduction in communication rounds. The mean per-client F1 was 0.67 in the case study against 0.68 for the baseline, and 11 clients scored higher than in the baseline, e.g., client 2's push-up F1 rose from 0% to 76% and the first-stopping client 18 improved overall F1 from 71% to 74% while cutting its communication by 47%. The authors take these results to show that early stopping balances generalization and personalization in FL for FitTech.
Load-bearing premise
The rule assumes that the validation F1-score computed during training is a faithful predictor of each client's test performance, but the paper does not define a validation split separate from the test split; if the plateau signal is noisy or does not track test F1, the 13% saving and 1% drop are artifacts of the tuning procedure rather than a general property of the method.
Editorial extensions
If this is right
- If the 13% saving generalizes, FL systems for fitness wearables can let individual clients skip rounds once their local performance plateaus, saving battery and bandwidth for roughly one F1 point.
- FedFitTech gives the FitTech community a common, reproducible baseline for comparing FL algorithms on real inertial-sensor fitness data.
- Because 37.5% of clients left the training loop while the global model continued to improve, a client's participation in FL should be treated as an adaptive decision, not a fixed commitment.
- Early stopping can serve as a lightweight form of personalization: 11 of 24 clients ended with better local F1 than the full baseline, and stopping prevented the global model from overriding their local patterns.
Reading between the lines
- The 13% communication saving is likely sensitive to the patience and threshold hyperparameters (5 and 0.01) and to the 100-round budget; a sweep over these values would show how much of the saving is method rather than tuning.
- The same F1-stability rule could be tested in other FL domains with heterogeneous clients, but the accuracy-versus-communication trade-off would need to be re-measured because this paper demonstrates it on one dataset and one model family.
- A stronger control would randomly drop the same number of client-rounds; if random dropout matches the 0.67 mean F1, the F1-stability signal is not doing the work attributed to it.
- Because the test split is the first 20% of each label's time series, the validation signal guiding early stopping likely comes from the same early session data; whether the rule helps on future, unseen exercise sessions remains an open question.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces FedFitTech, an open-source federated learning baseline for fitness tracking, implemented with the Flower framework and evaluated on the WEAR dataset using the TinyHAR model. The main technical contribution is a case study that applies client-side early stopping based on the stability of a validation F1-score over a sliding window. The authors report that this early stopping reduces total communication rounds by 13% while decreasing the mean client F1-score only from 68% to 67%, which they characterize as a negligible recognition cost. The paper also includes client-level F1 heatmaps and a discussion of communication savings and per-client performance changes. The central claim is that simple client-side early stopping can cut communication in federated fitness tracking with minimal mean F1 loss.
Significance. If the empirical claim is reliable, the paper offers a simple and lightweight communication-reduction mechanism for federated learning in a relatively underexplored domain, and the open-source baseline under Flower is a useful reproducibility contribution for future FitTech research. The use of a real fitness dataset and a small wearable-suitable model strengthens the practical relevance. However, the empirical support is currently limited by an undefined validation split, a single-run evaluation without statistical testing, and early-stopping hyperparameters tuned on the same dataset, so the headline 13%/1% trade-off is not yet established.
major comments (4)
- [Section 3.3] The manuscript defines only a train/test split (first 20% of each label's time series as test, remaining 80% as training) and never defines a validation split, yet the early stopping rule is described as using 'the stability of the validation F1-score over a sliding window.' If this validation F1 is computed on the test split, the early stopping decisions have access to the same labels on which the reported 1% F1 drop is measured, which is a form of test-set leakage. If it is computed on an undocumented portion of the training split, the paper provides no evidence that plateaus in this signal predict held-out test performance. Please define the validation split explicitly, describe its temporal construction, and confirm that no stopping decision used test labels; if the reported numbers were obtained using test-set access, they must be recomputed with a proper validation split.
- [Section 4, Figures 2-4] The evaluation is a single run with no variance or significance testing. The claim that a 1% mean F1 drop (0.68 vs 0.67 over 24 clients) is 'negligible' is not statistically supported; this difference is within the range of typical random fluctuation for such a small client population. Similarly, the 13% communication saving is a point estimate from one run. Please report results over multiple independent runs (or random seeds), provide means and standard deviations, and apply an appropriate paired test (e.g., a paired permutation test over clients) to support the trade-off claim.
- [Section 3.3, Early stopping] The early-stopping hyperparameters (patience=5, threshold=0.01, and the sliding-window size) were selected via 'several experiments' on the same dataset used for the final evaluation. This tuning procedure is not described, and without a separate tuning set or a sensitivity analysis it is unclear whether the reported 13% saving and 1% drop are properties of the method or artifacts of hyperparameters tuned to this specific benchmark. Please report the tuning protocol, the search space, and sensitivity of the communication saving and F1 drop to these hyperparameters.
- [Section 4.1, Fig. 3] The text states that 'the overall global model F1-score continues to increase, despite some clients being dropped,' but the dashed line in Fig. 3 is the mean of local client F1-scores, not the F1 of the global model evaluated on a fixed held-out set. Since clients that stop early are removed from the mean, an increase in this mean can occur merely because low-performing clients drop out. Please evaluate the global model on a common held-out test set (or clearly rephrase the claim to match what Fig. 3 actually shows).
minor comments (6)
- [Section 4.2, Figs. 5-6] Figures 5 and 6 are client-label F1-score heatmaps, but the text refers to them as 'confusion matrices' when discussing label L; please correct the terminology.
- [Section 3.3, Data splitting] The temporal splitting rationale is clear for train/test, but the paper should also state how the proposed validation set is constructed (per client? temporal order? fraction of the 80% training portion?) so that the early stopping procedure is reproducible.
- [Abstract and Section 4.2] The phrase 'negligible recognition cost by 1%' should specify that this is a drop in mean F1 from 0.68 to 0.67, and it would be helpful to include a confidence interval or standard deviation once multiple runs are available.
- [Section 2] The related-work discussion is mostly qualitative; a summary table comparing methods, datasets, and reported metrics would make the positioning of FedFitTech easier to evaluate.
- [Section 5 and repository link] Please include a version or commit hash for the public repository, and state the exact software and hardware environment used, so that the baseline and case study are fully reproducible.
- [Throughout] There are several language issues, e.g., 'researchers of [4]' should be 'researchers in [4],' and 'the result presents that some of the clients even have better' should be rephrased for clarity.
Circularity Check
Reported 1% recognition cost is the early-stopping threshold (0.01), and the stopping hyperparameters are tuned on the same dataset; the headline trade-off is partly determined by the rule itself.
-
self definitional
[Section 3.3 (Early stopping), Section 4.2 / Fig. 4, Abstract]
"we implemented early stopping based on the stability of the validation F1-score over a sliding window. The hyperparameters of this method are set based on our several experiments to determine the optimum values for this paper's case study. As a result of experimentation, the patience value is set to 5, and the threshold is set to 0.01 for the stopping criterion."
The stopping tolerance is 0.01 (1%), and the reported result is "maintaining the overall recognition performance at a negligible recognition cost by 1%". Because the rule stops when the sliding-window validation F1 changes by less than 0.01, the final F1 is constrained to sit within this tolerance of its plateau; the 1% recognition cost is therefore the threshold of the stopping rule itself rather than an independently measured outcome. This is reinforced by Fig. 4, where baseline and case-study mean F1 are 0.68 and 0.67. No validation split is described in Section 3.3's data-splitting paragraph, so if the validation F1 used for stopping comes from the same split as the reported F1, the evaluation is contaminated by construction.
-
fitted input called prediction
[Section 3.3 (Early stopping) and Abstract]
"The hyperparameters of this method are set based on our several experiments to determine the optimum values for this paper's case study. As a result of experimentation, the patience value is set to 5, and the threshold is set to 0.01 for the stopping criterion. ... The results show that this reduces the overall redundant communications by 13%, while maintaining the overall recognition performance at a negligible recognition cost by 1%."
The patience and threshold are tuned by "several experiments" on the same WEAR dataset, and the headline 13%/1% trade-off is then presented as the outcome of the case study. The communication saving is computed from the stopping rounds that those tuned parameters produce, so the abstract's quantitative claim is a fitted operating point, not a parameter-free prediction. Changing the tuned threshold or patience would change the reported saving and F1 cost, so the headline numbers cannot be read as an independent property of the early-stopping method.
full rationale
The paper's main derivation is an empirical comparison, not a formal derivation, but the central quantitative claim is partially circular. The early-stopping rule's threshold is 0.01, and the reported recognition cost is 1%—the same quantity by construction; training stops when validation F1 is stable within that tolerance, so the final F1 is within the tolerance of the plateau. Moreover, the stopping hyperparameters are explicitly tuned on the dataset via "several experiments", and no validation split is defined after the train/test split is described, leaving the source of the "validation F1" ambiguous; if it is the test split, the measured 1% cost is leaked. The 13% communication reduction is a genuine round-count computation, but it is conditional on the tuned stopping rule rather than an independent prediction. The paper is not otherwise circular: the model (TinyHAR), dataset (WEAR), framework (Flower), and early-stopping reference (Prechelt) are external, and the claims about the baseline itself are not defended by self-citation or uniqueness theorems. Score 6 because the recognition-cost headline reduces to the stopping tolerance, while the communication-saving headline retains empirical content.
Assumptions & free parameters
free parameters (3)
- early_stopping_patience =
5
- early_stopping_threshold =
0.01
- window_size =
100 samples (2 seconds)
assumptions (3)
- domain assumption The first 20% of each label in the time series forms a valid test set, and the remaining 80% captures the training distribution.
- ad hoc to paper Local validation F1 stability over a sliding window is a reliable early stopping signal in federated fitness tracking.
- domain assumption Treating re-recorded subjects 1/19 and 15/20 as separate devices yields a realistic 24-client federated learning setting.
Cite this review
Pith. "Pith review of FedFitTech: A Baseline in Federated Learning for Fitness Tracking." pith.science (2026). https://pith.science/paper/HPRMYT3J
@misc{pith2026250616840,
author = {Pith},
title = {Pith review of: FedFitTech: A Baseline in Federated Learning for Fitness Tracking},
year = {2026},
howpublished = {\url{https://pith.science/paper/HPRMYT3J}},
note = {Machine review of arXiv:2506.16840}
}
read the original abstract
The rapid evolution of sensors and resource-efficient machine learning models has spurred the widespread adoption of wearable fitness tracking devices. Equipped with inertial sensors, such devices can continuously capture physical movements for fitness technology (FitTech), enabling applications from sports optimization to preventive healthcare. Traditional Centralized Learning approaches to detect fitness activities struggle with data privacy concerns, regulatory restrictions, and communication inefficiencies. In contrast, Federated Learning (FL) enables a decentralized model training by communicating model updates rather than potentially private wearable sensor data. Applying FL to FitTech presents unique challenges, such as data imbalance, lack of labeled data, heterogeneous user activities, and trade-offs between personalization and generalization. To simplify research on FitTech in FL, we present the FedFitTech baseline, under the Flower framework, which is publicly available and widely used by both industry and academic researchers. Additionally, to illustrate its usage, this paper presents a case study that implements a system based on the FedFitTech baseline, incorporating a client-side early stopping strategy and comparing the results. For instance, this system allows wearable devices to optimize the trade-off between capturing common fitness activities and preserving individuals' nuances, thereby enhancing both the scalability and efficiency of privacy-aware fitness tracking applications. The results show that this reduces the overall redundant communications by 13%, while maintaining the overall recognition performance at a negligible recognition cost by 1%. Thus, the FedFitTech baseline creates a foundation for a wide range of new research and development opportunities in FitTech, and it is available as open source at: https://github.com/shreyaskorde16/FedFitTech
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
A tutorial on human activity recognition using body-worn inertial sensors.ACM Computing Surveys (CSUR), 46(3):1–33, 2014
Andreas Bulling, Ulf Blanke, and Bernt Schiele. A tutorial on human activity recognition using body-worn inertial sensors.ACM Computing Surveys (CSUR), 46(3):1–33, 2014
2014
-
[2]
The eu general data protection regulation (gdpr).A practical guide, 1st ed., 10(3152676), 2017
Paul Voigt and Axel Von dem Bussche. The eu general data protection regulation (gdpr).A practical guide, 1st ed., 10(3152676), 2017
work page 2017
-
[3]
A guide to the california consumer privacy act of 2018.A vailable at SSRN 3275571, 2018
Lydia de la Torre. A guide to the california consumer privacy act of 2018.A vailable at SSRN 3275571, 2018
work page 2018
-
[4]
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pages 1273–1282. PMLR, 2017
2017
-
[5]
Jessica Chia Liu, Jack Goetz, Srijan Sen, and Ambuj Tewari. Learning from others without sacrificing privacy: Simulation comparing centralized and federated machine learning on mobile health data.JMIR mHealth and uHealth, 9(3):e23728, 2021
work page 2021
-
[6]
Sannara Ek, François Portet, Philippe Lalanda, and German Vega. Evaluation and comparison of federated learning algorithms for human activity recognition on smartphones.Pervasive and Mobile Computing, 87:101714, 2022
work page 2022
-
[7]
KS Arikumar, Sahaya Beni Prathiba, Mamoun Alazab, Thippa Reddy Gadekallu, Sharnil Pandya, Javed Masood Khan, and Rajalakshmi Shenbaga Moorthy. Fl- pmi: federated learning-based person movement identification through wearable devices in smart healthcare systems.Sensors, 22(4):1377, 2022
work page 2022
-
[8]
Meta-har: Federated representation learning for human activity recognition
Chenglin Li, Di Niu, Bei Jiang, Xiao Zuo, and Jianming Yang. Meta-har: Federated representation learning for human activity recognition. InProceedings of the web conference 2021, pages 912–922, 2021
work page 2021
Show all 29 references
-
[9]
Protohar: Prototype guided personalized federated learning for human activity recognition.IEEE Journal of Biomedical and Health Informatics, 27(8), 2023
Dongzhou Cheng, Lei Zhang, Can Bu, Xing Wang, Hao Wu, and Aiguo Song. Protohar: Prototype guided personalized federated learning for human activity recognition.IEEE Journal of Biomedical and Health Informatics, 27(8), 2023
2023
-
[10]
Flower: A friendly federated learning research framework.arXiv preprint arXiv:2007.14390, 2020
Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez- Marques, Yan Gao, Lorenzo Sani, Hei Li Kwing, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane. Flower: A friendly federated learning research framework.arXiv preprint arXiv:2007.14390, 2020. F...
2007 arXiv
-
[11]
Jie Feng, Can Rong, Funing Sun, Diansheng Guo, and Yong Li. Pmf: A privacy- preserving human mobility prediction framework via federated learning.Pro- ceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(1):1–21, 2020
2020
-
[12]
A federated learning system with enhanced feature extraction for human activity recognition.Knowledge-Based Systems, 229:107338, 2021
Zhiwen Xiao, Xin Xu, Huanlai Xing, Fuhong Song, Xinhan Wang, and Bowen Zhao. A federated learning system with enhanced feature extraction for human activity recognition.Knowledge-Based Systems, 229:107338, 2021
2021
-
[13]
2d federated learning for personalized human activity recognition in cyber- physical-social systems.IEEE Transactions on Network Science and Engineering, 9 (6):3934–3944, 2022
Xiaokang Zhou, Wei Liang, Jianhua Ma, Zheng Yan, I Kevin, and Kai Wang. 2d federated learning for personalized human activity recognition in cyber- physical-social systems.IEEE Transactions on Network Science and Engineering, 9 (6):3934–3944, 2022
2022
-
[14]
Pengyuan Zhou, Hengwei Xu, Lik Hang Lee, Pei Fang, and Pan Hui. Are you left out? an efficient and fair federated learning for personalized profiles on wearable devices of inferior networking conditions.Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Tec...
2022
-
[15]
A profile similarity-based personalized federated learning method for wearable sensor-based human activity recognition.Information & Management, 61(7):103922, 2024
Yidong Chai, Haoxin Liu, Hongyi Zhu, Yue Pan, Anqi Zhou, Hongyan Liu, Jianwei Liu, and Yang Qian. A profile similarity-based personalized federated learning method for wearable sensor-based human activity recognition.Information & Management, 61(7):103922, 2024
2024
-
[16]
Can Bu, Lei Zhang, Hengtao Cui, Dongzhou Cheng, Hao Wu, and Aiguo Song. Learn from others and be yourself in federated human activity recognition via attention-based pairwise collaborations.IEEE Transactions on Instrumentation and Measurement, 73:1–15, 2024
2024
-
[17]
Clusterfl: a similarity-aware federated learning system for human activity recog- nition
Xiaomin Ouyang, Zhiyuan Xie, Jiayu Zhou, Jianwei Huang, and Guoliang Xing. Clusterfl: a similarity-aware federated learning system for human activity recog- nition. InProceedings of the 19th annual international conference on mobile systems, applications, and services, pages 5...
2021
-
[18]
Fedclar: Federated clustering for personalized sensor-based human activity recognition
Riccardo Presotto, Gabriele Civitarese, and Claudio Bettini. Fedclar: Federated clustering for personalized sensor-based human activity recognition. In2022 IEEE international conference on pervasive computing and communications (PerCom), pages 227–236. IEEE, 2022
2022
-
[19]
Feddl: Federated learning via dynamic layer sharing for human activity recognition
Linlin Tu, Xiaomin Ouyang, Jiayu Zhou, Yuze He, and Guoliang Xing. Feddl: Federated learning via dynamic layer sharing for human activity recognition. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, pages 15–28, 2021
2021
-
[20]
Protecting health monitoring privacy in fitness training: A fed- erated learning framework based on personalized differential privacy.Internet Technology Letters, 7(6):e499, 2024
Lifang Shao. Protecting health monitoring privacy in fitness training: A fed- erated learning framework based on personalized differential privacy.Internet Technology Letters, 7(6):e499, 2024
2024
-
[21]
Flrce: Resource-efficient federated learning with early-stopping strategy.IEEE Trans
Ziru Niu, Hai Dong, A Kai Qin, and Tao Gu. Flrce: Resource-efficient federated learning with early-stopping strategy.IEEE Trans. on Mobile Computing, 2024
2024
-
[22]
Flash: Concept drift adaptation in federated learning
Kunjal Panchal, Sunav Choudhary, Subrata Mitra, Koyel Mukherjee, Somdeb Sarkhel, Saayan Mitra, and Hui Guan. Flash: Concept drift adaptation in federated learning. InInternational Conference on Machine Learning, 2023
2023
-
[23]
Early stopping-but when? InNeural Networks: Tricks of the trade, pages 55–69
Lutz Prechelt. Early stopping-but when? InNeural Networks: Tricks of the trade, pages 55–69. Springer, 2002
2002
-
[24]
Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition.Sensors, 16(1):115, 2016
Francisco Javier Ordóñez and Daniel Roggen. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition.Sensors, 16(1):115, 2016
2016
-
[25]
Ensembles of deep lstm learners for activity recog- nition using wearables.Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 1(2):1–28, 2017
Yu Guan and Thomas Plötz. Ensembles of deep lstm learners for activity recog- nition using wearables.Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 1(2):1–28, 2017
2017
-
[26]
Tinyhar: A lightweight deep learning model designed for human activity recognition
Yexu Zhou, Haibin Zhao, Yiran Huang, Till Riedel, Michael Hefenbrock, and Michael Beigl. Tinyhar: A lightweight deep learning model designed for human activity recognition. InProceedings of the 2022 ACM International Symposium on Wearable Computers, pages 89–93, 2022
2022
-
[27]
Marius Bock, Hilde Kuehne, Kristof Van Laerhoven, and Michael Moeller. Wear: An outdoor sports dataset for wearable and egocentric activity recognition.Pro- ceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(4):1–21, 2024
2024
-
[28]
Beyond confusion: A fine-grained dialectical ex- amination of human activity recognition benchmark datasets.arXiv:2412.09037, 2024
Daniel Geissler, Dominique Nshimyimana, Vitor Fortes Rey, Sungho Suh, Bo Zhou, and Paul Lukowicz. Beyond confusion: A fine-grained dialectical ex- amination of human activity recognition benchmark datasets.arXiv:2412.09037, 2024
2024 arXiv
-
[29]
Demystifying impact of key hyper-parameters in federated learning: A case study on cifar-10 and fashionmnist.IEEE Access, 2024
Majid Kundroo and Taehong Kim. Demystifying impact of key hyper-parameters in federated learning: A case study on cifar-10 and fashionmnist.IEEE Access, 2024
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.