{"id":"13caec0f-4b79-4768-87a6-348d06de4640","arxiv_id":"2505.24415","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A musculoskeletal-simulation-based IMU augmentation pipeline with inverse-kinematics validation and threshold-optimized automatic labeling improves classification on one balanced dataset but yields mixed gains on imbalanced, ambiguously labeled datasets.","lead":"Researchers propose a data augmentation method that uses musculoskeletal simulations to generate synthetic IMU movement data for physiotherapy exercise evaluation. The method shows accuracy gains on a clearly labeled movement dataset, but mixed results on datasets with subjective or ambiguous labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Augmented-example labels are never validated independently; the automatic-label thresholds are fit to real data and then used to filter augmented data, making the downstream gains potentially dependent on an unverified label-transfer assumption.","rationale":"I agree with the reader's identification of the weakest assumption. The central claim depends on automatic labels being correct for generated data. Without independent validation of augmented labels, the contribution cannot be distinguished from a self-consistent labeling loop in which augmented examples are filtered to agree with the same fitted ruleset that defines their labels. The paper does provide useful evidence: FDE TATR is 0.933 versus 0.952 TRTR, the t-SNE plots show overlap, LOSO results improve on many subjects, and fine-tuning on FDE is near-perfect. However, these are strongest on the objective-label FDE dataset. The DS and HS TATR drops and the poor TRTA results already indicate either feature or label mismatch, and the automatic-label GMF1 values on real data are modest for DS (0.839) and HS (0.754). A positive expert-labeling result on augmented data would substantially strengthen the method; a negative result would mean the reported gains on DS and HS are not attributable to reliable automatic labeling. This is a scientific concern about the argument's weakest link, not an accusation, and it does not change the reader's CONDITIONAL verdict: the useful FDE evidence and honest limitation discussion justify conditional acceptance pending verification of the label-transfer assumption.","tokens_in":17180,"tokens_out":4462,"duration_ms":61440,"concrete_test":"Take a random stratified sample of, say, 200 generated augmented repetitions per dataset (drawn before or after the current filter; ideally both) and reconstruct them as skeletal animations in OpenSim. Have two or three physiotherapists, blinded to the automatic labels and to the sampled distribution class, independently assign FMS/FDE labels using the original criteria. Compute GMF1 (or Cohen's kappa) between the automatic labels and the expert labels on the augmented sample, and compare with the real-data GMF1 values in Table 1. If augmented-sample agreement is close to the real-data values, the label-transfer assumption is supported; if it is materially lower (e.g., a drop of more than 0.1, especially for DS and HS), the augmented labels are unreliable and the downstream classification and fine-tuning results need to be reinterpreted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing step is the automatic labeling of augmented examples. The thresholds are obtained by random search over 10 million combinations to maximize GMF1 against physiotherapist labels on real data (Methods, 'Automatic labeling'; Table 1: 0.958 FDE, 0.839 DS, 0.754 HS). These same thresholds are then applied to simulated, trajectory-modified examples, and the pipeline retains only augmented examples whose automatic label matches the class of the distribution from which the augmentation parameters were sampled ('Generation of augmented examples'). This makes the label of every augmented example, by construction, a product of the same fitted ruleset. There is no independent check that the ruleset transfers from real to simulated kinematics: the optimization target is real-data agreement, not agreement on augmented movements, and no held-out set is used for threshold selection. If the thresholds mislabel augmented movements in a systematic way, the classifier trained on augmented data learns the automatic ruleset rather than the physiotherapist construct, and the reported TATR/LOSO/fine-tuning gains rest on incorrect training targets. The paper's own results are consistent with this risk: TATR drops from 0.92 to 0.81 (DS) and 0.79 to 0.51 (HS), and the discussion notes that 'labeling of the augmented examples is significantly clearer than that of the real data,' which is an assumption rather than a measurement. The FDE results are less affected because labels are objective and GMF1 is high, but the general claim of 'automatic, reliable labeling' across exercises depends on a transfer that is never verified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a data augmentation method for IMU-based physiotherapy exercise classification. Starting from real IMU orientation trajectories, the method applies per-segment Euler-angle offsets and range scalings whose parameters are sampled from class- and segment-specific multivariate normal distributions, then projects the modified trajectories through OpenSim inverse kinematics to enforce biomechanical plausibility. Labels for the augmented examples are assigned by an automatic ruleset whose decision thresholds are optimized by random search over 10 million combinations to maximize the geometric mean of class-wise F1-scores against physiotherapist labels on the real data. The authors evaluate the augmented data with t-SNE visualizations, cross-dataset transfer experiments (TRTR, TATR, TRTA), leave-one-subject-out cross-validation, and patient-specific fine-tuning, using three datasets (Foot Drop exercise, Deep Squat, Hurdle Step) with increasing class imbalance and label ambiguity. The paper reports improvements in LOSOCV and fine-tuning scenarios when augmented data are added, alongside notable degradations in the transfer-only scenarios.","tokens_in":17445,"tokens_out":2237,"duration_ms":30097,"significance":"If the label-transfer premise holds, the method is a practically valuable contribution: it produces biomechanically plausible IMU trajectories without requiring MoCap or video data, it does not need a GPU, and it offers a route to enlarge minority classes in physiotherapy exercise datasets. The paper is unusually honest in reporting the settings where augmentation hurts (TATR drops for DS and HS, TRTA failures, subject 105 degradation), and the experimental design is clearly described with a sensible separation of train/test folds. The main weakness is that the automatic labeling of augmented examples rests on thresholds fitted to real data with no independent check on simulated kinematics; this is the paper's load-bearing assumption and it is not validated.","major_comments":[{"comment":"The automatic labeling thresholds are selected by random search over 10 million combinations to maximize GMF1 against physiotherapist labels on the real dataset (Table 1), and the same thresholds are subsequently applied to the augmented examples. The pipeline retains only augmented examples whose auto-label matches the class distribution from which the augmentation parameters were sampled. This makes the augmented labels a product of the same fitted ruleset by construction, and the optimization target is real-data agreement only; there is no held-out validation of threshold transfer to simulated kinematics. The paper's own discussion states that 'labeling of the augmented examples is significantly clearer than that of the real data,' which is an assumption, not a measurement. Because the downstream classification and fine-tuning gains rest on the correctness of these labels, this is a load-bearing gap. I recommend validating the auto-label on a subset of augmented examples with expert physiotherapist labels, or fitting thresholds on a leave-one-subject-out basis and reporting the resulting variability.","section":"Methods, Automatic labeling; Methods, Generation of augmented examples"},{"comment":"The abstract claims that augmented variants 'contribute to gains in classification accuracy, generalization to unseen subjects, and patient-specific fine-tuning from few examples,' but the controlled transfer experiments in Table 2 show substantial degradation. In TATR, the macro F1 drops from 0.92 to 0.81 for DS and from 0.79 to 0.51 for HS, and in TRTA real-trained models fail on augmented data, with HS class 1 at F1 = 0.00. These results do not support the strong claim that augmented data 'closely resemble real-world data' in a distributional sense. The paper hypothesizes that the augmented data have larger variance than the real data, but this is not quantified. Please either temper the abstract's claim or provide an analysis that reconciles the strong transfer failures with the LOSOCV gains, for example by reporting per-class feature-space distances or a calibrated metric of distribution shift.","section":"Results, Table 2 and discussion"},{"comment":"The LOSOCV results are the main positive evidence for the method, but they are presented without subject-level error bars or a statistical significance test across folds, and the reported gains are not uniform: subject 105 in the HS dataset shows a pronounced degradation, and several subjects show little or negative change. Given that the paper frames the method as improving generalization to unseen subjects, the single-subject degradation and the lack of confidence intervals should be addressed, e.g., by reporting per-subject confidence intervals, a paired test across subjects, or an analysis of why subject 105 is an outlier in terms of class distribution or ambiguity.","section":"Results, leave-one-subject-out cross-validation (Figure 6); Discussion"}],"minor_comments":[{"comment":"Typo: 'augmented variants closely resembles real-world data' should be 'closely resemble.'","section":"Abstract"},{"comment":"Typo: 'ouput layer' should be 'output layer.'","section":"Methods, Neural network architecture and training details"},{"comment":"The learning rate schedule is cited as '[QUELLE]', which is a placeholder and must be replaced with an actual reference.","section":"Methods, Patient specific finetuning"},{"comment":"The caption contains garbled text ('FMS 1 / FDE CE FMS 2 / FDE FD FMS 3 / FDE WSSL') that appears to be a legend leftover; please clean it up.","section":"Results, Figure 1 caption"},{"comment":"The sentence 'A plausible explanation for the observed discrepancies is possibly the different frequencies' is redundant; consider tightening.","section":"Discussion"},{"comment":"The availability statement says data are available 'on reasonable request' without mentioning source code or the augmentation pipeline; given the methodological focus, providing code or a detailed configuration would strengthen reproducibility.","section":"Data available"}],"recommendation":"major_revision","confidential_remarks":"The paper's central idea is promising and the evaluation is more thorough than typical for this area, but the automatic-labeling circularity is the main technical risk. The authors could address it with a relatively small expert-validation experiment; without that, the reported gains might reflect learning the auto-labeling ruleset rather than the physiotherapy construct. The citation for the Orthokidataset (reference 27) is incomplete ('tbd'), which should be fixed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the important thing: this paper has a real idea and an honest set of experiments, but its main general claim is supported by only one of the three datasets. The FDE results are credible and useful; the DS and HS results are partial and undermined by an unvalidated label-transfer assumption.\n\nWhat's new: the pipeline combines IMU-only affine modification of Euler angle trajectories with OpenSim inverse kinematics and a threshold-based automatic labeler. That combination is absent from the prior work cited, and it attacks a genuine bottleneck in physiotherapy exercise evaluation. The evaluation design is thoughtful—TRTR/TATR/TRTA to characterize distribution shift, LOSOCV, and patient-specific fine-tuning—and the paper is candid about subject-dependent effects and the poor HS class 1 performance. Credit where due: the discussion explicitly flags ambiguity as a core problem and admits that augmented labels are 'significantly clearer' than real data, which is more honesty than many papers give you.\n\nThe soft spot is load-bearing. The automatic labeling thresholds are selected by random search over 10 million combinations to maximize GMF1 on real data with physiotherapist labels. Those same thresholds are then applied to simulated, modified trajectories, and augmented examples are kept only if the automatic label matches the sampled class. There is no external validation that the fitted ruleset transfers from real to simulated kinematics. If the thresholds mislabel augmented movements systematically, the classifier trained on augmented data is learning the ruleset, not the physiotherapist construct. The TRTA results (e.g., 0.0 macro F1 for HS class 1) and the paper's own discussion suggest that augmented data sits in a different region of feature space. This doesn't sink the FDE experiments, where labels are objective and the auto-labeler reaches 0.958 GMF1, but it does mean the general 'boosting' claim is not established.\n\nSecondary issues: no code or data, a placeholder '[QUELLE]' citation, and an unpublished 'tbd' dataset reference. Also, the threshold optimization appears to use all real data, including left-out subjects in the LOSOCV, which could inflate the apparent benefit of augmented labels for those subjects. That is fixable, not a deal-breaker.\n\nWho should read it: anyone building IMU classifiers for physiotherapy, sports tech, or rehabilitation, especially with small labeled sets. The FDE fine-tuning results are the most compelling part. Send it to peer review; ask for external validation of augmented labels (even a small expert-labeled sample), code/data release, and a more measured abstract.","headline":"A promising augmentation pipeline with honest FDE results, but the auto-labeling of simulated data is never independently validated, so the general 'boosting' claim is only conditionally supported.","tokens_in":18043,"tokens_out":4760,"would_cite":true,"duration_ms":55392,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that musculoskeletal simulation combined with trajectory modification and automatic threshold-based labeling can generate realistic IMU movement data and thereby improve exercise-quality classification, generalization to…","keywords":["IMU data augmentation","musculoskeletal simulation","automatic exercise evaluation","inverse kinematics","physiotherapy","deep learning","Functional Movement Screening","patient-specific fine-tuning"],"falsifier":"Have independent physiotherapists blind-label a random sample of the augmented repetitions and compare their labels to the automatic labels per class and per dataset; substantial disagreement, especially on Hurdle Step class 1 (represented by only eight real examples), would falsify the claim that augmentation provides reliably labeled training data.","tokens_in":16861,"feed_emoji":"💪","tokens_out":6228,"duration_ms":78135,"temperature":0.7,"pith_summary":"The paper sets out to solve a data problem that blocks automated physiotherapy assessment: IMU recordings of exercises are scarce, imbalanced across quality classes, and hard to label. It proposes to generate new training examples by taking recorded movement trajectories, systematically shifting their starting posture and range of motion, and running them through a musculoskeletal inverse-kinematics simulation so the resulting movements respect anatomical joint constraints. Because the simulation changes which quality label a movement deserves, the method adds an automatic labeling step: kinematic outputs are checked against rules derived from the exercise's scoring criteria, with thresholds tuned to match expert labels on real data. Across three exercise datasets, the augmented examples resemble real data and, when added to training, improve classification, generalization to unseen subjects, and fine-tuning from two real repetitions.","feed_headline":"Musculoskeletal simulation yields realistic IMU training data","feed_subtitle":"Exercise-quality classifiers gain accuracy and adapt to new patients from two real examples.","key_machinery":"The mechanism has three linked parts. First, each recorded IMU orientation trajectory is converted to Euler angles and modified by $e_{s,\\mathrm{aug}} = (e_s[t] - e_s[0]) \\odot \\alpha + \\beta$, where the offset $\\beta$ encodes initial posture and the scaling $\\alpha$ encodes a target range of motion; both are sampled from class-, exercise-, and segment-specific multivariate normal distributions. Second, an inverse-kinematics solve against a full-body musculoskeletal model enforces anatomical constraints so the edited trajectory remains plausible. Third, automatic labeling evaluates kinematic parameters against per-exercise criteria, with decision thresholds found by random search over 10 million combinations to maximize geometric-mean F1 against physiotherapist labels on real data; generated examples are kept only when the automatic label agrees with the class sampled by the augmentation distribution.","core_discovery":"On the paper's own terms, the central claim is that a non-label-preserving augmentation pipeline—trajectory modification plus musculoskeletal simulation plus automatic labeling—can manufacture realistic IMU orientation sequences for physiotherapeutic exercises without relying on motion capture, video, or pose data. The evidence is that augmented-only training nearly matches real-only training on the Foot Drop exercise (macro F1 0.933 versus 0.952), that combining real and augmented data raises leave-one-subject-out performance for most participants, and that patient-specific fine-tuning with two real examples reaches near-perfect scores when augmented variants supply the missing classes. The paper also reports limits: on Deep Squat and Hurdle Step, where expert labels are ambiguous, gains shrink and augmented-only training degrades, indicating that label ambiguity rather than biomechanical fidelity is the binding constraint.","pith_inferences":["If the threshold-generalization premise holds, the same pipeline could be adapted to any exercise whose scoring criteria can be expressed as kinematic rules, turning clinical protocols into generators of labeled training data.","The subject-specific clustering visible in the dimensionality-reduction analysis suggests a further use the paper does not develop: augmented examples of a particular patient could serve as a data-driven prior for domain adaptation or home-monitoring anomaly detection.","One testable extension is to replace the hard threshold rules with a probabilistic labeler that outputs a confidence per class, which would let the augmentation filter retain ambiguous examples instead of discarding them and might soften the label-ambiguity bottleneck on Deep Squat and Hurdle Step."],"forward_implications":["Training sets for IMU-based exercise classifiers can be enlarged without new recording campaigns, including targeted expansion of minority classes.","Manual expert labeling of generated movements can be replaced by the threshold-based automatic labeling, reducing cost and enabling reproducible labels.","Pretrained networks can be personalized to a new patient from as few as two real repetitions when augmented variants of those repetitions supply the other classes.","The size of the gains depends on dataset properties: balanced, objectively labeled exercises benefit most, while subjectively labeled exercises are limited by label ambiguity.","Augmented data appear to cover a broader range of movement variance than the small real datasets, so combining real and augmented training is preferable to either alone."],"supporting_citations":[{"why":"Provides the Deep Squat and Hurdle Step IMU datasets, the orientation-estimation preprocessing, and the prior automatic-assessment architecture this work extends.","marker":"[8]"},{"why":"Provides the Foot Drop exercise dataset with objective motion-capture-based labels used as the cleanest test case.","marker":"[27]"},{"why":"Defines the Functional Movement Screening scoring criteria for the Deep Squat, which the automatic labeling ruleset encodes.","marker":"[9]"},{"why":"Defines the scoring criteria for the Hurdle Step, completing the ruleset used for automatic labeling.","marker":"[10]"},{"why":"Supplies the open-source inverse-kinematics solver that enforces biomechanical plausibility of augmented trajectories.","marker":"[31]"},{"why":"Supplies the full-body musculoskeletal model used by the inverse-kinematics solve.","marker":"[32]"},{"why":"Supplies the orientation filter used to convert raw FMS IMU data into quaternion trajectories.","marker":"[29]"},{"why":"Supplies the spherical linear interpolation used to resample orientation sequences to the network's fixed 256-step input.","marker":"[33]"},{"why":"Documents interrater variability in the Functional Movement Screen, supporting the paper's attribution of limited gains to label ambiguity.","marker":"[28]"}],"fun_headline_variants":["Simulated IMU data boosts exercise classifiers","Musculoskeletal simulation augments IMU data for exercise AI","Virtual IMU data improves physio exercise evaluation","Simulation-based augmentation enhances exercise classifiers","Data augmentation with musculoskeletal sim boosts IMU models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the automatic-labeling thresholds, fitted to reproduce physiotherapist labels on real recordings, remain correct when applied to simulated augmented movements; if they misfit, the generated training targets are unreliable and the reported gains are built on mislabeled data.","fun_headline_variants_meta":{"raw":{"variants":["Simulated IMU data boosts exercise classifiers","Musculoskeletal simulation augments IMU data for exercise AI","Virtual IMU data improves physio exercise evaluation","Simulation-based augmentation enhances exercise classifiers","Data augmentation with musculoskeletal sim boosts IMU models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000772,"raw_usage":{"total_tokens":3371,"prompt_tokens":853,"completion_tokens":2518,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":469,"completion_tokens_details":{"reasoning_tokens":2448}},"tokens_in":469,"tokens_out":2518,"duration_ms":22845,"temperature":1.0,"reasoning_tokens":2448,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:22:44.145778+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have independent physiotherapists blind-label a random sample of the augmented repetitions and compare their labels to the automatic labels per class and per dataset; substantial disagreement, especially on Hurdle Step class 1 (represented by only eight real examples), would falsify the claim that augmentation provides reliably labeled training data.","supporting_citations":[{"cited_title":"Orthokidataset","cited_arxiv_id":null,"evidence_quote":"Provides the Foot Drop exercise dataset with objective motion-capture-based labels used as the cleanest test case."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Functional Movement Screening scoring criteria for the Deep Squat, which the automatic labeling ruleset encodes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the scoring criteria for the Hurdle Step, completing the ruleset used for automatic labeling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the open-source inverse-kinematics solver that enforces biomechanical plausibility of augmented trajectories."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the full-body musculoskeletal model used by the inverse-kinematics solve."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the orientation filter used to convert raw FMS IMU data into quaternion trajectories."},{"cited_title":"C., Matheson, G","cited_arxiv_id":null,"evidence_quote":"Documents interrater variability in the Functional Movement Screen, supporting the paper's attribution of limited gains to label ambiguity."}],"review_version":1}