{"id":"035e1642-22f8-48a7-a670-fe5c9fbc876d","arxiv_id":"2607.04842","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Auto-generated label distributions let IMU exercise networks match one-hot classification while more reliably detecting ambiguous borderline repetitions and their competing classes.","lead":"Training IMU exercise classifiers on automatically generated label distributions matches one-hot classification while better recovering borderline ambiguity. Home physio feedback systems could flag near-boundary repetitions instead of forcing hard class decisions that feel arbitrary to patients.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"AGLD is both training target and evaluation reference, so Famb1/Ftop21 gains largely show the network can reproduce its own synthetic target rather than recover clinical ambiguity.","rationale":"The reader's weakest_assumption correctly isolates the uncalibrated, self-referential AGLD construction as the soft spot. I sharpen it one step: even if σc were perfect, using AGLD as both target and metric makes the Famb1/Ftop21 advantage partly tautological. The paper itself acknowledges the weak external signal (low precision, no direct ambiguity labels) and the heuristic σc. Classification parity remains well-supported and is not under attack. The concrete test (human ambiguity labels as independent reference) is the minimal experiment that would convert the detection claim from 'reproduces AGLD' into 'recovers clinical ambiguity.' Verdict stays CONDITIONAL for the same reason the reader gave; no upgrade or downgrade is warranted.","tokens_in":20796,"tokens_out":547,"duration_ms":4601,"concrete_test":"Collect direct binary ambiguity labels (or multi-rater score distributions) from physiotherapists on a held-out subset of DS/HS (or a new exercise), then recompute Famb1 and Ftop21 of both models against those human labels instead of AGLD. If the ambiguity approach no longer significantly outperforms the baseline, the detection claim does not transfer beyond the synthetic reference.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that the ambiguity approach 'detects ambiguity and the relevant classes more reliably' rests on Famb1 and Ftop21 (Section 2.6), which treat the AGLD entropy H(p) and top2(p) as ground truth. The same AGLD is the KLD training target (Eq. 2). Because the baseline collapses AGLD to one-hot while the ambiguity model is trained to match the full distribution, higher Famb1/Ftop21 is expected by construction whenever the network can approximate p; it does not independently establish that the recovered ambiguity matches human-perceived borderline cases. Section 3.5 and Figure 6 provide the only external check (rater disagreement on DS/HS) and show low precision (~0.3–0.4) with no direct ambiguity labels. Limitations 4.5 further note that σc is uncalibrated and thresholds are drawn independently, so the synthetic reference itself may be mis-scaled. Classification parity (Table 1) is solid; the ambiguity-detection half of the strongest claim is not.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes Automatically Generated Label Distributions (AGLD) for IMU-based exercise evaluation: a rule-based labeling pipeline is run with criterion thresholds sampled from N(μc, σc²) to produce a class distribution per repetition without a large human rater pool. A CNN is trained with Kullback–Leibler divergence to match the full AGLD (ambiguity approach) and compared to a one-hot cross-entropy baseline that collapses AGLD to its mode. On four datasets (RD, RGS, DS, HS), the ambiguity approach matches or exceeds baseline macro-F1 for the most relevant class (Table 1) and yields higher Famb1 and Ftop21 across intermediate entropy thresholds (Figs. 3–4), with hierarchical outcome trees at τ = 0.47 bit. The authors conclude that representing label distributions in the training target adds ambiguity information at no cost to classification.","tokens_in":21095,"tokens_out":1382,"duration_ms":14071,"significance":"If the AGLD is accepted as a useful proxy for borderline executions, the work is practically relevant for home-based physiotherapy feedback systems: it offers a concrete way to flag repetitions between categories and report competing classes rather than forcing a hard label. Strengths include evaluation on four real IMU datasets, participant/class/entropy-stratified five-fold CV, metrics reported across the full τ range rather than a single threshold, and an explicit hierarchical breakdown of detection errors. Classification parity (or better) under distributional training is a clean, useful empirical result. The main scientific contribution is the AGLD construction plus the demonstration that KLD training recovers distributional structure without harming mode classification; the clinical claim about human-perceived ambiguity is more provisional.","major_comments":[{"comment":"Sections 2.4, 2.6 and Eqs. (1)–(2): Famb1 and Ftop21 treat AGLD entropy H(p) and top2(p) as ground truth, while the ambiguity model is trained with DKL(p ∥ q) to reproduce that same p. The baseline collapses p to one-hot. Higher Famb1/Ftop21 is therefore largely expected once the network approximates p; it does not independently show recovery of clinical borderline cases. The abstract and conclusion should state this dependency explicitly (e.g., “more reliably recovers the AGLD-defined ambiguity”) or add an evaluation that does not use AGLD as both target and reference.","section":"§2.4–2.6, Eqs. (1)–(2); abstract/conclusion"},{"comment":"Section 3.5 and Figure 6 provide the only external check of AGLD entropy against human rater disagreement (DS/HS only). Precision stays ~0.3–0.4 across the informative τ range, and raters never labeled ambiguity directly—only scores. This is too weak to support the claim that the system “detects ambiguity … more reliably” in the clinical sense. Either collect direct ambiguous/unambiguous rater labels on a subset, or substantially qualify the claim and treat Figure 6 as a consistency check rather than validation.","section":"§3.5, Figure 6; §4.4"},{"comment":"Section 2.2 and Limitations 4.5: σc is fixed heuristically (5°, 5 cm, 0.5 m/s) with independent draws per criterion, and no sensitivity analysis is reported. Because AGLD entropy (and thus Famb1/Ftop21 and the τ = 0.47 split) is a direct function of σc (illustrated in Fig. 7), the ambiguity-detection results are not shown to be robust to this free parameter. A modest sensitivity study (e.g., scale σc by 0.5× and 2× and re-report Famb1/Ftop21 and classification) is needed for the load-bearing half of the main claim.","section":"§2.2, §4.5, Figure 7"}],"minor_comments":[{"comment":"HS FMS 1 has only eight repetitions (Fig. 1); the large Fcls1 gain (0.32→0.63 in Table 1) is correctly flagged as unreliable in §4.1 but still appears in the abstract’s unqualified “matched or exceeded … on all four datasets.” Soften the abstract wording for HS.","section":"Abstract; Table 1; §4.1"},{"comment":"The entropy threshold τ = 0.47 bit is motivated by a 0.9/0.1 two-class split (§2.4), but for K = 3 or 4 the same entropy can arise from other mass allocations. A short note on this would help readers interpret the outcome trees.","section":"§2.4, §3.4"},{"comment":"Related work on multi-annotator and noisy-label methods is discussed, but soft-label / label-smoothing baselines (training with a fixed soft target not derived from AGLD) are not compared. Even a brief discussion would clarify what is specific to AGLD versus any soft target.","section":"§1; §2.4"},{"comment":"Figure 5 and Appendix A trees are informative but dense; stating absolute counts of true positives for top-2 in the main text (not only percentages of parents) would aid comparison across datasets.","section":"Figure 5; Appendix A"},{"comment":"Minor: “ambivalence” appears in the outcome-tree node labels (Fig. 5, A1–A3) while the paper elsewhere uses “ambiguity”; unify terminology.","section":"Figure 5; Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The classification half of the paper is solid and publishable; the ambiguity-detection half is oversold relative to the circular evaluation design. I would accept after major revision if the authors reframe claims around recovering AGLD structure and add at least a σc sensitivity check—or, ideally, a small human ambiguity-label study on DS/HS. Scope fits a methods-oriented ML/sensors venue; less so a pure clinical journal without stronger external validation."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is narrow but real: you can put a full label distribution in the training target for IMU exercise scoring, keep (or slightly improve) macro-F1 versus a one-hot baseline, and get a network whose softmax entropy and top-2 actually track that distribution. That is the claim Table 1 and Figures 3–4 support.\n\nWhat is new relative to Lin et al. is the AGLD itself—100 rule-based “raters” with independent threshold noise around fitted criteria—plus an explicit one-hot baseline and an entropy/top-2 flagging procedure. The experimental hygiene is good: participant/class/entropy-stratified five-fold CV, metrics over the full τ range rather than one cherry-picked cut, hierarchical outcome trees, and a limitations section that names the uncalibrated σc and the missing direct ambiguity labels. Classification parity on RD/RGS/DS is clean; the HS jump is mostly FMS-1 with n=8, which they flag.\n\nThe soft spot is the one the stress-test hits, and it lands. Famb1 and Ftop21 treat AGLD entropy and top-2 as ground truth while the KLD model is trained to reproduce that same AGLD. Once the network approximates p, those gains are expected by construction; they do not independently show recovery of human-perceived borderline cases. Section 3.5 / Figure 6 is the only external check (rater disagreement on DS/HS) and shows low precision (~0.3–0.4) with no direct “is this ambiguous?” labels. σc is heuristic and uncorrelated across criteria. So accept “no cost to classification”; treat “detects clinical ambiguity more reliably” as provisional until someone calibrates the noise model or collects explicit ambiguity annotations.\n\nThis is for people building criteria-based IMU feedback systems or working on LDL without large rater pools. Math and citations look fine; data partly public, code not shipped. I would send it to peer review—methods contribution with honest scope—and ask referees to force a clearer separation between synthetic-target recovery and clinical ambiguity. Worth engaging if you work in this lane; not a general ML must-read.","headline":"Solid classification parity with a useful AGLD construction, but the headline ambiguity-detection gains are partly circular because AGLD is both train target and eval reference.","tokens_in":21752,"tokens_out":553,"would_cite":false,"duration_ms":9378,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Training IMU exercise classifiers on full label distributions flags borderline repetitions without hurting classification accuracy.","keywords":["Label Ambiguity","Label Distribution Learning","Physiotherapeutic Exercise Evaluation","IMU","Kullback-Leibler divergence","borderline repetitions","Functional Movement Screen"],"falsifier":"Gather a multi-rater exercise set in which each rater also marks every repetition as ambiguous or clear; if AGLD entropy and the network's predicted entropy then fail to rank the marked-ambiguous cases above the clear ones, the claim that AGLD captures real ambiguity fails.","tokens_in":21637,"feed_emoji":"🏋️","tokens_out":763,"duration_ms":12363,"temperature":0.7,"pith_summary":"Home physiotherapy systems that score exercises from wearable sensors currently force every repetition into one hard category, even when the movement sits near a boundary where trained raters disagree. This paper shows how to build a soft label distribution for each repetition automatically, by running a rule-based scorer many times with slightly varied thresholds that mimic rater noise. A network trained to match those full distributions equals or beats ordinary one-hot training when picking the main class on four IMU datasets, while also recovering which repetitions are ambiguous and which two classes compete. The practical payoff is feedback that can say a movement lies between two categories instead of issuing an arbitrary hard verdict that erodes patient trust.","feed_headline":"Soft labels flag borderline exercises at no accuracy cost","feed_subtitle":"Training on simulated rater distributions lets IMU systems report competing classes instead of forcing hard calls.","key_machinery":"Automatically Generated Label Distributions (AGLD): one hundred simulated raters that share the same physiotherapist-defined criteria but draw independent thresholds from normals around fitted means, so relative class frequencies become a soft target that encodes borderline ambiguity without a large human rater pool.","core_discovery":"A network trained with a Kullback-Leibler objective to reproduce automatically generated label distributions matches or exceeds a one-hot cross-entropy baseline on macro-F1 for the leading class across four IMU exercise datasets, and recovers ambiguous repetitions and their two competing classes more reliably from the predicted output distribution.","pith_inferences":["Simulated-rater distributions could transfer to other criteria-based clinical scores that suffer boundary disagreement without needing multi-annotator pools.","Calibrating the threshold widths against real multi-rater data would turn AGLD entropy into a drop-in proxy for interrater reliability on new exercises.","Soft targets of this form may reduce the apparent arbitrariness patients feel when nearly identical movements receive opposite hard scores.","Extending evaluation past the top-two classes is needed before multi-way ambiguity is fully represented."],"forward_implications":["Feedback systems can flag borderline repetitions and name both competing classes instead of forcing a single hard assignment.","Label-distribution training adds usable ambiguity information at no cost to classification of the leading class.","Predicted entropy can serve as a practical detector of ambiguous executions once a low threshold is chosen.","The same AGLD construction applies to both nominal fault categories and ordinal FMS scores."],"fun_headline_variants":["Soft labels detect borderline IMU exercises without accuracy loss","KL training on label distributions flags ambiguity in exercise data","Auto soft labels recover competing classes in IMU physio reps","Ambiguity approach matches one-hot baseline across four IMU sets","Label distributions expose borderline exercise form at no F1 cost"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The method assumes that sampling independent criterion thresholds from normal distributions with fixed heuristic widths produces label distributions whose entropy and top classes faithfully match the ambiguity human raters would actually perceive.","fun_headline_variants_meta":{"raw":{"variants":["Soft labels detect borderline IMU exercises without accuracy loss","KL training on label distributions flags ambiguity in exercise data","Auto soft labels recover competing classes in IMU physio reps","Ambiguity approach matches one-hot baseline across four IMU sets","Label distributions expose borderline exercise form at no F1 cost"]},"model":"grok-4.5","effort":"low","cost_usd":0.005924,"raw_usage":{"total_tokens":1524,"prompt_tokens":709,"num_sources_used":0,"completion_tokens":82,"cost_in_usd_ticks":59240000,"prompt_tokens_details":{"text_tokens":709,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":733,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":709,"tokens_out":82,"duration_ms":5692,"temperature":1.0,"reasoning_tokens":733,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T12:41:09.676520+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Gather a multi-rater exercise set in which each rater also marks every repetition as ambiguous or clear; if AGLD entropy and the network's predicted entropy then fail to rank the marked-ambiguous cases above the clear ones, the claim that AGLD captures real ambiguity fails.","supporting_citations":[],"review_version":1}