{"id":"879535ed-42e1-468d-97d6-36b7560df2d9","arxiv_id":"2505.20924","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Gradient-based label leakage attacks recover activity class labels from federated HAR updates with high accuracy, especially under sequential sampling, and standard local privacy defenses provide only partial protection.","lead":"Federated learning for wearable activity recognition is meant to keep sensor data on the device, but this paper shows that a curious server can often recover which activity labels a user's device was training on, using only the gradient updates it sends. Five published label-leakage attacks were tested on two wearable datasets, and several recovered class labels accurately even from fully trained models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed critical factors (class count, imbalance) rest on a confounded two-dataset comparison; within-dataset factor manipulation is missing.","rationale":"The reader's weakest assumption (server knows N and has white-box access) is explicitly stated in the paper and is plausible for an orchestrating server; it affects attack calibration but does not undermine the core leakage result. The more load-bearing concern is the internal validity of the paper's headline causal claims about the number of classes and class imbalance, which are based entirely on a confounded comparison of two datasets. The leakage benchmark itself remains useful and the sampling-strategy findings are well controlled, so the paper should not be rejected; however, the factor claims must be reworded as dataset-specific observations or supported by controlled within-dataset experiments. This concern aligns with a conditional verdict: the paper needs revision before the abstract's characterization of critical factors can be taken at face value.","tokens_in":11672,"tokens_out":10102,"duration_ms":112549,"concrete_test":"On the Wetlab dataset, construct controlled variants: (a) merge the 9 original classes into K super-classes (K=4, 9, 18) by relabeling sliding windows while keeping sensor data fixed; (b) rebalance Wetlab to match WEAR's class distribution via subsampling/oversampling. Rerun the EBI/LLBG single-step attacks under shuffled and sequential sampling. If ClassAcc does not vary systematically with K or with the rebalanced distribution, the claimed influence of class count and imbalance is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract's central claim is that the number of activity classes and the degree of class imbalance are critical factors influencing the extent of label leakage. The only evidence for these factors is the contrast between WEAR (18 classes) and Wetlab (9 classes). These datasets differ in sensor placement, activity semantics, recording protocol, class distributions, and model accuracy; the number of classes and imbalance are nowhere independently varied within a dataset. Consequently, any dataset-specific property (e.g., temporal autocorrelation, class-exclusivity of sequential batches, model accuracy) could be the true driver of the observed leakage difference. The within-dataset sampling-strategy comparison is properly controlled, but the class-count/imbalance causal claims are not. Since these factors are featured prominently in the abstract as a main finding, the analytical contribution is not yet supported: the experiments establish that label leakage occurs in these two HAR datasets, but not that these specific dataset properties govern it. The paper's limitation section does not acknowledge this confound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a benchmark study of five gradient-based label leakage attacks (LLG, LLG*, EBI, iLRG, LLBG) applied to federated inertial-based human activity recognition (HAR). The evaluation covers two HAR datasets (WEAR, Wetlab) and two architectures (DeepConvLSTM, TinyHAR), under shuffled, sequential, and balanced batch sampling, for trained and untrained models, with single- and multi-step local updates, and under local differential privacy defenses (gradient noise, clipping, and their combination). The authors propose a new class-averaged accuracy metric, ClassAcc, to mitigate majority-class bias in existing metrics. The main reported findings are that bias-based attacks are highly effective, sequential sampling is particularly leaky due to label-exclusive batches, and combined clipping plus noise provides only partial protection. The paper claims that the number of activity classes, class imbalance, and sampling strategy are critical factors determining label leakage, and it offers practical recommendations for privacy-aware federated HAR deployment.","tokens_in":11841,"tokens_out":4379,"duration_ms":45183,"significance":"If its central claims hold, this is a timely and useful empirical contribution: it transfers label leakage attacks from computer vision to HAR, demonstrates their practical effectiveness on real sensor data, and provides a new evaluation metric (ClassAcc) that is better suited to imbalanced activity datasets. The public release of code is a clear strength for reproducibility. However, the paper's most prominent quantitative claim—that reconstruction accuracies exceed 90% on two benchmark datasets even for trained models—is not supported by the reported numbers, and the causal attribution of leakage differences to the number of classes and class imbalance is confounded by the two-dataset comparison. These issues weaken the abstract and the deductive force of the conclusions, though the core experimental effort and the qualitative findings about sequential sampling remain valuable.","major_comments":[{"comment":"The abstract states that reconstruction accuracies reach 'well-above 90% on two benchmark datasets, even for trained models.' Table 1 does not support this: for trained models, the best bias-based ClassAcc is 80.71% on WEAR under sequential sampling and 70.76% on Wetlab under sequential sampling; none of the trained-model entries in the table reach 90%. Even untrained Wetlab bias-based performance is 81.67%. Please either correct the abstract to reflect the actual ranges (e.g., 'well above 90% on WEAR for untrained models under sequential sampling, and above 70% for trained models') or provide per-attack, per-architecture results that justify the stronger claim.","section":"Abstract; Section 4, Table 1"},{"comment":"The paper identifies 'the number of activity classes' and 'class imbalance' as critical factors influencing label leakage, but the only evidence is the contrast between WEAR (18 classes) and Wetlab (9 classes). These datasets differ in sensor placement, activity semantics, recording protocol, class distribution, temporal autocorrelation, and model accuracy, so the observed leakage difference is confounded. No within-dataset manipulation of class count or imbalance is performed. The limitations section does not acknowledge this confound. To make the causal claim defensible, the authors should either add controlled experiments (e.g., subsampling classes or rebalancing within a single dataset) or weaken the claim to an observed association rather than a demonstrated causal factor.","section":"Section 4 (Table 1) and Section 5 (Limitations)"},{"comment":"The text states that 'under sequential sampling, EBI and LLBG maintain ClassAcc levels above 90% on WEAR, even with trained models,' but Table 1 reports the bias-based average ClassAcc for WEAR trained sequential as 80.71%. If the claim refers to specific attacks or architectures, it should be made explicit and supported with the corresponding numbers; otherwise the text and table are inconsistent. This inconsistency is similar to the abstract overstatement and should be corrected in the same revision.","section":"Section 4, 'Single-step gradient updates'"}],"minor_comments":[{"comment":"Equation (4) uses vertical bars around a difference of boolean expressions without defining the absolute value or indicator function; please use explicit indicator notation, e.g., |I(p_gt_j > 0) - I(p_j > 0)|, for clarity.","section":"Section 3.4, Equation (4)"},{"comment":"The definition of ClassAcc is informal ('average of the per-class LnAcc computed across the complete dataset of a client'). A precise formula would help readers understand the difference from LnAcc, especially because the paper argues this metric is a contribution.","section":"Section 3.4"},{"comment":"The statement that applying clipping and noise to the last-layer gradients 'does not impair the prediction accuracy of trained models' is made without showing results in the paper. Please add a small table or a reference to the code repository output to substantiate this claim.","section":"Section 5, Limitations"},{"comment":"The word 'proofed' should be 'proved' or 'shown' ('LDP measures ... proofed to be partially effective').","section":"Section 6"},{"comment":"The caption says 'We report the average LnAcc difference between single step and multi-step gradient updates,' but the text in Section 4 mentions both ClassAcc and LnAcc differences. Please clarify which metric is actually plotted, or plot both consistently.","section":"Figure 3 caption"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope and the empirical setup is a reasonable first benchmark for label leakage in FL-HAR. The main concerns are not about the existence of leakage but about overstatement and causal attribution. The abstract's 'well-above 90% ... even for trained models' is contradicted by Table 1 and must be fixed. The class-count/imbalance claim would require either a new controlled experiment or a careful softening. The authors' own limitation section omits the confounding issue, which should be addressed explicitly. Given the clear reproducibility effort and the usefulness of the ClassAcc metric, the paper can become acceptable after a thorough revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper is the first to take known gradient-based label reconstruction attacks and run them properly on HAR data, with code, two architectures, and two datasets. That alone is useful. The central finding—sequential sampling makes label-exclusive batches and leakage is severe, shuffled batches are safer, and common LDP defenses only partially help—holds up on the numbers they present. I also like the ClassAcc metric; it is a real fix for the majority-class bias in LeAcc/LnAcc for imbalanced HAR data.\n\nThe soft spots are real but don't sink the paper. The abstract says 'well-above 90% on two benchmark datasets, even for trained models,' but the reported ClassAcc averages for trained models are ~80% on WEAR and ~70% on Wetlab under sequential sampling (Table 1). Individual EBI/LLBG may hit >90%, but that needs to be stated precisely. Second, the claim that the number of classes and class imbalance are 'critical factors' is not supported by their experiments. They only compare WEAR (18 classes) to Wetlab (9 classes); those datasets differ in sensor placement, activity semantics, recording protocol, and temporal dynamics. They never vary class count or imbalance within a dataset. The stress-test note is right; this is a confounded comparison, and the limitation section doesn't acknowledge it. The authors should either run a controlled within-dataset manipulation or soften the causal language to 'associated with.' Third, the multi-step experiment in Figure 3 lacks a same-data-size single-step baseline, so the effect of averaging multiple steps is entangled with the larger total batch.\n\nThe threat model (server knows batch size N and has white-box access) is clearly stated and plausible for a coordinating server. The code is released, though they use a pre-release version of the WEAR dataset, which makes reproduction less clean.\n\nBottom line: this is a competent experimental paper with a useful benchmark and a fair metric. It deserves a serious referee. The core leak finding is probably correct; the causal attributions need to be dialed back. I would accept it for review and ask for the fixes above.","headline":"First real benchmark of label leakage attacks on federated HAR; the core leak finding holds, but the causal claims about class count and imbalance rest on a confounded cross-dataset comparison and the abstract oversells trained-model accuracy.","tokens_in":12372,"tokens_out":2666,"would_cite":true,"duration_ms":27238,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated training leaks activity labels past 90 percent accuracy","keywords":["Federated Learning","Label Leakage","Gradient Inversion","Human Activity Recognition","Inertial Sensors","Local Differential Privacy","Class Imbalance","Label Reconstruction"],"falsifier":"Run the same five attacks against clients that use a class-weighted cross-entropy loss, or that locally vary the batch size per update while the server assumes a fixed $N$, and measure ClassAcc; if the reported above-90% reconstruction collapses, the leakage is an artifact of the assumed calibration rather than an inherent property of federated HAR gradients.","tokens_in":11459,"feed_emoji":"⌚","tokens_out":5894,"duration_ms":54050,"temperature":0.7,"pith_summary":"This paper tries to establish that federated learning for wearable human activity recognition is far more privacy-leaky than prior work assumed: an honest-but-curious server can reconstruct which activity labels a client's local batch contained purely from the gradient updates the client shares. Applying five known gradient-based label reconstruction attacks to two HAR benchmark datasets, the authors report class-averaged reconstruction accuracies above 90 percent for bias-based attacks on sequentially sampled batches, even when the model has been fully trained. The paper further claims that the number of activity classes, class imbalance, and the sampling strategy are the decisive factors, with label-exclusive batches created by sequential sampling being the most dangerous. Local differential privacy defenses such as gradient noise and clipping offer only partial protection, and the paper recommends delaying updates until batches cover diverse activity classes and combining clipping with noise.","feed_headline":"Federated training leaks activity labels past 90 percent accuracy","feed_subtitle":"Bias-based attacks recover activity labels from shared gradients; clipping plus noise only partly blocks them.","key_machinery":"The mechanism that carries the attacks is the linear structure of the final-layer bias gradients: for a batch of size $N$ with $\\lambda_i$ samples of class $i$, the bias gradient component for class $i$ is approximately $-\\lambda_i/N$ in an untrained model, so each bias component encodes the per-class label count. Bias-based attacks (EBI, LLBG, iLRG) estimate this per-sample gradient impact $m$ and iteratively reconstruct the label multiset by adding labels whose bias-gradient components remain minimal until the batch size $N$ is reached. Weight-based attacks (LLG, LLG*) use a similar sum-of-weight-gradient sign property but are weaker. The paper's analysis of sampling strategies and its ClassAcc metric are the tools that expose when this mechanism is most dangerous.","core_discovery":"The central claim is that label leakage in federated HAR is severe and systematically underestimated by metrics that favor majority classes. The authors show that bias-based attacks—EBI, LLBG, and iLRG—which read the bias-gradient components of the final classification layer, reconstruct per-batch label counts with high accuracy; under sequential sampling these attacks keep ClassAcc above 90% on the WEAR dataset even for a trained model. The discovery is that HAR's structural properties, especially the dominant NULL class and temporally ordered sliding windows that make sequential batches label-exclusive, turn a known computer-vision vulnerability into a critical privacy risk for wearable and stream-based applications. The paper also introduces ClassAcc, a class-averaged accuracy metric, to counter the misleadingly optimistic picture given by majority-biased LeAcc and LnAcc.","pith_inferences":["If the paper's assumptions are relaxed, a client-side protocol that uses hidden, randomized local batch sizes could substantially blunt bias-based attacks without altering the model; the paper explicitly excludes this case but does not test it.","The finding that label-exclusive batches drive leakage implies that client-side buffering that enforces class diversity before an update is sent could be a more practical privacy control than differential privacy tuning, and a direct test would measure ClassAcc under a diversity-constrained sampling policy.","Since bias-based attacks outperform weight-based ones, the paper's recommendation to drop bias terms could be evaluated as a standalone defense; an ablation on the two architectures would quantify how much leakage is actually removed.","The iLRG attack's volatility under LDP and multi-step averaging suggests a hard-to-model interaction between linear-equation reconstruction and gradient noise; studying that interaction may yield better adaptive defenses."],"forward_implications":["If the paper is right, any stream-based federated HAR deployment that sends updates after short recording sessions is exposing the user's current activity to the aggregating server.","Sequential or temporally ordered sampling should be treated as a privacy hazard, because label-exclusive batches make bias-gradient reconstruction nearly perfect.","LeAcc and LnAcc metrics are insufficient for HAR; class-balanced ClassAcc should be reported in any privacy evaluation on imbalanced sensor data.","Common LDP defenses selectively applied to the last layer do not provide reliable protection; the combination of clipping and noise reduces precision but class presence often remains inferable.","Training without bias terms in the classification layer is a suggested structural mitigation."],"supporting_citations":[{"why":"Supplies the LLBG bias-gradient attack and the clipping-plus-noise defense setup that the study applies and evaluates.","marker":"[8]"},{"why":"Provides the LLG and LLG* weight-based label leakage attacks that serve as weaker baselines.","marker":"[26]"},{"why":"Introduces the iLRG attack, the strongest method in several untrained-model settings.","marker":"[14]"},{"why":"Defines the federated learning protocol with multiple local steps (FedAVG) that underlies the update-sharing threat model.","marker":"[15]"},{"why":"WEAR dataset, one of the two evaluation benchmarks, with 18 participants and 18 sports activity classes.","marker":"[1]"},{"why":"Wetlab dataset, the second benchmark, with 9 laboratory activity classes and a NULL class.","marker":"[23]"},{"why":"DeepConvLSTM architecture used as one of the two model backbones.","marker":"[17]"},{"why":"TinyHAR architecture used as the second, resource-efficient model backbone.","marker":"[32]"}],"fun_headline_variants":["Label leak attacks hit 90% accuracy in federated HAR","Federated HAR leaks activity labels past 90% accuracy","Bias-based attacks recover activity labels in federated HAR","Federated learning leaks activity labels despite noise","Class imbalance fuels label leakage in federated HAR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the honest-but-curious server knows the exact local batch size $N$ and the full model architecture; if clients could hide or vary $N$, or used class-weighted losses, the per-class gradient impact estimates that all bias-based attacks rely on would no longer be calibrated and the reported accuracies would likely drop.","fun_headline_variants_meta":{"raw":{"variants":["Label leak attacks hit 90% accuracy in federated HAR","Federated HAR leaks activity labels past 90% accuracy","Bias-based attacks recover activity labels in federated HAR","Federated learning leaks activity labels despite noise","Class imbalance fuels label leakage in federated HAR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000642,"raw_usage":{"total_tokens":2920,"prompt_tokens":876,"completion_tokens":2044,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":1965}},"tokens_in":492,"tokens_out":2044,"duration_ms":13872,"temperature":1.0,"reasoning_tokens":1965,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:43:35.473696+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same five attacks against clients that use a class-weighted cross-entropy loss, or that locally vary the batch size per update while the server assumes a fixed $N$, and measure ClassAcc; if the reported above-90% reconstruction collapses, the leakage is an artifact of the assumed calibration rather than an inherent property of federated HAR gradients.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the iLRG attack, the strongest method in several untrained-model settings."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the federated learning protocol with multiple local steps (FedAVG) that underlies the update-sharing threat model."},{"cited_title":"Scholl, Matthias Wille, and Kristof Van Laerhoven","cited_arxiv_id":null,"evidence_quote":"Wetlab dataset, the second benchmark, with 9 laboratory activity classes and a NULL class."}],"review_version":1}