REVIEW 4 major objections 6 minor 1 cited by
Generalizable Sensor-Based Activity Recognition via Categorical Concept Invariant Learning
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper argues that aligning each sample's concept matrix—the per-feature, per-class contributions of features times classifier weights—with its class mean makes sensor-based activity recognition generalize across unseen people…
desk verdict A simple, plausible regularizer for domain-generalized HAR with real benchmark coverage, but the empirical margins are thin and the paper's own momentum equation contradicts its ablation text. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the concept matrix $M_i$, defined entrywise as $(M_i)_{j,c} = W_{j,c} z_j$: each entry is one feature's contribution to one class logit before summation. The argument is carried by two pieces working together: the loss $L_{\mathrm{CMS}}$, which penalizes within-class scatter of these matrices around their class mean, and the momentum update $\hat{M}_c^t = (1-\lambda)\hat{M}_c^{t-1} + \lambda \bar{M}_c$, where $\bar{M}_c$ is the mean concept matrix for class $c$ in the current batch. Together they provide a fine-grained, classifier-aware alignment that the paper's ablations show is stronger than aligning features alone or logits alone.
What would settle it
Train CCIL while tracking the rank and norm of the classifier weights $W$ and the per-class feature covariance; if $L_{\mathrm{CMS}}$ decreases mainly through shrinking feature or weight norms, or through a low-rank collapse of $W$, the claimed alignment mechanism is not what drives the accuracy gains. A direct check is to re-run the experiments with a norm-preserving reparameterization that keeps $\|W\|$ and $\|z\|$ approximately fixed and see whether the gains survive.
Extended reading notes
Core claim
The paper's central claim is that a well-generalized activity model should produce similar concept matrices for samples of the same activity, regardless of domain. For input sample $i$, the concept matrix is $(M_i)_{j,c} = W_{j,c} z_j$, the contribution of feature dimension $j$ to class logit $c$. CCIL adds the categorical concept invariance loss $L_{\mathrm{CMS}} = \frac{1}{N_b} \sum_c \sum_{\{i:y_i=c\}} \|M_i - \hat{M}_c\|^2$, where $\hat{M}_c$ is a class-mean concept matrix updated by momentum online. This simultaneously regularizes feature invariance and logit invariance, because the matrix entries weight features by the classifier's own importance. The paper reports that this loss, with no other architectural change, gives the best average accuracy on all four benchmark settings, surpassing the strongest baseline by 1.3 to 3.7 points depending on the setting and exceeding ERM by 7 to 10 points on cross-person benchmarks.
Load-bearing premise
The load-bearing premise is that the running per-activity mean concept matrix, updated from the model's own weights and features, is a stable and trustworthy target; if those weights or features drift or collapse, the loss can shrink without improving real generalization.
Editorial extensions
If this is right
- HAR models would gain several accuracy points on unseen users with only a few lines of code added to standard empirical risk minimization; the paper reports 7 to 10 point gains over ERM on cross-person benchmarks.
- The regularization is model-agnostic: the authors show it also improves results when the CNN backbone is replaced with a Transformer.
- Because CCIL does not need domain labels during training, it applies to mixed multi-user data where clean domain partitions are unavailable.
- Consistent gains under cross-position and cross-dataset settings imply the same invariant helps when sensor placement or collection device changes, not only when the person changes.
Reading between the lines
- The authors leave implicit that $L_{\mathrm{CMS}}$ is intra-class scatter minimization in the concept space; under that reading, adding a constraint on feature or weight norms would directly test whether the gains come from genuine alignment rather than from scale shrinkage.
- A testable extension is to apply the class-mean alignment per sensor axis or per body position, which could reveal which concept-matrix entries carry the domain-stable signal.
- Because the momentum target is coupled to the parameters being optimized, freezing $\hat{M}_c$ from a model pretrained on the source domains would isolate whether the target's drift matters; the paper's $\lambda=1$ ablation is a partial step in that direction.
- The same multiplicative feature-weight view appears in attention and gating layers, so concept-matrix invariance could be ported to those architectures beyond human activity recognition.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CCIL, a regularization method for sensor-based human activity recognition under domain shift. For each sample, it forms a D x C 'concept matrix' M = W * z from the feature vector z and classifier weights W, and adds a loss term L_CMS that penalizes the squared distance between each sample's concept matrix and a class-mean target M_hat_c, which is updated by momentum. The total objective is L = L_CE + alpha * L_CMS. The method is evaluated on DSADS, PAMAP2, USC-HAD, and UCI-HAR under cross-person, cross-position, cross-dataset, and one-person-to-another settings, and compared against ERM, DANN, CORAL, Mixup, GroupDRO, RSC, ANDMask, GILE, AdaRNN, and DIVERSIFY.
Significance. If the reported gains are statistically robust, CCIL would be a simple, model-agnostic regularizer with a clear intuition that goes beyond feature-invariance by also accounting for classifier weights. The paper has several strengths: coverage of four datasets and four shift settings, a source-domain validation protocol, an ablation study, a parameter-sensitivity analysis, a Transformer extensibility experiment, and pseudo-code. However, the central claim of consistent and substantial improvement over state-of-the-art methods is not yet established because the reported margins are small in several settings, no error bars are given, and the description of the momentum update is internally inconsistent. These issues are load-bearing for the paper's empirical contribution.
major comments (4)
- [Methodology, Eq. (5) vs. Ablation Study and Table 5] The momentum update in Eq. (5) is written as M_hat^t = (1 - lambda) * M_hat^{t-1} + lambda * batch_mean, which means lambda=1 replaces the target with the current batch and lambda=0 freezes the initial target. The ablation text states the opposite: 'W/lambda = 0' is described as using the mean value dynamically calculated from the current batch, and 'W/lambda = 1' is described as keeping M_hat fixed from the initial pretrained model. The numbers in Table 5 (W/lambda=0: 85.5, W/lambda=1: 89.3) and the sensitivity analysis in Figure 4 are consistent with the standard EMA convention in Eq. (5), not with the ablation text. Only one convention can be the one that produced Tables 1-4; please correct Eq. (5) or the text and explicitly state the exact update used.
- [Implementation Details and Tables 1-4] The paper states that 'All methods were adjusted to report the average best performance over three trials' and that hyperparameters are tuned per method and per dataset, but no standard deviations, per-seed results, or model-selection criterion are reported. This matters because the margins over the strongest baseline are small in three of the four settings: 1.3 points in Table 2 (cross-position DSADS), 1.5 points in Table 3 (cross-dataset average), and 2.1 points in Table 4 (one-person-to-another). With three trials and per-method hyperparameter tuning, these gaps are within typical run-to-run variation for HAR models. Please report the number of seeds, standard deviations, the exact rule for selecting the reported checkpoint (source validation vs. test-set selection), and ideally a paired statistical comparison.
- [Table 3 and Experimental Results] The claim that CCIL 'consistently' beats the state of the art is not supported on every target: in the cross-dataset setting, target 3 (PAMAP2) shows CCIL at 59.6% versus DIVERSIFY at 59.9%. The average is higher (52.6 vs. 51.1), but the per-target result contradicts the unqualified wording. Please qualify the claim to say CCIL achieves the best average accuracy while not winning every target domain.
- [Experimental Setup, One-person-to-another setting] The protocol description says four pairs of subjects are used - (0,1), (2,3), (4,5), and (6,7) - and that the model generalizes 'from the second subject in each pair to the first.' However, Table 4 reports only three target columns, labeled 0, 1, and 2 as DSADS, USC-HAD, and PAMAP2. It is unclear whether the reported accuracy is averaged over all four pairs for each dataset, or whether only one pair per dataset is used, and how the pair-to-target mapping is defined. Please specify the exact data split and the aggregation rule over the four pairs.
minor comments (6)
- [Throughout] There are several typographical errors, including 'vanialla' in the Methodology section, 'popuar' in the Problem Formulation section, 'model-agonistic' in the Experimental Results section, and 'E MR' in the Comparative Methods subsection. These should be corrected.
- [Methodology, Eq. (5)] The notation '|yi = c|' in Eq. (5) is undefined and is described as 'the sample corresponding to the c-th class.' It should be defined precisely as the number of samples in the batch whose label is c, and the denominator should be stated as a count.
- [Algorithm 1] In Step 6, the pseudo-code says 'Using output features z and classifier weight W to calculate the mean values M_hat,' but M_hat is an exponential moving average, not a direct batch computation. This step should be rewritten to match the corrected Eq. (5).
- [Ablation Study] The terms 'W/lambda = 0' and 'W/lambda = 1' are described in a way that is opposite to Eq. (5). After fixing the equation, the ablation text and the definition of 'initial pretrained model' need to be made consistent; otherwise the reader cannot determine whether pretraining was actually used.
- [Figure 1] The axes in Figure 1 are labeled '1' and '2' without explanation. Please label the axes as subject index and sensor position, or add a caption that defines these axes, so that the illustration of domain shift is clear.
- [Table 6] In Table 6, the column 'Subject' lists the number of subjects in each dataset; renaming it to 'Number of subjects' would avoid ambiguity, and the 'Sampling rate' column should state the unit (Hz) directly in the header or in the caption.
Circularity Check
No significant circularity: the CCIL regularizer is a training objective evaluated on unseen target domains, and the few self-citations are background references, not load-bearing arguments.
full rationale
The paper's central claim is empirical: adding the categorical concept-matrix similarity loss LCMS to cross-entropy improves domain-generalization accuracy on four HAR benchmarks. The regularizer directly enforces the stated key idea that same-class concept matrices should be similar, but that is a design choice, not a derived prediction; the reported accuracies are measured on held-out target domains (Tables 1-4), so no reported result is forced by construction from the loss. Hyperparameters α and λ are tuned and reported as sensitivities; they are not fitted parameters renamed as predictions. The momentum-updated class-mean matrix M_hat_c depends on the model being trained, which could permit degenerate solutions, but that is a potential robustness/correctness concern, not circularity. The self-citations (Huang et al. 2022; Wang et al. 2024) appear only in background statements and do not supply any load-bearing uniqueness theorem or ansatz. The internal inconsistency between Eq. (5) and the ablation text regarding λ = 0 versus λ = 1 is a reproducibility flaw, not a circularity flaw. No circular step can be exhibited from the paper's own equations or citations, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (2)
- alpha =
1 (best in sensitivity analysis)
- lambda =
0.9 (best)
assumptions (4)
- ad hoc to paper Concept matrices of samples from the same activity class should be similar across domains.
- domain assumption Momentum-updated class mean M_hat_c approximates the true class centroid across all source domains.
- domain assumption Source and target domains share the same activity label space and sensor channel layout.
- standard math L2 norm is an appropriate similarity measure for concept matrices.
invented entities (1)
-
Concept matrix M (element-wise product of feature vector and classifier weight matrix)
Cite this review
Pith. "Pith review of Generalizable Sensor-Based Activity Recognition via Categorical Concept Invariant Learning." pith.science (2026). https://pith.science/paper/SHEMT4CN
@misc{pith2026241213594,
author = {Pith},
title = {Pith review of: Generalizable Sensor-Based Activity Recognition via Categorical Concept Invariant Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/SHEMT4CN}},
note = {Machine review of arXiv:2412.13594}
}
read the original abstract
Human Activity Recognition (HAR) aims to recognize activities by training models on massive sensor data. In real-world deployment, a crucial aspect of HAR that has been largely overlooked is that the test sets may have different distributions from training sets due to inter-subject variability including age, gender, behavioral habits, etc., which leads to poor generalization performance. One promising solution is to learn domain-invariant representations to enable a model to generalize on an unseen distribution. However, most existing methods only consider the feature-invariance of the penultimate layer for domain-invariant learning, which leads to suboptimal results. In this paper, we propose a Categorical Concept Invariant Learning (CCIL) framework for generalizable activity recognition, which introduces a concept matrix to regularize the model in the training stage by simultaneously concentrating on feature-invariance and logit-invariance. Our key idea is that the concept matrix for samples belonging to the same activity category should be similar. Extensive experiments on four public HAR benchmarks demonstrate that our CCIL substantially outperforms the state-of-the-art approaches under cross-person, cross-dataset, cross-position, and one-person-to-another settings.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Subject Invariant Contrastive Learning for Human Activity Recognition
A subject-reweighted contrastive loss improves cross-subject generalization for human activity recognition across unimodal, multimodal, and supervised settings.
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Altun, K.; Barshan, B.; and Tun c el, O. 2010. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognition
work page 2010
- [4]
-
[5]
Cha, J.; Lee, K.; Park, S.; and Chun, S. 2022. Domain generalization by mutual-information regularization with pre-trained models. In ECCV
work page 2022
-
[6]
Chen, K.; Zhang, D.; Yao, L.; Guo, B.; Yu, Z.; and Liu, Y. 2021. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. ACM Computing Surveys
work page 2021
-
[7]
Chen, L.; Zhang, Y.; Song, Y.; Van Den Hengel, A.; and Liu, L. 2023. Domain generalization via rationale invariance. In ICCV
work page 2023
-
[8]
M.; Min, K.; Wang, H.; Piran, M
Dang, L. M.; Min, K.; Wang, H.; Piran, M. J.; and Moon, H. 2020. Sensor-based and vision-based human activity recognition: A comprehensive survey. Pattern Recognition
work page 2020
Show all 38 references
-
[9]
Du, Y.; Wang, J.; Feng, W.; Pan, S.; Qin, T.; Xu, R.; and Wang, C. 2021. Adarnn: Adaptive learning and forecasting of time series. In CIKM
2021
-
[10]
Ganin, Y.; Ustinova, E.; Ajakan, H.; Germain, P.; Larochelle, H.; Laviolette, F.; March, M.; and Lempitsky, V. 2016. Domain-adversarial training of neural networks. Journal of Machine Learning Research
2016
-
[11]
H.; Chignell, M.; Valaee, S.; and Liu, X
Gu, F.; Chung, M. H.; Chignell, M.; Valaee, S.; and Liu, X. 2021. A Survey on Deep Learning for Human Activity Recognition. ACM Computing Surveys
2021
-
[12]
Gulrajani, I.; and Lopez-Paz, D. 2021. In search of lost domain generalization. In ICLR
2021
-
[13]
Y.; Halloran, S.; and Pl \"o tz, T
Hammerla, N. Y.; Halloran, S.; and Pl \"o tz, T. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. In IJCAI
2016
-
[14]
He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR
2020
-
[15]
Huang, W.; Zhang, L.; Wu, H.; Min, F.; and Song, A. 2022. Channel-Equalization-HAR: a light-weight convolutional neural network for wearable sensor based human activity recognition. IEEE Transactions on Mobile Computing
2022
-
[16]
P.; and Huang, D
Huang, Z.; Wang, H.; Xing, E. P.; and Huang, D. 2020. Self-challenging improves cross-domain generalization. In ECCV
2020
-
[17]
Kong, Y.; and Fu, Y. 2022. Human action recognition and prediction: A survey. International Journal of Computer Vision
2022
-
[18]
M.; and Loog, M
Kouw, W. M.; and Loog, M. 2019. A review of domain adaptation without target labels. IEEE Transactions on Pattern Analysis and Machine Intelligence
2019
-
[19]
Lu, W.; Chen, Y.; Wang, J.; and Qin, X. 2021. Cross-domain activity recognition via substructural optimal transport. Neurocomputing
2021
-
[20]
Lu, W.; Wang, J.; Sun, X.; Chen, Y.; Ji, X.; Yang, Q.; and Xie, X. 2024. Diversify: A General Framework for Time Series Out-of-distribution Detection and Generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence
2024
-
[21]
J.; and Roggen, D
Ord \'o \ n ez, F. J.; and Roggen, D. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors
2016
-
[22]
Parascandolo, G.; Neitz, A.; ORVIETO, A.; Gresele, L.; and Sch \"o lkopf, B. 2020. Learning explanations that are hard to vary. In ICLR
2020
-
[23]
J.; Da, B.; and Miao, C
Qian, H.; Pan, S. J.; Da, B.; and Miao, C. 2019. A Novel Distribution-Embedded Neural Network for Sensor-Based Activity Recognition. In IJCAI
2019
-
[24]
J.; and Miao, C
Qian, H.; Pan, S. J.; and Miao, C. 2021. Latent independent excitation for generalizable sensor-based cross-person activity recognition. In AAAI
2021
-
[25]
Reiss, A.; and Stricker, D. 2012. Introducing a new benchmarked dataset for activity monitoring. In ISWC
2012
-
[26]
W.; Hashimoto, T
Sagawa, S.; Koh, P. W.; Hashimoto, T. B.; and Liang, P. 2019. Distributionally Robust Neural Networks. In ICLR
2019
-
[27]
Sun, B.; and Saenko, K. 2016. Deep coral: Correlation alignment for deep domain adaptation. In ECCV
2016
-
[28]
Sun, Z.; Ke, Q.; Rahmani, H.; Bennamoun, M.; Wang, G.; and Liu, J. 2022. Human action recognition from various data modalities: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence
2022
-
[29]
Vapnik, V. 1991. Principles of risk minimization for learning theory. Advances in neural information processing systems
1991
-
[30]
Wang, J.; Chen, Y.; Hao, S.; Peng, X.; and Hu, L. 2019. Deep learning for sensor-based activity recognition: A survey. Pattern recognition letters
2019
-
[31]
Wang, J.; Chen, Y.; Hu, L.; Peng, X.; and Philip, S. Y. 2018. Stratified transfer learning for cross-domain activity recognition. In PerCom
2018
-
[32]
Wang, J.; Lan, C.; Liu, C.; Ouyang, Y.; Qin, T.; Lu, W.; Chen, Y.; Zeng, W.; and Philip, S. Y. 2022. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering
2022
-
[33]
Wang, S.; Wang, J.; Xi, H.; Zhang, B.; Zhang, L.; and Wei, H. 2024. Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition. In IMWUT/Ubicomp
2024
-
[34]
Yu, Y.-C.; and Lin, H.-T. 2023. Semi-supervised domain adaptation with source label adaptation. In CVPR
2023
-
[35]
N.; and Lopez-Paz, D
Zhang, H.; Cisse, M.; Dauphin, Y. N.; and Lopez-Paz, D. 2018. mixup: Beyond Empirical Risk Minimization. In ICLR
2018
-
[36]
Zhang, M.; and Sawchuk, A. A. 2012. USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In IMWUT/Ubicomp
2012
-
[37]
Zhou, K.; Liu, Z.; Qiao, Y.; Xiang, T.; and Loy, C. C. 2022. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence
2022
-
[38]
Zhou, K.; Yang, Y.; Qiao, Y.; and Xiang, T. 2021. Domain Generalization with MixStyle. In ICLR
2021
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.