{"id":"01928195-9331-4690-9169-c214192a95df","arxiv_id":"2412.12216","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A soft-voting ensemble of SVM, decision tree, and MLP classifies six sitting postures plus standing from Azure Kinect joint angles with 98.1% F1 on a new 33,409-sample dataset.","lead":"This paper builds a sitting posture detector that uses an Azure Kinect depth camera to track nine skeleton joints, computes joint angles, and classifies seven postures with an ensemble of machine learning models, reaching 98.1% F1. A generalist would read it to see how a commodity depth sensor plus a voting ensemble can monitor office sitting habits in real time.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Participant-level leakage in 5-fold CV likely inflates the central 98.1% F1; the paper must report leave-participant-out evaluation.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing concern: the 5-fold cross-validation may not be participant-independent. I agree with that assessment, and it is the single most important threat to the central F1 claim. The dataset is constructed from sequential frames of 36 participants, so the effective sample size for generalization to new users is at most 36, not 33,409. A frame-level split can memorize person-specific geometry, and neither the reported confusion matrices nor the two-tester deployment test provide an unbiased estimate of performance on unseen participants. I also considered the inconsistency between the 10 features described in Section III-D and the 13 features mentioned in Section III-E, and the apparently post-hoc selection of the nine angle features in Section III-C; both are worth clarifying, but they are secondary to the evaluation protocol. The paper does have independent support in the form of a cited public repository, clear system deployment details, and a latency measurement, which makes the engineering claim plausible even if the headline F1 needs re-estimation. Because the requested fix is a re-run with participant grouping rather than a rejection of the method, the existing CONDITIONAL verdict is appropriate; no verdict adjustment is needed.","tokens_in":32458,"tokens_out":3171,"duration_ms":30651,"concrete_test":"Obtain the dataset and training script from the GitHub repository cited in Section III-A. Re-run the soft-voting ensemble (SVM polynomial kernel C=1, C4.5 decision tree, MLP 200-100-25, soft voting) using GroupKFold(n_splits=5) with participant ID as the group, keeping all other preprocessing and hyperparameters identical. Report the weighted F1 and the per-participant F1 range. Also inspect the original split code: if it uses StratifiedKFold or train_test_split without a groups argument, the leakage is confirmed. If leave-participant-out F1 remains near 98.1% with small per-participant variance, the concern is retired.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (98.1% F1 for the soft-voting ensemble on 33,409 samples) rests on the evaluation protocol in Section IV-A. The text says only that 5-fold cross-validation was used and never states that the split was grouped by participant. Because the samples are consecutive per-second frames from only 36 participants, roughly 928 frames per participant, a random frame-level split places highly correlated frames from the same person in both training and test folds. With features limited to nine joint angles plus head depth, per-person geometry (height, weight, camera distance, habitual posture variants) is partly person-identifying; the model can exploit these person-specific offsets rather than learn posture semantics. The only out-of-sample check, described in Section IV-C, uses two new testers and reports raw misclassification counts, not F1, so it cannot independently support the headline number. The paper's own limitations in Section V (limited participant diversity, controlled environment) acknowledge generalizability concerns but do not address the leakage. Thus the condition that would make the 98.1% claim true, approximate sample independence, is violated by construction, and the F1 is not demonstrated to transfer to unseen participants.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents SitPose, a sitting-posture and sedentary-behavior monitoring system based on the Azure Kinect depth camera. The system tracks nine skeleton joints, computes nine joint-angle features plus one head-depth feature, and classifies seven postures (six sitting variants plus standing) using several machine-learning models. The authors collected a dataset of 33,409 samples from 36 participants across four environments, compared SVM, decision tree, MLP, GBDT, TabNet, and a soft-voting ensemble of SVM, DT, and MLP, and report that the ensemble achieves the highest F1 score of 98.1% under 5-fold cross-validation. They also describe a deployed system with a reported latency of about 97 ms and a small deployment test with two previously unseen testers.","tokens_in":32664,"tokens_out":2593,"duration_ms":26363,"significance":"If the reported evaluation is valid, the paper offers a practical, privacy-conscious depth-camera system for office posture monitoring, with a publicly released dataset and code, a relatively large participant pool of 36 individuals, and a balanced gender ratio. The comparison across several established classifiers and the deployment details are useful engineering contributions. However, the central 98.1% F1 claim rests on an evaluation protocol whose train/test independence is not established, and the deployment test with only two unseen testers provides only weak external validation. The paper's significance is therefore conditional on strengthening the evaluation: participant-independent cross-validation, variance reporting, and a clearer account of post-hoc feature selection.","major_comments":[{"comment":"The cross-validation protocol is not described as participant-independent. The dataset consists of consecutive per-second frames from only 36 participants, so a random 5-fold frame-level split places highly correlated frames from the same person in both training and test folds. With features limited to joint angles and head depth, person-specific geometry and habitual posture variants can be memorized, inflating the reported F1. The paper must state whether the split was grouped by participant and, if not, provide a leave-participant-out evaluation (or a grouped 5-fold split) to support the 98.1% claim.","section":"Section IV-A, Section IV-B"},{"comment":"The headline F1 of 98.1% is reported as a single point estimate with no confidence interval, per-fold variance, or per-class breakdown. Given that the F1 values of MLP, DT, and TabNet are all 'around 97%', the 1-point difference attributed to the ensemble may be within fold-to-fold fluctuation. Report the mean and standard deviation across folds, and consider pairwise significance testing or at least per-fold F1 values.","section":"Section IV-B, Figure 7"},{"comment":"The feature set appears to have been selected after experiments on the same dataset: the text states 'after experiments we select' nine angle pairs, and the head-depth feature was added because it was found 'very useful for distinguishing between left sitting and right sitting postures.' This post-hoc feature selection on the full dataset can bias the cross-validation estimate. The authors should either describe a nested or hold-out procedure that accounts for feature selection, or clearly report how many features were considered and how the selection was validated.","section":"Section III-C, Section III-D"},{"comment":"The paper calls the dataset 'very uneven' and uses this to justify F1 instead of accuracy, but the class counts listed in Section III-D (ranging from 4,259 to 5,474 out of 33,409) are nearly balanced. This internal inconsistency should be corrected, and the F1 formulation in Eq. (11), which weights by class proportion, should be reconciled with the stated concern about imbalance.","section":"Section IV-A"},{"comment":"The deployment test with two testers who did not participate in dataset collection is the only genuinely unseen-participant evaluation, but it is reported only as raw misclassification counts for a subset of postures (e.g., 3 instances of sitting straight misjudged as leaning forward, 2 as hunching over). No precision, recall, F1, or accuracy is computed from these test data, and the number of repetitions per posture is 30, which is small. This test cannot independently support the 98.1% headline; report full per-posture results for the deployment test.","section":"Section IV-D"}],"minor_comments":[{"comment":"There is a typo: 'noteble' should be 'notable'.","section":"Section III-B"},{"comment":"The text contains 'opreated' and 'opreating' for 'operated' and 'operating'; please proofread the deployment section.","section":"Section IV-C"},{"comment":"The phrase 'nine joint spatical vectors' should be 'nine joint spatial vectors'.","section":"Section III-C"},{"comment":"Reference [4] is cited as 'Cao et al . [4].' with an extra period; please fix the citation punctuation.","section":"Section I"},{"comment":"The F1 formula is a weighted (per-class) F1, but the notation uses w_i without defining it clearly until the following sentence; consider stating that w_i = n_i / N before the equation.","section":"Section IV-A, Eq. (11)"},{"comment":"The limitations paragraph is honest about participant diversity and controlled environments, but it does not address the evaluation-protocol issue; a sentence acknowledging the need for participant-independent validation would strengthen the paper.","section":"Conclusion, Section V"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable engineering benchmark for IEEE Sensors Journal, but the central quantitative claim is not yet demonstrated because the cross-validation split is not shown to be participant-independent, and the deployment test is too small to serve as an external check. I recommend the authors be asked to rerun or re-report the evaluation with grouped cross-validation and variance metrics. The post-hoc feature selection also needs a clearer validation procedure. These are fixable within the scope of the manuscript; hence major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know first: the main asset here is the dataset—33,409 depth-skeleton samples from 36 participants across four environments, covering seven postures. The system description is concrete, the latency measurement (97 ms) is plausible, and the comparison against SVM, DT, MLP, GBDT, and TabNet is honest work. If the data and code actually ship on GitHub, that alone makes the paper worth engaging with. The authors give real credit to prior Kinect-based posture work and do not oversell the method's novelty.\n\nThe soft spot is load-bearing: the 5-fold cross-validation in Section IV-A is not described as participant-independent. The data are consecutive per-second frames from only 36 people, roughly 928 frames each. A random frame-level split will put the same person's highly correlated frames in both training and test folds. With only nine joint angles plus head depth as features, the model can latch onto person-identifying geometry instead of posture semantics. So the 98.1% F1 is very likely optimistic. The only genuine out-of-sample check, two new testers in Section IV-D, reports raw misclassification counts rather than F1, and it does show a handful of errors, so it cannot support the headline figure. Also, the feature selection is post hoc: the nine angle pairs and the head-depth feature were chosen “after experiments” on the same dataset, which adds another layer of optimism.\n\nThere is a smaller internal inconsistency worth noting: Section IV-A calls the dataset “very uneven,” but the class counts in Section III-D range from 4,259 to 5,474, which is close to balanced. That is a minor factual slip, not a fatal one. No variance or confidence intervals are reported for the 98.1%, which would be easy to fix.\n\nThe conclusion's stated limitations—limited participant diversity, controlled environment—are honest, but they do not address the leakage problem.\n\nWho is this for? People building office ergonomics tools and researchers who want a labeled sitting-posture dataset. The paper is not a scientific breakthrough, but it is a solid systems paper with a useful benchmark if the eval is fixed. It deserves peer review, not desk rejection, because the dataset and the system are real and the evaluation gap is repairable. A referee should ask for leave-participant-out results, error bars, and a clarification of the feature-selection procedure before the F1 number is trusted.","headline":"A useful sitting-posture dataset and a clearly described system, but the 98.1% F1 headline is likely inflated by a non-participant-independent cross-validation split.","tokens_in":33227,"tokens_out":1305,"would_cite":true,"duration_ms":14607,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A depth camera measuring nine joint angles, classified by a soft-voting ensemble of SVM, decision tree, and MLP, identifies seven office postures with 98.1% F1 in real time.","keywords":["sitting posture detection","sedentary behavior","Azure Kinect","depth camera","ensemble learning","soft voting","joint angle features","real-time posture monitoring"],"falsifier":"Re-run the same soft-voting model with leave-one-participant-out cross-validation on the released dataset; if the F1 drops well below 98.1%, the reported result was inflated by frame-level leakage.","tokens_in":32225,"feed_emoji":"🪑","tokens_out":9465,"duration_ms":75757,"temperature":0.7,"pith_summary":"This paper aims to show that a single off-the-shelf depth camera, without RGB images or chair sensors, can detect poor sitting posture and sedentary behavior reliably enough for daily office use. The authors collected 33,409 frames of skeletal data from 36 participants in six sitting postures plus standing, and found that a soft-voting ensemble of SVM, decision tree, and MLP, built on nine joint-angle features plus head depth, reaches an F1 score of 98.1%. They claim this beats each base model as well as the GBDT and TabNet comparisons. If correct, the system would give office workers a privacy-preserving, real-time coach that warns after a minute of bad posture and after an hour of uninterrupted sitting. The practical stakes are the musculoskeletal and chronic-disease risks linked to prolonged sitting.","feed_headline":"Depth camera plus soft-voting trio hits 98.1% F1 on office postures","feed_subtitle":"Nine joint angles plus head depth classify seven office postures in about 97 ms, flagging bad sitting and 1-hour sedentary streaks.","key_machinery":"The load-bearing mechanism is soft voting over three heterogeneous base learners—SVM, decision tree, and MLP—combined with a deliberately small feature set: nine spatial angles between selected upper-body joints (computed with $\\arccos$ of normalized vector dot products, plus auxiliary reference points) and the depth value of the head. Soft voting means each base model outputs class probabilities and the ensemble chooses the class with the highest summed probability, so the three models' different inductive biases cancel each other's mistakes. The depth camera's body-tracking SDK supplies the 3D joint coordinates that feed these angles, and the standing class acts as a seated/not-seated gate for sedentary detection.","core_discovery":"The central claim is that sitting posture can be recognized from depth-sensor skeleton data alone, using only nine spatial angles between upper-body joints and the head's depth value as features. On the authors' dataset, the soft-voting ensemble of SVM, decision tree, and MLP achieves the highest F1 score of 98.1%, outperforming SVM (89.1%), GBDT (93.9%), and the around-97% scores of decision tree, MLP, and TabNet. The deployed system classifies each frame in roughly 97 ms, and it converts per-frame predictions into practical alarms: if an abnormal posture occupies more than 75% of a minute, the system pops up a reminder, and after one cumulative hour of sitting it warns the user to stand up. The paper also contributes a public dataset of 33,409 labeled samples and a deployed Windows interface that reports posture counts and sedentary durations.","pith_inferences":["Not tested in the paper: a participant-level split of the 33,409 frames would give a fairer estimate of how the 98.1% F1 transfers to unseen people; frame-level 5-fold cross-validation may leak the same person's correlated frames into both training and test folds.","The paper measures detection accuracy and latency, not behavior change; whether pop-up reminders actually reduce poor posture or sedentary time is an open question that a longitudinal deployment study could answer.","Because the features are geometric angles of the upper body, the approach should transfer to other depth cameras with skeleton APIs; a direct cross-sensor test would tell whether the 98.1% result is sensor-specific."],"forward_implications":["Office posture monitoring can run on a depth camera alone, without RGB images or pressure-sensor seats, preserving privacy and easing deployment.","The soft-voting ensemble reduces the confusions that individual models make between similar postures such as hunched over versus leaning forward, lifting F1 from about 97% to 98.1%.","The system can operate in real time, with 97 ms end-to-end latency under typical office software load, making per-minute posture summaries and one-hour sedentary reminders feasible.","The released dataset of 33,409 labeled frames gives a common benchmark for later sitting-posture research using depth sensors.","Including standing as a seventh class lets the system tell whether the user is actually seated, which is necessary for trustworthy sedentary-behavior tracking."],"supporting_citations":[{"why":"Establishes Azure Kinect's depth accuracy against earlier Kinects, supporting the sensor choice for joint tracking.","marker":"[6]"},{"why":"Documents the body-tracking SDK's accuracy for skeleton joints, which SitPose relies on for all features.","marker":"[7]"},{"why":"Provides the Azure Kinect skeletal-tracking API used to collect the 3D joint coordinates.","marker":"[16]"},{"why":"Supplies the ergonomics literature from which the six sitting postures are chosen.","marker":"[19]"},{"why":"LightGBM is one of the comparison algorithms the soft-voting ensemble is shown to beat.","marker":"[22]"},{"why":"TabNet is the other comparison algorithm the soft-voting ensemble is shown to beat.","marker":"[23]"}],"fun_headline_variants":["Ensemble depth sensing nails 98.1% F1 on sitting postures","Kinect + soft-voting AI: 98.1% F1 for office posture","Real-time posture AI hits 98.1% F1 with depth camera","Soft-voting trio reads posture from depth in 97 ms","Depth-camera ensemble: 98.1% F1 on sitting poses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline 98.1% F1 assumes that splitting the collected video frames randomly into training and test folds gives a fair measure of performance; if frames from the same person appear in both folds, the score can be artificially high.","fun_headline_variants_meta":{"raw":{"variants":["Ensemble depth sensing nails 98.1% F1 on sitting postures","Kinect + soft-voting AI: 98.1% F1 for office posture","Real-time posture AI hits 98.1% F1 with depth camera","Soft-voting trio reads posture from depth in 97 ms","Depth-camera ensemble: 98.1% F1 on sitting poses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000637,"raw_usage":{"total_tokens":2925,"prompt_tokens":924,"completion_tokens":2001,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":1899}},"tokens_in":540,"tokens_out":2001,"duration_ms":11611,"temperature":1.0,"reasoning_tokens":1899,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:01:20.303376+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same soft-voting model with leave-one-participant-out cross-validation on the released dataset; if the F1 drops well below 98.1%, the reported result was inflated by frame-level leakage.","supporting_citations":[{"cited_title":"Skeletal tracking on azure kinect,","cited_arxiv_id":null,"evidence_quote":"Provides the Azure Kinect skeletal-tracking API used to collect the 3D joint coordinates."},{"cited_title":"Application of machine learning approaches for classifying sitting posture based on force and acceleration sensors,","cited_arxiv_id":null,"evidence_quote":"Supplies the ergonomics literature from which the six sitting postures are chosen."},{"cited_title":"Skeleton tracking accuracy and precision evaluation of kinect v1, kinect v2, and the azure kinect,","cited_arxiv_id":null,"evidence_quote":"Documents the body-tracking SDK's accuracy for skeleton joints, which SitPose relies on for all features."},{"cited_title":"Evaluation of the azure kinect and its comparison to kinect v1 and kinect v2,","cited_arxiv_id":null,"evidence_quote":"Establishes Azure Kinect's depth accuracy against earlier Kinects, supporting the sensor choice for joint tracking."},{"cited_title":"Lightgbm: A highly efficient gradient boosting decision tree,","cited_arxiv_id":null,"evidence_quote":"LightGBM is one of the comparison algorithms the soft-voting ensemble is shown to beat."},{"cited_title":"Tabnet: Attentive interpretable tabular learning,","cited_arxiv_id":null,"evidence_quote":"TabNet is the other comparison algorithm the soft-voting ensemble is shown to beat."}],"review_version":1}