{"id":"78b3327d-186d-42b2-ba83-a438a6389afa","arxiv_id":"2412.02725","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"emg2pose introduces the largest public sEMG hand pose dataset, 193 users with motion-capture labels, plus baselines and held-out user/stage generalization tasks.","lead":"The paper releases emg2pose, a large open dataset pairing wrist muscle signals (sEMG) with hand pose labels from motion capture, spanning 193 people and roughly 185 unique hours of recording. It also provides baselines and benchmark tasks for testing how well models generalize to new people and new hand movements.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"IK labels lack independent gold-standard validation, and the 12.7% failed-frame censoring may bias benchmark scores if failures are concentrated in hard, occlusion-heavy stages.","rationale":"The paper's strongest, most defensible contribution is scale: the explicit dataset statistics, per-hand frame counts, and comparisons to prior sEMG datasets make the 'largest publicly available sEMG hand pose dataset' claim plausible even with both hands counted separately. What would have to be true for the full central claim to hold is that the IK-based joint angles are high-quality ground truth. The reader's weakest assumption identified this exactly. I pressed the concern further by connecting the 12.7% IK failure rate to the stated evaluation rule that unavailable motion-capture time-points are skipped. Because the paper itself names occlusion and fist clenching as label-quality risks, a non-random failure pattern would censor difficult test frames and inflate all reported held-out errors, while also making the 29-stage diversity less complete than advertised. This is a correctness/completeness risk, not a disagreement with community consensus, and it is testable through independent label reconstruction. The concern does not change the reader's verdict: CONDITIONAL remains appropriate, since the dataset may still be valuable and largest in scale, but acceptance should be conditioned on an independent label-validation analysis or release of per-stage failure statistics.","tokens_in":22649,"tokens_out":5065,"duration_ms":54016,"concrete_test":"Stratified independent-label audit: sample roughly 100 stage recordings spanning all 29 stages and both hands; reconstruct joint angles from the raw 26-camera marker trajectories with an independent IK/calibration pipeline; compute per-joint mean absolute error between released labels and the independent reconstruction, and report per-stage failure rates. Then re-run the released vemg2pose tracking baseline on recovered frames that were previously skipped due to IK failure. If recovered-frame error exceeds reported test error by a material margin (e.g., more than 1 degree angular MAE), the skipped-frames censoring is confirmed and the reported benchmark scores are biased.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that emg2pose provides 'high-quality hand pose labels' (Abstract, Section 1) depends on the motion-capture IK pipeline being accurate enough to serve as ground truth. That accuracy is never established. Section 3.2 reports the IK solver failed for 12.7% of frames, typically due to simultaneously occluded markers, and Section 3.5 says time-points for which motion capture data are not available are skipped during training and evaluation. Appendix B.4 concedes that the camera-based method still suffers from occlusion, hindering label quality for gestures such as fist clenching. If IK failures are not uniformly random across stages, this evaluation protocol censors exactly the hardest, occlusion-heavy frames, making reported held-out errors optimistic and reducing the effective kinematic diversity of the benchmark. No independent gold-standard comparison is provided; the only reported accuracy figure (0.32 degrees filtered vs. unfiltered, Appendix A) measures smoothing, not IK error. The scale claim is plausible, but the label-fidelity claim is load-bearing and currently unvalidated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces emg2pose, a large benchmark dataset of wrist surface electromyography (sEMG) and hand pose labels obtained from a 26-camera motion capture rig. The dataset is reported to span 193 users, 370 hours, 29 kinematic stages, and 80M labeled frames, with pose labels given as joint angles. The authors provide three baselines (NeuroPose, SensingDynamics, and a new velocity-based vemg2pose model), define regression and tracking tasks, and evaluate generalization to held-out users, held-out stages, and held-out user-stage combinations. The paper also includes a datasheet, discussion of limitations, and links to code and data.","tokens_in":22891,"tokens_out":4540,"duration_ms":45316,"significance":"If the dataset and labels are as described, this is a valuable contribution to sEMG-based hand pose estimation and to benchmark research more broadly. The strongest assets are the scale relative to prior open sEMG datasets, the explicit three-axis generalization evaluation (users, stages, and user-stage combinations), the release of code and baseline models, and the unusually detailed documentation of collection, consent, preprocessing, and limitations. The paper is transparent about several weaknesses, including occlusion-induced label degradation, the default hand model used for landmark metrics, and the absence of seed variation in reported results. However, two issues need attention: the accuracy of the IK-derived pose labels is not validated against an independent gold standard, and the headline scale figures double-count the two hands. These issues affect the central claims and should be addressed before publication.","major_comments":[{"comment":"The central claim that emg2pose provides \"high-quality hand pose labels\" is not yet supported because the motion-capture inverse kinematics labels are never validated against an independent gold standard. Appendix A reports a 0.32 degree difference between filtered and unfiltered signals, which quantifies smoothing, not IK accuracy. Section 3.2 states that the IK solver failed on 12.7% of frames, typically due to simultaneously occluded markers, and Section 3.5 says those time-points are skipped during training and evaluation. Since Appendix B.4 concedes that occlusion \"hinders label quality for gestures such as fist clenching,\" the skipped frames are plausibly concentrated in the hardest kinematic conditions; if so, the reported held-out errors are optimistic and the effective kinematic diversity is reduced. I request an independent validation study on a subset (e.g., manual marker annotation, a second sensing modality, or synthetic marker-dropout analysis), a per-stage report of failure rates, and an evaluation of how results change when failed frames are handled differently.","section":"Section 3.2, Appendix A, Appendix B.4"},{"comment":"The headline scale figures double-count the two hands. Section 3.2 notes that hours count left- and right-hand data separately, and Table 2 lists both \"per hand\" and \"across hands\" frame counts, but the abstract and introduction use the per-hand number (370 hours, 80M frames) without qualification. Unique hours are roughly 185 and unique frames roughly 40M. The largest-sEMG-dataset claim probably survives (Atzori et al. [2014] reports 37 hours), but the comparison with CV datasets such as Sener et al. [2022] (111M frames) is misleading if 80M is used. Please state unique-count figures in the abstract and introduction, or explicitly define 370 hours and 80M frames as per-hand totals.","section":"Section 1, Table 2, Section 3.2"}],"minor_comments":[{"comment":"There are typos: \"sEMG meaures\" and \"between between predicted and ground truth fingertip locations.\"","section":"Section 3.5"},{"comment":"The stated stage duration is inconsistent: Section 3.2 says 45–120 s, Appendix A says 30–120 s, and Appendix B.2.1 says 45–60 s (freeform 60–120 s). Please reconcile.","section":"Appendix A, Section 3.2, Appendix B.2.1"},{"comment":"The name \"emg2pose\" is used both for the dataset and for the positional (non-velocity) baseline model in Table 7, which is confusing next to \"vemg2pose.\"","section":"Section 3.5 and Table 7"},{"comment":"The paper states that seed variance is negligible but does not report the underlying numbers; adding a sentence with the observed spread across seeds would strengthen reproducibility.","section":"Table 4 and Checklist 3(c)"}],"recommendation":"major_revision","confidential_remarks":"The main correctness risk is the unvalidated IK label accuracy and the resulting censoring of hard, occlusion-heavy frames; this is fixable with additional analysis. The double-counting of hours and frames in the headline is also fixable by rewording. I do not see problems with the citation pattern: the reliance on CTRL-labs at Reality Labs et al. [2024] is justified by the hardware connection, and the novelty disclosure appears adequate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead emg2pose. The dataset is real and the scale claim survives scrutiny once you do the arithmetic. 193 users and roughly 185 unique hours of wrist sEMG with motion-capture hand pose labels is the largest open resource in the field by a wide margin. The genuinely new piece is the held-out-stage generalization benchmark: no prior sEMG dataset tests whether a model can handle kinematic categories it never saw, and the paper shows performance degrades substantially when stages are excluded. That is a useful contribution.\n\nThe paper also does many small things right. The splits are thoughtful (held-out users, stages, and user-stage combinations), the baselines include open reimplementations of two established architectures plus a new velocity-based model that makes good sense for sEMG, and the ablation on training-set scale directly shows the value of the large dataset. The limitations section is honest about the default hand model for landmark metrics, the lack of wrist tracking, and the fact that occlusion still degrades label quality for some gestures.\n\nThe main soft spot is label validation. The abstract calls these \"high-quality hand pose labels,\" but the only quantitative support is a 0.32 degree difference between filtered and unfiltered joint angles, which measures smoothing, not IK accuracy. The 12.7% frame failure rate is disclosed, and those frames are skipped, but the paper never checks whether failures are concentrated in occlusion-heavy stages. If they are, the benchmark silently censors the hardest frames. That is a real gap, and it tempers the \"high-quality\" claim. I would like to see a small validation study, e.g., comparing IK results to a different tracking modality on a subset of participants.\n\nThe second issue is presentation. The abstract and introduction report \"370 hours\" and \"80M labelled frames,\" yet Table 3 notes that hours count both hands separately, and Table 2 shows the across-hands figure is 40M frames. So the headline numbers are double-counted. The dataset is still large at 185 hours, so this is easy to correct in revision, but the current framing overstates.\n\nThe paper is worth serious refereeing. The dataset fills a real gap, the benchmark design is sound, and the authors are transparent about limitations. My recommendation is to send it out, with a request for the label-validation analysis and corrected headline numbers. The central contribution does not depend on those fixes, but they determine whether the \"high-quality\" claim is defensible.","headline":"A genuinely large sEMG pose dataset with a novel held-out-stage benchmark; the label-validation gap and double-counted headline numbers are real but fixable.","tokens_in":23450,"tokens_out":2975,"would_cite":true,"duration_ms":29579,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"emg2pose releases a 193-user, 370-hour paired wrist-sEMG and hand-pose dataset, with held-out benchmarks showing generalization improves as user and behavior diversity grow.","keywords":["emg2pose","surface electromyography","hand pose estimation","benchmark dataset","domain generalization","wrist-worn sEMG","motion capture labels","pose regression"],"falsifier":"Take a random sample of several hundred frames from occlusion-heavy and fist-clench stages, have annotators independently label joint angles using a different capture method, and compare them with the dataset's inverse-kinematics angles; if the median per-joint difference approaches the 7 to 15 degree errors reported for the models, the benchmark's headline numbers mostly measure label noise.","tokens_in":22492,"feed_emoji":"🖐️","tokens_out":10969,"duration_ms":97429,"temperature":0.7,"pith_summary":"The paper's aim is to remove the data bottleneck that keeps sEMG hand-pose models from working for people and gestures they were not trained on. It releases emg2pose, claimed to be the largest public dataset pairing wrist sEMG with hand-pose labels: 193 users, 370 hours, 751 sessions, 29 movement stages, and 80 million labelled frames, a scale it argues is comparable to major computer-vision hand datasets. Alongside the data it defines three held-out generalization tasks (unseen users, unseen stages, and unseen user-stage combinations) and reports baselines, including a new velocity-predicting model whose errors shrink as training users and stages increase. A sympathetic reader would care because this turns sEMG-to-pose from a small-data personalization problem into a community-scale benchmark, bringing the field to the scale standards that have driven vision-based hand tracking.","feed_headline":"193 users, 370 hours of wrist sEMG get hand-pose labels","feed_subtitle":"New benchmark pairs wrist muscle signals with motion-capture hand poses and tests generalization to unseen users and movements.","key_machinery":"The load-bearing mechanism is the paired recording pipeline: a 16-channel, 2 kHz wrist sEMG band worn simultaneously with 19 reflective markers per hand tracked by 26 cameras, whose 3D positions are converted by an inverse-kinematics solver with a personalized hand model into 20 joint-angle degrees of freedom, then filtered and resampled to 2 kHz. On the modelling side, vemg2pose carries the baseline results: a causal strided convolutional featurizer built from time-depth separable convolutions turns sEMG into features at 50 Hz, and an autoregressive LSTM predicts joint angular velocities that are integrated into angles; for tracking the ground-truth initial pose seeds the integrator, while for regression the first 250 ms of angles are also predicted. The velocity representation is what lets one model handle both tasks and keeps predictions smooth.","core_discovery":"The central claim is that a dataset large and diverse enough to span user anatomy, sensor placement, and hand kinematics makes it possible to learn continuous hand pose from wrist muscle signals for people and movement types never seen in training. The paper reports 193 users, 370 hours, 29 stages, and 80 million labelled frames, with 16-channel 2 kHz sEMG synchronized to 26-camera motion-capture labels within 10 ms; it calls this the largest open sEMG pose dataset and comparable in scale to large vision hand datasets. On held-out users, stages, and user-stage combinations, the velocity-based vemg2pose baseline achieves mean joint-angle errors of about 12.2, 15.2, and 15.8 degrees in regression and 7.7, 11.2, and 11.0 degrees in tracking, beating reimplementations of prior sEMG pose networks. Scale experiments show held-out error decreasing as training users or stages are added, which the authors present as evidence that breadth across these axes, not just dataset size, drives generalization.","pith_inferences":["If the inverse-kinematics labels contain noise of the same order as the reported errors, and the solver failed on 12.7 percent of frames without an independent gold-standard check, then part of the measured error is label error; an independent label audit on a few hundred frames would separate the two.","The velocity-integration design hints that other derivative-sensing wearables, such as inertial, ultrasound, or impedance sensors, could borrow the same architecture whenever the measurement responds to movement rather than to static pose.","The user-plus-stage held-out split could be adopted as a general time-series domain-generalization benchmark beyond sEMG, because it cleanly separates shift in the signal source from shift in the output behavior.","The paper's own limitation notes imply that adding real-world signal aggressors such as sweat, electrode contact changes, and muscle fatigue, as well as wrist tracking, will be needed before the benchmark reflects in-the-wild performance; those are testable extensions rather than demonstrated results."],"forward_implications":["On emg2pose's held-out splits, sEMG alone supports continuous hand-pose estimation for unseen users at roughly 12.2 degrees mean joint-angle error in regression and 7.7 degrees in tracking, with the velocity model outperforming both reimplemented prior architectures.","The three test sets let researchers measure generalization to new anatomy, new kinematics, and both at once; the user-plus-stage condition is the paper's proposed proxy for real-world deployment.","Dataset scale is shown to be causally linked to generalization: subsampling training users or stages degrades held-out performance, so further scaling along these axes should keep reducing error.","Stages designed to confound vision systems, including occlusion and hand-hand or hand-object interaction, do not degrade sEMG tracking, indicating the modality covers cases where cameras fail.","The open benchmark and baselines give the community a shared platform for exploring sequence models, probabilistic decoding, and personalization for biosignal interfaces."],"supporting_citations":[{"why":"Supplies the sEMG-RD wrist band hardware used for collection and evidence that this device supports generalized sEMG classification across thousands of users.","marker":"[CTRL-labs at Reality Labs et al., 2024]"},{"why":"Provides the marker-based hand tracking and personalized inverse-kinematics pipeline that generates the dataset's joint-angle labels.","marker":"[Han et al., 2018]"},{"why":"Defines the NeuroPose architecture and the earlier MyoBand pose-regression setting that emg2pose extends and reimplements as a baseline.","marker":"[Liu et al., 2021]"},{"why":"Introduces the SensingDynamics high-density sEMG pose-regression method reimplemented as the second baseline.","marker":"[Sîmpetru et al., 2022a]"},{"why":"The largest prior open sEMG dataset with pose regression labels; it is the scale and pose-label comparison emg2pose is designed to surpass.","marker":"[Atzori et al., 2014]"},{"why":"Contributes the time-depth separable convolution block used in the vemg2pose featurizer.","marker":"[Hannun et al., 2019]"},{"why":"Motivates the 15 Hz low-pass filter applied to the joint angles to remove tracking jitter.","marker":"[Ingram et al., 2008]"},{"why":"Provides one of the large vision hand and activity datasets used in the frame-count and subject-count scale comparison.","marker":"[Sener et al., 2022]"},{"why":"Provides the other large-scale vision dataset used to argue emg2pose matches the scale of computer-vision hand datasets.","marker":"[Yu et al., 2020]"}],"fun_headline_variants":["Largest sEMG hand-pose dataset: 193 users, 370 hrs","Wrist muscle signals decode hand poses for unseen users","sEMG benchmark: 80M frames, 29 stages, 193 users","From wrist EMG to hand pose: a benchmark for generalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The motion-capture pipeline that turns marker positions into joint angles is accurate enough to serve as ground truth for all 193 users, even though the solver failed on 12.7 percent of frames and its angle estimates were never checked against an independent gold standard.","fun_headline_variants_meta":{"raw":{"variants":["Largest sEMG hand-pose dataset: 193 users, 370 hrs","Wrist muscle signals decode hand poses for unseen users","sEMG benchmark: 80M frames, 29 stages, 193 users","From wrist EMG to hand pose: a benchmark for generalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000956,"raw_usage":{"total_tokens":4117,"prompt_tokens":1027,"completion_tokens":3090,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":3011}},"tokens_in":643,"tokens_out":3090,"duration_ms":21964,"temperature":1.0,"reasoning_tokens":3011,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:53:31.769723+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of several hundred frames from occlusion-heavy and fist-clench stages, have annotators independently label joint angles using a different capture method, and compare them with the dataset's inverse-kinematics angles; if the median per-joint difference approaches the 7 to 15 degree errors reported for the models, the benchmark's headline numbers mostly measure label noise.","supporting_citations":[],"review_version":1}