{"id":"aa3d0a3a-8ecb-434b-92b0-bbd0b3572c3a","arxiv_id":"2506.01757","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Sampling RGB at 10Hz and 3D hand pose at 30Hz preserves egocentric action recognition accuracy while reducing CPU usage by roughly 3x compared with all-30Hz input.","lead":"This paper tests how often a wearable device needs to sample RGB video and 3D hand poses to recognize actions correctly, and measures the CPU cost of each setup. It finds that dropping the RGB rate to 10Hz while keeping hand pose at 30Hz keeps accuracy nearly unchanged while cutting CPU usage by about 3x.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'nearly same F1-score' claim in §3.3 rests on single-run point estimates with no error bars; on the small H2O validation set, run-to-run variance could exceed the reported accuracy gap, so the central trade-off is not yet established.","rationale":"The paper is a clearly presented empirical study with a plausible qualitative finding: downsampling the expensive RGB stream while keeping the cheap hand pose stream at high frequency can preserve accuracy and cut CPU time. The CPU scaling with RGB frame rate is internally consistent with the architecture description, since LeViT inference dominates the hand pose MLP. However, the central quantitative claim has two pillars: accuracy preservation and CPU reduction. The CPU pillar is a relative measurement on a server CPU, and while the reader's hardware-proxy concern is legitimate, it is secondary because the relative reduction would likely transfer if RGB dominates on-device. The accuracy pillar is the more load-bearing weakness: Figure 2 shows single macro-F1 points with no error bars, and H2O is a small dataset, so the 'nearly the same' comparison could be run-to-run noise. This concern affects the empirical validity of the headline claim regardless of deployment hardware, and it can be settled directly by a multi-seed experiment. The reader's CONDITIONAL verdict already captures the need for reproducibility and on-device validation; my stress test reinforces the accuracy-variance issue as the most fundamental unverified assumption, so the verdict should remain CONDITIONAL pending the concrete check.","tokens_in":4832,"tokens_out":6984,"duration_ms":72693,"concrete_test":"Retrain the four key configurations (30/30, 10/30, 10/10, and RGB-only 30 Hz) with at least five independent seeds under the same training protocol, and report the macro F1 mean and standard deviation on the H2O validation set. If the 10/30 mean is within one standard deviation of the 30/30 mean, the 'nearly the same F1-score' claim holds; if the gap exceeds the noise, the efficiency recommendation must be re-qualified. Additionally, publish the numeric CPU-usage values for the one-second inference window, since Figure 2's log scale makes the 'approximately 3×' reduction impossible to verify numerically.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.3, the headline result that configurations using 10 Hz RGB and 10–30 Hz hand pose 'achieve nearly the same F1-score' as the full 30 Hz RGB / 30 Hz hand pose setup, while reducing CPU usage by approximately 3×, is supported only by single point estimates in Figure 2. The paper reports no seeds, confidence intervals, or standard deviations for the macro F1 scores, and the H2O validation set is small. Run-to-run training variance could easily be larger than the accuracy gap that the 'minimal to no loss' statement depends on. Without a noise characterization, the accuracy-preservation half of the claim is not a measurable statement. The CPU reduction is a relative measurement on a single thread of an AMD EPYC 7742 and may transfer to XR hardware if RGB inference dominates, but the CPU half is secondary: if the F1 gap is within noise, the entire efficiency argument becomes unverifiable. The reader's hardware-proxy concern is valid but less fundamental, because the relative CPU scaling is plausible and the paper's central contribution is the accuracy-efficiency trade-off.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical study of egocentric action recognition (EAR) on the H2O dataset using a two-stream architecture: an RGB stream based on a LeViT-256-distilled feature extractor with a Temporal MLP, and a 3D hand-pose stream based on an MLP feature extractor with a Temporal MLP. The authors systematically vary the sampling frequencies of RGB (30/10/3/1 Hz) and hand pose (30/10/3/1 Hz) inputs, measure macro F1-score and CPU usage on a single thread of an AMD EPYC 7742, and report that configurations with 10 Hz RGB and 10–30 Hz hand pose achieve nearly the same F1-score as the full 30 Hz RGB + 30 Hz hand pose setup while reducing CPU usage by approximately 3×. The paper concludes that modality-aware sampling rates are a promising efficiency lever for real-time EAR on resource-constrained XR devices.","tokens_in":5021,"tokens_out":4174,"duration_ms":43305,"significance":"If the central trade-off claim is reliable, the paper identifies a simple and actionable design insight: keeping the cheap hand-pose stream at high frequency while downsampling the expensive RGB stream can preserve accuracy while cutting computational cost. The study is clearly motivated by a real deployment constraint (Magic Leap 2) and the plotted trend in Figure 2 is internally consistent: RGB frequency dominates CPU usage, and the hand-pose stream mitigates accuracy loss at low RGB rates. The paper is also commendably concrete in reporting CPU usage per configuration, which makes the efficiency trend directly visible. However, the significance is currently limited by methodological weaknesses: the accuracy-preservation half of the claim rests on single-run point estimates with no error characterization, and the CPU numbers come from a server CPU rather than the target XR device. These issues are fixable within the manuscript's scope, but they must be addressed before the central claim can be considered established.","major_comments":[{"comment":"The headline claim that configurations with 10 Hz RGB and 10–30 Hz hand pose achieve 'nearly the same F1-score' as the full 30/30 setup is supported only by single point estimates in Figure 2. The paper reports no multiple seeds, standard deviations, confidence intervals, or significance tests, and it does not state the size of the H2O validation set used. On a small validation set, run-to-run training variance can easily exceed the reported accuracy gap, so the 'minimal to no loss' statement is not currently a measurable claim. Please report the mean and variance over at least three to five runs, or otherwise provide a statistical characterization (e.g., confidence intervals for the macro F1 difference), and state the validation set size and label distribution.","section":"§3.3, Figure 2"},{"comment":"CPU usage is measured on a single thread of an AMD EPYC 7742 server CPU, yet the paper motivates the study by deployment on Magic Leap 2, which has a mobile/embedded CPU with different relative costs for a ViT (LeViT) versus an MLP hand-pose model. The reported 3× CPU reduction depends on the assumption that RGB feature extraction dominates the total cost on the target hardware. Because the paper does not validate this proxy on-device, the practical efficiency claim may not transfer to the intended platform. Please either include on-device measurements (even a small subset) or add a clear discussion of how the relative CPU cost might differ on embedded CPUs and how that would affect the reported trade-off.","section":"§2.2, Figure 2 caption"},{"comment":"The manuscript omits many details needed to reproduce the experiments: training hyperparameters (number of epochs, batch size, optimizer, learning rate schedule), GPU data-parallel setup, the exact mechanism for reducing sampling frequencies (frame skipping versus retraining with shorter sequences), the hand-pose keypoint representation (which keypoints, how many, coordinate frame, and whether they are ground-truth or predicted), and the H2O train/validation split and preprocessing specifics. For an empirical study whose conclusions are quantitative trends, these omissions make it impossible for a reader to assess the robustness of the results or to reproduce the curves in Figure 2. Please add a dedicated experimental-details section or supplementary material addressing these points.","section":"§2.1–§2.4"}],"minor_comments":[{"comment":"The caption states 'over a one-second input window,' but Section 3 says all sequence models were trained on 2-second sequences (60 time steps at 30 Hz). Please clarify whether CPU usage is measured per second of input or per full sequence, and ensure consistent wording.","section":"Figure 2 caption"},{"comment":"The sentence 'For a fixed fHP, the F1-score declines more sharply as fRGB decreases, though the hand pose stream helps mitigate this drop' is confusing: if the hand-pose stream mitigates the drop, the decline should be less sharp than in the RGB-only case, not more. Please rephrase to compare the multimodal model against the RGB-only model explicitly.","section":"§3.3"},{"comment":"The F1 scores of the single-frame models (RegNet, LeViT-256, LeViT-256-distilled, HP-MLP, FusionNet) are not reported numerically; only qualitative statements are given. Include a table with the exact macro F1 and CPU usage values for these models so readers can contextualize the sequence-model results.","section":"§3.1"},{"comment":"The sentence 'only shared augmentations are applied' is vague. Specify which augmentations are considered shared (e.g., random cropping, flipping, rotation) and which are modality-specific, as this affects the cross-modal consistency claim.","section":"§2.4"},{"comment":"The paper uses 'macro F1-score' but does not define it; please state that it is the unweighted average of per-class F1 scores, since the class distribution may be imbalanced in the H2O dataset.","section":"§3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short empirical study that could be valuable to the on-device EAR community, but the central claim currently lacks statistical validation and on-device hardware evidence. The absence of error bars and the server-CPU proxy are fixable with additional experiments; I do not see a fundamental methodological flaw. I recommend major revision rather than rejection, and I suggest the editor ensure the revised version includes reproducibility details and either on-device measurements or a carefully scoped caveat."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a small, clearly-written empirical paper on sampling rates for RGB + 3D hand pose in egocentric action recognition. The genuinely new thing is the systematic joint sweep of both rates; prior work combining these modalities didn't vary sampling frequencies. The main finding—10 Hz RGB with 10–30 Hz hand pose gives roughly the same macro F1 as 30 Hz/30 Hz while cutting CPU about 3x—is plausible and nicely presented in Figure 2. If true, it's a useful design lever for XR glasses.\n\nWhat it does well: the architecture is simple (LeViT-256-distilled + Temporal MLP for RGB, MLP + Temporal MLP for hand pose), the experiments cover a reasonable grid of frequencies, and the distinction between RGB-only and multimodal is clean. The CPU scaling story (roughly 3x reduction per 3x drop in RGB rate) is consistent and makes sense.\n\nWhere it's soft: the headline accuracy claim rests on single-run point estimates. No seeds, no error bars, no repeated experiments. The H2O validation set is small, and run-to-run variance could easily exceed the accuracy gap between 10 Hz RGB + 30 Hz hand pose and the full-rate setup. Without a noise characterization, 'nearly the same F1-score' is not a measurable statement. That's the load-bearing problem, not the CPU proxy. The server-CPU measurement (single-thread AMD EPYC) is a legitimate concern for deployment, but as a relative measure it's more defensible; the bigger issue is that the entire efficiency argument depends on the accuracy-preservation half being real.\n\nAlso missing: training/architecture details (sequence window, hyperparameters, temporal MLP specifics), and no code or reproduction protocol. That's fixable but should be required before this is cited as fact.\n\nBottom line: this is work worth engaging with, not dismissing. The idea is sound and the empirical direction is right, but the paper is not yet at the level where the quantitative claim can be trusted. For peer review: yes, send it out—a good referee can push for variance info, implementation details, and ideally on-device CPU numbers. If the authors supply those, this becomes a solid modest contribution; if not, it stays a preprint with a promising but unverified result.","headline":"A useful but under-supported empirical trade-off map: the claimed 3x CPU saving at matched accuracy needs error bars and on-device validation before it can be relied on.","tokens_in":5613,"tokens_out":2274,"would_cite":true,"duration_ms":21077,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that egocentric action recognition can cut CPU usage by about 3x by feeding hand pose at 30 Hz while dropping RGB to 10 Hz, with nearly no loss in F1-score.","keywords":["egocentric action recognition","multimodal learning","sampling frequency","3D hand pose","vision transformer","temporal MLP","CPU efficiency","wearable XR devices"],"falsifier":"Measure inference time and CPU usage of the same two-stream models on an actual wearable or embedded CPU, and compare the 30 Hz RGB + 30 Hz hand pose configuration with the 10 Hz RGB + 30 Hz hand pose configuration; if the latter does not show roughly a 3x CPU reduction at near-equal macro F1, the paper's central efficiency claim is not portable to its target hardware. A second check is reproducing the frequency sweep on the test set to confirm the F1 plateau at 10 Hz RGB.","tokens_in":4594,"feed_emoji":"⚡","tokens_out":5273,"duration_ms":50890,"temperature":0.7,"pith_summary":"This paper claims that in egocentric action recognition, the heavy RGB video stream can be sampled far less often than usual if a light 3D hand-pose stream is kept at a higher rate, and that this preserves recognition accuracy while cutting CPU usage by about 3x. The authors build a two-stream model on RGB frames and 3D hand keypoints, then systematically vary the sampling frequencies of both modalities on a two-hand manipulation dataset. Their central finding is that 10 Hz RGB combined with 10–30 Hz hand pose matches the F1-score of the full 30 Hz RGB and 30 Hz hand pose configuration while using roughly a third of the CPU. The practical reason to care is that wearable AR glasses lack dedicated hardware acceleration, so a modality-aware sampling strategy offers a cheap efficiency lever for real-time on-device action recognition.","feed_headline":"RGB down to 10 Hz keeps accuracy, cuts CPU 3x","feed_subtitle":"For egocentric action recognition, keeping hand pose at 30 Hz while dropping RGB to 10 Hz preserves F1 and triples efficiency.","key_machinery":"The central object is a two-stream 'Multimodal Temporal MLP' (MM-TMLP): an RGB stream built on a LeViT vision-transformer feature extractor followed by a Temporal MLP, and a hand-pose stream built on an MLP feature extractor paired with a second Temporal MLP; the final-time-step outputs of the two streams are concatenated and classified. The mechanism doing the work is the relative sampling frequency of the two modalities. Because the RGB stream is the computationally expensive branch and the hand-pose stream is cheap, holding hand pose at a high rate while downsampling RGB shifts the compute budget without losing the motion cues that hand keypoints provide.","core_discovery":"On the paper's own terms, the discovery is a measured trade-off curve: for a fixed hand-pose frequency, lowering the RGB sampling rate causes a gradual, not catastrophic, F1 decline, while CPU usage falls roughly threefold with each threefold reduction in RGB frequency; and for a fixed RGB frequency, adding the hand-pose stream improves F1 regardless of its sampling rate. The headline configuration, 10 Hz RGB with 10–30 Hz hand pose, reaches nearly the same macro F1-score as the fully sampled 30 Hz RGB + 30 Hz hand pose model while reducing CPU usage by approximately 3x. This is presented as evidence that multimodal input strategies, with frequency as a design lever, make efficient real-time egocentric action recognition feasible on resource-constrained wearable devices.","pith_inferences":["If the CPU-cost ratios transfer to mobile CPUs, the same sampling lever could let other high-cost modalities (e.g., higher-resolution RGB or optical flow) be downsampled while a cheap modality such as gaze, head pose, or audio carries the temporal signal—an extension the paper does not test.","The 3x CPU gain is measured on a server CPU single thread, not on the target wearable; an on-device benchmark is the natural next experiment, since the relative cost of the ViT versus the hand-pose MLP may differ on embedded hardware.","Because the effect is about a cheap stream compensating for a slow expensive stream, the result may generalize beyond egocentric action recognition to any multimodal sequence task with heterogeneous per-modality costs, though that generalization is speculative.","A direct testable extension: on the same dataset, replacing hand pose with an equally cheap modality (e.g., head-gaze keypoints) and re-running the frequency sweep would show whether the compensation is specific to hand motion or general to any auxiliary stream."],"forward_implications":["A 10 Hz RGB stream with 10–30 Hz hand pose can replace a 30 Hz RGB + 30 Hz hand pose setup on resource-limited hardware with nearly unchanged macro F1.","CPU usage scales roughly threefold with each threefold change in RGB sampling frequency, so deployment can pick an operating point on the accuracy–cost curve.","For any fixed RGB frequency, adding hand pose improves F1 regardless of hand-pose sampling frequency, so the hand-pose branch is a reliable accuracy supplement.","RGB-only sequence models lose accuracy more sharply as RGB frequency drops than multimodal models do, indicating that hand pose cushions the loss.","Modality-aware sampling is a practical alternative to model compression for reducing on-device CPU cost."],"supporting_citations":[{"why":"Supplies the two-hand manipulation dataset and action labels used for all training and evaluation.","marker":"[7]"},{"why":"Provides the LeViT vision-transformer feature extractor whose computational cost motivates RGB downsampling.","marker":"[4]"},{"why":"Inspires the Temporal MLP architecture that models temporal dependencies in both streams.","marker":"[2]"},{"why":"Distillation produces the LeViT-256-distilled model chosen as the RGB feature extractor.","marker":"[5]"},{"why":"Prior work combining RGB with 3D hand poses that the paper builds on.","marker":"[11]"},{"why":"The two-stream design pattern for video action recognition that underlies the architecture.","marker":"[13]"},{"why":"RegNet serves as the alternative RGB backbone baseline for the accuracy versus CPU trade-off.","marker":"[10]"},{"why":"Evidence that more modalities do not always help, motivating the targeted two-stream choice.","marker":"[16]"}],"fun_headline_variants":["10 Hz RGB, 30 Hz pose: 3x CPU cut, F1 preserved","Egocentric action: 10 Hz RGB, 30 Hz pose, 3x CPU drop","Slow RGB, fast hand pose: 3x CPU savings in egocentric recognition","Multimodal trick: slow RGB, fast pose, 3x CPU efficiency","F1 intact with 10 Hz RGB and 30 Hz pose, CPU down 3x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that CPU usage measured on a single thread of a server CPU faithfully represents the relative computational cost of the RGB and hand-pose streams on the wearable device where the system is meant to run.","fun_headline_variants_meta":{"raw":{"variants":["10 Hz RGB, 30 Hz pose: 3x CPU cut, F1 preserved","Egocentric action: 10 Hz RGB, 30 Hz pose, 3x CPU drop","Slow RGB, fast hand pose: 3x CPU savings in egocentric recognition","Multimodal trick: slow RGB, fast pose, 3x CPU efficiency","F1 intact with 10 Hz RGB and 30 Hz pose, CPU down 3x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002334,"raw_usage":{"total_tokens":8962,"prompt_tokens":877,"completion_tokens":8085,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":7971}},"tokens_in":493,"tokens_out":8085,"duration_ms":55767,"temperature":1.0,"reasoning_tokens":7971,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:34:23.816082+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure inference time and CPU usage of the same two-stream models on an actual wearable or embedded CPU, and compare the 30 Hz RGB + 30 Hz hand pose configuration with the 10 Hz RGB + 30 Hz hand pose configuration; if the latter does not show roughly a 3x CPU reduction at near-equal macro F1, the paper's central efficiency claim is not portable to its target hardware. A second check is reproducing the frequency sweep on the test set to confirm the F1 plateau at 10 Hz RGB.","supporting_citations":[{"cited_title":"H2o: Two hands manipulating objects for first person interaction recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the two-hand manipulation dataset and action labels used for all training and evaluation."},{"cited_title":"Levit: a vision transformer in convnet’s clothing for faster inference","cited_arxiv_id":null,"evidence_quote":"Provides the LeViT vision-transformer feature extractor whose computational cost motivates RGB downsampling."},{"cited_title":"Avatars grow legs: Generating smooth human motion from sparse tracking in- puts with diffusion model","cited_arxiv_id":null,"evidence_quote":"Inspires the Temporal MLP architecture that models temporal dependencies in both streams."},{"cited_title":"On the utility of 3d hand poses for action recognition","cited_arxiv_id":null,"evidence_quote":"Prior work combining RGB with 3D hand poses that the paper builds on."},{"cited_title":"Two-stream con- volutional networks for action recognition in videos","cited_arxiv_id":null,"evidence_quote":"The two-stream design pattern for video action recognition that underlies the architecture."},{"cited_title":"Holoassist: an egocentric human interaction dataset for interactive ai assistants in the real world","cited_arxiv_id":null,"evidence_quote":"Evidence that more modalities do not always help, motivating the targeted two-stream choice."}],"review_version":1}