Pith. sign in

REVIEW 5 major objections 6 minor 57 references

THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Sampling

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A wearable camera system can capture every hand-activity segment and match full-video accuracy while processing only 3% of the RGB stream, driven by thermal detection of activity transitions.

desk verdict A plausible, well-motivated wearable sampling system whose headline 3%-data/95%-F1 result is credible but rests on an unmeasured transition-detector, plus some internal power numbers that need reconciling. read the letter →

arxiv 2507.06442 v1 pith:BMX2BXWM submitted 2025-07-08 cs.CV

Soroush Shahi , Farzad Shahabi , Rama Nabulsi , Glenn Fernandes , Aggelos Katsaggelos , Nabil Alshurafa This is my paper
classification cs.CV
keywords computervisionhand-objectreasoningegocentricvideoadaptiveframesamplingthermalsensingwearablecameraactivityrecognitionvision-languagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

THOR is a system for wearable cameras that tries to make continuous hand-activity monitoring practical by recording almost none of the RGB stream. Its claim is that a low-resolution thermal camera can act as a gatekeeper: a model trained to make heat snapshots of the same activity look alike and snapshots of different activities look different reveals when the wearer switches activities, so THOR raises the RGB frame rate only during transitions and drops it back down during sustained activity. The same thermal image localizes the hand and object, so the system crops a small patch instead of processing the full frame. In an in-the-wild study with 14 participants and 30 activities, THOR-High captures every annotated activity segment while using 3.07% of the RGB data, and the captioning model reaches 95% F1 against 94% F1 for full-video processing, with 78% lower end-to-end latency and a reported 48% reduction in overall system power. If these numbers hold in longer deployments, they suggest that longitudinal wearable observation of health-risk behaviors such as eating, smoking, and screen use can be both accurate and energy-lean.

What carries the argument

The load-bearing mechanism is the thermal similarity score: a contrastively trained lightweight backbone maps each low-resolution thermal frame to a 64-dimensional embedding; cosine similarity between the current embedding and a sliding window of past embeddings is min-max normalized, and its complement sets the RGB frame rate via $\mathrm{FPS}_{\mathrm{new}} = \mathrm{FPS}_{\min} + (\mathrm{FPS}_{\max} - \mathrm{FPS}_{\min}) \cdot (1 - s)$. The same thermal image is thresholded with Otsu's method to produce a heat mask that defines the hand-object crop, so both temporal and spatial sampling are derived from a single low-power thermal stream. Recognition is carried by a fine-tuned vision-language model that receives the cropped patch and outputs short activity captions, with a larger teacher model and synthetic hand-object images used to create the fine-tuning data.

What would settle it

Take a held-out participant whose protocol runs visually similar activities back to back (for example, reading then texting, or wiping then washing) and count how many short segments are captured with at least four frames under the THOR-High policy; the full-coverage claim predicts 100%, so any systematically missed pair type would show the transition detector is the weak link.

Watch

Extended reading notes

Core claim

The central claim is that activity boundaries carry most of the information needed for sparse capture: if you know when an activity changes, you can sample densely only there, and a thermal pose embedding can supply that timing. The paper demonstrates that 64-dimensional embeddings from a MobileNetV2 trained with contrastive loss on 130 thermal-labeled activities cluster sustained activity while a rolling-window cosine-similarity drop flags transitions; this similarity score is mapped to the RGB frame rate by $\mathrm{FPS}_{\mathrm{new}} = \mathrm{FPS}_{\min} + (\mathrm{FPS}_{\max} - \mathrm{FPS}_{\min}) \cdot (1 - s)$, where $s$ is the normalized similarity. It also shows that the thermal heat mask localizes the hand and object, so a small patch suffices for recognition, and a fine-tuned ~2-billion-parameter vision-language model can caption that patch directly. On the authors' in-the-wild study the pipeline captures all segments with 3.07% of RGB data and recognizes 30 activities at 95% F1, compared with 94% F1 for the same VLM given the full video; on a public egocentric corpus the generated captions land semantically close to human narrations.

Load-bearing premise

The load-bearing premise (Section 4.1.4) is that every activity transition is reliably visible as a drop in cosine similarity between neighboring thermal hand/body embeddings, so the RGB sampler speeds up exactly when a switch occurs, yet the paper never measures transition-detection precision or recall directly.

Editorial extensions

If this is right

  • A wearable with THOR's policy would spend most of its time in a low-power state, activating the RGB sensor mainly around activity transitions, so continuous all-day observation becomes plausible for battery-constrained devices.
  • Storage and post-processing volume drop to roughly 3% of the original stream, which makes long-term studies and review of wearable footage far less burdensome.
  • The vision-language model classifies from a single cropped patch, so the recognition step runs in 1.47 seconds per query instead of 6.63 seconds, making real-time captioning feasible on a smartphone rather than a server.
  • Because captions are free-form, the same model covers both fine-grained and coarse-grained activities, allowing one pipeline to spot a snack, a meal, or a full cooking episode without a fixed label set.
  • Recognition accuracy is essentially unchanged (95% versus 94% full-video F1), so the efficiency gains do not appear to come at the cost of activity classification quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the paper explicitly defers precise boundary delineation to future work (Section 7.5.1), but the same thermal similarity score that raises the frame rate should also mark the start and end of an activity segment, which would yield timestamped segments without ever touching RGB data.
  • Inference: because RGB capture is cropped to the hand-object patch, the stored record contains far less background and bystander information than full-frame video; a re-identification study comparing full frames against THOR's patches would quantify this unmeasured privacy benefit.
  • Inference: the 3% data figure depends on the thermal embedding having been trained on the activities it later encounters; stress-testing the frozen sampler on a new cohort with novel activities would show how much of the saving transfers, since transitions between unseen posture pairs may not trigger a similarity dip.
  • Inference: the free-form captions could support user-defined behavior queries such as 'eating while using the phone' through semantic matching rather than keyword bags; the paper sketches this as a future application but does not measure retrieval precision on long free-living recordings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. THOR combines a low-resolution thermal camera with an RGB camera on a wearable device. A contrastively trained MobileNetV2 embedding maps thermal frames to pose-similarity scores; THOR I uses drops in cosine similarity to raise the RGB sampling rate at activity transitions and Otsu-based thermal masking to crop hand-object patches. THOR II is a Qwen2-VL model fine-tuned with LoRA on Ego4D patches and synthetic hand-object images, and it generates short activity captions from the patches. The paper reports that THOR-High uses 3.07% of the original RGB pixel data, captures 100% of 2,221 activity segments in a 14-participant in-the-wild study, and achieves 95% F1 versus 94% for a full-video Qwen2-VL baseline, with reductions in latency and power. Generalization is also evaluated on Ego4D via caption-to-narration cosine similarity.

Significance. If the claims hold, THOR would be a useful contribution to energy-efficient, privacy-conscious wearable activity monitoring: the core idea of using a low-power thermal sentinel to drive both temporal and spatial RGB sampling is well motivated, and the evaluation includes a real device, a held-out 14-participant study, and an external large-scale dataset. The design of THOR II is also commendable: it is trained on Ego4D and synthetic data and then tested on the study data, avoiding circularity, and the teacher-student synthetic-data pipeline is a practical way to reduce annotation cost. The main weakness is that the headline 3%-data/100%-coverage result is not supported by a direct evaluation of the transition detector on which it depends, and several central hyperparameters and statistical quantities are not reported. The evidence is promising but not yet sufficient to support the strong claims as stated.

major comments (5)
  1. [§4.1.4, Algorithm 1, §6.1.2] The central claim that THOR-High captures 100% of activity segments using 3.07% of the data rests entirely on the thermal embedding transition detector, yet no precision or recall of that detector is reported. Section 4.1.3 reports only NMI (0.64) and Figure 4 is an illustrative embedding, not a detection curve; Section 7.5.1 explicitly defers boundary delineation to future work. The concrete failure mode is that similar-pose consecutive activities, such as reading versus texting or computer use versus gaming, may not produce a sufficient cosine-similarity drop, so the FPS boost may not fire and a short segment could fall below the four-frame coverage criterion of Section 4.5.1. Please add a direct evaluation that compares detected transition events against the annotated segment boundaries, reporting per-participant transition precision/recall and specifically testing confusable activity pairs.
  2. [Algorithm 1, Table 1, §4.4.2] The data-usage figures in Table 1 and the coverage results in Figure 6 depend on hyperparameters FPSmin, FPSmax, the window size W, and the epsilon in Algorithm 1, but none of these values are disclosed for THOR-High, THOR-Mid, or THOR-Low. Because the variants are defined by adjusting exactly these parameters, the 3.07% figure is not reproducible, and it is also unclear whether these values were tuned to guarantee the four-frame coverage criterion. Please report the exact settings for each variant and include a sensitivity analysis showing how data usage and coverage change as W and the FPS bounds vary.
  3. [Table 4, §6.2.1] The headline comparison of 95% F1 for THOR II versus 94% for the full-video Qwen2-VL baseline is reported as a single mean without any measure of variance. With a one-point difference, the claim that the two are 'comparable' is not statistically supported. Please report per-participant precision, recall, and F1 distributions, confidence intervals, and a paired significance test across the 14 participants, or explicitly state that the difference is within noise.
  4. [§3.2, Table 2] The prototype is described as using an 'MLX90614 thermal sensor' combined with 32x24 low-resolution thermal frames and thermal-frame embeddings. The MLX90614 is a single-pixel infrared thermometer, not a 32x24 thermal imager, so this specification is internally inconsistent with the method. If the intended sensor is a thermal array such as the MLX90640, please correct the hardware description and ensure the measured power consumption in Table 2 corresponds to that sensor; if a different sensor was used, state its model and resolution explicitly.
  5. [§5.3.2, Table 4] The Qwen2-VL baseline used for the full-video comparison is not fully specified. Section 5.3.2 says all baseline models are trained on the study dataset with leave-one-out validation, while Section 6.2.2 describes Qwen2-VL as sharing the same architecture but using the entire video data; it is unclear whether the baseline was fine-tuned with the same LoRA recipe and whether it processes one full frame, a sequence of frames, or full-resolution patches. Please clarify the exact input and training protocol for the Qwen2-VL baseline, since fairness of the 95%-versus-94% comparison depends on it.
minor comments (6)
  1. [§6.1 heading] The heading 'THOR I: Adaptive Saptio-temporal Sampling' contains a typo: 'Saptio' should be 'Spatio'.
  2. [Figure 4 caption] The Figure 4 caption appears to contain a duplicated sentence; the caption should state the clusters-and-transitions interpretation once.
  3. [Table 1] The caption for Table 1 should explicitly define what 'percentage of data utilized' counts: pixels, bytes, or frames, and whether both temporal downsampling and spatial cropping are included in the denominator and numerator.
  4. [§4.2, Algorithm 1] The notation around the similarity score is inconsistent: the text defines alpha as 1 - s and then calls s a similarity factor, while Algorithm 1 computes an inverse similarity from a min-max normalized rolling average. Please unify the notation and define all variables, including W and epsilon, in one place.
  5. [§4.4.1] The 'head-tail classification' method used to derive the short/medium/long segment bins is not described, so the thresholds of one minute and 2.7 minutes cannot be reproduced; please provide the exact procedure or a reference.
  6. [§6.2.3, Figure 9] The Ego4D generalization result reports cosine similarities without variance or a comparison against a non-THOR captioning baseline, so it is difficult to judge how much of the similarity comes from the general VLM rather than from the sampling and fine-tuning contributions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 3%-data/95%-F1 claims are empirical outputs, with the transition-detector recall gap being a validity limitation rather than a circular reduction.

full rationale

THOR's central derivation chain is not circular. THOR I's transition detector is an empirically trained contrastive model on thermal embeddings, evaluated via leave-one-participant-out cross-validation (§4.1.3); THOR II is fine-tuned on Ego4D and synthetic data and tested on the 14-participant study (§5.2, §6.2.1), so the reported 95% F1 is not an input. The 3% data-utilization and segment-coverage figures are measurements from the algorithm's output, not parameters fitted to guarantee those numbers; the uniform-sampling baselines are independently tuned to the same four-frame coverage criterion (§4.5.1, §4.5.2). The main weakness—that transition-detection precision/recall is never directly measured (§4.1.3, §7.5.1) and that similar-pose transitions could be missed—is a validity/robustness gap, not a circular reduction: no equation or definition makes the coverage claim true by construction. Self-citations to HabitSense, SmartAct, and ActiSight are contextual and not load-bearing for the core spatio-temporal sampling contribution.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central accuracy claim (95% F1 with 3% data) rests on several empirically chosen parameters whose values are not fully reported, and on domain assumptions about thermal pose discriminability and patch sufficiency. No new physical entities are introduced.

free parameters (6)
  • FPSmin, FPSmax, window W (THOR-High/Mid/Low) = not reported
    These control the sampling rate and therefore the reported 3% data utilization; they are tuned per variant to guarantee coverage of short, medium, or long segments (Section 4.4.2).
  • Embedding dimension = 64
    Chosen empirically for the contrastive encoder (Section 4.1.1).
  • LoRA rank/alpha/dropout = 8 / 8 / 0.05
    Fine-tuning hyperparameters for Qwen2-VL (Section 5.2.3).
  • Patch expansion margin = 20 pixels
    Ad hoc expansion of the thermal mask to define the RGB crop (Section 4.3).
  • Activity segment bin thresholds = 1 and 2.7 minutes
    Head-tail classification defines short/medium/long segments and conditions the variant design (Section 4.4.1).
  • Activity keyword lists = hand-built
    Manual keyword bags per activity used for caption matching; the choice influences F1 scores (Section 5.3.1, Appendix A).
assumptions (6)
  • domain assumption Thermal frames encode body/hand pose sufficiently to discriminate hand activities
    The contrastive model in Section 4.1 maps thermal frames to embeddings that cluster by activity; if thermal information is not discriminative for some activity pairs, transition detection fails.
  • domain assumption Activity transitions coincide with drops in thermal pose similarity
    The sampling trigger in Section 4.2 increases FPS when similarity falls; this is the core premise of THOR I and is not directly validated with transition-level precision/recall.
  • domain assumption Cropped hand-object patches retain enough visual context for VLM recognition
    Patch-based inference in Section 5.1 assumes the crop captures all necessary cues; the related-work section itself notes that background masking can remove context needed to disambiguate similar activities.
  • domain assumption Synthetic hand-object images and teacher-generated QA pairs are a valid training proxy for real interactions
    THOR II is fine-tuned on Affordance Diffusion images paired with Qwen2-VL-72B captions (Section 5.2.1); no ablation isolates the contribution of synthetic data.
  • standard math Otsu thresholding separates body heat from background in the thermal image
    Used for the binary heat mask in Section 4.3; standard, but the small 32x24 resolution may make the assumption fragile in warm environments.
  • domain assumption MLX90614 thermal readings at 4 fps provide temporal resolution adequate for transition detection
    The hardware choice in Section 3.2 determines the ability to catch short activities; no analysis of the minimum detectable segment length is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Sampling." pith.science (2026). https://pith.science/paper/BMX2BXWM

@misc{pith2026250706442,
  author       = {Pith},
  title        = {Pith review of: THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BMX2BXWM}},
  note         = {Machine review of arXiv:2507.06442}
}
read the original abstract

Wearable cameras are increasingly used as an observational and interventional tool for human behaviors by providing detailed visual data of hand-related activities. This data can be leveraged to facilitate memory recall for logging of behavior or timely interventions aimed at improving health. However, continuous processing of RGB images from these cameras consumes significant power impacting battery lifetime, generates a large volume of unnecessary video data for post-processing, raises privacy concerns, and requires substantial computational resources for real-time analysis. We introduce THOR, a real-time adaptive spatio-temporal RGB frame sampling method that leverages thermal sensing to capture hand-object patches and classify them in real-time. We use low-resolution thermal camera data to identify moments when a person switches from one hand-related activity to another, and adjust the RGB frame sampling rate by increasing it during activity transitions and reducing it during periods of sustained activity. Additionally, we use the thermal cues from the hand to localize the region of interest (i.e., the hand-object interaction) in each RGB frame, allowing the system to crop and process only the necessary part of the image for activity recognition. We develop a wearable device to validate our method through an in-the-wild study with 14 participants and over 30 activities, and further evaluate it on Ego4D (923 participants across 9 countries, totaling 3,670 hours of video). Our results show that using only 3% of the original RGB video data, our method captures all the activity segments, and achieves hand-related activity recognition F1-score (95%) comparable to using the entire RGB video (94%). Our work provides a more practical path for the longitudinal use of wearable cameras to monitor hand-related activities and health-risk behaviors in real time.

Figures

Figures reproduced from arXiv: 2507.06442 by the authors.

Figure 2
Figure 2. Example activity categories and associated objects from our study (15 classes shown). Our method uses a thermal [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the THOR system. A wearable RGB camera is augmented with a downward-facing thermal sensor to [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. By using a low-resolution patch as input, the model processes fewer visual tokens compared to full-image input (A–B). [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figures from the paper (6 more)
Figure 6
Figure 6. Figure 6: Coverage of segments with di!erent length using di!erent variations of THOR and baselines methods. and activity transitions, which might be in$uenced by factors such as attention span. For instance, P12 exhibited frequent transitions during daily routines, resulting in…
Figure 7
Figure 7. Figure 7: Power consumption profile of di!erent components in THOR I (A) and THOR II (B) compared to baseline methods. 0 2 4 6 8 0 2 4 6 8 (C) THOR II Patch Processing Latency (D) THOR II On-device Inference Latency Patch Full-image Latency (s) Latency (s) -71% # -78% # (A) THOR…
Figure 8
Figure 8. Figure 8: Input (A) and end-to-end (B) Latency of THOR II compared to baseline method. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Cosine similarity between narrations provided by annotators (A1 vs A2) and our method. 6.2.3 Generalization. We evaluate the generalization ability of our method by comparing the cosine similarity between the generated captions and human￾provided narrations in the Ego4…
Figure 10
Figure 10. Figure 10: Accuracy of di!erent THOR variants in classifying 30 hand-related activities, compared to using the full video. Note that each variant is designed for segments of di!erent lengths; therefore, their accuracies should not be directly compared to one another. during full…
Figure 11
Figure 11. Figure 11: Overview of our synthetic data generation pipeline. We feed the image of common objects into a di [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 49 canonical work pages

  1. [1]

    Saghir Alfasly, Jian Lu, Chen Xu, Zaid Al-Huda, Qingtang Jiang, Zhaosong Lu, and Charles K Chui. 2023. FastPicker: Adaptive independent two-stage video-to-video summarization for e￿cient action recognition. Neurocomputing 516 (2023), 231–244

    work page 2023

  2. [2]

    Rawan Alharbi, Sougata Sen, Ada Ng, Nabil Alshurafa, and Josiah Hester. 2022. Actisight: wearer foreground extraction using a practical rgb-thermal wearable. In 2022 IEEE International Conference on Pervasive Computing and Communications (PerCom) . IEEE, 237–246

    work page 2022

  3. [3]

    Rawan Alharbi, Tammy Stump, Nilofar Vafaie, Angela Pfammatter, Bonnie Spring, and Nabil Alshurafa. 2018. I can’t be myself: e￿ects of wearable cameras on the capture of authentic behavior in the wild. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 2, 3 (2018), 1–40

    work page 2018

  4. [4]

    Sven Bambach, Stefan Lee, David J Crandall, and Chen Yu. 2015. Lending a hand: Detecting hands and recognizing activities in complex egocentric interactions. In Proceedings of the IEEE international conference on computer vision . 1949–1957

    work page 2015

  5. [5]

    Siddhant Bansal, Michael Wray, and Dima Damen. 2024. Hoi-ref: Hand-object interaction referral in egocentric vision. arXiv preprint arXiv:2404.09933 (2024)

    arXiv 2024

  6. [6]

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. 2021. Is space-time attention all you need for video understanding?. In ICML, Vol. 2. 4

    2021

  7. [7]

    Rebekah Carter. 2024. Snap Spectacles 5 Review: The Latest Snap AR Glasses. https://www.xrtoday.com/augmented-reality/snap- spectacles-5-review-the-latest-snap-ar-glasses/. Accessed: 2025-04-28

    work page 2024

  8. [8]

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geo￿rey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning . PmLR, 1597–1607

    work page 2020

Show all 57 references
  1. [9]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition . Ieee, 248–255

    2009

  2. [10]

    Fangqiang Ding, Yunzhou Zhu, Xiangyu Wen, Gaowen Liu, and Chris Xiaoxuan Lu. 2024. ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images. arXiv preprint arXiv:2403.09871 (2024)

    2024 arXiv

  3. [11]

    Aiden R Doherty, Steve E Hodges, Abby C King, Alan F Smeaton, Emma Berry, Chris JA Moulin, Siân Lindley, Paul Kelly, and Charlie Foster. 2013. Wearable cameras in health: the state of the art and future possibilities. American journal of preventive medicine 44, 3 (2013), 320–323

    2013

  4. [12]

    Shiwei Fang, Ketan Mayer-Patel, and Shahriar Nirjon. 2019. ZenCam: Context-driven control of autonomous body cameras. In 2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS) . IEEE, 41–48

    2019

  5. [13]

    Glenn J Fernandes, Jiayi Zheng, Mahdi Pedram, Christopher Romano, Farzad Shahabi, Blaine Rothrock, Thomas Cohen, Helen Zhu, Tanmeet S Butani, Josiah Hester, et al . 2024. HabitSense: A privacy-aware, AI-enhanced multimodal wearable platform for mHealth applications. Proceeding...

    2024

  6. [14]

    Luke Gemming, Jennifer Utter, and Cliona Ni Mhurchu. 2015. Image-assisted dietary assessment: a systematic review of the evidence. Journal of the Academy of Nutrition and Dietetics 115, 1 (2015), 64–77

    2015

  7. [15]

    Amir Ghodrati, Babak Ehteshami Bejnordi, and Amirhossein Habibian. 2021. Frameexit: Conditional early exiting for e￿cient video recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 15608–15618

    2021

  8. [16]

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al . 2022. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer ...

    2022

  9. [17]

    Cathal Gurrin, Alan F Smeaton, Aiden R Doherty, et al . 2014. Lifelogging: Personal big data. Foundations and Trends® in information retrieval 8, 1 (2014), 1–125. THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Sampling • 21

    2014

  10. [18]

    Grace Harmon. 2024. Ray-Ban Meta Smart Glasses get an on-device assistant, with battery life limitations. https://www.emarketer.com/ content/ray-ban-meta-smart-glasses-on-device-assistant--with-battery-life-limitations. Accessed: 2025-04-28

    2024

  11. [19]

    Mohamed Abul Hassan and Edward Sazonov. 2020. Selective content removal for egocentric wearable camera in Nutritional Studies. IEEE Access 8 (2020), 198615–198623

    2020

  12. [20]

    Mingzhou He, Haojie Wang, Shuchang Zhou, Qingbo Wu, King Ngi Ngan, Fanman Meng, and Hongliang Li. 2024. Inertial Strengthened CLIP model for Zero-shot Multimodal Egocentric Activity Recognition. In 2024 Asia Paci￿c Signal and Information Processing Association Annual Summit an...

    2024

  13. [21]

    Keita Higuchi, Ryo Yonetani, and Yoichi Sato. 2017. Egoscanning: Quickly scanning ￿rst-person videos with egocentric elastic timelines. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems . 6536–6546

    2017

  14. [22]

    Steve Hodges, Lyndsay Williams, Emma Berry, Shahram Izadi, James Srinivasan, Alex Butler, Gavin Smyth, Narinder Kapur, and Ken Wood. 2006. SenseCam: A retrospective memory aid. In UbiComp 2006: Ubiquitous Computing: 8th International Conference, UbiComp 2006 Orange County, CA,...

    2006

  15. [23]

    Elad Ho￿er and Nir Ailon. 2015. Deep metric learning using triplet network. In Similarity-based pattern recognition: third international workshop, SIMBAD 2015, Copenhagen, Denmark, October 12-14, 2015. Proceedings 3 . Springer, 84–92

    2015

  16. [24]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al . 2022. Lora: Low-rank adaptation of large language models. ICLR 1, 2 (2022), 3

    2022

  17. [25]

    Tatsuya Ishihara, Kris M Kitani, Wei-Chiu Ma, Hironobu Takagi, and Chieko Asakawa. 2015. Recognizing hand-object interactions in wearable camera videos. In 2015 IEEE International Conference on Image Processing (ICIP) . IEEE, 1349–1353

    2015

  18. [26]

    Bruno Korbar, Du Tran, and Lorenzo Torresani. 2019. Scsampler: Sampling salient clips from video for e￿cient action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 6232–6242

    2019

  19. [27]

    Amel Ksibi, Ala Saleh D Alluhaidan, Amina Salhi, and Sahar A El-Rahman. 2021. Overview of lifelogging: current challenges and advances. IEEE Access 9 (2021), 62630–62641

    2021

  20. [28]

    Viet-Duc Le, Van-Nam Hoang, Tien-Thanh Nguyen, Van-Hung Le, Thanh-Hai Tran, Hai Vu, and Thi-Lan Le. 2023. Hand activity recognition from automatic estimated egocentric skeletons combining slow fast and graphical neural networks. Vietnam Journal of Computer Science 10, 01 (2023...

    2023

  21. [29]

    Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision . 1833–1844

    2021

  22. [30]

    Jintao Lin, Haodong Duan, Kai Chen, Dahua Lin, and Limin Wang. 2022. Ocsampler: Compressing videos to one clip with single-step sampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 13894–13903

    2022

  23. [31]

    Ralph Maddison, Susie Cartledge, Michelle Rogerson, Nicole Sylvia Goedhart, Tarveen Ragbir Singh, Christopher Neil, Dinh Phung, and Kylie Ball. 2019. Usefulness of wearable cameras as a tool to enhance chronic disease self-management: scoping review. JMIR mHealth and uHealth 7...

    2019

  24. [32]

    Yue Meng, Chung-Ching Lin, Rameswar Panda, Prasanna Sattigeri, Leonid Karlinsky, Aude Oliva, Kate Saenko, and Rogerio Feris. 2020. Ar-net: Adaptive frame resolution for e￿cient action recognition. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–2...

    2020

  25. [33]

    Meta Platforms, Inc. 2025. Battery life on Ray-Ban Meta AI Glasses. https://www.meta.com/help/ai-glasses/303057485648146/. Accessed: 2025-04-28

    2025

  26. [34]

    Shinya Michibata, Katsufumi Inoue, Michifumi Yoshioka, and Atsushi Hashimoto. 2020. Cooking activity recognition in egocentric videos with a hand mask image branch in the multi-stream cnn. In Proceedings of the 12th Workshop on Multimedia for Cooking and Eating Activities. 1–6

    2020

  27. [35]

    Franziska Mueller, Dushyant Mehta, Oleksandr Sotnychenko, Srinath Sridhar, Dan Casas, and Christian Theobalt. 2017. Real-time hand tracking under occlusion from an egocentric rgb-d sensor. In Proceedings of the IEEE international conference on computer vision . 1154–1163

    2017

  28. [36]

    Thi Hoa Cuc Nguyen, Jean-Christophe Nebel, Gordon Hunter, and Francisco Florez-Revuelta. 2018. Automated detection of hands and objects in egocentric videos, for ambient assisted living applications. In 2018 14th International Conference on Intelligent Environments (IE). IEEE, 91–94

    2018

  29. [37]

    Evangelos Niforatos, Veranika Lim, Christian Vuerich, Marc Langheinrich, and Agon Bexheti. 2015. PulseCam: Biophysically Driven Life Logging. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (Copenhagen,...

    2015

  30. [38]

    Takehiko Ohkawa, Kun He, Fadime Sener, Tomas Hodan, Luan Tran, and Cem Keskin. 2023. Assemblyhands: Towards egocentric activity understanding via 3d hand pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 12999–13008

    2023

  31. [39]

    Baoqi Pei, Guo Chen, Jilan Xu, Yuping He, Yicheng Liu, Kanghua Pan, Yifei Huang, Yali Wang, Tong Lu, Limin Wang, et al . 2024. Egovideo: Exploring egocentric foundation model and downstream adaptation. arXiv preprint arXiv:2406.18070 (2024). 22 • Shahi et al

    2024 arXiv

  32. [40]

    Hamed Pirsiavash and Deva Ramanan. 2012. Detecting activities of daily living in ￿rst-person camera views. In 2012 IEEE conference on computer vision and pattern recognition . IEEE, 2847–2854

    2012

  33. [41]

    Rafael Possas, Sheila Pinto Caceres, and Fabio Ramos. 2018. Egocentric activity recognition on a budget. In Proceedings of the IEEE conference on computer vision and pattern recognition . 5967–5976

    2018

  34. [42]

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. 2021. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on compute...

    2021

  35. [43]

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition . 4510–4520

    2018

  36. [44]

    Giovanni Schiboni, Fabio Wasner, and Oliver Amft. 2018. A privacy-preserving wearable camera setup for dietary event spotting in free-living. In 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) . IEEE, 872–877

    2018

  37. [45]

    Soroush Shahi, Mahdi Pedram, Glenn Fernandes, and Nabil Alshurafa. 2022. Smartact: energy e￿cient and real-time hand-to-mouth gesture detection using wearable rgb-t. In 2022 IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE, 1–4

    2022

  38. [46]

    Dandan Shan, Jiaqi Geng, Michelle Shu, and David F Fouhey. 2020. Understanding human hands in contact at internet scale. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 9869–9878

    2020

  39. [47]

    Shuhan Tan, Tushar Nagarajan, and Kristen Grauman. 2023. Egodistill: Egocentric head motion distillation for e￿cient video under- standing. Advances in Neural Information Processing Systems 36 (2023), 33485–33498

    2023

  40. [48]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al . 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024)

    2024 arXiv

  41. [49]

    GJ Welk. 2002. Physical activity assessments for health-related research. Human Kinetics (2002)

    2002

  42. [50]

    Zuxuan Wu, Caiming Xiong, Chih-Yao Ma, Richard Socher, and Larry S Davis. 2019. Adaframe: Adaptive frame selection for fast video recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1278–1287

    2019

  43. [51]

    Boshen Xu, Ziheng Wang, Yang Du, Zhinan Song, Sipeng Zheng, and Qin Jin. 2024. Egonce++: Do egocentric video-language models really understand hand-object interactions? arXiv preprint arXiv:2405.17719 (2024)

    2024 arXiv

  44. [52]

    Jilan Xu, Yifei Huang, Junlin Hou, Guo Chen, Yuejie Zhang, Rui Feng, and Weidi Xie. 2024. Retrieval-augmented egocentric video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 13525–13536

    2024

  45. [53]

    Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birch￿eld, Jiaming Song, Shubham Tulsiani, and Sifei Liu. 2023. A￿ordance di￿usion: Synthesizing hand-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 22479–22489

    2023

  46. [54]

    Lingzhi Zhang, Shenghao Zhou, Simon Stent, and Jianbo Shi. 2022. Fine-grained egocentric hand-object segmentation: Dataset, model, and applications. In European Conference on Computer Vision . Springer, 127–145

    2022

  47. [55]

    Yanyi Zhang, Xinyu Li, and Ivan Marsic. 2021. Multi-label activity recognition using activity-speci￿c features and activity correlations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 14625–14635

    2021

  48. [56]

    Yuan Zhi, Zhan Tong, Limin Wang, and Gangshan Wu. 2021. Mgsampler: An explainable sampling strategy for video action recognition. In Proceedings of the IEEE/CVF International conference on Computer Vision . 1513–1522

    2021

  49. [57]

    Generate 3 extremely short question and answer about the hand interaction. Your response must be in JSON

    Chengzhang Zhong, Amy R Reibman, Hansel Mina Cordoba, and Amanda J Deering. 2019. Hand-hygiene activity recognition in egocentric video. In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) . IEEE, 1–6. THOR: Thermal-guided Hand-Object Reasoning via ...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.