Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Hierarchical and Multimodal Data for Daily Activity Understanding

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new public dataset records 200 hours of daily activity from 20 synchronized sensors with three-level hierarchical labels, and its benchmarks show that wearable sensors retain accuracy where vision models collapse.

desk verdict A genuinely useful multimodal daily-activity dataset—if the release-status contradiction in the appendix is fixed, it deserves to be a standard benchmark. read the letter →

arxiv 2504.17696 v4 pith:73NTN5N5 submitted 2025-04-24 cs.CV cs.AI

Ghazal Kaviani , Yavuz Yarici , Seulgi Kim , Mohit Prabhushankar , Ghassan AlRegib , Mashhour Solh , Ameya Patil This is my paper · ORCID
classification cs.CVcs.AI
keywords multimodaldatasethierarchicalactivityrecognitionsensorfusionactioncounterfactualtemporallocalizationanticipationcross-viewrobustnesswearablesensors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces DARai, a public, open-source dataset of continuous, unscripted daily-activity recordings from 50 people in 10 home environments, captured by 20 sensor streams. The dataset's load-bearing design is its three-level annotation hierarchy—activities, actions, and procedures—together with 'action counterfactuals,' alternative executions of the same activity that let researchers test how sensors and models handle natural variation. The paper argues that if this resource is used, the community gets a testbed where multimodal fusion, hierarchical recognition, temporal localization, and anticipation can be studied on realistic long-range data. Its benchmarks suggest that body-worn sensors such as insole pressure are more robust to task granularity and viewpoint changes than cameras, a finding that matters for privacy-preserving activity understanding.

What carries the argument

The central object is the DARai dataset itself: a synchronized, multimodal corpus in which 20 sensor streams (RGB, depth, radar, IMUs, EMG, insole pressure, biomonitors, gaze, microphones, environmental sensors) are time-aligned to frame-level annotation boundaries at three nested levels—L1 activities (goal-level tasks), L2 actions (shared sub-steps), and L3 procedures (exact execution steps). The design includes predefined hierarchical decomposition rules and action-counterfactual instances, where the same activity is performed under different conditions, so that sensor-specific variation can be studied. The benchmark suite built on top—transformer, convolutional, and recurrent models for recognition, localization, and anticipation—is the demonstration mechanism that turns the data into evidence about modality robustness.

What would settle it

Measure the actual inter-stream timestamp disagreement on a released sample: if the median absolute difference between host clocks and sensor internal clocks at procedure boundaries exceeds the median duration of L3 labels, the hierarchy claims cannot be validated at their stated granularity.

Watch

Extended reading notes

Core claim

The central claim is that DARai is the largest available daily-activity dataset in number of sensors and modalities, and that its design exposes systematic limitations of vision-centric activity understanding. With 200+ hours from 50 participants, 10 environments, and 20 modalities, annotated at three levels (18 activities, 44 actions, 98 procedures) with shared lower-level labels across higher-level classes, the paper demonstrates that visual models drop sharply from L1 to L2/L3 accuracy while wearable modalities such as insole pressure and EMG degrade less; that cross-view camera testing collapses accuracy (from 89% to 15% in one case) while cross-body wearable transfer degrades less; and that fusion of wearable signals sustains accuracy at fine granularity (over 42% at L3). The paper's benchmarks also show that action anticipation at the procedure level benefits from longer observation, while action-level anticipation plateaus. All of these are claims about what the dataset enables and what it reveals.

Load-bearing premise

All conclusions hinge on the host-clock synchronization being accurate enough that every sensor stream's timestamps align with the frame-level L1/L2/L3 boundaries; if alignment error exceeds the duration of short L3 procedures, the hierarchical and temporal benchmarks are built on misaligned inputs.

Editorial extensions

If this is right

  • Non-visual modalities (insole, EMG, IMU, gaze) can sustain fine-grained recognition where RGB and depth models collapse, suggesting privacy-preserving activity understanding without cameras is feasible.
  • Long-horizon procedural anticipation improves with more observation while action-level anticipation plateaus, meaning models should treat hierarchy levels differently.
  • The counterfactual design provides paired samples that expose contextual bias, such as models misclassifying counterfactual reading-on-couch actions as phone conversations due to background.
  • The dataset's fluid, unscripted boundaries make temporal localization harder than in scripted datasets like Breakfast or 50 Salads, providing a stricter test for segmentation methods.
  • The fixed cross-subject split and released loaders let researchers compare methods on a common benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the synchronization and annotation boundaries hold at the stated precision, DARai could become a standard evaluation for sensor-fusion and hierarchical-learning research, and the cross-view/cross-body gaps imply that deployment of vision models in homes needs either multi-view setups or modality complementarity.
  • A testable extension of the paper's findings would be to check whether the L3 robustness of wearables generalizes to activities outside the ten environments, since the environments are all kitchens, living spaces, and offices.
  • The counterfactual design could support causal-style analyses of what sensor signals actually drive a prediction, moving from accuracy comparisons to attribution comparisons.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces DARai, a multimodal and hierarchically annotated dataset of daily human activities, and presents a set of machine-learning benchmarks on it. The dataset consists of over 200 hours of continuous, partly unscripted recordings from 50 participants in 10 environments, with 18 L1 activities, 44 L2 actions, and 98 L3 procedures, collected from cameras, depth/radar sensors, wearable IMUs, EMG, insole pressure, biomonitors, and gaze tracking. The authors report unimodal and multimodal recognition results across the three hierarchy levels, cross-view and cross-body robustness experiments, temporal localization, and short- and long-horizon action anticipation, and they release code, loaders, splits, and a dataset DOI via IEEE Dataport. The central claim is that DARai is the largest publicly available dataset in terms of the number of sensors and modalities and that its hierarchical, unscripted design enables studying action-counterfactual variations and fine-grained temporal dependencies. The manuscript is accompanied by extensive appendices on collection hardware, annotation workflow, benchmark details, and failure cases, including an honest account that some multimodal time-series classification attempts were unsuccessful.

Significance. If the release-completeness and synchronization issues are resolved, DARai would be a valuable community resource. Its strengths are concrete: the dataset, code, loaders, and cross-subject splits are public, the hierarchy is large (160 labels across three levels), the recording setup covers an unusually wide set of modalities under one protocol, and the benchmark suite spans recognition, localization, anticipation, and cross-domain robustness. The paper also makes falsifiable empirical observations, such as the sharp visual-model degradation from L1 to L3 and the smaller degradation of wearable modalities, and it compares against external datasets such as Breakfast and 50 Salads. The explicit statement that action counterfactuals are descriptive rather than causal is a useful clarification. However, the paper's central promise of a fully released, hierarchically annotated dataset is currently weakened by internal inconsistencies about what is actually available and about the synchronization accuracy that underpins the cross-modal and temporal benchmarks.

major comments (4)
  1. [Appendix C.2; Appendix A.3; Abstract] The release-status statements are internally contradictory. Appendix C.2 ends with 'We plan to release the losslessly compressed version of the visual data along with the full hierarchical labels,' which implies that the full L1/L2/L3 annotations are not yet part of the public artifact, whereas the Abstract, Section 3.3, and Appendix A.3 state that the dataset with hierarchical labels and code is publicly available. Because every benchmark in Sections 5–7 and the counterfactual analysis in Section 7.3 presuppose the availability of frame-level L3 labels, the paper must either release those labels in the IEEE Dataport artifact or explicitly restrict all empirical claims to the currently released subset.
  2. [Section 1 (Challenges); Appendix D] The synchronization figures are inconsistent by two orders of magnitude: Section 1 claims 'a synchronization drift of 1 ms in 24-hour timespan,' while Appendix D states that the NTP-aligned host clocks are 'maintained under 0.1 seconds per day.' Since the cross-modal and temporal benchmarks in Sections 6 and 7 rely on frame-level alignment of all streams with L2/L3 boundaries, the authors should report the actual measured drift, the measurement method, and the worst-case alignment error relative to the shortest L3 procedure duration.
  3. [Abstract; Section 1; Section 3.1; Appendix D.1] The reported sensor count is inconsistent: the Abstract says 20 sensors, Contribution 1 says 20 data modalities from 12 sensors, Section 3.1 says 12 sensor devices, and Appendix D says 16 sensor devices were used while D.1 says 12 distinct sensor types were selected. The 'largest available dataset in terms of the number of sensors and data modalities' claim depends on these counts, so the authors need to disambiguate device units, sensor types, and derived modalities and use one consistent set of numbers.
  4. [Appendix E (Annotation Workflow)] No inter-annotator agreement or boundary-consistency metric is reported for the L1/L2/L3 frame-level annotations. Given that the dataset's central contribution is a three-level hierarchy with fine-grained L3 boundaries, the authors should report agreement statistics (e.g., kappa or segmental F1) and the number of annotators per clip to support the reliability of the annotation-based benchmarks.
minor comments (5)
  1. [Section 6.1] The text says '20% of the procedures (L3) are shared between higher-level activities (L1) and actions (L2),' which conflicts with the Abstract's 14.2% of L3 procedures shared between L2 actions; clarify the denominator and definition of sharing.
  2. [Appendix G.1.2] This section states that multimodal time-series classification 'was not successful' on L1 classes, but Section 6.2 and Table 8 report L1 accuracies for EMG + Bio and other sensor groups; the relationship between the failed experiment and the reported results should be reconciled.
  3. [Table 20 caption] Table 20 reports results on a 50% subset of L2 and L3 labels, while Table 6 reports all labels; clarify which label subset is used in each benchmark and why.
  4. [Abstract] The sentence 'multi-sensor and and its inclusion' contains a duplicated 'and' and should be corrected.
  5. [Section 3.2; Figure 2] The example in Figure 2 says the L2 action 'Rest' contains no L3 procedures, but Table 4 lists 98 L3 labels; state how non-procedural activities are represented at L3 (e.g., no label versus an explicit 'other' category).

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: DARai is an empirical dataset and benchmark report; no prediction reduces to a fitted parameter or to a self-citation.

full rationale

DARai is a data-resource paper. Its contributions are the collection, annotation, and public release of a multimodal hierarchical dataset, plus benchmark measurements. There is no derivation chain in which an output quantity is constructed from the same quantity or from a parameter fitted to it. Recognition, localization, and anticipation numbers in Sections 5-7 are empirical evaluations on the released data, and Table 29 compares DARai directly against external benchmarks (Breakfast and 50 Salads), so the headline results are self-contained against outside evidence. The self-citations (Kokilepersaud et al. 2023-2025, Kim et al. 2025-2026, Yarici et al. 2025, Prabhushankar et al. 2020-2024) appear as motivational context, related-work statements, or incidental references in evaluation-protocol citations; none of these is the sole justification for a central claim, and none imports an unverified ansatz or uniqueness theorem. No fitted input is renamed as a prediction: the cross-view, cross-body, fusion, and anticipation curves are measured outcomes. Internal inconsistencies exist (e.g., the 1 ms/24h sync claim in Section 1 vs. the NTP 'under 0.1 seconds per day' statement in Appendix D; and Appendix C.2's sentence 'We plan to release the losslessly compressed version of the visual data along with the full hierarchical labels,' which complicates the paper's public-release claims). These are verifiability and correctness concerns, not circularity: they do not make any result true by construction. Accordingly, no circular step is identified and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 1 invented entities

The paper's central claims rest on three assumptions it does not independently verify: synchronization of heterogeneous streams, validity of annotator labels without agreement metrics, and the authors' predefined decomposition rules. No fitted free parameters enter the central claims, since the paper reports measurements on its released dataset rather than a derivation; benchmark hyperparameters are standard practice and do not constitute fitted claims. One conceptual construct, action counterfactuals, is introduced and is openly flagged as descriptive rather than causal.

assumptions (3)
  • domain assumption Global timestamp alignment via NTP-synchronized host clocks is accurate enough to align all sensor streams to frame-level annotation boundaries.
    Invoked in Section 3.3 and Appendix D. The paper states 1 ms drift over 24 hours in Section 1 but 0.1 s/day in Appendix D. All cross-modal fusion and temporal benchmark results depend on this alignment being correct.
  • domain assumption Annotator-selected L1/L2/L3 labels and boundaries are reliable ground truth.
    Appendix E describes the annotation workflow and a 50% quality-control recheck, but no inter-annotator agreement metric is reported, so label reliability is asserted rather than measured.
  • ad hoc to paper The predefined decomposition rules (L1 activity to L2 actions to L3 procedures) capture how activities are actually structured in daily life.
    Sections 2 and 3.2 define the hierarchy through author-chosen rules. The claimed sharing statistics (22.7% of L2 and 14.2% or 20% of L3) are properties of this rule set, not of an external standard.
invented entities (1)
  • Action counterfactuals (descriptive) independent evidence
    purpose: Labels for observed alternative executions of the same L1 activity, either with the same L3 procedures but different L2 actions or with varied L2/L3 ordering; used to test generalization to unseen variations.
    Introduced in Sections 1 and 3.1, and explicitly stated to be descriptive, not causal. The falsifiable handle is the released recordings of the same participants doing enforced variations (heavy versus light box), which can be re-annotated or re-analyzed independently. The construct is a relabeling of existing variation in the data, not a new physical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hierarchical and Multimodal Data for Daily Activity Understanding." pith.science (2026). https://pith.science/paper/73NTN5N5

@misc{pith2026250417696,
  author       = {Pith},
  title        = {Pith review of: Hierarchical and Multimodal Data for Daily Activity Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/73NTN5N5}},
  note         = {Machine review of arXiv:2504.17696}
}
read the original abstract

Daily Activity Recordings for Artificial Intelligence (DARai, pronounced "Dahr-ree") is a multimodal, hierarchically annotated dataset constructed to understand human activities in real-world settings. DARai consists of continuous scripted and unscripted recordings of 50 participants in 10 different environments, totaling over 200 hours of data from 20 sensors including multiple camera views, depth and radar sensors, wearable inertial measurement units (IMUs), electromyography (EMG), insole pressure sensors, biomonitor sensors, and gaze tracker. To capture the complexity in human activities, DARai is annotated at three levels of hierarchy: (i) high-level activities (L1) that are independent tasks, (ii) lower-level actions (L2) that are patterns shared between activities, and (iii) fine-grained procedures (L3) that detail the exact execution steps for actions. The dataset annotations and recordings are designed so that 22.7% of L2 actions are shared between L1 activities and 14.2% of L3 procedures are shared between L2 actions. The overlap and unscripted nature of DARai allows counterfactual activities in the dataset. Experiments with various machine learning models showcase the value of DARai in uncovering important challenges in human-centered applications. Specifically, we conduct unimodal and multimodal sensor fusion experiments for recognition, temporal localization, and future action anticipation across all hierarchical annotation levels. To highlight the limitations of individual sensors, we also conduct domain-variant experiments that are enabled by DARai's multi-sensor and counterfactual activity design setup. The code, documentation, and dataset are available at the dedicated DARai website: https://alregib.ece.gatech.edu/software-and-datasets/darai-daily-activity-recordings-for-artificial-intelligence-and-machine-learning/

Figures

Figures reproduced from arXiv: 2504.17696 by the authors.

Figure 1
Figure 1. DARai multimodal data collected to capture and reflect action counterfactual variations across modalities. Hierarchical structure of DARai data, demonstrated using an example activity: Moving an Object. Two variations are shown, moving a heavy box (blue) and a light box (green), along with their corresponding lower hierarchy levels. For instance, variations in insole pressure and forearm muscle interactions are obse… view at source ↗
Figure 1
Figure 1. Therefore, integrating various data modalities help mitigate the limitations of a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. A sequence of frames depicting an example of an annotated activity sample: L1 activity ’Sleeping’ with a sequence of L2 actions: ’Prepare for Activity’ and ’Rest’, and L3 procedures: ’Lay Down’ and ’Use Blanket’. Boundaries indicate the start and end times of each annotation level within the corresponding upper-level video sample. Hierarchy Level Number of Unique Labels Example L_1: Activities 18 "Moving an Object" … view at source ↗
Figures from the paper (32 more)
Figure 3
Figure 3. Figure 3: Overview of machine learning applications using the multimodal hierarchical DARai dataset. This figure highlights key aspects explored in this work, including temporal modeling, multimodal modeling, and hierarchical modeling, as well as potential applications for the m…
Figure 4
Figure 4. Figure 4: Illustration of the unique information provided by various data modalities, highlighting the distinct insights each contributes to modeling human activity and state patterns. training and testing. To maintain consistency across all experiments, the sets of subject IDs …
Figure 5
Figure 5. Figure 5: Comparison of same-view and cross-view inference performance for ResNet, MViT-s, and Swin-t using visual data. The results show top-1 accuracy for each model when tested on the same camera view and cross views. data from a sensor worn on one side of the body, e.g., a h…
Figure 6
Figure 6. Figure 6: for the IMU and EMG sensors [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Comparison of Top-1 accuracy results for individual data modalities and data modality pairs across 3 level of DARai hierarchy. 6.1 Unimodal Analysis and Comparison [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Activity confusion matrices for unimodal and multimodal settings. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Feeding different length of sample sequence for action anticipation task. (a) shows short-term action anticipation task, and (b) shows long-term action anticipation task. 7.1 Short- and Long-Horizon Activity Anticipation Action anticipation predicts future actions from…
Figure 10
Figure 10. Figure 10: Action anticipation accuracy across different observation rates based on partially observed L1-level sequences at L2 (Action level) and L3 (Procedure level). The left graph represents L2-level action anticipation for both anticipating over the next 8 seconds (short-te…
Figure 11
Figure 11. Figure 11: This figure shows the comparison of confusion matrices demonstrating action anticipation performance when predicting next L2 labels (a) and L3 labels (b). 21 [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Action counterfactual cases demonstrating performance comparisons under different observation rates. (a) Comparison between ‘Carrying light/small object’ (factual) and ‘Carrying heavy/big object’ (counter￾factual). (b) Comparison between ‘Reading at desk’ (factual) an…
Figure 14
Figure 14. Figure 14: Overall layout of the livingroom and home-office with sensor setup [PITH_FULL_IMAGE:figures/full_fig_p036_14.png]
Figure 15
Figure 15. Figure 15: Variation in light condition. A) Shows morning light with open and close curtain. B) shows noon light with open and closed curtain. In our setup, as the cameras are fixed, we did not collect any IMU data from these cameras. Unfortunately, a limitation of the Kinect ca…
Figure 16
Figure 16. Figure 16: Azure Kinect DK Lidar Camera Our Lidar camera, the L515, is a compact model designed for indoor use and manufactured by Intel [PITH_FULL_IMAGE:figures/full_fig_p037_16.png]
Figure 17
Figure 17. Figure 17: Lidar camera L515 Heart Rate: This signal is derived from the ECG waveform by counting the peaks of each QRS complex. R-R Interval: This signal, calculated as the inverse of heart rate, is defined as the time elapsed between two successive R-waves of the QRS complex. …
Figure 18
Figure 18. Figure 18: Electrode placement 38 [PITH_FULL_IMAGE:figures/full_fig_p038_18.png]
Figure 19
Figure 19. Figure 19: PupilLabs Neon calibration-free eye tracking module [PITH_FULL_IMAGE:figures/full_fig_p039_19.png]
Figure 20
Figure 20. Figure 20: Insoles with 8 point pressure sensors 39 [PITH_FULL_IMAGE:figures/full_fig_p039_20.png]
Figure 21
Figure 21. Figure 21: Step 1, selecting the Level 1 label associated with the video clip. [PITH_FULL_IMAGE:figures/full_fig_p042_21.png]
Figure 22
Figure 22. Figure 22: Step 2, selecting associated level 2 label with the video clip and the higher level label. When annotators identify the L1 label, they will access hierarchical labels specific to that instance of the L1 label, as depicted in [PITH_FULL_IMAGE:figures/full_fig_p042_22.png]
Figure 23
Figure 23. Figure 23: Step 3, selecting associated level 3 and then level 4 label with the video clip and the higher level label. 42 [PITH_FULL_IMAGE:figures/full_fig_p042_23.png]
Figure 24
Figure 24. Figure 24: Class distributions of Level 2 actions (left) and Level 3 procedures (right) in the DARai hierarchy. The corresponding class labels are listed in Appendix E in the same order as presented in the plot. 55 [PITH_FULL_IMAGE:figures/full_fig_p055_24.png]
Figure 25
Figure 25. Figure 25: Schedule [PITH_FULL_IMAGE:figures/full_fig_p056_25.png]
Figure 26
Figure 26. Figure 26: Blurred Face Example 56 [PITH_FULL_IMAGE:figures/full_fig_p056_26.png]
Figure 27
Figure 27. Figure 27: Demographic and BMI distribution of DARai’s subjects [PITH_FULL_IMAGE:figures/full_fig_p057_27.png]
Figure 28
Figure 28. Figure 28: Participants exhaustion and dehydration levels after completing one recording session 57 [PITH_FULL_IMAGE:figures/full_fig_p057_28.png]
Figure 29
Figure 29. Figure 29: Participants’ experience of comfortness with each sensor rated from 0 to 5. 0 means ’did not notice it’ and 5 means ’very obtrusive’ [PITH_FULL_IMAGE:figures/full_fig_p058_29.png]
Figure 30
Figure 30. Figure 30: Overview of the GAF image pipeline 58 [PITH_FULL_IMAGE:figures/full_fig_p058_30.png]
Figure 31
Figure 31. Figure 31: Combining modalities to learn a shared representation space enables the model to learn from different patterns for the same activity classes and improves the accuracy of activity understanding. 59 [PITH_FULL_IMAGE:figures/full_fig_p059_31.png]
Figure 32
Figure 32. Figure 32: This visualization depicts the definition of segmentation task. [PITH_FULL_IMAGE:figures/full_fig_p060_32.png]
Figure 33
Figure 33. Figure 33: This visualization highlights the comparison between L2 segmentation perfor￾mance and L3 segmentation performance [PITH_FULL_IMAGE:figures/full_fig_p060_33.png]
Figure 34
Figure 34. Figure 34: This visualization depicts the definition of anticipation task. [PITH_FULL_IMAGE:figures/full_fig_p060_34.png]
Figure 35
Figure 35. Figure 35: This visualization highlights the comparison between L2 anticipation performance and L3 anticipation performance. 61 [PITH_FULL_IMAGE:figures/full_fig_p061_35.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-level and Multi-modal Action Anticipation

    cs.CV 2025-06 reject novelty 4.0 of 10

    A multi-modal action anticipation model with a clustering-based fine-grained text generator and temporal consistency loss reports state-of-the-art gains, though key claims are not fully supported by the provided evidence.

Reference graph

Works this paper leans on

32 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [2]

    doi: 10.3390/s21186037

    ISSN 1424-8220. doi: 10.3390/s21186037. URL https://www.mdpi.com/1424-8220/21/18/60

    doi:10.3390/s21186037

  2. [4]

    doi: 10.3390/s23052715

    ISSN 1424-8220. doi: 10.3390/s23052715. URLhttp://dx.doi .org/10.3390/s23052715. Sourav Garg, Niko Suenderhauf, and Michael Milford. Semantic–geometric visual place recognition: a new perspective for reconciling opposing views.The International Journal of Robotics Research, 41(6):573–598,

    work page doi:10.3390/s23052715

  3. [5]

    The choice of the number of subsequences was made based on the original data length and sampling rate

    The resulting feature maps are then fed into two self-attention blocks, each with two heads. The choice of the number of subsequences was made based on the original data length and sampling rate. Timeseries Image RepresentationWe extract features from these 1D data using Gramian Angular Field (GAF) to create a 2D representation from the signal sequences. ...

    work page 2015

  4. [8]

    Qiuhong Ke, Mario Fritz, and Bernt Schiele

    URL https://dx.doi.org/10.21227/ecnr-hy49. Qiuhong Ke, Mario Fritz, and Bernt Schiele. Time-conditioned action anticipation in one shot. InCVPR, pages 9925–9934,

    work page doi:10.21227/ecnr-hy49

  5. [9]

    Seulgi Kim, Ghazal Kaviani, Mohit Prabhushankar, and Ghassan AlRegib

    doi: 10.1109/IJCB52358.2021.9484410. Seulgi Kim, Ghazal Kaviani, Mohit Prabhushankar, and Ghassan AlRegib. Multi-level and multi-modal action anticipation.arXiv preprint arXiv:2506.02382,

    arXiv 2021

  6. [12]

    Probing Fine-Grained Action Understanding and Cross-View Generalization of Foundation Models

    Thinesh Thiyakesan Ponbagavathi, Kunyu Peng, and Alina Roitberg. Probing fine-grained action understanding and cross-view generalization of foundation models.arXiv preprint arXiv:2407.15605,

    work page Pith review arXiv

  7. [13]

    Counterfactual Gradients-based Quantification of Prediction Trust in Neural Networks

    Mohit Prabhushankar and Ghassan AlRegib. Counterfactual gradients-based quantification of prediction trust in neural networks.arXiv preprint arXiv:2405.13758,

    work page Pith review arXiv

  8. [15]

    Vladimir M Sloutsky

    doi: 10.1109/TPAMI.2023.3284080. Vladimir M Sloutsky. From perceptual categories to concepts: What develops?Cognitive science, 34(7):1244–1286,

    arXiv 2023

Show all 32 references
  1. [16]

    Combining embedded accelerometers with com- puter vision for recognizing food preparation activities

    Sebastian Stein and Stephen J McKenna. Combining embedded accelerometers with com- puter vision for recognizing food preparation activities. InProceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, pages 729–738,

    2013

  2. [17]

    URL https://dx.doi.org/10.21227/ehzr-w794. D. Temel, G. Kwon, M. Prabhushankar, and G. AlRegib. CURE-TSR: Challenging unreal and real environments for traffic sign recognition. InNeural Information Processing Systems (NeurIPS) Workshop on Machine Learning for Intelligent Trans...

    doi

  3. [19]

    Keze Wang, Xiaolong Wang, Liang Lin, Meng Wang, and Wangmeng Zuo

    doi: 10.1109/ACCESS.202 3.3282311. Keze Wang, Xiaolong Wang, Liang Lin, Meng Wang, and Wangmeng Zuo. 3d human activity recognition with reconfigurable convolutional neural networks. InProceedings of the 22nd ACM international conference on Multimedia, pages 97–106,

    doi

  4. [20]

    Zhiguang Wang and Tim Oates

    URLhttps://cvpr.thecvf.com/virtual/2023/tutorial/18564. Zhiguang Wang and Tim Oates. Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. InWorkshops at the twenty-ninth AAAI conference on artificial intelligence,

    2023

  5. [21]

    Input Sequence Length

    Segmenting untrimmed procedural activities into lower-level action sequences is inherently difficult due to occlusion, camera viewpoints, and unstructured transitions. As seen in Table 1 of (Singhania et al., 2023), existing datasets such as Breakfast (MoF 0.69), 50Salads (MoF...

    2023

  6. [22]

    Jinglin Xu, Guohao Zhao, Sibo Yin, Wenhao Zhou, and Yuxin Peng

    doi: 10.1109/ACCESS.2020.3032699. Jinglin Xu, Guohao Zhao, Sibo Yin, Wenhao Zhou, and Yuxin Peng. Finesports: A multi-person hierarchical sports video dataset for fine-grained action understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2020

  7. [23]

    Sub- ject invariant contrastive learning for human activity recognition.arXiv preprint arXiv:2507.03250,

    Yavuz Yarici, Kiran Kokilepersaud, Mohit Prabhushankar, and Ghassan AlRegib. Sub- ject invariant contrastive learning for human activity recognition.arXiv preprint arXiv:2507.03250,

    arXiv

  8. [25]

    The license associated with our dataset is a Creative Commons International 4 license

    The code is associated with an MIT License. The license associated with our dataset is a Creative Commons International 4 license. A.3 Maintenance Plan The dataset is provided on IEEE Dataport as an open access dataset and IEEE subscription is not required to access or downloa...

    2006

  9. [26]

    The optimal configuration is then passed as parameters during recording

    We can calibrate this camera by adjusting sensor parameters and modes, balancing exposure, and setting white points, among other settings. The optimal configuration is then passed as parameters during recording. We have 36 DARai: Daily Activity Recordings for AI and ML applica...

    2023

  10. [27]

    Calibration of this camera involves adjusting the RGB and depth settings separately and then aligning the outputs of the two sensors to achieve higher quality depth and RGB frames

    It features a separate RGB sensor and a laser emitter for depth calculation. Calibration of this camera involves adjusting the RGB and depth settings separately and then aligning the outputs of the two sensors to achieve higher quality depth and RGB frames. We tested these cam...

    2000

  11. [28]

    The results for different architectures can be found in Appendix G

    For multichannel time series signals (IMU, EMG, insole, biomonitor, and gaze data), we use a transformer encoder (Khaertdinov et al., 2021). The results for different architectures can be found in Appendix G. All signals are resampled to a fixed-size sequence, with 100 to 2000...

    2021

  12. [30]

    47 Ka viani, Yarici, Kim, Prabhushankar, AlRegib, Solh and Patil Average Class Accuracy of L2 and L3 Classess for visual data Data Modalities Training Model View Level 2 Level3 RGB R3Dcam 1 0.30 0.029 RGB R3Dcam 2 0.72 0.67 RGB mvit_v2_scam 1 0.027 0.017 RGB mvit_v2_scam 2 0.5...

    2020

  13. [32]

    As the dataset was recorded at 15 frames per second (fps), we parsed the data by sampling one frame every 15 frames

    For this task, we used RGB frame features extracted from each sample video and a window size of eight consecutive frames to benchmark the decomposition of activity samples into lower-level steps (L2 and L3 labels) from different camera views. As the dataset was recorded at 15 ...

    2021

  14. [1995]

    doi: 10.1145/219717.219748

    ISSN 0001-0782. doi: 10.1145/219717.219748. URLhttps://doi.org/10.114 5/219717.219748. David L Mills.Computer network time synchronization: the network time protocol. CRC press,

  15. [2016]

    A large-scale varying-view rgb-d action dataset for arbitrary-view human action recognition

    Yanli Ji, Feixiang Xu, Yang Yang, Fumin Shen, Heng Tao Shen, and Wei-Shi Zheng. A large-scale varying-view rgb-d action dataset for arbitrary-view human action recognition. arXiv preprint arXiv:1904.10681,

    1904 arXiv

  16. [2017]

    Xia, C.C

    L. Xia, C.C. Chen, and JK Aggarwal. View invariant human action recognition using histograms of 3d joints. InComputer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, pages 20–27. IEEE,

    2012

  17. [2019]

    doi: 10.1109/TI TS.2019.2931429

    ISSN 1524-9050. doi: 10.1109/TI TS.2019.2931429. Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. InProceedings of the IEEE conference on Computer Vision and Pattern Recogniti...

    2019

  18. [2020]

    Coarse to fine multi-resolution temporal convolutional network.arXiv preprint arXiv:2105.10859,

    Dipika Singhania, Rahul Rahaman, and Angela Yao. Coarse to fine multi-resolution temporal convolutional network.arXiv preprint arXiv:2105.10859,

    arXiv

  19. [2021]

    Siddhant Bansal, Chetan Arora, and C. V. Jawahar. My view is the best view: Procedure learning from egocentric videos. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors,Computer Vision – ECCV 2022, pages 657–675, Cham,

    2022

  20. [2022]

    Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Tri- antafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. arXiv prepri...

    arXiv

  21. [2023]

    URL https://arxiv.org/abs/2304.05170. Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. Scaling egocentric vision: The epic-kitchens dataset. In...

    arXiv

  22. [2024]

    URLhttps://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.13680

    doi: https://doi.org/10.1111/ exsy.13680. URLhttps://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.13680. 25 Ka viani, Yarici, Kim, Prabhushankar, AlRegib, Solh and Patil Ghazal Kaviani, Yavuz Yarici, Mohit Prabhushankar, Ghassan AlRegib, Mashhour Solh, and Ameya Patil. Darai: ...

    doi

  23. [2025]

    Towards automatic learning of procedures from web instructional videos.Proceedings of the AAAI Conference on Artificial Intelligence, 32 (1), Apr

    Luowei Zhou, Chenliang Xu, and Jason Corso. Towards automatic learning of procedures from web instructional videos.Proceedings of the AAAI Conference on Artificial Intelligence, 32 (1), Apr. 2018a. doi: 10.1609/aaai.v32i1.12342. URLhttps://ojs.aaai.org/index.php /AAAI/article/...

    doi

  24. [2026]

    Clinically labeled contrastive learning for oct biomarker classification

    Kiran Kokilepersaud, Stephanie Trejo Corona, Mohit Prabhushankar, Ghassan AlRegib, and Charles Wykoff. Clinically labeled contrastive learning for oct biomarker classification. IEEE Journal of Biomedical and Health Informatics, 27(9):4397–4408, 2023a. doi: 10.110 9/JBHI.2023.3...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.