Pith. sign in

REVIEW 3 major objections 4 minor 49 references

Domain Generalization using Action Sequences for Egocentric Action Recognition

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Action sequences that repeat across kitchens let egocentric action recognition generalize to unseen environments.

desk verdict A credible sequence-based DG method with genuinely thorough ablations, but the headline +2.4 gain is not trustworthy until the model-selection protocol is disclosed. read the letter →

arxiv 2506.17685 v1 pith:HZCMOUJE submitted 2025-06-21 cs.CV

Amirshayan Nasirimajd , Chiara Plizzari , Simone Alberto Peirone , Marco Ciccone , Giuseppe Averta , Barbara Caputo This is my paper · ORCID
classification cs.CV
keywords domaingeneralizationegocentricactionrecognitionsequencesmaskedsequencereconstructioncross-modallearningSeqDGEPIC-KITCHENS-100EGTEA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that egocentric action recognition models fail in new environments because they latch onto visual cues specific to one kitchen, and that the fix is to exploit the fact that goal-directed action sequences—take eggs, crack them, fry them—repeat across kitchens even when appearances differ. To test this, the authors build SeqDG, which trains a transformer on windows of actions, masks the central action, and reconstructs it from surrounding visual and textual features, while a SeqMix augmentation swaps in same-label actions from other domains. On EPIC-KITCHENS-100 this yields a +2.4% relative average improvement in cross-domain action accuracy over its source-only baseline, and on EGTEA it reaches 74.0% Top-1 accuracy, surpassing previous methods. Text is used only during training, so at inference the model needs only video.

What carries the argument

The central object is the action sequence $S_i = \{a_{i-W/2}, \ldots, a_i, \ldots, a_{i+W/2}\}$, an ordered window of $W$ actions centered on the target, each represented by frozen visual and textual features. The mechanism is masked cross-modal reconstruction, called SeqRec, combined with a mixing augmentation, SeqMix. In SeqRec, the central action's features are zeroed and two transformer decoders reconstruct them, with the visual decoder cross-attending to unmasked textual features and the textual decoder cross-attending to unmasked visual features; the classification head uses a CLS token. SeqMix replaces an action with a same-label action from another source domain with probability 0.5. The full objective is $\mathcal{L} = \mathcal{L}_C + \lambda_{rV}\mathcal{L}_{rV} + \lambda_{rT}\mathcal{L}_{rT}$, jointly training classification, visual reconstruction, and textual reconstruction. This machinery is what forces the encoder to rely on relational, domain-agnostic structure instead of per-domain appearance.

What would settle it

Run SeqDG and its source-only baseline on a benchmark whose action orderings differ by domain (or on a subset of EPIC-KITCHENS-100 selected for low sequence repetition), and check whether the +2.4% improvement disappears or reverses; alternatively, compute the correlation between per-domain sequence repetition counts and SeqDG's gain and look for a near-zero slope.

Watch

Extended reading notes

Core claim

Human actions do not occur in isolation: the order in which people perform goal-directed steps, such as opening the fridge, picking up milk, opening the lid, and pouring milk, stays consistent across kitchens and users even when the visual appearance shifts. SeqDG turns this regularity into a training objective: features from $W$ surrounding actions are encoded through a transformer, the central action's visual and textual features are masked, and two decoders reconstruct the masked features by cross-attending to the other, unmasked modality. This reconstruction pushes the encoder to capture temporal and cross-modal dependencies rather than domain-specific appearance, and SeqMix further widens the training distribution by replacing the target action with a same-label action from a different source domain. The result is a model that generalizes to unseen kitchens and temporal shifts without ever seeing target data, and that outperforms both source-only baselines and existing domain-adaptation methods.

Load-bearing premise

The method assumes that the order in which people perform actions is largely the same across environments; if a deployment setting has no repeated action sequences, the reconstruction objective will learn source-specific statistics rather than general temporal structure.

Editorial extensions

If this is right

  • Sequence context alone, without text, already improves cross-domain accuracy by +1.3%, so the temporal ordering of actions is itself a generalizable cue.
  • SeqDG outperforms UDA methods that use unlabeled target data during training, suggesting target-free domain generalization can be competitive with adaptation.
  • Gains are larger for Noun and Action metrics than for Verb, consistent with the idea that manipulated objects persist across consecutive actions more reliably than verbs.
  • The approach improves results across TSN, I3D, TSM, and TBN backbones, so the sequence-reconstruction objective is not tied to one visual encoder.
  • Because text is absent at inference, the method can be deployed with video-only inputs, and test-time language-model filtering adds little.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If sequence regularity is the active ingredient, cross-domain gains should shrink on datasets with weakly ordered or random activities; a direct test would correlate per-domain sequence repetition counts with SeqDG's improvement over its baseline.
  • SeqMix's constraint that swapped actions share a label may leak label information across domains; an ablation swapping same-verb but different-noun actions could isolate how much of the gain comes from the mixing rule rather than cross-modal reconstruction.
  • The same recipe could transfer to procedure-heavy first-person tasks such as surgery or assembly, provided the domain's sequence repetition rate is checked first.
  • The choice of text encoder matters little (CLIP, MiniLM, BERT give similar results), which suggests the reconstruction objective is doing the work rather than the specific language model.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes SeqDG, a domain generalization method for egocentric action recognition. The key idea is that action sequences are more consistent across visual domains than individual action appearances. SeqDG uses two components: SeqRec, a visual-text sequence reconstruction objective that masks the central action and reconstructs it from surrounding actions using cross-attention between visual and textual features, and SeqMix, a data augmentation that replaces an action in a sequence with another action from a different source domain but with the same label. The training loss combines a classification loss on the central action with two reconstruction losses. The method is evaluated on EPIC-KITCHENS-100 in a cross-domain setting and on EGTEA in an intra-domain setting, reporting improvements over source-only baselines and prior methods, including a +2.4 Action Top-1 gain on EPIC-KITCHENS-100 and 74.0 Top-1 accuracy on EGTEA.

Significance. The idea of leveraging action-sequence consistency across visual domains is timely and intuitively plausible, especially for scripted activities such as cooking. The paper is thorough in its experimental coverage: ablations of each component, multiple modalities (RGB, Flow, Audio), several backbones (TSN, I3D, TSM, TBN), sequence lengths, language models, and loss weights, and the authors release code and data. If the empirical claims survive unbiased model selection, SeqDG would be a solid contribution to egocentric domain generalization. At present, however, the central empirical claims are weakened by apparent target-validation-based hyperparameter selection and by the absence of uncertainty estimates, and the reported gains over the nearest competitors are small.

major comments (3)
  1. [§4.4, Tables 1, 4, 6, 7, 8, Fig. 5] All cross-domain results are reported on the EPIC-KITCHENS-100 target validation set, as stated in the caption of Table 1, and the ablation and hyperparameter analyses in Tables 4, 6, 7, 8 and Figure 5 do not indicate that a source-only validation split was used for model selection. Because the values of λ_rV, λ_rT, sequence length W, SeqMix probability p, language model, and backbone are all selected using these target-validation numbers, the reported +2.4 Action Top-1 improvement over Source Only is not an unbiased estimate for truly unseen environments. This is load-bearing: in Table 8, varying W from 3 to 7 changes RGB-only Action accuracy by 0.7 points around the chosen W=5, which is the same order as the headline gain. Please specify the model-selection protocol and either re-run the experiments with source-only validation for all hyperparameter choices or report the full grid of results on a source-only validation split.
  2. [§4.4, Tables 1 and 3] No variance or significance is reported. In Table 1, SeqDG's Action Top-1 accuracy (20.6) differs from EgoZAR (20.3) and RNA (20.0) by at most 0.6 points, and in Table 3 the EGTEA lead over MTCN is 1.5 points. Without standard deviations across seeds or significance tests, these gaps are within the range of noise typical of this benchmark, so the claims of 'state-of-the-art' performance and '+2.4%' improvement are not yet supported. Please report means and standard deviations over at least three runs and, for the main SOTA comparisons, a paired significance test or confidence intervals. In addition, the Source Only baselines differ across method blocks in Table 1 (e.g., Action accuracies of 19.2, 19.0, 19.6, 18.2, 19.4), so comparing relative improvements across methods is not a controlled comparison.
  3. [§3.2, §4.3] The textual narrations used for L_rT are described in §4.3 as 'simple verb and noun combinations', which are effectively close to the action labels themselves. If this is the case, the reconstruction objective L_rT in Eq. (11) may be little more than masked-label prediction rather than a domain-agnostic semantic prior, which weakens the claim that text is 'less tied to the domain' (§3). This does not invalidate the method, but it affects the interpretation of SeqRec and of the SeqDG (No text) ablation. Please clarify what the textual branch encodes, compare against a variant that uses only label embeddings for the text branch, or evaluate on a benchmark with genuinely free-form narrations.
minor comments (4)
  1. [Abstract] The abstract states '+2.4% relative average improvement', but Table 1 shows an absolute increase of 2.4 percentage points (18.2 to 20.6 Action Top-1), which is a relative improvement of about 13.2%. Please use the correct terminology.
  2. [Eq. (5), §4.3] Equation (5) gives Z_V ∈ R^{(W+2)×D} and the text says 'append a learnable classification token', while §4.3 mentions separate classification tokens for Verb and Noun. Please clarify how many tokens are appended and align the dimension notation.
  3. [Fig. 2] The caption of Figure 2 says the counts are for 'sub-sequences of any smaller (or equal) length that repeat across domains'; this makes the reported numbers difficult to interpret. Please state whether the counts are unique repeated sequences or total occurrences, and whether shorter sequences are counted within longer ones.
  4. [§3.5, Tables 2 and 3] Section 3.5 states that textual annotations are not used at inference time, but Tables 2 and 3 report results for SeqDG† and MTCN† that use a language model at test time to filter predictions. Please clarify what the LM filtering uses and whether this contradicts the earlier statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SeqDG's reported gains are measured on held-out target splits, and its auxiliary reconstruction losses are not equivalent to the classification output.

full rationale

SeqDG's derivation chain is empirical rather than analytic. The method trains a sequence encoder with a classification head on the central action, auxiliary visual/text reconstruction losses, and a label-conditioned sequence mixing augmentation, all on source domains, and then evaluates on disjoint target kitchens. No reported quantity is algebraically derived from a fitted parameter or from the target label by construction: the +2.4% Action Top-1 gain on EPIC-KITCHENS-100 cross-domain and the +0.6% gain over SOTA on EGTEA are measured accuracies on held-out evaluation splits. The reconstruction targets are masked visual and text features (Eq. 6-7), not the classification logits, and text is not used at inference; the no-text ablation (SeqDG No text, +1.3%) shows the text-based reconstruction is not the sole source of the effect. Self-citations to CIR, EgoZAR, and RNA appear as baselines or as background support for domain-shift intuition, not as load-bearing uniqueness theorems or as justification of the architecture. The main validity concern noted in review—possible hyperparameter selection on the target validation split—would be evaluation leakage rather than circularity, and the paper does not expose any equation or fitted parameter that reduces the prediction to its input.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

SeqDG introduces no new physical or mathematical entities. All assumptions are standard background in the video-understanding literature or are domain assumptions about the plausibility of sequence consistency across kitchens.

free parameters (4)
  • lambda_rV = 1.0
    Weight for the visual reconstruction loss LrV; tuned over the range 0.5-4.5 in Figure 5 and set to 1 for all main results.
  • lambda_rT = 1.0
    Weight for the text reconstruction loss LrT; tuned similarly in Figure 5 and set to 1.
  • sequence_length W = 5
    Number of actions in the sequence; chosen based on Table 8, best over W=3,5,7 for RGB-only EPIC-KITCHENS-100.
  • SeqMix probability p = 0.5
    Probability of replacing an action in a sequence during SeqMix augmentation (Section 3.3).
assumptions (4)
  • standard math Transformer self-attention and cross-attention layers operate as standard (Eqs. 2-8).
    The architecture relies on vanilla attention blocks; no modification is proven, so correctness is assumed from prior work.
  • domain assumption Action sequences are frequently repeated across different kitchens and users (Section 1, Figure 2).
    This repeatability is the empirical justification for the entire approach; if sequences differ across domains, the reconstruction objective may not transfer.
  • domain assumption Text narrations in the source domain are available and are less domain-dependent than pixels (Section 3.2).
    The SeqRec objective conditions reconstruction on text features; if narrations are absent or domain-specific, the method's benefits hinge on the visual-only variant (SeqDG No text) for support.
  • domain assumption Pre-extracted features (TBN, SlowFast) contain enough information about actions and are used consistently across methods (Section 4.3).
    All comparisons share this feature foundation, but the method does not train the visual backbone, so gains are relative to features that may already encode domain-specific cues.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Domain Generalization using Action Sequences for Egocentric Action Recognition." pith.science (2026). https://pith.science/paper/HZCMOUJE

@misc{pith2026250617685,
  author       = {Pith},
  title        = {Pith review of: Domain Generalization using Action Sequences for Egocentric Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HZCMOUJE}},
  note         = {Machine review of arXiv:2506.17685}
}
read the original abstract

Recognizing human activities from visual inputs, particularly through a first-person viewpoint, is essential for enabling robots to replicate human behavior. Egocentric vision, characterized by cameras worn by observers, captures diverse changes in illumination, viewpoint, and environment. This variability leads to a notable drop in the performance of Egocentric Action Recognition models when tested in environments not seen during training. In this paper, we tackle these challenges by proposing a domain generalization approach for Egocentric Action Recognition. Our insight is that action sequences often reflect consistent user intent across visual domains. By leveraging action sequences, we aim to enhance the model's generalization ability across unseen environments. Our proposed method, named SeqDG, introduces a visual-text sequence reconstruction objective (SeqRec) that uses contextual cues from both text and visual inputs to reconstruct the central action of the sequence. Additionally, we enhance the model's robustness by training it on mixed sequences of actions from different domains (SeqMix). We validate SeqDG on the EGTEA and EPIC-KITCHENS-100 datasets. Results on EPIC-KITCHENS-100, show that SeqDG leads to +2.4% relative average improvement in cross-domain action recognition in unseen environments, and on EGTEA the model achieved +0.6% Top-1 accuracy over SOTA in intra-domain action recognition.

Figures

Figures reproduced from arXiv: 2506.17685 by the authors.

Figure 1
Figure 1. Egocentric Action Recognition (EAR) models struggle in classifying [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Number of repeated Verb, Noun and Action sequences in the EPIC [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. SeqDG architecture. We are given visual and textual inputs X V and X T . A classification token CLS is appended to the visual input for classifica￾tion. Visual inputs are fed to an encoder ENCV , resulting in intermediate visual embeddings Z V i , while textual features are passed through an identity function to get Z T i . The latter are masked (Z¯ V i and Z¯ T i ) and fed to two separate decoders DECV and DECT for… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: SeqMix. Given a sequence from domain di , we replace action ai in the sequence with a different action aj from another sequence belonging to a different domain dj . features and σ(·) is the softmax function. The decoders DECV and DECT are trained by minimizing the L2 d…
Figure 5
Figure 5. Figure 5: Parameter analysis of the weights associated with the visual and tex [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 42 canonical work pages

  1. [1]

    Personalized dy- namics models for adaptive assistive navigation systems,

    E. OhnBar, K. Kitani, and C. Asakawa, “Personalized dy- namics models for adaptive assistive navigation systems,” inCORL, 2018

    work page 2018

  2. [2]

    Egocentric future localization,

    H. S. Park, J.-J. Hwang, Y . Niu, and J. Shi, “Egocentric future localization,” inCVPR, 2016

    work page 2016

  3. [3]

    An outlook into the future of egocentric vision,

    C. Plizzari, G. Goletto, A. Furnari, S. Bansal, F. Ragusa, G. M. Farinella, D. Damen, and T. Tommasi, “An outlook into the future of egocentric vision,”IJCV, 2024

    work page 2024

  4. [4]

    What can a cook in italy teach a mechanic in india? action recognition generalisation over scenarios and locations,

    C. Plizzari, T. Perrett, B. Caputo, and D. Damen, “What can a cook in italy teach a mechanic in india? action recognition generalisation over scenarios and locations,” inICCV, 2023

    work page 2023

  5. [5]

    Egocentric zone-aware ac- tion recognition across environments,

    S. A. Peirone, G. Goletto, M. Planamente, A. Bottino, B. Caputo, and G. Averta, “Egocentric zone-aware ac- tion recognition across environments,”Pattern Recogni- tion Letters, 2025

    work page 2025

  6. [6]

    A deeper dive into what deep spatiotemporal networks encode: Quantifying static vs. dynamic information,

    M. Kowal, M. Siam, M. A. Islam, N. D. Bruce, R. P. Wildes, and K. G. Derpanis, “A deeper dive into what deep spatiotemporal networks encode: Quantifying static vs. dynamic information,” inCVPR, 2022

    work page 2022

  7. [7]

    Unbiased look at dataset bias,

    A. Torralba and A. A. Efros, “Unbiased look at dataset bias,” inCVPR, 2011

    work page 2011

  8. [8]

    Hierarchical schemas and goals in the control of sequential behavior

    R. P. Cooper and T. Shallice, “Hierarchical schemas and goals in the control of sequential behavior.” 2006

    work page 2006

Show all 49 references
  1. [9]

    With a little help from my temporal context: Multimodal egocentric action recognition,

    E. Kazakos, J. Huh, A. Nagrani, A. Zisserman, and D. Damen, “With a little help from my temporal context: Multimodal egocentric action recognition,” inBMVC, 2021

    2021

  2. [10]

    Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100,

    D. Damen, H. Doughty, G. M. Farinella, A. Furnari, J. Ma, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, and M. Wray, “Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100,”IJCV, 2022

    2022

  3. [11]

    Multi-modal domain adaptation for fine-grained action recognition,

    J. Munro and D. Damen, “Multi-modal domain adaptation for fine-grained action recognition,” inCVPR, 2020

    2020

  4. [12]

    Spatio-temporal contrastive domain adaptation for action recognition,

    X. Song, S. Zhao, J. Yang, H. Yue, P. Xu, R. Hu, and H. Chai, “Spatio-temporal contrastive domain adaptation for action recognition,” inCVPR, 2021

    2021

  5. [13]

    Learning cross-modal contrastive features for video domain adaptation,

    D. Kim, Y .-H. Tsai, B. Zhuang, X. Yu, S. Sclaroff, K. Saenko, and M. Chandraker, “Learning cross-modal contrastive features for video domain adaptation,” in ICCV, 2021

    2021

  6. [14]

    Quo vadis, action recog- nition? a new model and the kinetics dataset,

    J. Carreira and A. Zisserman, “Quo vadis, action recog- nition? a new model and the kinetics dataset,” inCVPR, 2017. 8

    2017

  7. [15]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” inICCV, 2019

    2019

  8. [16]

    Temporal segment networks: Towards good practices for deep action recognition,

    L. Wang, Y . Xiong, Z. Wang, Y . Qiao, D. Lin, X. Tang, and L. Van Gool, “Temporal segment networks: Towards good practices for deep action recognition,” inECCV, 2016

    2016

  9. [17]

    Tsm: Temporal shift module for efficient video understanding,

    J. Lin, C. Gan, and S. Han, “Tsm: Temporal shift module for efficient video understanding,” inICCV, 2019

    2019

  10. [18]

    Synchronization is all you need: Exocentric-to-egocentric transfer for temporal action seg- mentation with unlabeled synchronized video pairs,

    C. Quattrocchi, A. Furnari, D. Di Mauro, M. V . Giuffrida, and G. M. Farinella, “Synchronization is all you need: Exocentric-to-egocentric transfer for temporal action seg- mentation with unlabeled synchronized video pairs,” in ECCV, 2024

    2024

  11. [19]

    Joint-bone fusion graph convolutional network for semi-supervised skeleton action recognition,

    Z. Tu, J. Zhang, H. Li, Y . Chen, and J. Yuan, “Joint-bone fusion graph convolutional network for semi-supervised skeleton action recognition,”IEEE TMM, 2022

    2022

  12. [20]

    Motion-driven visual tempo learning for video-based action recognition,

    Y . Liu, J. Yuan, and Z. Tu, “Motion-driven visual tempo learning for video-based action recognition,”IEEE TIP, 2022

    2022

  13. [21]

    Multimodal distillation for egocentric ac- tion recognition,

    G. Radevski, D. Grujicic, M. Blaschko, M.-F. Moens, and T. Tuytelaars, “Multimodal distillation for egocentric ac- tion recognition,” inICCV, 2023

    2023

  14. [22]

    Exploring missing modality in multimodal egocentric datasets,

    M. Ramazanova, A. Pardo, H. Alwassel, and B. Ghanem, “Exploring missing modality in multimodal egocentric datasets,”arXiv preprint arXiv:2401.11470, 2024

    2024 arXiv

  15. [24]

    Audiovisual slowfast networks for video recog- nition,

    F. Xiao, Y . J. Lee, K. Grauman, J. Malik, and C. Feicht- enhofer, “Audiovisual slowfast networks for video recog- nition,”arXiv preprint arXiv:2001.08740, 2020

    2001 arXiv

  16. [25]

    Tim: A time interval machine for audio- visual action recognition,

    J. Chalk, J. Huh, E. Kazakos, A. Zisserman, and D. Damen, “Tim: A time interval machine for audio- visual action recognition,” inCVPR, 2024

    2024

  17. [26]

    Masked video and body-worn imu autoencoder for egocentric action recognition,

    M. Zhang, Y . Huang, R. Liu, and Y . Sato, “Masked video and body-worn imu autoencoder for egocentric action recognition,” inECCV, 2024

    2024

  18. [27]

    Long-form video-language pre-training with multimodal temporal contrastive learning,

    Y . Sun, H. Xue, R. Song, B. Liu, H. Yang, and J. Fu, “Long-form video-language pre-training with multimodal temporal contrastive learning,” inNeurIPS, 2022

    2022

  19. [28]

    Temporal attentive alignment for large-scale video domain adaptation,

    M.-H. Chen, Z. Kira, G. AlRegib, J. Yoo, R. Chen, and J. Zheng, “Temporal attentive alignment for large-scale video domain adaptation,” inICCV, 2019

    2019

  20. [29]

    Interact before align: Leveraging cross-modal knowledge for do- main adaptive action recognition,

    L. Yang, Y . Huang, Y . Sugano, and Y . Sato, “Interact before align: Leveraging cross-modal knowledge for do- main adaptive action recognition,” inCVPR, 2022

    2022

  21. [30]

    Videodg: Generalizing temporal relations in videos to novel domains,

    Z. Yao, Y . Wang, J. Wang, S. Y . Philip, and M. Long, “Videodg: Generalizing temporal relations in videos to novel domains,”IEEE TPAMI, 2021

    2021

  22. [31]

    Relative norm alignment for tackling domain shift in deep multi-modal classification,

    M. Planamente, C. Plizzari, S. A. Peirone, B. Caputo, and A. Bottino, “Relative norm alignment for tackling domain shift in deep multi-modal classification,”IJCV, 2024

    2024

  23. [32]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” in CVPR, 2022

    2022

  24. [33]

    Videomae: Masked autoencoders are data-efficient learners for self- supervised video pre-training,

    Z. Tong, Y . Song, J. Wang, and L. Wang, “Videomae: Masked autoencoders are data-efficient learners for self- supervised video pre-training,” inNeurIPS, 2022

    2022

  25. [34]

    Videomae v2: Scaling video masked autoencoders with dual masking,

    L. Wang, B. Huang, Z. Zhao, Z. Tong, Y . He, Y . Wang, Y . Wang, and Y . Qiao, “Videomae v2: Scaling video masked autoencoders with dual masking,” inCVPR, 2023

    2023

  26. [35]

    Multimodal masked autoencoders learn transferable representations,

    X. Geng, H. Liu, L. Lee, D. Schuurmans, S. Levine, and P. Abbeel, “Multimodal masked autoencoders learn transferable representations,”arXiv preprint arXiv:2205.14204, 2022

    2022 arXiv

  27. [36]

    Language-driven representation learning for robotics,

    S. Karamcheti, S. Nair, A. S. Chen, T. Kollar, C. Finn, D. Sadigh, and P. Liang, “Language-driven representation learning for robotics,”arXiv preprint arXiv:2302.12766, 2023

    2023 arXiv

  28. [37]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inNeurIPS, 2017

    2017

  29. [38]

    mixup: Beyond empirical risk minimization,

    H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez- Paz, “mixup: Beyond empirical risk minimization,”arXiv preprint arXiv:1710.09412, 2017

    2017 arXiv

  30. [39]

    Scaling egocentric vision: The epic-kitchens dataset,

    D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Per- rett, W. Price, and M. Wray, “Scaling egocentric vision: The epic-kitchens dataset,” inECCV, 2018

    2018

  31. [40]

    In the eye of beholder: Joint learning of gaze and actions in first person video,

    Y . Li, M. Liu, and J. M. Rehg, “In the eye of beholder: Joint learning of gaze and actions in first person video,” in ECCV, 2018

    2018

  32. [41]

    Epic-fusion: Audio-visual temporal binding for egocen- tric action recognition,

    E. Kazakos, A. Nagrani, A. Zisserman, and D. Damen, “Epic-fusion: Audio-visual temporal binding for egocen- tric action recognition,” inICCV, 2019

    2019

  33. [42]

    Tem- poral relational reasoning in videos,

    B. Zhou, A. Andonian, A. Oliva, and A. Torralba, “Tem- poral relational reasoning in videos,” inECCV, 2018

    2018

  34. [43]

    Bert: Pre-training of deep bidirectional transformers for lan- guage understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for lan- guage understanding,”arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  35. [44]

    Multitask learning to improve egocentric action recognition,

    G. Kapidis, R. Poppe, E. van Dam, L. Noldus, and R. Veltkamp, “Multitask learning to improve egocentric action recognition,” inICCVW, 2019. 9

    2019

  36. [45]

    Multi- fiber networks for video recognition,

    Y . Chen, Y . Kalantidis, J. Li, S. Yan, and J. Feng, “Multi- fiber networks for video recognition,” inECCV, 2018

    2018

  37. [46]

    Learning spatiotemporal attention for egocentric action recognition,

    M. Lu, D. Liao, and Z.-N. Li, “Learning spatiotemporal attention for egocentric action recognition,” inICCVW, 2019

    2019

  38. [47]

    Integrating human gaze into attention for egocentric activity recognition,

    K. Min and J. J. Corso, “Integrating human gaze into attention for egocentric activity recognition,” inWACV, 2021

    2021

  39. [48]

    Egocentric action recognition by capturing hand-object contact and object state,

    T. Shiota, M. Takagi, K. Kumagai, H. Seshimo, and Y . Aono, “Egocentric action recognition by capturing hand-object contact and object state,” inWACV, 2024

    2024

  40. [49]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” inICML

  41. [50]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers,

    W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou, “Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers,” inNeurIPS, 2020. 10

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.