Pith. sign in

REVIEW 4 major objections 5 minor 89 references

Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectives

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that one hierarchical graph architecture with a temporal-distance-gated convolution can solve five egocentric video tasks at different time scales, outperforming single-task, multi-task, and task-translation baselines on…

desk verdict A fair incremental follow-up to EgoPack with a genuinely useful GNN layer, but the headline MQ gains are small and unreplicated. read the letter →

arxiv 2502.02487 v1 pith:RFPOPH4B submitted 2025-02-04 cs.CV

Simone Alberto Peirone , Francesca Pistilli , Antonio Alliegro , Tatiana Tommasi , Giuseppe Averta This is my paper · ORCID
classification cs.CV
keywords egocentricvisionvideounderstandingmulti-tasklearninggraphneuralnetworkstemporalactionlocalizationrecognitionlong-termanticipationEgo4D
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that egocentric video understanding is inherently multi-scale — a viewer simultaneously recognizes what is happening, notices object state changes, and anticipates what comes next — and that a single model should be able to do all of it. Hier-EgoPack extends the EgoPack 'backpack of skills' idea with a hierarchical temporal backbone whose new Temporal Distance Gated Convolution (TDGC) layer explicitly encodes how far apart in time two video segments are and whether one lies in the past or future. Using one generic architecture and frozen EgoVLP features, the paper reports the best or second-best numbers among the compared non-LLM baselines on five Ego4D benchmarks, covering clip-level tasks (action recognition, object state change classification, point of no return, long-term anticipation) and a frame-level task (moment queries), while beating single-task, multi-task, and task-translation baselines. The practical bet is that task knowledge can be abstracted into frozen prototypes and reused across temporal granularities, so a model trained once on known tasks can learn a new task cheaply.

What carries the argument

The load-bearing object is the Temporal Distance Gated Convolution (TDGC) layer, a graph convolution for temporal graphs. For a root node $i$ and neighbor $j$, it forms $s_{ij} = \operatorname{sign}(p_e^{(l)}[i]-p_e^{(l)}[j])$ to mark past versus future and $w_{ij} = \operatorname{MLP}(|p_e^{(l)}[i]-p_e^{(l)}[j]|)$ to weight by temporal distance, then aggregates $x'_j = \phi(W_n^\mathsf{T} x_j^{(l)} + b_n)$ into $x_i^{(l+1)} = W_r^\mathsf{T} x_i^{(l)} + \operatorname{mean}_{j\in\bar{N}(i)}(s_{ij}(w_{ij}\odot x'_j)) + b_r$. The temporal backbone stacks TDGC layers and mean-pools alternate nodes to halve resolution, producing progressively coarsened graphs $\{G^{(1)},\ldots,G^{(L)}\}$; fine-grained tasks read the first stage while variable-resolution tasks like Moment Queries read all stages. The second mechanism is the EgoPack-style backpack: prototypes are formed by forwarding action-annotated clips through each task's neck and averaging per (verb, noun) label, then frozen; a novel task's features are refined by SAGE message passing with their nearest prototypes.

What would settle it

Run each benchmark comparison with at least ten random seeds and report the per-seed spread. Specifically, if the validation-set Moment Queries average mAP of Hier-EgoPack and ActionFormer overlap within one standard deviation, the claimed cross-task advantage is not established; the same test applies to the 0.019 noun edit-distance gap versus HierVL on Long Term Anticipation.

Watch

Extended reading notes

Core claim

The central claim is that cross-task knowledge transfer in egocentric video does not require task-specific backbones or tasks of matching temporal granularity. Hier-EgoPack trains one shared temporal backbone on support tasks, distills each task's viewpoint into frozen prototypes indexed by (verb, noun) action labels, and lets a novel task refine its features by retrieving the nearest prototypes and passing messages over that graph. On five Ego4D tasks the paper reports that this setup outperforms its EgoPack predecessor, a task-translation baseline built in the style of EgoT2, and the VSGN and ActionFormer methods on Moment Queries, while reaching the best noun and action edit distance among the vision-based LTA methods compared. The qualitative analysis supports the transfer story: for a novel Moment Queries task, the support tasks activate quite different prototypes, whereas fine-grained tasks show higher activation consensus, suggesting the model is genuinely drawing complementary perspectives rather than a single common feature.

Load-bearing premise

The reported gains over the closest baselines are small on several key metrics — Moment Queries average mAP 21.0 versus 20.6 for ActionFormer and Long Term Anticipation noun edit distance 0.716 versus 0.735 for HierVL — and the paper reports three-seed averages without standard deviations, so the load-bearing assumption is that these gaps are larger than run-to-run variance rather than noise.

Editorial extensions

If this is right

  • One shared temporal backbone can serve tasks that read a single stage of the hierarchy (AR, OSCC, PNR, LTA) and tasks that read all stages (MQ), with only small task-specific necks and heads attached.
  • A novel task learned from frozen task prototypes transfers better than finetuning a multi-task model or translating features between independently trained single-task models.
  • Both temporal gates in TDGC carry the gain: removing the past/future sign drops MQ average mAP from 20.21 to 15.12, and removing the distance weighting drops it to 19.52 in the ablations.
  • On Ego4D validation, Hier-EgoPack with EgoVLP features reports higher Moment Queries average mAP than VSGN and ActionFormer (21.0 versus 11.4 and 20.6), and its full model has far fewer parameters than the ASL alternative (37.1M versus 350.7M).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the graph construction only needs segment features and timestamps, the TDGC rule should transfer to any sequential video domain where event ordering matters, such as surgical or broadcast footage; testing it there would show whether the benefit is egocentric-specific or a general inductive bias.
  • The frozen-prototype design points toward a continual-learning recipe: new tasks can be added later without revisiting support-task supervision. A testable prediction is that activation consensus between support tasks, computed as in the paper, predicts which novel tasks will benefit most from the backpack.
  • The authors emphasize that TDGC shares weights across past and future while gating only the aggregation, a choice that lets the same event semantics be projected once; an extension would be to inspect whether this property improves sample efficiency when fewer action annotations are available.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Hier-EgoPack, an extension of the authors' prior EgoPack framework that replaces the flat temporal backbone with a multi-stage hierarchical GNN and introduces a Temporal Distance Gated Convolution (TDGC) layer. The architecture processes dense temporal graphs, progressively subsamples them, and shares the backbone across five Ego4D tasks (AR, OSCC, PNR, LTA, MQ) while retaining EgoPack's prototype-based cross-task knowledge transfer. Experiments report results against Ego4D baselines, EgoT2, EgoPack, and a self-constructed Task-Translation baseline, together with ablations of TDGC components, pooling, temporal threshold, and GNN layer choice on the Moment Queries task. The main claims are that the hierarchical backbone enables reasoning across temporal granularities, that TDGC gives a strong 'sense of time', and that Hier-EgoPack significantly outperforms VSGN and ActionFormer on MQ and achieves state-of-the-art vision-only LTA noun edit distance.

Significance. If the central claims are supported, the paper would make a useful contribution to egocentric video understanding by showing that a single hierarchical graph architecture with frozen visual features can transfer task prototypes across fine-grained and long-horizon tasks. The TDGC layer is a sensible and well-motivated design: it incorporates both sign and magnitude of temporal distance in message passing, and the ablation in Table 4 shows clear degradations when either component is removed. The authors also standardize graph construction across tasks, which is a genuine improvement over the task-dependent node construction in EgoPack. The qualitative prototype-activation analysis is a strength: it gives concrete evidence that different support tasks activate different prototypes, particularly for MQ. However, the empirical evidence for the headline claims is currently fragile: the margins over the closest baselines are small (e.g., 0.4 mAP on MQ validation, 0.5 mAP on MQ test, 0.019 noun ED over HierVL), no standard deviations or significance tests are reported, and no ablation removes the hierarchy itself.

major comments (4)
  1. [Sec. 4.2 and Table 5] The claim in Sec. 4.5 that Hier-EgoPack 'significantly outperforms VSGN and ActionFormer' is not supported by the evidence reported. On MQ validation the margin over ActionFormer is 21.0 vs 20.6 mAP (Table 5), and on test it is 18.0 vs 17.5; these differences are within the range one would expect from seed variance, yet the paper only reports three-seed averages without standard deviations. The same issue applies to LTA (Table 6), where the noun-ED gain over HierVL is 0.716 vs 0.735 from a single test-set evaluation, and the verb-ED value is actually worse than EgoPack (0.726 vs 0.721). Please report per-seed results or confidence intervals and, where appropriate, a paired significance test (e.g., across the three seeds) before using the word 'significantly'.
  2. [Sec. 4.4, Tables 3 and 4] No ablation isolates the contribution of the hierarchy itself. Tables 3 and 4 vary the number of TDGC layers per stage, pooling strategy, temporal threshold, and GNN layer, but all these variants use the same multi-stage backbone with subsampling. A flat single-stage version of the same architecture (same TDGC layers, same features, same NMS, but no temporal subsampling) is never reported on MQ. Without this control, the paper does not establish that the hierarchical structure, rather than TDGC or other task-specific design choices, drives the MQ improvement over EgoPack and single-task baselines. The only flat comparison, Table 2 with Omnivore features, shows mixed and mostly negligible differences (e.g., OSCC 71.77 vs 71.83, LTA verb ED 0.729 vs 0.728), so it does not support the hierarchy claim either.
  3. [Sec. 4.2.2, Eq. (10)] The Task-Translation baseline is self-constructed, and the description is too underspecified to assess whether it is a strong baseline. Eq. (10) defines the attention mask, but the number of transformer layers, number of attention heads, hidden size, training epochs, learning rate, and whether the Single Task models are frozen or fine-tuned are not reported. Since the comparison against Task-Translation is used to argue that prototype-based abstraction is superior to translation, the baseline must be implemented at least as carefully as the proposed method. Please provide full hyperparameters and, ideally, include a variant trained from the same MTL initialization as Hier-EgoPack to reduce the chance that the gap is due to baseline tuning.
  4. [Sec. 4.3, Table 1] The headline result in Table 1 is largely driven by a single task, MQ, where Hier-EgoPack improves 0.5 mAP over the Task-Translation baseline (21.0 vs 20.5) and 0.8 over Single Task (21.0 vs 20.2). For the fine-grained tasks, the differences are small and inconsistent: OSCC accuracy is 75.60 vs 76.42 for Task-Translation, and PNR localization error is identical to Single Task (0.61). The abstract and introduction claim that Hier-EgoPack 'effectively solves these diverse tasks simultaneously' and 'achieves strong performance' on all five benchmarks, but the per-task results do not consistently show an advantage. The authors should either temper these claims or provide evidence that the MQ gains are not an artifact of the chosen baselines and settings.
minor comments (5)
  1. [Sec. 3.5] The sentence 'Eq. 3.5 is applied to features from all l stages' appears to refer to Eq. (9); please correct the cross-reference.
  2. [Sec. 4.2.1] The NMS sigma value is taken from ActionFormer and from a challenge report [77], but no sensitivity analysis is provided; since NMS configuration can notably affect mAP on MQ, a sentence explaining the choice in the text of the paper (rather than only in a cited report) would be helpful.
  3. [Table 1 and Sec. 4.3] The definitions of MTL, MTL+FT, and MTL+HT are given only in prose; adding the acronyms to a table caption or a small legend would improve readability, especially because MTL+HT is not mentioned again after Sec. 4.3.
  4. [Fig. 6 and Sec. 4.6] The color legend in Fig. 6 is difficult to read at print size, and the caption text mentions 'top 50 most predicted labels' but the y-axis is not labeled. Please enlarge font sizes and clarify what the rows represent.
  5. [Sec. 4.1] The AR task is described as not being an official Ego4D task and as 'initially introduced by EgoT2 using the LTA annotations'; this is important context and should be stated earlier, in the task description paragraph, rather than in a footnote.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method's components are defined independently of the evaluated benchmarks, and the evaluation is external.

full rationale

Hier-EgoPack's derivation chain is self-contained. The TDGC layer (Eq. 3-5) is defined from relative temporal distances (s_ij, w_ij) and a mean aggregation; the hierarchical backbone is a standard progressive subsampling; the prototype backpack averages support-task features by (verb, noun) labels and retrieves them via k-NN (Eq. 8-9). None of these components takes the target metrics (MQ mAP, LTA edit distance, OSCC accuracy) as inputs, and no parameter is fitted to the reported benchmarks. The evaluation uses held-out Ego4D validation/test splits and external baselines (VSGN, ActionFormer, HierVL, EgoT2), so the improvements are not forced by construction. Self-citations to the authors' prior EgoPack are used as a starting point and as a baseline, not as the sole justification for the claimed novelty; EgoPack is a published peer-reviewed method whose comparison is direct rather than definitional. The reported small margins without standard deviations are a statistical-robustness concern, not a circularity: they do not reduce any prediction to its input. I find no circular step.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard GNN machinery plus the EgoPack prototype mechanism. The main free parameters are graph connectivity and layer count, both tuned on MQ. No new physical or conceptual entities are introduced.

free parameters (4)
  • Temporal distance threshold tau = 2
    Defines edges in the temporal graph; chosen via ablation on MQ (Table 3), not derived.
  • Number of TDGC layers per stage N_l = 2
    Ablation shows performance saturates after two layers (Table 3); used for all tasks.
  • NMS sigma = 2.0
    Copied from ActionFormer and prior work [77], empirically found to reduce penalty on near-replicate predictions.
  • k for prototype k-NN retrieval = not specified
    The number of closest prototypes used in Eq. 9 is not stated in the paper, leaving an unspecified free parameter.
assumptions (3)
  • domain assumption Pre-extracted frozen features (EgoVLP or Omnivore) contain sufficient information for all downstream tasks.
    The feature extractor is never fine-tuned; all task reasoning relies on these features (Sec. 4.2).
  • ad hoc to paper Prototypes built from AR verb-noun annotations provide a useful common-thread abstraction for all support tasks.
    Sec. 3.4 collects prototypes only from action recognition samples, assuming AR is the common thread; this is an inherited EgoPack design choice.
  • standard math Graph message passing as defined in Eqs. 3-5 is a valid differentiable operation for learning.
    Standard GNN operations; no unusual mathematical assumptions beyond differentiability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectives." pith.science (2026). https://pith.science/paper/RFPOPH4B

@misc{pith2026250202487,
  author       = {Pith},
  title        = {Pith review of: Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RFPOPH4B}},
  note         = {Machine review of arXiv:2502.02487}
}
read the original abstract

Our comprehension of video streams depicting human activities is naturally multifaceted: in just a few moments, we can grasp what is happening, identify the relevance and interactions of objects in the scene, and forecast what will happen soon, everything all at once. To endow autonomous systems with such a holistic perception, learning how to correlate concepts, abstract knowledge across diverse tasks, and leverage tasks synergies when learning novel skills is essential. A significant step in this direction is EgoPack, a unified framework for understanding human activities across diverse tasks with minimal overhead. EgoPack promotes information sharing and collaboration among downstream tasks, essential for efficiently learning new skills. In this paper, we introduce Hier-EgoPack, which advances EgoPack by enabling reasoning also across diverse temporal granularities, which expands its applicability to a broader range of downstream tasks. To achieve this, we propose a novel hierarchical architecture for temporal reasoning equipped with a GNN layer specifically designed to tackle the challenges of multi-granularity reasoning effectively. We evaluate our approach on multiple Ego4d benchmarks involving both clip-level and frame-level reasoning, demonstrating how our hierarchical unified architecture effectively solves these diverse tasks simultaneously.

Figures

Figures reproduced from arXiv: 2502.02487 by the authors.

Figure 1
Figure 1. Overview of the Hier-EgoPack architecture. First, the video is converted into a graph representation G (0) whose node embeddings are extracted using a frozen video features extractor. The graph is then processed by the hierarchical temporal backbone Mt, shared by all the tasks, to progressively learn higher level representations of the input video {G(1) , G (2) , . . . , G (L)}. The node embeddings of these graphs a… view at source ↗
Figure 2
Figure 2. Temporal Distance Gated Convolution layer (TDGC), specif￾ically designed to integrate past and future events grounding (sij ) and to reason about the temporal distance between nodes (wij ) in the aggregation step. the index of the stage of the hierarchical temporal backbone. Overall, the output of the temporal backbone Mt maps the input graph G (0) to a set of graphs: Mt : G (0) → {G(1) , G (2) , . . . , G (L) }, (2… view at source ↗
Figure 3
Figure 3. Learning a novel task with a backpack. After the Multi-Task training phase, we extract a set of prototypes Pk that summarize what the network has learned from each support task Tk, like a backpack of skills that we can carry over. In this Cross-Tasks Interaction phase, the network can peek at these different task-perspective to enrich the learning of the novel task. 3.3 Task-specific components The temporal backbone… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Activation frequency for the task-specific prototypes from different support tasks. We focus on the Top-20 most activated prototypes across the support tasks. LTA and OSCC have more uniform activations across different support tasks, i.e., they look at similar prototyp…
Figure 5
Figure 5. Figure 5: Activations consensus for different novel tasks. Activations consensus between two support tasks is defined as the percentage of their prototypes corresponding to the same label activated by the two tasks. Fine-grained tasks, i.e., AR, OSCC and LTA, have higher average…
Figure 6
Figure 6. Figure 6: Activation frequency of the prototypes from the support tasks when the novel task is Moment Queries (MQ). For each task from the MTL pre-training phase, we plot the distribution of closest prototypes in the Hier-EgoPack interaction phase. For readability, we restrict o…
Figure 7
Figure 7. Figure 7: Activation frequency of the prototypes from the support tasks when the novel task is OSCC, separately for the positive and negative correct predictions. Positive samples tend to focus more on prototypes whose verb could be associated with an object state change, e.g., …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 75 canonical work pages

  1. [1]

    Multiview transformers for video recognition,

    S. Yan, X. Xiong, A. Arnab, Z. Lu, M. Zhang, C. Sun, and C. Schmid, “Multiview transformers for video recognition,” in CVPR, 2022

    2022

  2. [2]

    Anticipative feature fusion transformer for multi-modal action anticipation,

    Z. Zhong, D. Schneider, M. Voit, R. Stiefelhagen, and J. Beyerer, “Anticipative feature fusion transformer for multi-modal action anticipation,” in WACV, 2023

    2023

  3. [3]

    Actionformer: Localizing moments of actions with transformers,

    C.-L. Zhang, J. Wu, and Y. Li, “Actionformer: Localizing moments of actions with transformers,” in ECCV, 2022

    2022

  4. [4]

    Ubernet: Training a universal convolutional neu- ral network for low-, mid-, and high-level vision using diverse datasets and limited memory,

    I. Kokkinos, “Ubernet: Training a universal convolutional neu- ral network for low-, mid-, and high-level vision using diverse datasets and limited memory,” in CVPR, 2017

    2017

  5. [5]

    Egocentric video task translation,

    Z. Xue, Y. Song, K. Grauman, and L. Torresani, “Egocentric video task translation,” in CVPR, 2023

    2023

  6. [6]

    A backpack full of skills: Egocentric video understanding with diverse task perspectives,

    S. A. Peirone, F. Pistilli, A. Alliegro, and G. Averta, “A backpack full of skills: Egocentric video understanding with diverse task perspectives,” in CVPR, 2024

    2024

  7. [7]

    Test of time: Instilling video-language models with a sense of time,

    P . Bagad, M. Tapaswi, and C. G. Snoek, “Test of time: Instilling video-language models with a sense of time,” in CVPR, 2023

    2023

  8. [8]

    Ego4d: Around the world in 3,000 hours of egocentric video,

    K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Gird- har, J. Hamburger, H. Jiang, M. Liu, X. Liu et al., “Ego4d: Around the world in 3,000 hours of egocentric video,” in CVPR, 2022

    2022

Show all 89 references
  1. [9]

    The evolution of first person vision methods: A survey,

    A. Betancourt, P . Morerio, C. S. Regazzoni, and M. Rauterberg, “The evolution of first person vision methods: A survey,” IEEE TCSVT, 2015

    2015

  2. [10]

    An outlook into the future of egocentric vision,

    C. Plizzari, G. Goletto, A. Furnari, S. Bansal, F. Ragusa, G. M. Farinella, D. Damen, and T. Tommasi, “An outlook into the future of egocentric vision,” IJCV, 2024

    2024

  3. [11]

    The epic-kitchens dataset: Collection, challenges and baselines,

    D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, and M. Wray, “The epic-kitchens dataset: Collection, challenges and baselines,” IEEE TP AMI, 2021

    2021

  4. [12]

    In the eye of the beholder: A survey of models for eyes and gaze,

    D. W. Hansen and Q. Ji, “In the eye of the beholder: A survey of models for eyes and gaze,” IEEE TP AMI, 2009

    2009

  5. [13]

    Epic-tent: An egocentric video dataset for camping tent assembly,

    Y. Jang, B. Sullivan, C. Ludwig, I. Gilchrist, D. Damen, and W. Mayol-Cuevas, “Epic-tent: An egocentric video dataset for camping tent assembly,” in ICCVW, 2019

    2019

  6. [14]

    Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100,

    D. Damen, H. Doughty, G. M. Farinella, A. Furnari, J. Ma, E. Kaza- kos, D. Moltisanti, J. Munro, T. Perrett, W. Price, and M. Wray, “Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100,” IJCV, 2022

    2022

  7. [15]

    Assembly101: A large-scale multi-view video dataset for understanding procedural activities,

    F. Sener, D. Chatterjee, D. Shelepov, K. He, D. Singhania, R. Wang, and A. Yao, “Assembly101: A large-scale multi-view video dataset for understanding procedural activities,” in CVPR, 2022

    2022

  8. [16]

    Egocen- tric vision-based action recognition: A survey,

    A. N ´u˜nez-Marcos, G. Azkune, and I. Arganda-Carreras, “Egocen- tric vision-based action recognition: A survey,” Neurocomputing, 2022

    2022

  9. [17]

    Rolling-unrolling lstms for action anticipation from first-person video,

    A. Furnari and G. M. Farinella, “Rolling-unrolling lstms for action anticipation from first-person video,” IEEE TP AMI, 2020

    2020

  10. [18]

    Anticipative video transformer,

    R. Girdhar and K. Grauman, “Anticipative video transformer,” in ICCV, 2021

    2021

  11. [19]

    Next- active-object prediction from egocentric videos,

    A. Furnari, S. Battiato, K. Grauman, and G. M. Farinella, “Next- active-object prediction from egocentric videos,” Journal of Visual Communication and Image Representation, 2017

    2017

  12. [20]

    Improving action segmentation via graph-based temporal reasoning,

    Y. Huang, Y. Sugano, and Y. Sato, “Improving action segmentation via graph-based temporal reasoning,” in CVPR, 2020

    2020

  13. [21]

    Spotem: Efficient video search for episodic memory,

    S. K. Ramakrishnan, Z. Al-Halah, and K. Grauman, “Spotem: Efficient video search for episodic memory,” in ICLR, 2023

    2023

  14. [22]

    Amego: Active memory from long egocentric videos,

    G. Goletto, T. Nagarajan, G. Averta, and D. Damen, “Amego: Active memory from long egocentric videos,” in ECCV, 2024

    2024

  15. [23]

    Egoschema: A diag- nostic benchmark for very long-form video language understand- ing,

    K. Mangalam, R. Akshulakov, and J. Malik, “Egoschema: A diag- nostic benchmark for very long-form video language understand- ing,” NeurIPS, 2023

    2023

  16. [24]

    Egotaskqa: Understanding human tasks in egocentric videos,

    B. Jia, T. Lei, S.-C. Zhu, and S. Huang, “Egotaskqa: Understanding human tasks in egocentric videos,” NeurIPS, 2022

    2022

  17. [25]

    Multi-modal domain adaptation for fine-grained action recognition,

    J. Munro and D. Damen, “Multi-modal domain adaptation for fine-grained action recognition,” in CVPR, 2020

    2020

  18. [26]

    Interact before align: Leveraging cross-modal knowledge for domain adaptive action recognition,

    L. Yang, Y. Huang, Y. Sugano, and Y. Sato, “Interact before align: Leveraging cross-modal knowledge for domain adaptive action recognition,” in CVPR, 2022

    2022

  19. [27]

    Temporal attentive alignment for large-scale video domain adap- tation,

    M.-H. Chen, Z. Kira, G. AlRegib, J. Yoo, R. Chen, and J. Zheng, “Temporal attentive alignment for large-scale video domain adap- tation,” in ICCV, 2019

    2019

  20. [28]

    What can a cook in italy teach a mechanic in india? action recognition generalisation over scenarios and locations,

    C. Plizzari, T. Perrett, B. Caputo, and D. Damen, “What can a cook in italy teach a mechanic in india? action recognition generalisation over scenarios and locations,” in ICCV, 2023

    2023

  21. [29]

    Relative norm alignment for tackling domain shift in deep multi-modal classification,

    M. Planamente, C. Plizzari, S. A. Peirone, B. Caputo, and A. Bot- tino, “Relative norm alignment for tackling domain shift in deep multi-modal classification,” IJCV, 2024

    2024

  22. [30]

    Human action recognition from various data modalities: A re- view,

    Z. Sun, Q. Ke, H. Rahmani, M. Bennamoun, G. Wang, and J. Liu, “Human action recognition from various data modalities: A re- view,” IEEE TP AMI, 2023

    2023

  23. [31]

    Listen to look: Action recognition by previewing audio,

    R. Gao, T.-H. Oh, K. Grauman, and L. Torresani, “Listen to look: Action recognition by previewing audio,” in CVPR, 2020

    2020

  24. [32]

    Egocentric video-language pretraining,

    K. Q. Lin, J. Wang, M. Soldan, M. Wray, R. Yan, E. Z. XU, D. Gao, R.-C. Tu, W. Zhao, W. Kong et al. , “Egocentric video-language pretraining,” in NeurIPS, 2022

    2022

  25. [33]

    Egovlpv2: Egocentric video- language pre-training with fusion in the backbone,

    S. Pramanick, Y. Song, S. Nag, K. Q. Lin, H. Shah, M. Z. Shou, R. Chellappa, and P . Zhang, “Egovlpv2: Egocentric video- language pre-training with fusion in the backbone,” inICCV, 2023

    2023

  26. [34]

    Hiervl: Learning hierarchical video-language embeddings,

    K. Ashutosh, R. Girdhar, L. Torresani, and K. Grauman, “Hiervl: Learning hierarchical video-language embeddings,” in CVPR, 2023

    2023

  27. [35]

    Learning video representations from large language models,

    Y. Zhao, I. Misra, P . Kr ¨ahenb ¨uhl, and R. Girdhar, “Learning video representations from large language models,” in CVPR, 2023

    2023

  28. [36]

    A survey of convo- lutional neural networks: analysis, applications, and prospects,

    Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convo- lutional neural networks: analysis, applications, and prospects,” IEEE transactions on neural networks and learning systems , 2021

    2021

  29. [37]

    A survey of the recent architectures of deep convolutional neural networks,

    A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, “A survey of the recent architectures of deep convolutional neural networks,” Artificial intelligence review, 2020

    2020

  30. [38]

    Recent advances in convolutional neural networks,

    J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai et al., “Recent advances in convolutional neural networks,” PR, 2018

    2018

  31. [39]

    Dynamic edge-conditioned filters in convolutional neural networks on graphs,

    M. Simonovsky and N. Komodakis, “Dynamic edge-conditioned filters in convolutional neural networks on graphs,” in CVPR, 2017

    2017

  32. [40]

    Dynamic graph cnn for learning on point clouds,

    Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM Transactions on Graphics, 2019

    2019

  33. [41]

    Graph learning in robotics: a survey,

    F. Pistilli and G. Averta, “Graph learning in robotics: a survey,” IEEE Access, 2023

    2023

  34. [42]

    Molecular graph convolutions: moving beyond fingerprints,

    S. Kearnes, K. McCloskey, M. Berndl, V . Pande, and P . Riley, “Molecular graph convolutions: moving beyond fingerprints,” Journal of computer-aided molecular design, 2016

    2016

  35. [43]

    Graph neural networks for social recommendation,

    W. Fan, Y. Ma, Q. Li, Y. He, E. Zhao, J. Tang, and D. Yin, “Graph neural networks for social recommendation,” inThe world wide web conference, 2019

    2019

  36. [44]

    Learning to simulate complex physics with graph networks,

    A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P . Battaglia, “Learning to simulate complex physics with graph networks,” in ICML, 2020

    2020

  37. [45]

    Graph convolutional networks for temporal action lo- calization,

    R. Zeng, W. Huang, M. Tan, Y. Rong, P . Zhao, J. Huang, and C. Gan, “Graph convolutional networks for temporal action lo- calization,” in ICCV, 2019

    2019

  38. [46]

    Stacked spatio- temporal graph convolutional networks for action segmentation,

    P . Ghosh, Y. Yao, L. Davis, and A. Divakaran, “Stacked spatio- temporal graph convolutional networks for action segmentation,” in WACV, 2020

    2020

  39. [47]

    Action graphs: Weakly- supervised action localization with graph convolution networks,

    M. Rashid, H. Kjellstrom, and Y. J. Lee, “Action graphs: Weakly- supervised action localization with graph convolution networks,” in WACV, 2020

    2020

  40. [48]

    All about knowledge graphs for actions,

    P . Ghosh, N. Saini, L. S. Davis, and A. Shrivastava, “All about knowledge graphs for actions,” arXiv preprint arXiv:2008.12432 , 2020. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13

    2008 arXiv

  41. [49]

    Egocentric object manipulation graphs,

    E. Dessalene, M. Maynord, C. Devaraj, C. Fermuller, and Y. Aloi- monos, “Egocentric object manipulation graphs,” arXiv preprint arXiv:2006.03201, 2020

    2006 arXiv

  42. [50]

    Forecasting action through contact representations from first person video,

    E. Dessalene, C. Devaraj, M. Maynord, C. Fermuller, and Y. Aloi- monos, “Forecasting action through contact representations from first person video,” IEEE TP AMI, 2021

    2021

  43. [51]

    Ego-topo: Environment affordances from egocentric video,

    T. Nagarajan, Y. Li, C. Feichtenhofer, and K. Grauman, “Ego-topo: Environment affordances from egocentric video,” in CVPR, 2020

    2020

  44. [52]

    Multitask learning,

    R. Caruana, “Multitask learning,” Machine learning , vol. 28, pp. 41–75, 1997

    1997

  45. [53]

    A survey on multi-task learning,

    Y. Zhang and Q. Yang, “A survey on multi-task learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 12, pp. 5586–5609, 2021

    2021

  46. [54]

    Video task decathlon: Unifying image and video tasks in autonomous driving,

    T. E. Huang, Y. Liu, L. Van Gool, and F. Yu, “Video task decathlon: Unifying image and video tasks in autonomous driving,” in ICCV, 2023

    2023

  47. [55]

    Mutual context network for jointly estimating egocentric gaze and action,

    Y. Huang, M. Cai, Z. Li, F. Lu, and Y. Sato, “Mutual context network for jointly estimating egocentric gaze and action,” IEEE TIP, 2020

    2020

  48. [56]

    Efficiently identifying task groupings for multi-task learning,

    C. Fifty, E. Amid, Z. Zhao, T. Yu, R. Anil, and C. Finn, “Efficiently identifying task groupings for multi-task learning,” in NeurIPS, 2021

    2021

  49. [57]

    A unified sequence interface for vision tasks,

    T. Chen, S. Saxena, L. Li, T.-Y. Lin, D. J. Fleet, and G. E. Hinton, “A unified sequence interface for vision tasks,” in NeurIPS, 2022

    2022

  50. [58]

    Adamv-moe: Adaptive multi-task vision mixture-of- experts,

    T. Chen, X. Chen, X. Du, A. Rashwan, F. Yang, H. Chen, Z. Wang, and Y. Li, “Adamv-moe: Adaptive multi-task vision mixture-of- experts,” in ICCV, 2023

    2023

  51. [59]

    Deep multitask learning with progressive parameter sharing,

    H. Shi, S. Ren, T. Zhang, and S. J. Pan, “Deep multitask learning with progressive parameter sharing,” in ICCV, 2023

    2023

  52. [60]

    Unihcp: A unified model for human- centric perceptions,

    Y. Ci, Y. Wang, M. Chen, S. Tang, L. Bai, F. Zhu, R. Zhao, F. Yu, D. Qi, and W. Ouyang, “Unihcp: A unified model for human- centric perceptions,” in CVPR, 2023

    2023

  53. [61]

    Learning with whom to share in multi-task feature learning,

    Z. Kang, K. Grauman, and F. Sha, “Learning with whom to share in multi-task feature learning,” in ICML, 2011

    2011

  54. [62]

    Learning to branch for multi- task learning,

    P . Guo, C.-Y. Lee, and D. Ulbricht, “Learning to branch for multi- task learning,” in ICML, 2020

    2020

  55. [63]

    Which tasks should be learned together in multi-task learning?

    T. Standley, A. Zamir, D. Chen, L. Guibas, J. Malik, and S. Savarese, “Which tasks should be learned together in multi-task learning?” in ICML, 2020

    2020

  56. [64]

    Adashare: Learning what to share for efficient deep multi-task learning,

    X. Sun, R. Panda, R. Feris, and K. Saenko, “Adashare: Learning what to share for efficient deep multi-task learning,” in NeurIPS, 2020

    2020

  57. [65]

    Multitask learning to improve egocentric action recognition,

    G. Kapidis, R. Poppe, E. van Dam, L. Noldus, and R. Veltkamp, “Multitask learning to improve egocentric action recognition,” in ICCVW, 2019

    2019

  58. [66]

    Interactive prototype learning for egocentric action recognition,

    X. Wang, L. Zhu, H. Wang, and Y. Yang, “Interactive prototype learning for egocentric action recognition,” in ICCV, 2021

    2021

  59. [67]

    Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,

    A. Kendall, Y. Gal, and R. Cipolla, “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in CVPR, 2018

    2018

  60. [68]

    Grad- norm: Gradient normalization for adaptive loss balancing in deep multitask networks,

    Z. Chen, V . Badrinarayanan, C.-Y. Lee, and A. Rabinovich, “Grad- norm: Gradient normalization for adaptive loss balancing in deep multitask networks,” in ICML, 2018

    2018

  61. [69]

    Gra- dient adversarial training of neural networks,

    A. Sinha, Z. Chen, V . Badrinarayanan, and A. Rabinovich, “Gra- dient adversarial training of neural networks,” arXiv preprint arXiv:1806.08028, 2018

    2018 arXiv

  62. [70]

    Dy- namic task prioritization for multitask learning,

    M. Guo, A. Haque, D.-A. Huang, S. Yeung, and L. Fei-Fei, “Dy- namic task prioritization for multitask learning,” in ECCV, 2018

    2018

  63. [71]

    Gradient surgery for multi-task learning,

    T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn, “Gradient surgery for multi-task learning,” in NeurIPS, 2020

    2020

  64. [72]

    Mti-net: Multi- scale task interaction networks for multi-task learning,

    S. Vandenhende, S. Georgoulis, and L. Van Gool, “Mti-net: Multi- scale task interaction networks for multi-task learning,” in ECCV, 2020

    2020

  65. [73]

    An overview of multi-task learning in deep neural networks,

    S. Ruder, “An overview of multi-task learning in deep neural networks,” arXiv preprint arXiv:1706.05098, 2017

    2017 arXiv

  66. [74]

    Video self-stitching graph network for temporal action localization,

    C. Zhao, A. K. Thabet, and B. Ghanem, “Video self-stitching graph network for temporal action localization,” in ICCV, 2021

    2021

  67. [75]

    Omnivore: A single model for many visual modalities,

    R. Girdhar, M. Singh, N. Ravi, L. van der Maaten, A. Joulin, and I. Misra, “Omnivore: A single model for many visual modalities,” in CVPR, 2022

    2022

  68. [77]

    Nms threshold matters for ego4d moment queries–2nd place solution to the ego4d moment queries challenge 2023,

    L. Sui, F. Mu, and Y. Li, “Nms threshold matters for ego4d moment queries–2nd place solution to the ego4d moment queries challenge 2023,” arXiv preprint arXiv:2307.02025, 2023

    2023 arXiv

  69. [78]

    Focal loss for dense object detection,

    T.-Y. Ross and G. Doll ´ar, “Focal loss for dense object detection,” in CVPR, 2017

    2017

  70. [79]

    Distance-iou loss: Faster and better learning for bounding box regression,

    Z. Zheng, P . Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-iou loss: Faster and better learning for bounding box regression,” in AAAI, 2020

    2020

  71. [80]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” in ICCV, 2019

    2019

  72. [81]

    Quo vadis, action recognition? a new model and the kinetics dataset,

    J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in CVPR, 2017

    2017

  73. [82]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in ICLR, 2017

    2017

  74. [83]

    Graph attention networks,

    P . Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P . Li `o, and Y. Bengio, “Graph attention networks,” in ICLR, 2018

    2018

  75. [84]

    Inductive representation learning on large graphs,

    W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in NeurIPS, 2017

    2017

  76. [85]

    Signed graph convolutional net- works,

    T. Derr, Y. Ma, and J. Tang, “Signed graph convolutional net- works,” in 2018 IEEE International Conference on Data Mining (ICDM), 2018

    2018

  77. [86]

    Action sensitivity learning for temporal action localization,

    J. Shao, X. Wang, R. Quan, J. Zheng, J. Yang, and Y. Yang, “Action sensitivity learning for temporal action localization,” in ICCV, 2023

    2023

  78. [87]

    Action sensitivity learning for the ego4d episodic memory challenge 2023,

    J. Shao, X. Wang, R. Quan, and Y. Yang, “Action sensitivity learning for the ego4d episodic memory challenge 2023,” arXiv preprint arXiv:2306.09172, 2023

    2023 arXiv

  79. [88]

    Intention-conditioned long- term human egocentric action anticipation,

    E. V . Mascar ´o, H. Ahn, and D. Lee, “Intention-conditioned long- term human egocentric action anticipation,” in WACV, 2023

    2023

  80. [89]

    Antgpt: Can large language models help long-term action anticipation from videos?

    Q. Zhao, S. Wang, C. Zhang, C. Fu, M. Q. Do, N. Agarwal, K. Lee, and C. Sun, “Antgpt: Can large language models help long-term action anticipation from videos?” in ICLR, 2024

    2024

  81. [90]

    Palm: Predicting actions through language models,

    S. Kim, D. Huang, Y. Xian, O. Hilliges, L. Van Gool, and X. Wang, “Palm: Predicting actions through language models,” in ECCV, 2024. Simone Alberto Peironereceived the B.Sc. and M.Sc. degrees in Computer Engineering from the Polytechnic University of Turin, in 2020 and 2022 re...

    2024

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.