Pith. sign in

REVIEW 13 cited by

Hypergraph Transformer for Skeleton-based Action Recognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2211.09590 v5 pith:DUFY6IUP submitted 2022-11-17 cs.CV

Yuxuan Zhou , Zhi-Qi Cheng , Chao Li , Yanwen Fang , Yifeng Geng , Xuansong Xie , Margret Keuper This is my paper · ORCID
classification cs.CV
keywords graphgcnsmodelshypergraphmodelself-attentionstate-of-the-artaction
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Skeleton-based action recognition aims to recognize human actions given human joint coordinates with skeletal interconnections. By defining a graph with joints as vertices and their natural connections as edges, previous works successfully adopted Graph Convolutional networks (GCNs) to model joint co-occurrences and achieved superior performance. More recently, a limitation of GCNs is identified, i.e., the topology is fixed after training. To relax such a restriction, Self-Attention (SA) mechanism has been adopted to make the topology of GCNs adaptive to the input, resulting in the state-of-the-art hybrid models. Concurrently, attempts with plain Transformers have also been made, but they still lag behind state-of-the-art GCN-based methods due to the lack of structural prior. Unlike hybrid models, we propose a more elegant solution to incorporate the bone connectivity into Transformer via a graph distance embedding. Our embedding retains the information of skeletal structure during training, whereas GCNs merely use it for initialization. More importantly, we reveal an underlying issue of graph models in general, i.e., pairwise aggregation essentially ignores the high-order kinematic dependencies between body joints. To fill this gap, we propose a new self-attention (SA) mechanism on hypergraph, termed Hypergraph Self-Attention (HyperSA), to incorporate intrinsic higher-order relations into the model. We name the resulting model Hyperformer, and it beats state-of-the-art graph models w.r.t. accuracy and efficiency on NTU RGB+D, NTU RGB+D 120, and Northwestern-UCLA datasets.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Heterogeneous Skeleton-Based Action Representation Learning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A unified self-supervised model lifts 2D skeletons to 3D, adds skeleton-specific prompts to unify joint sets, and fuses semantic motion encoding to recognize actions from heterogeneous skeleton formats, surpassing pri...

  2. SKI Models: Skeleton Induced Vision-Language Embeddings for Understanding Activities of Daily Living

    cs.CV 2025-02 conditional novelty 6.0 of 10

    SKI models distill skeleton-language knowledge into video-language encoders, boosting zero-shot ADL action recognition accuracy by up to 7.8 percentage points while discarding skeletons at inference.

  3. Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognition

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A text-conditioned diffusion model over skeleton features gives a new state of the art for zero-shot skeleton-based action recognition.

  4. STAR: Skeletal Token Alignment and Rearrangement for Interaction Recognition

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Skeleton-only interaction recognition that aligns skeleton and video features during training outperforms prior state-of-the-art on four benchmarks.

  5. Beyond Pairwise Relations: Dynamic Manipulation Hypergraphs for Vision-Based Human Activity Recognition

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Dynamic manipulation hypergraphs with time-varying multi-entity templates outperform matched pairwise graphs and static hypergraphs on fine-grained manipulation recognition.

  6. UniSTFormer: Unified Spatio-Temporal Lightweight Transformer for Efficient Skeleton-Based Action Recognition

    cs.CV 2025-08 conditional novelty 5.0 of 10

    UniSTFormer merges spatial and temporal attention for skeleton-based action recognition, achieving 91.0% on NTU-60 with 0.5M parameters and 1.1 GFLOPs.

  7. Evolving Skeletons: Motion Dynamics in Action Recognition

    cs.CV 2025-01 conditional novelty 5.0 of 10

    Taylor-transformed skeletons improve ST-GCN accuracy but reduce Hyperformer accuracy on NTU-60/120, indicating that motion-injected inputs do not universally benefit skeleton-based action recognition models.

  8. CascadeFormer: A Family of Two-stage Cascading Transformers for Skeleton-based Human Action Recognition

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A masked-pretrained skeleton transformer with a second fine-tuning transformer and cross-attention fusion reaches 94.66% on Penn Action, 91.16% on N-UCLA, and 81.01%/88.17% on NTU RGB+D 60 cross-subject/cross-view.

  9. PoseGRAF: Geometric-Reinforced Adaptive Fusion for Monocular 3D Human Pose Estimation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    PoseGRAF fuses joint and bone-direction features through cross-attention and dynamic fusion, reporting marginal state-of-the-art MPJPE improvements on Human3.6M and MPI-INF-3DHP.

  10. HFGCN:Hypergraph Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition

    cs.CV 2025-01 conditional novelty 4.0 of 10

    HFGCN groups skeleton joints by body part and core distance, then uses hypergraph attention and convolution to reach top-1 accuracy of 93.1% on NTU RGB+D X-Sub.

  11. FLASH: Efficient Impact Fall Detection with Unified Hypergraph State-Space Model

    cs.CV 2026-07 reject novelty 3.0 of 10

    FLASH uses single-matrix hypergraph convolutions plus Mamba to detect fall impact frames efficiently, but it is less accurate than its own dual-hypergraph predecessor.

  12. 3D Skeleton-Based Action Recognition: A Review

    cs.CV 2025-06 reject novelty 3.0 of 10

    A task-oriented review of skeleton-based action recognition that reorganizes known methods along a data processing pipeline and contains no new experimental result.

  13. FreqMixFormerV2: Lightweight Frequency-aware Mixed Transformer for Human Skeleton Action Recognition

    cs.CV 2024-12 conditional novelty 3.0 of 10

    FreqMixFormerV2 compresses the earlier FreqMixFormer skeleton-action transformer from 2.0M to 1.2M parameters, giving up about 0.3 to 0.8 accuracy points on NTU and NW-UCLA benchmarks.

Pith tools