An Effective End-to-End Solution for Multimodal Action Recognition

Caifeng Shan; Songping Wang; Xiantao Hu; Yueming Lyu

REVIEW 4 major objections 4 minor 36 references

An Effective End-to-End Solution for Multimodal Action Recognition

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Channel-concatenated RGB/thermal/depth inputs, a Temporal Shift Module backbone, and a stack of averaging and augmentation tricks achieve 99% top-1 and 100% top-5 accuracy on the ICPR 2024 multimodal action recognition leaderboard.

desk verdict A competent competition writeup that assembles known components into a 99% Top-1 recipe; the main soft spots are an unreliable architecture formula and test-set tuning, but the recipe is useful. read the letter →

arxiv 2506.09345 v1 pith:72AD5BL4 submitted 2025-06-11 cs.CV

Songping Wang , Xiantao Hu , Yueming Lyu , Caifeng Shan This is my paper

classification cs.CV

keywords multimodalactionrecognitionTemporalShiftModuleRGB-depth-thermalfusionchannelconcatenationtransferlearningstochasticweightaveragingtest-timeaugmentationmodelensembling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets the ICPR 2024 tri-modal action recognition competition, where the training set is small (2,000 videos over 20 classes) and test accuracy is the goal. It argues that a simple end-to-end pipeline—channel-concatenating RGB, thermal, and depth frames and feeding them through a Temporal Shift Module (TSM) network built on 2D CNN backbones—can reach 99% top-1 and 100% top-5 accuracy on the competition leaderboard. The key to the result is not a novel fusion architecture but a combination of transfer learning from three large video/image datasets, weight averaging, ensembling, and inference-time augmentation. A sympathetic reader would care because the result suggests near-perfect multimodal action classification may be achievable with computationally modest 2D CNN-based models plus careful training and inference engineering.

What carries the argument

The load-bearing mechanism is the Temporal Shift Module (TSM) combined with channel concatenation: for each video, the preprocessed RGB, thermal, and depth frames are stacked along the channel dimension and fed into a TSM network built on ResNet-50 or ResNet-101 backbones. TSM shifts a fraction of feature channels along the temporal axis, yielding temporal modeling at 2D CNN cost. The paper formalizes the fusion as LogitsR, LogitsI, LogitsD = Fθ(Cat(XR, XI, XD)) with a weighted cross-entropy loss over the three modality logits, although it does not specify how a single concatenated-input network emits three separate logits. The remaining machinery consists of the knowledge-enhanced pretraining, SWA across saved checkpoints, test-time augmentation, twice temporal sampling, and full-resolution inference that the ablation study shows to be the source of the accuracy gains.

What would settle it

Re-implement the pipeline exactly as described—TSM-Res50 with ImageNet and Kinetics400 pretraining, channel-concatenated RGB/thermal/depth inputs, 8 temporal segments, the reported hyperparameters, and the stated inference tricks—and check two things: whether the network can actually produce three separate modality logits for Eq. (1), and whether removing the external pretraining drops top-1 accuracy from about 95% to about 83% as reported. If either check fails, the paper's attribution of its accuracy to this particular recipe collapses.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an off-the-shelf Temporal Shift Module classifier, given channel-concatenated RGB/thermal/depth inputs and pretrained on ImageNet, Kinetics400, and Something-somethingV2, can be pushed to essentially perfect classification on the competition test set. The ablation study shows a clear progression: the base multimodal model reaches 83% top-1; adding test-time augmentation lifts it to 83.4%; adding the pretraining knowledge brings it to 95%; combining ensemble learning and stochastic weight averaging reaches 98.2%; twice sampling adds 0.4 points; and full-resolution inference brings the final top-1 accuracy to 99%, with top-5 at 100%. The paper attributes the gains to the combination of channel-level multimodal fusion, TSM's efficient spatio-temporal modeling, and a stack of prediction-enhancement techniques.

Load-bearing premise

The central assumption is that one TSM network, after receiving the channel-concatenated RGB, thermal, and depth frames, is able to output three separate modality scores that are then weighted together, and the paper never explains how those three scores are obtained from that single input.

Editorial extensions

If this is right

A channel-concatenation fusion with a TSM backbone can be competitive with far more complex multimodal fusion architectures on small tri-modal datasets.
Transfer learning from large RGB video and image datasets is the single largest accuracy booster in this pipeline, worth about 12 points of top-1 accuracy.
Inference-time enhancements—TTA, SWA, ensembling, twice sampling, and full-resolution processing—stack additively to lift accuracy from 83% to 99%.
TSM's 2D-CNN computational profile makes this near-perfect accuracy potentially deployable with modest compute compared to 3D CNN alternatives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the architecture really uses one shared trunk for all modalities, the three logits in Eq. (1) likely come from separate classifiers or from splitting the final fused logits; a natural test is whether three explicit per-modality branches would match or beat the reported numbers.
The 12-point jump from pretraining suggests that on small multimodal datasets, representation quality matters more than fusion design; a promising extension is to test larger or self-supervised video backbones in the same pipeline.
Because top-5 accuracy is already 100%, the residual top-1 errors are concentrated in a few confusable classes; an error analysis on the 1% remaining mistakes could reveal systematic confusions that a class-balanced loss or prototype-based correction might fix.
The reported inference size of 256x256 is below the native depth resolution (640x360), so feeding higher-resolution inputs at test time is a cheap potential way to close the remaining gap.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing to know: this paper is a competition report, not a method paper. It combines TSM, ResNet backbones, channel concatenation, weighted late fusion, SWA, ensemble, TTA, and a handful of data tricks, and reports 99% Top-1 / 100% Top-5 on the ICPR 2024 tri-modal action recognition leaderboard. The pipeline is competent and the ablation in Figure 6 is genuinely informative, showing a steady climb from 83% to 99% as you add pretraining, ensemble+SWA, twice sampling, and full-resolution inference. If you are a practitioner stuck on a small tri-modal video set, this is a useful checklist.

What's actually new is the configuration and the specific leaderboard number. No new module, no new loss, no new fusion mechanism. The authors are upfront that they are using existing techniques, which I respect. The pretraining and augmentation details are specific enough to reproduce if you had the exact data split.

The soft spots are real but not fatal. Eq. (1) is the biggest one: it says one concatenated input produces three separate modality logits (LogitsR, LogitsI, LogitsD), but the text never describes a multi-head architecture or any mechanism that would separate them after concatenation. The stress-test note is correct that this is not a realizable model as written. In practice they may be training separate single-modality models and averaging, or using a multi-head TSM, but the paper doesn't say. That needs fixing in a revision.

Second, the hyperparameters — alpha, segment count, input size, and the SWA checkpoint selection — are chosen using test-set accuracy (the leaderboard or a held-out test set). That inflates the gains in Figure 6 and makes the 99% number less meaningful. No external baselines are reported, only their own base model, so we don't know how the competition's other entries fared. The self-citation to their own published version (ref [29]) is a bit awkward, but the published paper likely contains the same content.

The central empirical claim — that this pipeline achieves 99% on that test set — is plausible and I have no reason to doubt it, but it is not independently verifiable from the paper. No code, no checkpoints, no per-class breakdown.

Who should read it: someone entering a similar multimodal competition or working on small video datasets with depth/thermal inputs. It does not advance the science but it documents a practical recipe.

Recommendation: this deserves a serious referee, but with a clear request to fix the architecture description and report a validation split or error bars. I would not cite it in my own work.

Referee Report

4 major / 4 minor

Summary. This manuscript describes a tri-modal (RGB, TIR, depth) action recognition solution developed for the ICPR 2024 competition. The method processes the three modalities by channel concatenation, feeds them into TSM-based ResNet-50 and ResNet-101 models, applies weighted fusion of the authors' claimed per-modality logits, and augments training and inference with pretraining, stochastic weight averaging, test-time augmentation, ensembling, twice temporal sampling, and a so-called full-resolution inference. The paper reports Top-1 accuracy of 99% and Top-5 accuracy of 100% on the competition leaderboard, with an ablation study in Figure 6 showing incremental gains from 83% to 99%.

Significance. If the reported results are reproducible, the paper constitutes a strong engineering baseline for tri-modal action recognition under a very limited training set, combining established techniques in a sensible pipeline. The clear ablation structure and the use of standard, well-motivated components are positive aspects. However, the significance is substantially limited by the lack of a code release, by incomplete specification of the fusion mechanism and hyperparameters, and by the fact that the central architectural equation does not describe a realizable standard TSM model as written. The paper is best viewed as a competition report rather than a methodological advance, and its claims need to be supported by a more precise and reproducible model description.

major comments (4)

[§3.2, Eq. (1)] Equation (1) states that a single network F_theta applied to channel-concatenated RGB/TIR/depth frames produces three separate logit vectors (LogitsR, LogitsI, LogitsD), but no multi-head architecture, split of the final fully connected layer, or auxiliary loss is described anywhere in Section 3.2. A standard TSM classifier with one classification head outputs a single logit vector over action classes, so after Cat(X_R, X_I, X_D) the per-modality identities are not recoverable from the output. Because Eq. (2) and the alpha-tuning experiment in Section 4.2 depend on these per-modality logits, the described model is not reproducible as written; please specify the actual mechanism (e.g., three classification heads, feature-level fusion, or separate single-modality models) and state how each logit vector is obtained.
[§4.2, Eq. (2)] The fusion weights gamma and beta in Eq. (2) are never reported; the text says only that experiments were conducted on alpha, the depth coefficient. It is therefore impossible to reproduce the weighted fusion, and the claim that alpha=0.2 is the optimal allocation is incomplete without stating the fixed values (or ranges) of gamma and beta and whether they were tuned at all.
[§4.3, Figure 6] The incremental gains in Figure 6 are extremely small relative to the test-set size (500 videos): the steps from 98.2% to 98.6% to 99% correspond to two or three samples, and no confidence intervals, repeated trials, or significance tests are reported. The central claim of superiority rests on a single leaderboard submission, so the reader cannot distinguish real improvement from noise; please add uncertainty estimates or repeated evaluations where possible.
[§3.2, Inference process] The 'full-resolution inference' is said to feed 256x256 frames, yet the native resolutions of the three modalities are Depth 640x360, TIR 320x256, and RGB 455x256. Please clarify what full-resolution means, how the three modalities are resized or aligned, and how this step differs from the input-size experiment in Figure 3(C), where 256 is already identified as an inference input size.

minor comments (4)

[Figure 2 and §3.2] The figure caption uses 'Contact' while the text says 'concatenate'; please make the terminology consistent and clarify what the 'Contact' operation does in the diagram.
[§3.1] The data augmentation hyperparameters (crop scales, flip probability, normalization statistics) are not specified; providing these values would improve reproducibility.
[§3.2, Training] The pretraining details for Kinetics400, ImageNet, and Something-somethingV2 are omitted (e.g., which layers are initialized, whether the backbone is frozen, and for how many epochs); please include these details.
[References] Reference [34] appears to duplicate reference [32]; please consolidate the duplicate entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the reported accuracy is an external leaderboard measurement and the only self-citation is non-load-bearing.

full rationale

This is an empirical competition-report paper with no formal derivation that could reduce to its inputs. The central claim is a measured leaderboard accuracy (99% Top-1, 100% Top-5) plus an ablation stack; the reported numbers are measurements, not predictions derived from a fitted model. The only self-citation is reference [29], which is the authors' own ICPR-published version of the same solution, cited in an opening list of DNN successes; it is not used to justify any load-bearing premise, uniqueness claim, or architectural choice. Equations (1)-(2) are under-specified, because a channel-concatenated TSM is not shown to produce separate per-modality logits, but under-specification is a reproducibility issue, not circularity: nothing in the equations is defined in terms of the target result. Hyperparameters such as alpha, segment count, input size, and SWA weight selection are tuned against test-set accuracy in Section 4.2, which raises overfitting and selection concerns, but it does not make the reported accuracy equivalent by construction to the tuned parameters. No circular step with a quotable equation-level reduction was found.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The paper is an empirical engineering report, so the ledger contains no invented entities and no mathematical axioms. The central claim depends on a small set of hand-selected hyperparameters (alpha, segment count, input size, SWA weight selection) and on three domain assumptions: that channel concatenation with TSM is an effective fusion, that large-scale pretraining transfers to the small tri-modal dataset, and that using the test set for model selection is an acceptable way to report accuracy.

free parameters (6)

alpha (depth logit weight) = 0.2
Set by experiment in Section 4.2 (Figure 3A); directly controls the final prediction in Eq. (2).
beta and gamma (RGB and TIR logit weights) = Not stated
Appear in Eq. (2) but no values or tuning procedure are reported, so the fusion rule is incomplete.
Number of video segments = 8
Selected by a sweep in Section 4.2 (Figure 3B); the paper reports best performance at 8 or 10 segments.
Inference input size = 256x256
Selected by a sweep in Section 4.2 (Figure 3C); larger sizes deviate from training input and hurt accuracy.
SWA checkpoint selection = Top 3 weights per model
Weights are chosen based on test-set accuracy (Section 3.2 and Section 4.2), which is selection on the evaluation data.
Ensemble combination weights = Not stated
TSM-Res50 and TSM-Res101 predictions are ensembled, but the combination formula or weights are not given.

assumptions (3)

domain assumption Channel-wise concatenation of RGB, TIR, and Depth frames followed by a TSM network is an effective fusion strategy.
Invoked in Section 3.2 Eq. (1); no evidence is provided that this fusion is optimal or that per-modality logits remain separable after concatenation.
domain assumption Pretraining on ImageNet, Kinetics400, and Something-SomethingV2 transfers to the small ICPR RGB-TIR-depth dataset.
Used throughout Section 3.2; transfer is assumed without analysis of the domain gap between the pretraining data and thermal or depth modalities.
domain assumption The competition test set can be used for hyperparameter and checkpoint selection without invalidating the reported accuracy as a fair estimate.
Section 4.2 and Figure 4 explicitly describe using test-set accuracy to identify superior weights, which presumes this is an acceptable protocol.

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Effective End-to-End Solution for Multimodal Action Recognition." pith.science (2026). https://pith.science/paper/72AD5BL4

@misc{pith2026250609345,
  author       = {Pith},
  title        = {Pith review of: An Effective End-to-End Solution for Multimodal Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/72AD5BL4}},
  note         = {Machine review of arXiv:2506.09345}
}

read the original abstract

Recently, multimodal tasks have strongly advanced the field of action recognition with their rich multimodal information. However, due to the scarcity of tri-modal data, research on tri-modal action recognition tasks faces many challenges. To this end, we have proposed a comprehensive multimodal action recognition solution that effectively utilizes multimodal information. First, the existing data are transformed and expanded by optimizing data enhancement techniques to enlarge the training scale. At the same time, more RGB datasets are used to pre-train the backbone network, which is better adapted to the new task by means of transfer learning. Secondly, multimodal spatial features are extracted with the help of 2D CNNs and combined with the Temporal Shift Module (TSM) to achieve multimodal spatial-temporal feature extraction comparable to 3D CNNs and improve the computational efficiency. In addition, common prediction enhancement methods, such as Stochastic Weight Averaging (SWA), Ensemble and Test-Time augmentation (TTA), are used to integrate the knowledge of models from different training periods of the same architecture and different architectures, so as to predict the actions from different perspectives and fully exploit the target information. Ultimately, we achieved the Top-1 accuracy of 99% and the Top-5 accuracy of 100% on the competition leaderboard, demonstrating the superiority of our solution.

Figures

Figures reproduced from arXiv: 2506.09345 by the authors.

**Figure 1.** Comparison between single-modal action recognition and multi-modal action recognition. To promote the development of this technology, the ICPR 2024 Multimodal Action Recognition competition provides a platform for researchers to apply the latest algorithms and techniques to multimodal pattern recognition tasks. This not only offers researchers the opportunity to showcase and share their achievements but also fosters… view at source ↗

**Figure 2.** The framework of our proposed solution. Temporal Sample1 and Temporal Sample2 represent two temporal samplings of multimodal videos, TR1 and TR2 represent different data augmentation methods, TTA1 and TTA2 represent Test-Time augmentation techniques, SWA represents stochastic weight averaging technique, Kinetics400 and ImageNet, SomeV2 correspond to different pre-training knowledge, where SomeV2 is the abbreviatio… view at source ↗

**Figure 3.** The results of Top-1 accuracy under three different parameter adjustments. Subplot (A) shows the impact of varying the output coefficient (α) on model performance. Subplot (B) illustrates how the number of video segments (seg) affects accuracy. Subplot (C) demonstrates the effect of different input sizes (size) on the model’s performance. the video recognition model. Although this approach increases computational … view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: The figure presents the impact of training epochs on the Top-1 accuracy for two scenarios: without pretraining (left) and with Kinetics400 pretraining (right). action recognition capabilities. In the experiments, we primarily utilize Top-1 accuracy as the metric for ev…

**Figure 5.** Figure 5: Subplot (A) presents the effect of different sampling strategies on the model’s performance. Subplot (B) compares the accuracy of various backbone network architectures. Training Epochs We conducted experiments on The influence of training epochs both with and without…

**Figure 6.** Figure 6: The figure illustrates the performance improvements of the model under different strategies. Starting from the base model, various enhancements such as Test-Time Augmentation (TTA), Enhanced knowledge (pre-training knowledge), ensemble technique combined with stochas…

**Figure 7.** Figure 7: Our proposed method compared to the baseline method in the identification visualization results on a multimodal video. The first and second rows show the prediction results of the base method on the RGB and TIR modalities, respectively, while the last two rows display…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 25 canonical work pages

[29]

In: International Conference on Pattern Recogni- tion

Wang, S., Rao, H., Hu, X., Lyu, Y., Shan, C.: An effective end-to-end solution for multimodal action recognition. In: International Conference on Pattern Recogni- tion. pp. 324–338. Springer (2025)

work page 2025
[1]

In: APSIPA ASC

Ardianto, S., Hang, H.M.: Multi-view and multi-modal action recognition with learned fusion. In: APSIPA ASC. pp. 1601–1604. IEEE (2018)

work page 2018
[2]

In: ICCV

Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lučić, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV. pp. 6836–6846 (2021)

2021
[3]

Multimedia Tools and Applications78(5), 5919–5939 (2019)

Avola, D., Bernardi, M., Foresti, G.L.: Fusing depth and colour information for human action recognition. Multimedia Tools and Applications78(5), 5919–5939 (2019)

work page 2019
[4]

Bertasius, G., Wang, H., Torresani, L.: Is space-time attention all you need for video understanding? In: ICML. vol. 2, p. 4 (2021)

2021
[5]

In: CVPR

Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: CVPR. pp. 6299–6308 (2017) 14 Songping Wang et al

work page 2017
[6]

Multimedia Tools and Applications76, 4405–4425 (2017)

Chen, C., Jafari, R., Kehtarnavaz, N.: A survey of depth and inertial sensor fusion for human action recognition. Multimedia Tools and Applications76, 4405–4425 (2017)

work page 2017
[7]

IEEE Access10, 104190–104201 (2022)

Cheng, Q., Liu, Z., Ren, Z., Cheng, J., Liu, J.: Spatial-temporal information ag- gregation and cross-modality interactive learning for rgb-d-based human action recognition. IEEE Access10, 104190–104201 (2022)

work page 2022

Show all 36 references

[8]

In: CVPR (2009)

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large- Scale Hierarchical Image Database. In: CVPR (2009)

2009
[9]

In: ICCV

Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recogni- tion. In: ICCV. pp. 6202–6211 (2019)

2019
[10]

something something

Goyal, R., Ebrahimi Kahou, S., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al.: The" something something" video database for learning and evaluating visual common sense. In: ICCV. pp. 5842–5850 (2017)

2017
[11]

In: CVPR

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR. pp. 770–778 (2016)

2016
[12]

arXiv preprint arXiv:1704.04861 (2017)

Howard,A.G.:Mobilenets:Efficientconvolutionalneuralnetworksformobilevision applications. arXiv preprint arXiv:1704.04861 (2017)

2017 arXiv
[13]

arXiv preprint arXiv:1803.05407 (2018)

Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., Wilson, A.G.: Aver- aging weights leads to wider optima and better generalization. arXiv preprint arXiv:1803.05407 (2018)

2018 arXiv
[14]

Jiang, Z., Rozgic, V., Adali, S.: Learning spatiotemporal features for infrared action recognitionwith3dconvolutionalneuralnetworks.In:CVPRW.pp.115–123(2017)

2017
[15]

arXiv preprint arXiv:1705.06950 (2017)

Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)

2017 arXiv
[16]

In: ICCV

Lin, J., Gan, C., Han, S.: Tsm: Temporal shift module for efficient video under- standing. In: ICCV. pp. 7083–7093 (2019)

2019
[17]

In: MICCAI Challenge on Fast and Low-Resource Semi-supervised Abdominal Organ Segmentation, pp

Meng, H., Zhao, H., Yang, D., Wang, S., Li, Z.: Coarse to fine segmentation method enables accurate and efficient segmentation of organs and tumor in abdominal ct. In: MICCAI Challenge on Fast and Low-Resource Semi-supervised Abdominal Organ Segmentation, pp. 115–129. Springer (2023)

2023
[18]

IJCV118, 217–239 (2016)

Palmero, C., Clapés, A., Bahnsen, C., Møgelmose, A., Moeslund, T.B., Escalera, S.: Multi-modal rgb–depth–thermal human body segmentation. IJCV118, 217–239 (2016)

2016
[19]

NIPS27(2014)

Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recog- nition in videos. NIPS27(2014)

2014
[20]

In: ICME

Song, S., Lan, C., Xing, J., Zeng, W., Liu, J.: Skeleton-indexed deep multi-modal feature learning for high performance human action recognition. In: ICME. pp. 1–6. IEEE (2018)

2018
[21]

TPAMI45(3), 3200–3225 (2022)

Sun, Z., Ke, Q., Rahmani, H., Bennamoun, M., Wang, G., Liu, J.: Human ac- tion recognition from various data modalities: A review. TPAMI45(3), 3200–3225 (2022)

2022
[22]

In: CVPR

Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: CVPR. pp. 6450–6459 (2018)

2018
[23]

In: ICMEW

Vu, D.Q., Le, N., Wang, J.C.: Self-supervised learning via multi-transformation classification for action recognition. In: ICMEW. pp. 1–6. IEEE (2024)

2024
[24]

IEEE Access11, 53880–53898 (2023) An Effective End-to-End Solution for Multimodal Action Recognition 15

Wang, C., Yan, J.: A comprehensive survey of rgb-based and skeleton-based human action recognition. IEEE Access11, 53880–53898 (2023) An Effective End-to-End Solution for Multimodal Action Recognition 15

2023
[25]

In: ECCV

Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., Van Gool, L.: Temporal segment networks: Towards good practices for deep action recognition. In: ECCV. pp. 20–36. Springer (2016)

2016
[26]

arXiv preprint arXiv:2504.14921 (2025)

Wang, S., Liu, H., Lyu, Y., Hu, X., He, Z., Wang, W., Shan, C., Wang, L.: Fast adversarial training with weak-to-strong spatial-temporal consistency in the fre- quency domain on videos. arXiv preprint arXiv:2504.14921 (2025)

2025 arXiv
[27]

In: 2024 IEEE International Conference on Multimedia and Expo (ICME)

Wang, S., Liu, H., Zhao, H.: Public-domain locator for boosting attack transfer- ability on videos. In: 2024 IEEE International Conference on Multimedia and Expo (ICME). pp. 1–6. IEEE (2024)

2024
[28]

arXiv preprint arXiv:2504.12129 (2025)

Wang, S., Lyu, Y., Liu, S., Li, N., Tong, T., Sun, H., Shan, C.: Anti-aesthetics: Pro- tecting facial privacy against customized text-to-image synthesis. arXiv preprint arXiv:2504.12129 (2025)

2025 arXiv
[30]

arXiv preprint arXiv:2503.06276 (2025)

Wang, S., Yue, X., Lyu, Y., Shan, C.: Exploring adversarial transferability between kolmogorov-arnold networks. arXiv preprint arXiv:2503.06276 (2025)

2025 arXiv
[31]

In: CVPR

Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: CVPR. pp. 7794–7803 (2018)

2018
[32]

IEEE Transactions on Pattern Analysis and Machine Intelligence45(9), 10898–10912 (2023)

Wei, X., Wang, S., Yan, H.: Efficient robustness assessment via adversarial spatial- temporal focus on videos. IEEE Transactions on Pattern Analysis and Machine Intelligence45(9), 10898–10912 (2023)

2023
[33]

In: ECCV

Xie, S., Sun, C., Huang, J., Tu, Z., Murphy, K.: Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. In: ECCV. pp. 305–321 (2018)

2018
[34]

arXiv preprint arXiv:2301.00896 (2023)

Xingxing, W., Songping, W., Huanqian, Y.: Efficient robustness assessment via ad- versarial spatial-temporal focus on videos. arXiv preprint arXiv:2301.00896 (2023)

2023 arXiv
[35]

arXiv preprint arXiv:1910.04867 (2019)

Zhai, X., Puigcerver, J., Kolesnikov, A., Ruyssen, P., Riquelme, C., Lucic, M., Djo- longa, J., Pinto, A.S., Neumann, M., Dosovitskiy, A., et al.: A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint arXiv:1910.04867 (2019)

2019 arXiv
[36]

TOMM 18(3), 1–24 (2022)

Zhu, X., Zhu, Y., Wang, H., Wen, H., Yan, Y., Liu, P.: Skeleton sequence and rgb frame based multi-modality feature fusion network for action recognition. TOMM 18(3), 1–24 (2022)

2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [29]

In: International Conference on Pattern Recogni- tion

Wang, S., Rao, H., Hu, X., Lyu, Y., Shan, C.: An effective end-to-end solution for multimodal action recognition. In: International Conference on Pattern Recogni- tion. pp. 324–338. Springer (2025)

work page 2025

[2] [1]

In: APSIPA ASC

Ardianto, S., Hang, H.M.: Multi-view and multi-modal action recognition with learned fusion. In: APSIPA ASC. pp. 1601–1604. IEEE (2018)

work page 2018

[3] [2]

In: ICCV

Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lučić, M., Schmid, C.: Vivit: A video vision transformer. In: ICCV. pp. 6836–6846 (2021)

2021

[4] [3]

Multimedia Tools and Applications78(5), 5919–5939 (2019)

Avola, D., Bernardi, M., Foresti, G.L.: Fusing depth and colour information for human action recognition. Multimedia Tools and Applications78(5), 5919–5939 (2019)

work page 2019

[5] [4]

Bertasius, G., Wang, H., Torresani, L.: Is space-time attention all you need for video understanding? In: ICML. vol. 2, p. 4 (2021)

2021

[6] [5]

In: CVPR

Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: CVPR. pp. 6299–6308 (2017) 14 Songping Wang et al

work page 2017

[7] [6]

Multimedia Tools and Applications76, 4405–4425 (2017)

Chen, C., Jafari, R., Kehtarnavaz, N.: A survey of depth and inertial sensor fusion for human action recognition. Multimedia Tools and Applications76, 4405–4425 (2017)

work page 2017

[8] [7]

IEEE Access10, 104190–104201 (2022)

Cheng, Q., Liu, Z., Ren, Z., Cheng, J., Liu, J.: Spatial-temporal information ag- gregation and cross-modality interactive learning for rgb-d-based human action recognition. IEEE Access10, 104190–104201 (2022)

work page 2022