Pith. sign in

REVIEW 5 major objections 5 minor 80 references

DESign: Dynamic Context-Aware Convolution and Efficient Subnet Regularization for Continuous Sign Language Recognition

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A sign language recognition system using only RGB video reaches state-of-the-art word error rates on three benchmarks.

desk verdict Solid CSLR paper with two genuinely useful modules; the SOTA claim is softer than it looks because the final receptive field was chosen on Test WER and all results are single runs. read the letter →

arxiv 2507.03339 v1 pith:CDEPSPBS submitted 2025-07-04 cs.CV cs.AI

Sheng Liu , Yiheng Yu , Yuan Feng , Min Xu , Zhelun Jin , Yining Jiang , Tiantian Yuan This is my paper · ORCID
classification cs.CVcs.AI
keywords continuoussignlanguagerecognitiondynamicconvolutioncontext-awarekernelgenerationconnectionisttemporalclassificationsubnetregularizationCTCspikeworderrorrateRGB-only
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to improve continuous sign language recognition (CSLR) with only RGB video, without auxiliary cues such as keypoints, optical flow, or text. It proposes two mechanisms: Dynamic Context-Aware Convolution (DCAC), which generates a distinct convolution kernel per frame from intra-frame attention and inter-frame context, and Subnet Regularization CTC (SR-CTC), which applies extra CTC losses to intermediate network stages to stop training from overfitting to a single dominant alignment path. Together these form DESign, and the paper reports state-of-the-art word error rates on PHOENIX14, PHOENIX14-T, and CSL-Daily. If the results hold, the practical payoff is cheaper, deployable sign language recognition that matches or beats systems using extra modalities and large pretrained video backbones.

What carries the argument

The load-bearing mechanism is DCAC's Context-Aware Kernel Generator (CAKG), which fuses Intra-frame Attention and Inter-frame Context Awareness to produce a unique kernel for every frame, combined with a static-branch dual design borrowed from bi-volution. SR-CTC is the second mechanism: it appends light-weight spatial downsampling and temporal modeling to stages 2-4, applies a shared classifier so multi-scale features map to a common class-prototype space, and adds the resulting CTC losses to the total loss, regularizing alignment paths at no inference cost. Together these mechanisms target the two diagnosed failure modes: static convolution kernels that cannot adapt to diverse signing, and CTC spike/dominant-path overfitting in which only a few frames receive gradient.

What would settle it

Train the baseline and measure per-frame softmax entropy (or peakiness) at stages 2, 3, and 4 and at the final output; if the shallow-stage distributions are not systematically flatter than the final output's, the mechanism by which SR-CTC spreads gradients loses its stated basis.

Watch

Extended reading notes

Core claim

The central claim is that DESign achieves state-of-the-art performance on PHOENIX14, PHOENIX14-T, and CSL-Daily while using only a ResNet34 backbone and RGB input, surpassing even multi-cue methods that rely on optical flow, keypoints, text, and pretraining on large action-recognition datasets. On the PHOENIX benchmarks DESign reaches 17.1% Dev and 17.4% Test WER on PHOENIX14, 16.5% Dev and 18.2% Test WER on PHOENIX14-T, and 25.3% Dev and 24.2% Test WER on CSL-Daily. The paper attributes the gains to two complementary modules: DCAC adapts convolutional weights per frame using context, and SR-CTC regularizes CTC training through auxiliary subnetwork supervision, with ablations showing each module contributes 1.3-2.1% absolute WER reduction over the same baseline.

Load-bearing premise

The whole SR-CTC gain rests on the premise that shallow network stages produce flatter, less peaked per-frame predictions, so auxiliary CTC losses on those stages spread gradients across many alignment paths rather than reinforcing a single dominant path.

Editorial extensions

If this is right

  • On the PHOENIX14 benchmark, adding DCAC alone cuts WER by 1.3%/1.5% (Dev/Test) and SR-CTC alone by 1.4%/0.6%, and together they reduce WER by 2.0%/1.9% over the same baseline (Table IV).
  • SR-CTC improves four existing CSLR methods (SEN, TLP, CorrNet, CorrNet+) when plugged in, with up to 1.4%/2.1% WER reduction on TLP, at zero extra inference cost (Table X).
  • DESign outperforms multi-cue systems such as SignVTCL on PHOENIX14 Dev/Test despite using only RGB and a smaller backbone, suggesting that context-aware dynamic convolution can substitute for additional modalities.
  • Because the shared classifier acts as class-wise prototypes, features from different stages are implicitly aligned in the same semantic space, which is what lets shallow stages receive direct semantic feedback.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same SR-CTC recipe could be transferred to other CTC-based sequence tasks, such as speech recognition or handwriting recognition, where posterior spiking and dominant-path overfitting are also known issues.
  • The paper's observation that boundary frames need more kernel adaptation than steady-state frames suggests a selective kernel-sharing variant of DCAC that shares weights across near-identical adjacent frames could cut computation with minimal accuracy loss.
  • A direct test of the regularization hypothesis would be to check whether per-frame prediction entropy at shallow stages is systematically higher than at the final layer on a baseline model; the paper's mechanism rests on that flatness assumption.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes DESign, a continuous sign language recognition (CSLR) framework with two main contributions. The first, Dynamic Context-Aware Convolution (DCAC), generates per-frame convolutional kernels conditioned on both intra-frame attention and inter-frame contextual information, and combines them with a static convolution branch. The second, Subnet Regularization CTC (SR-CTC), applies auxiliary CTC losses to intermediate subnetworks during training, with a shared classifier and lightweight spatial/temporal modules that are used only at training time. The method is evaluated on PHOENIX14, PHOENIX14-T, and CSL-Daily, where the authors claim state-of-the-art WER using only RGB input and no auxiliary datasets. Same-backbone ablations in Table IV show that each module contributes over a ResNet34 baseline.

Significance. The paper is a solid empirical study of two complementary mechanisms: a temporal-context-aware dynamic convolution and a training-only CTC regularizer. The strength of the paper is the ablation structure: Table IV isolates the gains of DCAC and SR-CTC on the same backbone, Table X demonstrates that SR-CTC is plug-and-play across several published CSLR methods, and Table XIV partially addresses backbone fairness by reporting ResNet18 results. The supplementary derivation of FLOPs/parameters is also useful. If the reported gains survive a proper model-selection protocol and repeated-seed evaluation, DESign would be a valuable contribution to CSLR. However, the headline state-of-the-art claim is currently fragile because the final configuration appears to be selected using Test-set numbers, all results are single runs, and the SOTA claim is stated without consistently restricting it to video-only methods.

major comments (5)
  1. [§IV.C, Table VI] The final temporal receptive field configuration is selected using Test performance. Table VI shows that L=[5,9,13] achieves the best Dev WER (17.6) but L=[3,7,11] achieves the best Test WER (18.0), and the text states that L=[3,7,11] is adopted 'taking both performance and FLOPs into consideration.' Since the Test set is used to choose the architecture, the Test WERs reported in Tables II–III are selected statistics rather than unbiased estimates of generalization. Please specify a Dev-only selection protocol (or pre-register the configuration), report the consequences of applying that protocol to Table VI, and clearly state which Test numbers are exploratory.
  2. [§IV.B, Tables II–III] The claim of state-of-the-art performance is not consistently restricted to video-only methods. In Table II, the multi-cue method SignVTCL achieves a lower Test WER than DESign on PHOENIX14-T (17.9 vs 18.2). In Table III, SignVTCL is better than DESign on both Dev and Test for CSL-Daily (24.3/24.1 vs 25.3/24.2). The abstract and the contributions say DESign 'achieves state-of-the-art performance' without the single-cue qualifier. Please either restrict the SOTA claim explicitly to RGB-only, single-cue methods or handle the multi-cue results in the comparison narrative.
  3. [§IV.B, Table XIII] The main SOTA comparisons are not same-backbone. DESign uses ResNet34, while most prior methods in Tables II–III use ResNet18, as acknowledged in Table XIII. This confounds the comparison of method contributions with backbone capacity. Table XIV provides a reassuring same-backbone ResNet18 result on PHOENIX14 (DESign Test 17.7 vs CorrNet+ 18.2), but no equivalent same-backbone comparisons are provided for PHOENIX14-T or CSL-Daily. Please add these comparisons or temper the SOTA claim to the datasets and backbones actually compared.
  4. [§IV.B–C, Tables II–IV, VI–X] All reported WER numbers appear to come from single training runs; no error bars or seed information is given. The margins over the strongest video-only baselines are small in several cases (e.g., 0.7–1.0% absolute WER on PHOENIX14 and PHOENIX14-T Test, and 0.7% on CSL-Daily Test), and CSLR training is known to have non-negligible run-to-run variance. Please report means and standard deviations over at least three seeds for the main comparisons and ablations, or otherwise demonstrate that the reported margins exceed training noise.
  5. [§III.C, Supplement IX] The mechanism claim of SR-CTC is that shallow stages produce flatter per-frame CTC distributions, which spreads gradients over more alignment paths. The supporting evidence in Supplement IX (Fig. 9) consists of temporal-correlation heatmaps, which are not a direct measure of CTC posterior flatness or of gradient spread over paths. Please add a quantitative measure, such as the entropy of per-frame posteriors at each stage or the effective number of alignments contributing to the gradient, with and without SR-CTC. This would substantiate the mechanism that motivates the module.
minor comments (5)
  1. [§III.A] There is a typo: 'whith' should be 'with' in the sentence describing the temporal features.
  2. [§III.B, Eq. (3)] The notation GAPH,W→1,1 is not defined at first use; please spell out the global average pooling operation, for example GAP over spatial dimensions H and W.
  3. [§III.C and Table VIII] The text says 'Tab. VIII provides strong evidence supporting this claim,' but Table VIII is an ablation of classifier sharing, not direct evidence that shallow features are flatter. Please rephrase or point to the appropriate figures/supplement.
  4. [Fig. 4 caption] The caption does not indicate which color corresponds to which stage; please add a legend or label the curves.
  5. [§IV.B, paragraph on SignVTCL] The sentence 'surpassing SignVTCL despite its simpler design' is accurate for PHOENIX14 but not for PHOENIX14-T Test and CSL-Daily, where SignVTCL remains better on Test. Please qualify this claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical SOTA claim and module ablations are self-contained against external benchmarks; self-citations are motivational only.

full rationale

The paper's central claims are empirical WER results on PHOENIX14, PHOENIX14-T, and CSL-Daily, plus ablations showing DCAC and SR-CTC each improve WER over the same-backbone baseline. No claimed 'prediction' is derived from a fitted constant or from an identity that makes the output equal to the input by construction. The ablation tables (Tables IV-X) compare actual measured WERs under different configurations, and the final SOTA numbers are external benchmark measurements rather than consequences of the chosen hyperparameters. The hyperparameters lambda=0.1, r=16, n=6, and L=[3,7,11] are engineering choices selected empirically; Table VI even reports that L=[5,9,13] gives a better Dev WER while L=[3,7,11] gives a better Test WER, and the paper selects the latter 'taking both performance and FLOPs into consideration.' This is a model-selection and evaluation-protocol concern, not circularity, because the reported Test WER is not forced by the choice of L through any equation. The self-citations [20] and [21] are used only to motivate that sign language has strong temporal contextual dependencies; the architecture of DCAC and the design of SR-CTC are not justified by those citations as load-bearing premises. SR-CTC is explicitly credited to the external Intermediate CTC work [28], and the classifier-sharing idea is attributed to the external SMKD work [10]. The paper therefore does not reduce its central claim to a self-citation chain, a renamed known result, or a fitted input called a prediction. It may have correctness or reproducibility risks (single runs, test-set-informed configuration choice), but those are not circularity under the stated criteria.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest on standard deep learning practice (ImageNet-pretrained backbones, CTC loss) and about six hyperparameters tuned on the validation/test sets. No new theoretical entities are introduced; the derivation of CTC gradients in the supplement is a standard textbook result. The main domain assumption, that shallow features yield flatter CTC distributions, is supported by visualizations but not proven.

free parameters (6)
  • lambda (SR-CTC loss weight) = 0.1
    Set empirically in Sec. III.D to balance the auxiliary CTC losses.
  • reduction ratio r = 16
    Chosen based on prior dynamic convolution works [17],[44], Sec. IV.A.
  • number of experts n = 6
    Empirically determined in Sec. IV.A; affects kernel capacity.
  • temporal receptive fields L=[L2,L3,L4] = [3,7,11]
    Ablated in Table VI; chosen for best Dev/Test WER and FLOPs.
  • stage placement for DCAC and SR-CTC = stages 2,3,4
    Selected by ablations in Tables VII and IX; stage 1 excluded due to low-level features.
  • classifier sharing configuration = all shared, unfrozen
    Ablated in Table VIII; configuration 3 chosen.
assumptions (4)
  • standard math CTC gradient is dominated by high-probability alignment paths (Eq. 25)
    Derived from the CTC loss definition in Sec. VIII; standard result.
  • domain assumption Shallow-stage features produce flatter per-frame distributions, spreading CTC gradients across more alignment paths
    Stated in Sec. III.C and supplementary IX; supported by Figs. 8-9 but assumed in the SR-CTC design.
  • domain assumption Cross-frame motion and contextual dependencies are necessary for accurate CSLR
    Motivation for DCAC in Sec. I; based on prior CSLR literature [11],[14],[20],[21].
  • domain assumption ImageNet-pretrained ResNet34 provides a suitable feature extractor
    Standard CSLR practice, used for all experiments; influences results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DESign: Dynamic Context-Aware Convolution and Efficient Subnet Regularization for Continuous Sign Language Recognition." pith.science (2026). https://pith.science/paper/CDEPSPBS

@misc{pith2026250703339,
  author       = {Pith},
  title        = {Pith review of: DESign: Dynamic Context-Aware Convolution and Efficient Subnet Regularization for Continuous Sign Language Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CDEPSPBS}},
  note         = {Machine review of arXiv:2507.03339}
}
read the original abstract

Current continuous sign language recognition (CSLR) methods struggle with handling diverse samples. Although dynamic convolutions are ideal for this task, they mainly focus on spatial modeling and fail to capture the temporal dynamics and contextual dependencies. To address this, we propose DESign, a novel framework that incorporates Dynamic Context-Aware Convolution (DCAC) and Subnet Regularization Connectionist Temporal Classification (SR-CTC). DCAC dynamically captures the inter-frame motion cues that constitute signs and uniquely adapts convolutional weights in a fine-grained manner based on contextual information, enabling the model to better generalize across diverse signing behaviors and boost recognition accuracy. Furthermore, we observe that existing methods still rely on only a limited number of frames for parameter updates during training, indicating that CTC learning overfits to a dominant path. To address this, SR-CTC regularizes training by applying supervision to subnetworks, encouraging the model to explore diverse CTC alignment paths and effectively preventing overfitting. A classifier-sharing strategy in SR-CTC further strengthens multi-scale consistency. Notably, SR-CTC introduces no inference overhead and can be seamlessly integrated into existing CSLR models to boost performance. Extensive ablations and visualizations further validate the effectiveness of the proposed methods. Results on mainstream CSLR datasets (i.e., PHOENIX14, PHOENIX14-T, CSL-Daily) demonstrate that DESign achieves state-of-the-art performance.

Figures

Figures reproduced from arXiv: 2507.03339 by the authors.

Figure 1
Figure 1. A comparison between (a) existing convolution methods in CSLR and (b) our Dynamic Context-Aware Convolution (DCAC). Existing convolutional approaches in CSLR [11], [14], [23] share the same kernel across frames and lack the ability to dynamically adjust weights based on input samples. In contrast, DCAC adaptively adjusts the convolution kernel for each frame according to its contextual information, exhibiting strong… view at source ↗
Figure 2
Figure 2. An overview of DESign. The architecture comprises a frame-level feature extractor (2D CNN), followed by a 1D CNN and a two-layer BiLSTM for [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The architecture of the Context-Aware Kernel Generator (CAKG), as [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Illustration of classifier-sharing strategy in SR-CTC. This design [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: In the first example (top row), SR-CTC corrects errors [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of recognition results among Baseline, Baseline+SR-CTC, Baseline+DCAC, and DESign (i.e., Baseline+SR-CTC+DCAC) on [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Heatmap comparison between DESign and prior methods (i.e., Baseline, TLP [27], SEN [23], CorrNet [11], and CorrNet+ [14]). Red areas denote [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Comparison of frame-wise gradient L2 norms for a video sample [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Temporal correlation heatmaps of features from different stages. From left to right, the plots correspond to stage 2, stage 3, stage 4, and the final [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 11
Figure 11. Figure 11: Gradient distribution and zero-value ratio across different stages in [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 10
Figure 10. Figure 10: Comparison of FLOPs, inference time, and memory consumption [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 12
Figure 12. Figure 12: Visualization of weight variations along the context dimension (set to 7 in this case) generated by CAKG across different frames during inference. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Frame-wise visualization of the weight variations generated by [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 65 canonical work pages

  1. [1]

    Speech recognition techniques for a sign language recognition system,

    P. Dreuw, D. Rybach, T. Deselaers, M. Zahedi, and H. Ney, “Speech recognition techniques for a sign language recognition system,” hand, vol. 60, no. 80, 2007

    work page 2007

  2. [2]

    Automatic sign language analysis: A survey and the future beyond lexical meaning,

    S. C. Ong and S. Ranganath, “Automatic sign language analysis: A survey and the future beyond lexical meaning,” IEEE Transactions on Pattern Analysis & Machine Intelligence , vol. 27, no. 06, pp. 873–891, 2005

    work page 2005

  3. [3]

    Sign language and linguistic universals, wendy sandler and diane lillo-martin,

    T. Reagan, “Sign language and linguistic universals, wendy sandler and diane lillo-martin,” Studies in Second Language Acquisition , vol. 29, no. 4, p. 624–625, 2007

    work page 2007

  4. [4]

    Reviewing 25 years of continuous sign language recognition research: Advances, challenges, and prospects,

    S. Alyami, H. Luqman, and M. Hammoudeh, “Reviewing 25 years of continuous sign language recognition research: Advances, challenges, and prospects,” Information Processing & Management , vol. 61, no. 5, p. 103774, 2024

    2024

  5. [5]

    Orientation histograms for hand gesture recognition,

    W. T. Freeman and M. Roth, “Orientation histograms for hand gesture recognition,” in International workshop on automatic face and gesture recognition, vol. 12. Citeseer, 1995, pp. 296–301

    work page 1995

  6. [6]

    Discriminative exemplar coding for sign language recognition with kinect,

    C. Sun, T. Zhang, B.-K. Bao, C. Xu, and T. Mei, “Discriminative exemplar coding for sign language recognition with kinect,” IEEE Transactions on Cybernetics , vol. 43, no. 5, pp. 1418–1428, 2013

    work page 2013

  7. [7]

    Pose-based sign language recognition using gcn and bert,

    A. Tunga, S. V . Nuthalapati, and J. Wachs, “Pose-based sign language recognition using gcn and bert,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision , 2021, pp. 31–40

    work page 2021

  8. [8]

    Signbert: pre-training of hand-model-aware representation for sign language recognition,

    H. Hu, W. Zhao, W. Zhou, Y . Wang, and H. Li, “Signbert: pre-training of hand-model-aware representation for sign language recognition,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 11 087–11 096

    work page 2021

Show all 80 references
  1. [9]

    Visual alignment constraint for continuous sign language recognition,

    Y . Min, A. Hao, X. Chai, and X. Chen, “Visual alignment constraint for continuous sign language recognition,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , 2021, pp. 11 522–11 531

    2021

  2. [10]

    Self-mutual distillation learning for continuous sign language recognition,

    A. Hao, Y . Min, and X. Chen, “Self-mutual distillation learning for continuous sign language recognition,” 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pp. 11 283–11 292, 2021. [On- line]. Available: https://api.semanticscholar.org/CorpusID:244073884

    2021

  3. [11]

    Continuous sign language recognition with correlation network,

    L. Hu, L. Gao, Z. Liu, and W. Feng, “Continuous sign language recognition with correlation network,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023, pp. 2529– 2539

    2023

  4. [12]

    Two-stream network for sign language recognition and translation,

    Y . Chen, R. Zuo, F. Wei, Y . Wu, S. Liu, and B. K.-W. Mak, “Two-stream network for sign language recognition and translation,” ArXiv, vol. abs/2211.01367, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:253254833

    2022 arXiv

  5. [13]

    Tcnet: Continuous sign language recognition from trajectories and correlated regions,

    H. Lu, A. A. Salah, and R. Poppe, “Tcnet: Continuous sign language recognition from trajectories and correlated regions,” 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:268531355

    2024

  6. [14]

    Corrnet+: Sign language recognition and translation via spatial-temporal correlation,

    L. Hu, W. Feng, L. Gao, Z. Liu, and L. Wan, “Corrnet+: Sign language recognition and translation via spatial-temporal correlation,” arXiv preprint arXiv:2404.11111 , 2024

    2024 arXiv

  7. [15]

    Dynamic filter networks,

    X. Jia, B. De Brabandere, T. Tuytelaars, and L. V . Gool, “Dynamic filter networks,” Advances in neural information processing systems , vol. 29, 2016

    2016

  8. [16]

    Dynamic convolution: Attention over convolution kernels,

    Y . Chen, X. Dai, M. Liu, D. Chen, L. Yuan, and Z. Liu, “Dynamic convolution: Attention over convolution kernels,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11 030–11 039

    2020

  9. [17]

    Omni-dimensional dynamic convolution,

    C. Li, A. Zhou, and A. Yao, “Omni-dimensional dynamic convolution,” arXiv preprint arXiv:2209.07947 , 2022

    2022 arXiv

  10. [18]

    Tada! temporally-adaptive convolutions for video understanding,

    Z. Huang, S. Zhang, L. Pan, Z. Qing, M. Tang, Z. Liu, and M. H. Ang Jr, “Tada! temporally-adaptive convolutions for video understanding,”arXiv preprint arXiv:2110.06178, 2021

    2021 arXiv

  11. [19]

    Temporally-adaptive models for efficient video understanding,

    Z. Huang, S. Zhang, L. Pan, Z. Qing, Y . Zhang, Z. Liu, and M. H. Ang Jr, “Temporally-adaptive models for efficient video understanding,” arXiv preprint arXiv:2308.05787, 2023

    2023 arXiv

  12. [20]

    Improving continuous sign language recognition via cross-frame interactions in expanded contextual spaces,

    Y . Yu, S. Liu, Y . Feng, M. Xu, Z. Jin, and X. Yang, “Improving continuous sign language recognition via cross-frame interactions in expanded contextual spaces,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2025...

    2025

  13. [21]

    Olmd: Orientation-aware long-term motion decoupling for con- tinuous sign language recognition,

    ——, “Olmd: Orientation-aware long-term motion decoupling for con- tinuous sign language recognition,” in Proceedings of the AAAI Confer- ence on Artificial Intelligence , vol. 39, no. 9, 2025, pp. 9707–9715

    2025

  14. [22]

    Tam: Temporal adaptive module for video recognition,

    Z. Liu, L. Wang, W. Wu, C. Qian, and T. Lu, “Tam: Temporal adaptive module for video recognition,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 13 708–13 718

    2021

  15. [23]

    Self-emphasizing network for continuous sign language recognition,

    L. Hu, L. Gao, Z. Liu, and W. Feng, “Self-emphasizing network for continuous sign language recognition,” in AAAI Conference on Artificial Intelligence, 2022. [Online]. Available: https://api.semanticscholar.org/ CorpusID:254096222

    2022

  16. [24]

    A deep neural framework for continuous sign language recognition by iterative training,

    R. Cui, H. Liu, and C. Zhang, “A deep neural framework for continuous sign language recognition by iterative training,” IEEE Transactions on Multimedia , vol. 21, pp. 1880–1891, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:68149654

    2019

  17. [25]

    Iterative alignment network for continuous sign language recognition,

    J. Pu, W. gang Zhou, and H. Li, “Iterative alignment network for continuous sign language recognition,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 4160–4169,

    2019

  18. [26]

    Deep radial embedding for visual sequence learning

    Y . Min, P. Jiao, Y . Li, W. Xiaotao, L. LEI, X. Chai, and X. Chen, “Deep radial embedding for visual sequence learning.” vol. 13666, 2022, pp. 240–256

    2022

  19. [27]

    Temporal lift pooling for continuous sign language recognition,

    L. Hu, L. Gao, Z. Liu, and W. Feng, “Temporal lift pooling for continuous sign language recognition,” in European Conference on Computer Vision, 2022. [Online]. Available: https://api.semanticscholar. org/CorpusID:250626845

    2022

  20. [28]

    Intermediate loss regularization for ctc- based speech recognition,

    J. Lee and S. Watanabe, “Intermediate loss regularization for ctc- based speech recognition,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 6224–6228

    2021

  21. [29]

    Cosign: Explor- ing co-occurrence signals in skeleton-based continuous sign language recognition,

    P. Jiao, Y . Min, Y . Li, X. Wang, L. Lei, and X. Chen, “Cosign: Explor- ing co-occurrence signals in skeleton-based continuous sign language recognition,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 20 619–20 629

    2023

  22. [30]

    Cvt-slr: Contrastive visual-textual transformation for sign language recognition with variational alignment,

    J. Zheng, Y . Wang, C. Tan, S. Li, G. Wang, J. Xia, Y . Chen, and S. Z. Li, “Cvt-slr: Contrastive visual-textual transformation for sign language recognition with variational alignment,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 2...

    2023

  23. [31]

    Spatial-temporal multi-cue net- work for sign language recognition and translation,

    H. Zhou, W. Zhou, Y . Zhou, and H. Li, “Spatial-temporal multi-cue net- work for sign language recognition and translation,” IEEE Transactions on Multimedia, vol. 24, pp. 768–779, 2022. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 13

    2022

  24. [32]

    Learning sign language by watching tv (using weakly aligned subtitles),

    P. Buehler, A. Zisserman, and M. Everingham, “Learning sign language by watching tv (using weakly aligned subtitles),” in 2009 IEEE Confer- ence on Computer Vision and Pattern Recognition . IEEE, 2009, pp. 2961–2968

    2009

  25. [33]

    Video-based signer- independent arabic sign language recognition using hidden markov models,

    M. AL-Rousan, K. Assaleh, and A. Tala’a, “Video-based signer- independent arabic sign language recognition using hidden markov models,” Applied Soft Computing , vol. 9, no. 3, pp. 990–999, 2009. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S1568494609000209

    2009

  26. [34]

    C2slr: Consistency-enhanced continuous sign language recognition,

    R. Zuo and B. Mak, “C2slr: Consistency-enhanced continuous sign language recognition,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022, pp. 5121–5130

    2022

  27. [35]

    C2st: Cross-modal contextualized sequence transduction for continuous sign language recognition,

    H. Zhang, Z. Guo, Y . Yang, X. Liu, and D. Hu, “C2st: Cross-modal contextualized sequence transduction for continuous sign language recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 21 053–21 062

    2023

  28. [36]

    Joint ctc-attention based end- to-end speech recognition using multi-task learning,

    S. Kim, T. Hori, and S. Watanabe, “Joint ctc-attention based end- to-end speech recognition using multi-task learning,” in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, pp. 4835–4839

    2017

  29. [37]

    Auto-avsr: Audio-visual speech recognition with automatic labels,

    P. Ma, A. Haliassos, A. Fernandez-Lopez, H. Chen, S. Petridis, and M. Pantic, “Auto-avsr: Audio-visual speech recognition with automatic labels,” in ICASSP 2023-2023 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  30. [38]

    End-to-end audio-visual speech recognition with conformers,

    P. Ma, S. Petridis, and M. Pantic, “End-to-end audio-visual speech recognition with conformers,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 7613–7617

    2021

  31. [39]

    Relaxing the conditional independence as- sumption of ctc-based asr by conditioning on intermediate predictions,

    J. Nozaki and T. Komatsu, “Relaxing the conditional independence as- sumption of ctc-based asr by conditioning on intermediate predictions,” arXiv preprint arXiv:2104.02724 , 2021

    2021 arXiv

  32. [40]

    Re-sign: Re-aligned end-to-end sequence modelling with deep recurrent cnn-hmms,

    O. Koller, S. Zargaran, and H. Ney, “Re-sign: Re-aligned end-to-end sequence modelling with deep recurrent cnn-hmms,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 4297–4305

    2017

  33. [41]

    Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential paral- lelism in sign language videos,

    O. Koller, N. C. Camgoz, H. Ney, and R. Bowden, “Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential paral- lelism in sign language videos,” IEEE transactions on pattern analysis and machine intelligence , vol. 42, no. 9, pp. 2306–2320, 2019

    2019

  34. [42]

    Distilling cross-temporal contexts for continuous sign language recognition,

    L. Guo, W. Xue, Q. Guo, B. Liu, K. Zhang, T. Yuan, and S. Chen, “Distilling cross-temporal contexts for continuous sign language recognition,” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 10 771–10 780, 2023. [Online]. Available: https://api...

    2023

  35. [43]

    Signvtcl: multi-modal continuous sign language recog- nition enhanced by visual-textual contrastive learning,

    H. Chen, J. Wang, Z. Guo, J. Li, D. Zhou, B. Wu, C. Guan, G. Chen, and P.-A. Heng, “Signvtcl: multi-modal continuous sign language recog- nition enhanced by visual-textual contrastive learning,” arXiv preprint arXiv:2401.11847, 2024

    2024 arXiv

  36. [44]

    Condconv: Conditionally parameterized convolutions for efficient inference,

    B. Yang, G. Bender, Q. V . Le, and J. Ngiam, “Condconv: Conditionally parameterized convolutions for efficient inference,” Advances in neural information processing systems , vol. 32, 2019

    2019

  37. [45]

    Bi-volution: a static and dynamic coupled filter,

    X. Hu, X. Chen, B. Ni, T. Li, and Y . Liu, “Bi-volution: a static and dynamic coupled filter,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, 2022, pp. 960–968

    2022

  38. [46]

    A dynamic convolutional layer for short range weather prediction,

    B. Klein, L. Wolf, and Y . Afek, “A dynamic convolutional layer for short range weather prediction,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2015, pp. 4840–4848

    2015

  39. [47]

    Hypernetworks,

    D. Ha, A. Dai, and Q. V . Le, “Hypernetworks,” arXiv preprint arXiv:1609.09106, 2016

    2016 arXiv

  40. [48]

    Connection- ist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connection- ist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning , 2006, pp. 369–376

    2006

  41. [49]

    Interaug: augmenting noisy intermediate predictions for ctc-based asr,

    Y . Nakagome, T. Komatsu, Y . Fujita, S. Ichimura, and Y . Kida, “Interaug: augmenting noisy intermediate predictions for ctc-based asr,” arXiv preprint arXiv:2204.00174, 2022

    2022 arXiv

  42. [50]

    Cr-ctc: Consistency regularization on ctc for improved speech recognition,

    Z. Yao, W. Kang, X. Yang, F. Kuang, L. Guo, H. Zhu, Z. Jin, Z. Li, L. Lin, and D. Povey, “Cr-ctc: Consistency regularization on ctc for improved speech recognition,” arXiv preprint arXiv:2410.05101 , 2024

    2024 arXiv

  43. [51]

    Boundary and context aware training for cif-based non-autoregressive end-to-end asr,

    F. Yu, H. Luo, P. Guo, Y . Liang, Z. Yao, L. Xie, Y . Gao, L. Hou, and S. Zhang, “Boundary and context aware training for cif-based non-autoregressive end-to-end asr,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2021, pp. 328–334

    2021

  44. [52]

    Cass-nat: Ctc alignment-based single step non-autoregressive transformer for speech recognition,

    R. Fan, W. Chu, P. Chang, and J. Xiao, “Cass-nat: Ctc alignment-based single step non-autoregressive transformer for speech recognition,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 5889–5893

    2021

  45. [53]

    Knowledge transfer from pre-trained language models to cif-based speech recognizers via hierarchical distillation,

    M. Han, F. Chen, J. Shi, S. Xu, and B. Xu, “Knowledge transfer from pre-trained language models to cif-based speech recognizers via hierarchical distillation,” arXiv preprint arXiv:2301.13003 , 2023

    2023 arXiv

  46. [54]

    Cif: Continuous integrate-and-fire for end-to- end speech recognition,

    L. Dong and B. Xu, “Cif: Continuous integrate-and-fire for end-to- end speech recognition,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2020, pp. 6079–6083

    2020

  47. [55]

    Advances in joint ctc- attention based end-to-end speech recognition with a deep cnn encoder and rnn-lm,

    T. Hori, S. Watanabe, Y . Zhang, and W. Chan, “Advances in joint ctc- attention based end-to-end speech recognition with a deep cnn encoder and rnn-lm,” arXiv preprint arXiv:1706.02737 , 2017

    2017 arXiv

  48. [56]

    Squeeze-and-excitation networks,

    J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141

    2018

  49. [57]

    Subunets: End-to-end hand shape and continuous sign language recognition,

    N. C. Camg ¨oz, S. Hadfield, O. Koller, and R. Bowden, “Subunets: End-to-end hand shape and continuous sign language recognition,” 2017 IEEE International Conference on Computer Vision (ICCV) , pp. 3075–3084, 2017. [Online]. Available: https://api.semanticscholar.org/ CorpusID...

    2017

  50. [58]

    Adabrowse: Adaptive video browser for efficient continuous sign language recognition,

    L. Hu, L. Gao, Z. Liu, C.-M. Pun, and W. Feng, “Adabrowse: Adaptive video browser for efficient continuous sign language recognition,” Proceedings of the 31st ACM International Conference on Multimedia ,

  51. [59]

    Slowfast network for continuous sign language recognition,

    J. Ahn, Y . Jang, and J. S. Chung, “Slowfast network for continuous sign language recognition,” in IEEE International Conference on Acoustics, Speech and Signal Processing , 2024, pp. 3920–3924

    2024

  52. [60]

    Signgraph: A sign sequence is worth graphs of nodes,

    S. Gan, Y . Yin, Z. Jiang, H. Wen, L. Xie, and S. Lu, “Signgraph: A sign sequence is worth graphs of nodes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024

    2024

  53. [61]

    Signbert+: Hand-model-aware self-supervised pre-training for sign language understanding,

    H. Hu, W. Zhao, W. Zhou, and H. Li, “Signbert+: Hand-model-aware self-supervised pre-training for sign language understanding,” IEEE Transactions on Pattern Analysis &; Machine Intelligence , vol. 45, no. 09, pp. 11 221–11 239, sep 2023

    2023

  54. [62]

    Gloss prior guided visual feature learning for continuous sign language recog- nition,

    L. Guo, W. Xue, B. Liu, K. Zhang, T. Yuan, and D. Metaxas, “Gloss prior guided visual feature learning for continuous sign language recog- nition,” IEEE Transactions on Image Processing , vol. 33, pp. 3486– 3495, 2024

    2024

  55. [63]

    Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers,

    Forster, Jens, Ney, Hermann, Koller, and Oscar, “Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers,” Computer vision and image under- standing: CVIU, vol. 141, pp. 108–125, 2015

    2015

  56. [64]

    Neural sign language translation,

    N. C. Camgoz, S. Hadfield, O. Koller, H. Ney, and R. Bowden, “Neural sign language translation,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2018, pp. 7784–7793

    2018

  57. [65]

    Improving sign language translation with monolingual data by sign back-translation,

    H. Zhou, W. Zhou, W. Qi, J. Pu, and H. Li, “Improving sign language translation with monolingual data by sign back-translation,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 1316–1325

    2021

  58. [66]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 770–778, 2015. [Online]. Available: https://api.semanticscholar.org/CorpusID:206594692

    2016

  59. [67]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255, 2009. [Online]. Available: https://api.semanticscholar.org/CorpusID:57246310

    2009

  60. [68]

    Video- based sign language recognition without temporal segmentation,

    J. Huang, W. gang Zhou, Q. Zhang, H. Li, and W. Li, “Video- based sign language recognition without temporal segmentation,” in AAAI Conference on Artificial Intelligence , 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:9005234

    2018

  61. [69]

    Swin-mstp: Swin transformer with multi- scale temporal perception for continuous sign language recognition,

    S. Alyami and H. Luqman, “Swin-mstp: Swin transformer with multi- scale temporal perception for continuous sign language recognition,” Neurocomputing, vol. 617, p. 129015, 2025

    2025

  62. [70]

    Uni-sign: Toward unified sign language understanding at scale,

    Z. Li, W. Zhou, W. Zhao, K. Wu, H. Hu, and H. Li, “Uni-sign: Toward unified sign language understanding at scale,” arXiv preprint arXiv:2501.15187, 2025

    2025 arXiv

  63. [71]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 6202–6211

    2019

  64. [72]

    Rethinking spatiotem- poral feature learning: Speed-accuracy trade-offs in video classification,

    S. Xie, C. Sun, J. Huang, Z. Tu, and K. Murphy, “Rethinking spatiotem- poral feature learning: Speed-accuracy trade-offs in video classification,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 305–321. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO...

    2018

  65. [73]

    Grad-cam: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision, vol. 128, pp. 336 – 359, 2016. [Online]. Available: https://api.semanticsch...

    2016

  66. [74]

    Dynamical semantic enhancement network for continuous sign language recognition,

    S. Wang, L. Guo, and W. Xue, “Dynamical semantic enhancement network for continuous sign language recognition,” Multimedia Systems, vol. 30, p. 313, 2024

    2024

  67. [75]

    Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size,

    F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size,” arXiv preprint arXiv:1602.07360 , 2016

    2016 arXiv

  68. [76]

    Deep layer aggregation,

    F. Yu, D. Wang, E. Shelhamer, and T. Darrell, “Deep layer aggregation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2403–2412. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 15 Supplementary Materials This supplementar...

    2018

  69. [79]

    1 and Fig

    FLOPs: The following lists all FLOPs involved in the DCAC dynamic branch, corresponding to Alg. 1 and Fig. 3 in the main paper: FLOPs(Unfold) =Ci × T × H × W × ktkhkw, FLOPs(GAP) =Ci × T × H × W, FLOPs(FC) =Ci × Ci r × T, FLOPs(BN) = Ci r × T, FLOPs(FCs) = Ci r × Ci G + Co + k...

    2015

  70. [80]

    Parameters: According to Fig. 3 in the main text, the parameter count of each trainable module is as follows: Params(FC) =Ci × Ci r , Params(BN) = Ci r , Params(FCs) = Ci r × Ci G + Co + kt + n , Params(Experts) =n × Co × Ci G × ktkhkw, Params(Conv1) =C 2 i , Params(Conv2) = C...

    2015

  71. [2019]

    Available: https://api.semanticscholar.org/CorpusID: 195443370

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 195443370

  72. [2023]

    Available: https://api.semanticscholar.org/CorpusID: 260926078

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 260926078

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.