Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PGG-SLT shows that LLM-generated pseudo glosses, reordered by weak video supervision, can substitute for nearly all expert gloss annotations in sign language translation.

desk verdict A practical, well-ablated LLM-based pseudo-gloss pipeline for sign language translation that beats gloss-free SOTA; the reordering step has a real self-training flavor that needs one clean decoupling ablation before the SOTA claim is fully trusted. read the letter →

arxiv 2505.15438 v1 pith:HBDN2GNM submitted 2025-05-21 cs.CV

Jianyuan Guo , Peike Li , Trevor Cohn This is my paper · ORCID
classification cs.CV
keywords signlanguagetranslationpseudoglossin-contextlearningweaklysupervisedreorderingCTClossgloss-freelargemodelsvideo-textalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that expert gloss annotations, the costly intermediate layer used to train sign-language translation systems, can be replaced by pseudo glosses produced by a large language model prompted with only a few dozen example text-gloss pairs. It claims that these machine-written glosses become good enough when their order is corrected using weak visual supervision: a classifier trained without timestamps predicts which glosses appear in each video, and a greedy algorithm reorders the LLM output to match the inferred temporal sequence. The resulting three-stage pipeline, video to pseudo gloss, pseudo gloss to text, then joint fine-tuning, is claimed to surpass previous gloss-free methods on Phoenix14T and How2Sign, and to sit within about one BLEU point of fully gloss-supervised baselines on Phoenix14T. A reader should care because, if true, the bottleneck of expert glossing, which is expensive, rare, and language-specific, is largely removed, making translation systems easier to scale to new sign languages and datasets.

What carries the argument

The load-bearing object is the reordered pseudo gloss sequence $\widetilde{G}_{V}^{\text{target}}$, produced by three linked components. First, a large language model is prompted in-context with a handful of text-gloss example pairs to draft a pseudo gloss sequence from the spoken-language reference sentence. Second, because that draft follows written-language word order, a weakly supervised multi-label classifier maps video-frame features to a probability distribution over the pseudo gloss vocabulary, trained with a max-pooled binary cross-entropy loss, frequency-aware class weighting, and an $\ell^1$ temporal smoothing penalty; max-pooling over the vocabulary yields a frame-wise gloss, which is condensed into a reference gloss by filtering to words present in the draft and merging consecutive duplicates. Third, a greedy two-pointer algorithm (Algorithm 1) reorders the draft so that words shared with the reference gloss appear in the order of their first video-predicted occurrence. The resulting $\widetilde{G}_{V}^{\text{target}}$ is used as the target for a Connectionist Temporal Classification (CTC) loss during Sign2Gloss pre-training and as the source sequence for training the Gloss2Text translator, so the pseudo gloss plays the role that expert gloss annotations played in earlier methods.

What would settle it

Take the Phoenix14T development videos, where true glosses and their temporal order are available, and compare the reordered pseudo glosses used as CTC targets against the true gloss sequence; if a large fraction of reordered glosses have the same order as the true gloss only by chance, or if removing the reordering step changes final BLEU-4 by less than one point, the paper's explanation of why the method works would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that pseudo glosses generated from text by an LLM, then reordered using video-derived temporal cues, can stand in for human gloss annotations in sign language translation. Concretely, the LLM is prompted in-context with a handful of text-gloss example pairs to produce a draft gloss sequence from the spoken-language reference; because that draft follows written-language word order rather than sign order, the paper treats it as an unordered label set and trains a weakly supervised multi-label classifier on video frames to predict which glosses are present. A greedy pointer algorithm then reorders the draft to match the frame-wise predictions, and the reordered sequence is used as the target for a Connectionist Temporal Classification (CTC) loss during Sign2Gloss pre-training and for training the Gloss2Text translator. The final Sign2Text stage fine-tunes the whole model end-to-end while dropping the CTC loss. The paper reports that this pipeline outperforms prior gloss-free methods and nearly closes the gap to gloss-supervised translation, with best Phoenix14T test BLEU-4 of 27.32, above the previous gloss-free baseline of 26.02 and within about one point of the gloss-supervised baseline of 28.39.

Load-bearing premise

The load-bearing premise is that a classifier trained only with video-level labels can reliably recover the order of signs in each video; if its frame-wise order predictions are noisy, the reordered glosses used as training targets are corrupt and the improvement would have to be explained by something else.

Editorial extensions

If this is right

  • If the claim holds, a few dozen gloss annotations (30 from Phoenix14T, about 0.4% of the training split) are enough to bootstrap a competitive sign-language translation system, so datasets that currently lack glosses can be used with intermediate-supervision training.
  • The gap between gloss-free and gloss-based translation narrows from several BLEU points to roughly one point on Phoenix14T, making gloss-free pipelines a viable default for new languages.
  • The same recipe transfers to How2Sign, a benchmark with no gloss annotations at all, where LLM pseudo glosses built with examples borrowed from another ASL corpus improve over prior gloss-free methods.
  • Using a stronger decoder-only translation model in the Gloss2Text stage yields further gains, showing that the method benefits from the translation capacity of large language models.
  • At inference time the model needs no glosses at all, outputting spoken-language text directly from video, so the added supervision does not complicate deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because the draft glosses come from text alone, the method implicitly assumes the written reference encodes most of the semantic content of the signing; a natural next step is to let the LLM also see a few predicted video keywords or frames before drafting, which could recover signs that never appear in the written sentence.
  • Editorial extension: the reordering classifier's frame-wise predictions are never directly checked against true gloss timestamps, and Phoenix14T's ground-truth annotations make that check possible; a diagnostic comparing predicted gloss order to true sign order would isolate how much of the gain comes from order correction versus vocabulary filtering.
  • Editorial extension: the training schedule that drops CTC supervision in the final epochs suggests a curriculum interpretation, coarse gloss-level alignment early and free-form language modeling late, and one could test whether a gradually annealed CTC weight works as well as the abrupt cutoff.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes PGG-SLT, a sign language translation method that avoids full gloss annotation by using an LLM to generate pseudo glosses from spoken-language text via in-context learning with a small number of text-gloss pairs. A weakly supervised classifier on video features reorders the pseudo glosses to better match temporal sign order, and a three-stage training pipeline (Sign2Gloss with CTC, Gloss2Text, Sign2Text) is then applied. Experiments on Phoenix14T and How2Sign report BLEU4 improvements over prior gloss-free methods and competitive results with gloss-based methods.

Significance. If the reordering mechanism is reliable, the work is significant: it shows that LLM-generated pseudo glosses plus weak video-based reordering can substitute for most expert gloss annotations, reducing annotation cost from thousands of glosses to a few dozen. The ablations in Tables 6a-6e support the contributions of in-context examples, the reordering operation, and the classifier constraints. However, the paper releases no code, reports no error bars or significance tests, and leaves a load-bearing ambiguity about whether the reordering classifier and the downstream encoder share updating parameters.

major comments (4)
  1. [§3.3, Algorithm 1, §3.4] The reordering classifier and the downstream encoder appear to share the same vision encoder ψV, and the paper does not state whether ψV is frozen while ϕcls is trained or whether the reordered targets are generated offline before Stage 1 begins. If ψV is updated during classifier training, the reordered pseudo glosses used as CTC targets in Stage 1 are derived from features that the encoder is later trained to reproduce, so the +0.9 BLEU4 gain from reordering in Table 6d could partly reflect self-confirmation rather than genuine temporal alignment. Please state explicitly that the classifier is trained on a frozen ψV and that Algorithm 1 is run offline to fix the targets; otherwise, add an ablation that reorders using an independently trained or different visual encoder and report the resulting test BLEU4.
  2. [§4.2, Tables 4 and 5] The headline improvement over the previous gloss-free SOTA is small: 0.83 BLEU4 on Phoenix14T test (26.85 vs. 26.02) and 0.8 BLEU4 on How2Sign test (13.7 vs. 12.9), with no standard deviations, multiple seeds, or significance tests. For SLT models, run-to-run BLEU variation is typically comparable to these margins, so the abstract's claim of consistently outperforming previous gloss-free frameworks is not yet statistically supported. Please report repeated-run statistics or bootstrap confidence intervals for the main comparisons.
  3. [Abstract, §3.2, §4.2] The abstract states that the method 'eliminates the need for human-annotated glosses,' but the method uses 30 gold text-gloss pairs from Phoenix14T and 20 pairs from 2M-Flores-ASL for How2Sign as in-context examples. The paper should either present a zero-example variant as the main system or revise the claim to 'reduces the need for' or 'requires only a few dozen glosses,' since the current wording contradicts the experimental setup described in the paper itself.
  4. [§3.3, Table 6e] The classifier ablation in Table 6e is run with ground-truth glosses (with order information removed), not with the LLM-generated pseudo glosses that the actual pipeline uses. Because the LLM drafts are noisier, the reported precision/recall and WER may overstate the quality of the reordering signal in the real pipeline. Although Table 6d partially addresses this by showing WER changes on LLM glosses, please also report classifier metrics on the actual LLM pseudo-gloss bag labels, or explain why the true-gloss evaluation is representative.
minor comments (6)
  1. [Abstract] The abstract contains a typo: 'SLT mode' should read 'SLT model'.
  2. [Table 6d caption] The caption writes '30 paris' where '30 pairs' is intended.
  3. [Table 1 header] The header 'V ocabulary' has an errant space and should be 'Vocabulary'.
  4. [§4.2, How2Sign paragraph] The claimed '6.1 BLEU4 improvement' over SSVP-SLT does not match the numbers in Table 5 unless the comparison is to the non-† SSVP-SLT entry (7.0 BLEU4); please clarify which configuration is used.
  5. [Table 10(b)] The listed total trainable parameters (32,244,996) do not match the sum of the listed components (32,283,332); please correct the arithmetic.
  6. [Algorithm 1] The initialization 'target ← [ ] (empty sequence of length |L|)' is ambiguous because subsequent operations append to target; clarify whether target is a fixed-size list with placeholders or a growable sequence.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the LLM-generated pseudo glosses are externally grounded, the reordering targets are fixed before Stage 1, and final BLEU is evaluated independently.

full rationale

The derivation chain is anchored outside the model. Pseudo gloss content is produced by prompting an LLM with 30 text-gloss pairs from the training set, so the lexical targets do not come from the video encoder's own outputs (Sec. 3.2). The reordering stage (Sec. 3.3) trains a multi-label classifier with the LLM pseudo gloss set as weak labels and uses the classifier's frame-wise predictions only to permute those fixed pseudo glosses; the reordered sequence is then used as the CTC target in Stage 1 (Sec. 3.4). The paper describes extracting features with the vision encoder and training the classifier before the three-stage pipeline, and it defines Stage 1 as the stage that trains the vision encoder, so the reordered target is not a function of the final trained encoder. This is a self-distillation/pseudo-labeling design rather than a case where an output is defined in terms of the same quantity it is claimed to predict. The reported gains are measured on held-out BLEU/ROUGE, which is external to both the pseudo-gloss vocabulary and the classifier's frame-wise outputs. The paper even limits reliance on noisy pseudo gloss by dropping the CTC loss in the final epochs (Sec. 3.4). No load-bearing self-citation, imported uniqueness theorem, or fitted-parameter-as-prediction step appears. A residual concern is whether the classifier's inferred order is correct enough to be a good supervisor, but that is a robustness/correctness question, not circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The 'pseudo gloss' is a derived representation, not an invented entity. The main load-bearing assumptions are about LLM gloss quality and the reliability of the weakly supervised reordering.

free parameters (2)
  • w_base (frequency-aware weighting offset) = not stated
    In Eq. 8, the BCE weighting depends on w_base and log(fmax/fk). w_base is chosen by hand and the frequencies are computed from the LLM pseudo gloss drafts, so the weighting is fitted to the pseudo-label distribution.
  • Number of in-context example pairs = 30 for Phoenix14T, 20 for How2Sign
    The number of few-shot examples is a hand-picked hyperparameter; the paper shows performance improves with more examples (Table 6a).
assumptions (3)
  • domain assumption LLM pseudo glosses are a reliable proxy for true glosses in content and lexical choice
    Sec. 3.2 assumes LLMs can infer gloss-like annotations from text with a few examples; Table 6a shows BLEU4 of only 10.7-17.9 against true glosses, so this is a weak approximation.
  • domain assumption The frame-wise classifier's ordering is accurate enough to reorder pseudo glosses
    Sec. 3.3 trains a classifier with BCE on unordered pseudo-gloss sets, then uses its frame-wise predictions to perform greedy reordering (Algorithm 1). If these predictions are noisy, the CTC targets are corrupted.
  • domain assumption CTC on reordered pseudo glosses improves visual encoder training
    Stage 1 and 3.1 assume that enforcing sequential alignment to pseudo glosses generalizes to true gloss alignment, which is the core supervision mechanism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation." pith.science (2026). https://pith.science/paper/HBDN2GNM

@misc{pith2026250515438,
  author       = {Pith},
  title        = {Pith review of: Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HBDN2GNM}},
  note         = {Machine review of arXiv:2505.15438}
}
read the original abstract

Sign Language Translation (SLT) aims to map sign language videos to spoken language text. A common approach relies on gloss annotations as an intermediate representation, decomposing SLT into two sub-tasks: video-to-gloss recognition and gloss-to-text translation. While effective, this paradigm depends on expert-annotated gloss labels, which are costly and rarely available in existing datasets, limiting its scalability. To address this challenge, we propose a gloss-free pseudo gloss generation framework that eliminates the need for human-annotated glosses while preserving the structured intermediate representation. Specifically, we prompt a Large Language Model (LLM) with a few example text-gloss pairs using in-context learning to produce draft sign glosses from spoken language text. To enhance the correspondence between LLM-generated pseudo glosses and the sign sequences in video, we correct the ordering in the pseudo glosses for better alignment via a weakly supervised learning process. This reordering facilitates the incorporation of auxiliary alignment objectives, and allows for the use of efficient supervision via a Connectionist Temporal Classification (CTC) loss. We train our SLT mode, which consists of a vision encoder and a translator, through a three-stage pipeline, which progressively narrows the modality gap between sign language and spoken language. Despite its simplicity, our approach outperforms previous state-of-the-art gloss-free frameworks on two SLT benchmarks and achieves competitive results compared to gloss-based methods.

Figures

Figures reproduced from arXiv: 2505.15438 by the authors.

Figure 1
Figure 1. The training pipeline comprises three stages. Pseudo glosses are generated by LLMs and reordered using weakly supervised learning paradigms (see Sec. 3). thereby alleviating the challenges of direct video-to-text mapping and enhancing overall translation performance. Experiments demonstrate that our method outperforms existing gloss-free approaches and provides a viable alternative to traditional gloss-based methods… view at source ↗
Figure 2
Figure 2. and [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Example prompt used for pseudo gloss generation with Gemini 1.5 Pro [44]. The prompt contains a few example text-gloss pairs to guide the LLM in generating well-structured glosses for the query text. Detailed prompt formatting can be found in [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The prompt sent to the LLM for generating pseudo glosses includes two example pairs. These two examples serve as references to guide the LLM in producing accurate pseudo glosses during generation. The text marked in red represents spoken language from the datasets, whi…
Figure 5
Figure 5. Figure 5: The prompt sent to the LLM for generating pseudo glosses without providing example pairs. The text marked in red represents spoken language from the datasets, which should be replaced during each iteration. A.2 Greedy Reordering Strategy Here, we detail our Greedy Reor…
Figure 6
Figure 6. Figure 6: Example prompt that we feed to Gemini 1.5 Pro [44] for pseudo gloss generation on How2Sign [16]. Text this is dr art bowler and this has been how to tell if you have low self esteem Gloss doctor a r t b o w l e r how tell self esteem low Text so i you know i had to jus…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Sign Language QA benchmarks are introduced from PHOENIX14T and CSL-Daily via template-generated questions, and a question-conditioned baseline outperforms video-language and cascaded baselines.

Reference graph

Works this paper leans on

55 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Massively multilingual neural machine translation

    Roee Aharoni, Melvin Johnson, and Orhan Firat. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V olume 1 (Long and Short Papers), pages 3874–3884, 2019

    work page 2019

  3. [3]

    Bbc-oxford british sign language dataset

    Samuel Albanie, Gül Varol, Liliane Momeni, Hannah Bull, Triantafyllos Afouras, Himel Chowdhury, Neil Fox, Bencie Woll, Rob Cooper, Andrew McParland, et al. Bbc-oxford british sign language dataset. arXiv preprint arXiv:2111.03635, 2021

    arXiv 2021

  4. [4]

    Gesture and the nature of language

    David F Armstrong, William C Stokoe, and Sherman E Wilcox. Gesture and the nature of language . Cambridge University Press, 1995

    work page 1995

  5. [5]

    Curriculum learning

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceed- ings of the 26th annual international conference on machine learning , pages 41–48, 2009

    work page 2009

  6. [6]

    Weakly supervised deep detection networks

    Hakan Bilen and Andrea Vedaldi. Weakly supervised deep detection networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2846–2854, 2016

    work page 2016

  7. [7]

    Weakly supervised action labeling in videos under ordering constraints

    Piotr Bojanowski, Rémi Lajugie, Francis Bach, Ivan Laptev, Jean Ponce, Cordelia Schmid, and Josef Sivic. Weakly supervised action labeling in videos under ordering constraints. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 , pages 628–643. Springer, 2014

    work page 2014

  8. [8]

    Neural sign language translation

    Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. Neural sign language translation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7784–7793, 2018

    work page 2018

Show all 55 references
  1. [9]

    Sign language transformers: Joint end-to-end sign language recognition and translation

    Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10023–10033, 2020

    2020

  2. [10]

    A simple multi-modality transfer learning baseline for sign language translation

    Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, and Stephen Lin. A simple multi-modality transfer learning baseline for sign language translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5120–5130, 2022

    2022

  3. [11]

    Two-stream network for sign language recognition and translation

    Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, and Brian Mak. Two-stream network for sign language recognition and translation. Advances in Neural Information Processing Systems , 35:17043–17056, 2022

    2022

  4. [12]

    Factorized learning assisted with large language model for gloss-free sign language translation

    Zhigang Chen, Benjia Zhou, Jun Li, Jun Wan, Zhen Lei, Ning Jiang, Quan Lu, and Guoqing Zhao. Factorized learning assisted with large language model for gloss-free sign language translation. arXiv preprint arXiv:2403.12556, 2024

    2024 arXiv

  5. [13]

    Sign language recognition

    Helen Cooper, Brian Holt, and Richard Bowden. Sign language recognition. In Visual Analysis of Humans: Looking at People, pages 539–562. Springer, 2011

    2011

  6. [14]

    2m-belebele: Highly multilingual speech and american sign language comprehension dataset

    Marta R Costa-jussà, Bokai Yu, Pierre Andrews, Belen Alastruey, Necati Cihan Camgoz, Joe Chuang, Jean Maillard, Christophe Ropers, Arina Turkantenko, and Carleigh Wood. 2m-belebele: Highly multilingual speech and american sign language comprehension dataset. arXiv preprint arX...

    2024 arXiv

  7. [15]

    Recurrent convolutional neural networks for continuous sign language recognition by staged optimization

    Runpeng Cui, Hu Liu, and Changshui Zhang. Recurrent convolutional neural networks for continuous sign language recognition by staged optimization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7361–7369, 2017

    2017

  8. [16]

    How2sign: a large-scale multimodal dataset for continuous american sign language

    Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro-i Nieto. How2sign: a large-scale multimodal dataset for continuous american sign language. In Proceedings of the IEEE/CVF conference on computer visio...

    2021

  9. [17]

    Wildcat: Weakly supervised learning of deep convnets for image classification, pointwise localization and segmentation

    Thibaut Durand, Taylor Mordan, Nicolas Thome, and Matthieu Cord. Wildcat: Weakly supervised learning of deep convnets for image classification, pointwise localization and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 642...

    2017

  10. [18]

    Llms are good sign language translators

    Jia Gong, Lin Geng Foo, Yixuan He, Hossein Rahmani, and Jun Liu. Llms are good sign language translators. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18362–18372, 2024

    2024

  11. [19]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks

    Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning , pages 369–376, 2006

    2006

  12. [20]

    Multi-stream keypoint attention network for sign language recognition and translation

    Mo Guan, Yan Wang, Guangkun Ma, Jiarui Liu, and Mingzu Sun. Multi-stream keypoint attention network for sign language recognition and translation. arXiv preprint arXiv:2405.05672, 2024

    2024 arXiv

  13. [21]

    An efficient sign language translation using spatial configuration and motion dynamics with llms

    Eui Jun Hwang, Sukmin Cho, Junmyeong Lee, and Jong C Park. An efficient sign language translation using spatial configuration and motion dynamics with llms. arXiv preprint arXiv:2408.10593, 2024

    2024 arXiv

  14. [22]

    Lost in translation, found in context: Sign language translation with contextual cues

    Youngjoon Jang, Haran Raajesh, Liliane Momeni, Gül Varol, and Andrew Zisserman. Lost in translation, found in context: Sign language translation with contextual cues. arXiv preprint arXiv:2501.09754, 2025

    2025 arXiv

  15. [23]

    Visual alignment pre-training for sign language translation

    Peiqi Jiao, Yuecong Min, and Xilin Chen. Visual alignment pre-training for sign language translation. In European Conference on Computer Vision, pages 349–367. Springer, 2024

    2024

  16. [24]

    Cosign: Exploring co- occurrence signals in skeleton-based continuous sign language recognition

    Peiqi Jiao, Yuecong Min, Yanan Li, Xiaotao Wang, Lei Lei, and Xilin Chen. Cosign: Exploring co- occurrence signals in skeleton-based continuous sign language recognition. In Proceedings of the IEEE/CVF international conference on computer vision , pages 20676–20686, 2023

    2023

  17. [25]

    Google’s multilingual neural machine translation system: Enabling zero-shot translation

    Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association fo...

    2017

  18. [26]

    The kinetics human action video dataset

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017

    2017 arXiv

  19. [27]

    Neural sign language translation based on human keypoint estimation

    Sang-Ki Ko, Chang Jo Kim, Hyedong Jung, and Choongsang Cho. Neural sign language translation based on human keypoint estimation. Applied sciences, 9(13):2683, 2019

    2019

  20. [28]

    Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential parallelism in sign language videos

    Oscar Koller, Necati Cihan Camgoz, Hermann Ney, and Richard Bowden. Weakly supervised learning with multi-stream cnn-lstm-hmms to discover sequential parallelism in sign language videos. IEEE transactions on pattern analysis and machine intelligence , 42(9):2306–2320, 2019

    2019

  21. [29]

    Deep hand: How to train a cnn on 1 million hand images when your data is continuous and weakly labelled

    Oscar Koller, Hermann Ney, and Richard Bowden. Deep hand: How to train a cnn on 1 million hand images when your data is continuous and weakly labelled. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3793–3802, 2016

    2016

  22. [30]

    Re-sign: Re-aligned end-to-end sequence modelling with deep recurrent cnn-hmms

    Oscar Koller, Sepehr Zargaran, and Hermann Ney. Re-sign: Re-aligned end-to-end sequence modelling with deep recurrent cnn-hmms. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4297–4305, 2017

    2017

  23. [31]

    Weakly supervised learning of actions from transcripts

    Hilde Kuehne, Alexander Richard, and Juergen Gall. Weakly supervised learning of actions from transcripts. Computer Vision and Image Understanding, 163:78–89, 2017

    2017

  24. [32]

    Tspnet: Hierarchical feature learning via temporal semantic pyramid for sign language translation

    Dongxu Li, Chenchen Xu, Xin Yu, Kaihao Zhang, Benjamin Swift, Hanna Suominen, and Hongdong Li. Tspnet: Hierarchical feature learning via temporal semantic pyramid for sign language translation. Advances in Neural Information Processing Systems , 33:12034–12045, 2020

    2020

  25. [33]

    Uni-sign: Toward unified sign language understanding at scale

    Zecheng Li, Wengang Zhou, Weichao Zhao, Kepeng Wu, Hezhen Hu, and Houqiang Li. Uni-sign: Toward unified sign language understanding at scale. arXiv preprint arXiv:2501.15187, 2025

    2025 arXiv

  26. [34]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004

    2004

  27. [35]

    The emerging trends of multi-label learning

    Weiwei Liu, Haobo Wang, Xiaobo Shen, and Ivor W Tsang. The emerging trends of multi-label learning. IEEE transactions on pattern analysis and machine intelligence , 44(11):7955–7974, 2021

    2021

  28. [36]

    Multilingual denoising pre-training for neural machine translation

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics , 8:726–742, 2020. 20

    2020

  29. [37]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  30. [38]

    Sign language recognition: A deep survey

    Razieh Rastgoo, Kourosh Kiani, and Sergio Escalera. Sign language recognition: A deep survey. Expert Systems with Applications, 164:113794, 2021

    2021

  31. [39]

    Action sets: Weakly supervised action segmentation without ordering constraints

    Alexander Richard, Hilde Kuehne, and Juergen Gall. Action sets: Weakly supervised action segmentation without ordering constraints. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 5987–5996, 2018

    2018

  32. [40]

    Towards privacy-aware sign language translation at scale

    Phillip Rust, Bowen Shi, Skyler Wang, Necati Cihan CamgÃk, z, and Jean Maillard. Towards privacy-aware sign language translation at scale. arXiv preprint arXiv:2402.09611, 2024

    2024 arXiv

  33. [41]

    Open-domain sign language translation learned from online video

    Bowen Shi, Diane Brentari, Greg Shakhnarovich, and Karen Livescu. Open-domain sign language translation learned from online video. arXiv preprint arXiv:2205.12870, 2022

    2022 arXiv

  34. [42]

    Deafness and hereditary hearing loss overview

    Richard JH Smith, A Eliot Shearer, Michael S Hildebrand, Guy Van Camp, et al. Deafness and hereditary hearing loss overview. GeneReviews, 1993

    1993

  35. [43]

    Sign language translation from instructional videos

    Laia Tarrés, Gerard I Gállego, Amanda Duarte, Jordi Torres, and Xavier Giró-i Nieto. Sign language translation from instructional videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5625–5635, 2023

    2023

  36. [44]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  37. [45]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  38. [46]

    Youtube-asl: A large-scale, open-domain american sign language-english parallel corpus

    Dave Uthus, Garrett Tanzer, and Manfred Georg. Youtube-asl: A large-scale, open-domain american sign language-english parallel corpus. Advances in Neural Information Processing Systems , 36:29029–29047, 2023

    2023

  39. [47]

    Huggingface’s transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  40. [48]

    Sign2gpt: Leveraging large language models for gloss-free sign language translation

    Ryan Wong, Necati Cihan Camgoz, and Richard Bowden. Sign2gpt: Leveraging large language models for gloss-free sign language translation. arXiv preprint arXiv:2405.04164, 2024

    2024 arXiv

  41. [49]

    Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification

    Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, and Kevin Murphy. Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. In Proceedings of the European conference on computer vision (ECCV) , pages 305–321, 2018

    2018

  42. [50]

    Sign language translation with iterative prototype

    Huijie Yao, Wengang Zhou, Hao Feng, Hezhen Hu, Hao Zhou, and Houqiang Li. Sign language translation with iterative prototype. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15592–15601, 2023

    2023

  43. [51]

    Gloss attention for gloss-free sign language translation

    Aoxiong Yin, Tianyun Zhong, Li Tang, Weike Jin, Tao Jin, and Zhou Zhao. Gloss attention for gloss-free sign language translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2551–2562, 2023

    2023

  44. [52]

    Sltunet: A simple unified model for sign language translation

    Biao Zhang, Mathias Müller, and Rico Sennrich. Sltunet: A simple unified model for sign language translation. arXiv preprint arXiv:2305.01778, 2023

    2023 arXiv

  45. [53]

    Conditional sentence generation and cross-modal reranking for sign language translation

    Jian Zhao, Weizhen Qi, Wengang Zhou, Nan Duan, Ming Zhou, and Houqiang Li. Conditional sentence generation and cross-modal reranking for sign language translation. IEEE Transactions on Multimedia, 24:2662–2672, 2021

    2021

  46. [54]

    Gloss-free sign language translation: Improving from visual-language pretraining

    Benjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. Gloss-free sign language translation: Improving from visual-language pretraining. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision , pages 2087...

    2023

  47. [55]

    Improving sign language translation with monolingual data by sign back-translation

    Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. Improving sign language translation with monolingual data by sign back-translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1316–1325, 2021. 21

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.